EMNLP2026

Abstract:
Machine unlearning aims to remove specified training data while preserving retained behavior, but gradient-based updates often induce a forgetting-retention tradeoff. This motivates our first-principles objective: optimize forgetting while keeping the retaining loss locally unchanged, formalized as retain invariance. We prove an equivalence under optimizer-induced geometry: the retain loss is locally invariant if and only if the update is orthogonal to the subspace spanned by retain gradients. Crucially, this geometry differs from Euclidean geometry because optimizer preconditioning rescales raw gradients before parameter updates. Based on this characterization, we propose Geometric-disentanglement Unlearning (GU), a plug-and-play projection module that filters any candidate forget update without modifying the underlying unlearning objective. GU projects the candidate update under the optimizer-induced metric to enforce first-order retain invariance and, under a trust-region constraint, gives the steepest forget direction among retain-preserving updates. Across 72 TOFU configurations, GU improves or preserves retain-set Extraction Strength in 91.7% and model utility in 86.1%, with gains transferring to MUSE and WMDP. Optimizer-metric GU outperforms its Euclidean counterpart on all four metrics, while running 5.1× faster than the per-sample-QR baseline. Code is available at https://github.com/Lemutisme/Geometric-Unlearning.

Abstract:
Agents learn from experience, distilling it into memories, skills, or abstractions that improve which action they propose. The consequence of that action is discovered only after execution, when a long-horizon error has already become irreversible. We introduce WEVA (World-model Evolving Agents), a framework that makes gained experience predictive. WEVA learns from interaction traces what information is predictive and how actions transform the resulting state, using a quasi-symbolic representation over entities, preconditions, and outcomes. Specifically, before acting, EVA predicts the states induced by candidate action sequences, discards those that violate learned preconditions, and executes the first action of the highest-scoring surviving branch. The agreement between predicted and realised states delivers a model-trust signal that regulates when lookahead is reliable and when further interaction is more informative. WEVA improves task success while executing fewer actions, reduces trajectories that terminate in externally verified constraint violations, and detects changes in environmental dynamics before they are reflected in task success. These results position WEVA as a step towards agents that predict the consequences of their decisions and verify them against what they hold about the world before acting.

Abstract:
Large language models (LLMs) are emerging as versatile foundations for generative retrieval (GR), a paradigm that directly generates relevant document identifiers (docids) given a query. Existing ways of using LLMs for GR typically rely on heavy post-hoc training to fit a relevance data prior, often leading to overfitting and poor cross-domain generalization. Can retrieval, like other language tasks, be performed without retriever-specific training using LLMs? We conduct a preliminary study and find that constrained decoding, required to ensure valid docids, often misaligns with the model’s natural free-generation patterns. We propose FreeGR, a plug-and-play framework that adapts retrieval data to a LLM’s habitual generation behavior. FreeGR consists of three components: an LLM self-annotated docid generator, an LLM-preferred input context designer, and two decoding strategies that trade off retrieval effectiveness and efficiency. It requires no retriever-specific parameter optimization, is model-agnostic, and scales across LLM families and sizes while preserving their generative capabilities. Experiments on the NQ, MS MARCO, and BEIR datasets show that FreeGR achieves competitive retrieval effectiveness compared with several trained generative and dense retrievers, without retriever-specific parameter optimization.

Abstract:
Integrating external tools enables Large Language Models (LLMs) to interact with real-world environments and solve complex tasks. Given the growing scale of available tools, effective tool retrieval is essential to mitigate constraints of LLMs’ context windows and ensure computational efficiency. Existing approaches typically treat tool retrieval as a traditional ad-hoc retrieval task, matching user queries against the entire raw tool documentation. In this paper, we identify three fundamental challenges that limit the effectiveness of this paradigm: (i) the incompleteness and structural inconsistency of tool documentation; (ii) the significant semantic and granular mismatch between user queries and technical tool documents; and, most importantly, (iii) the multi-aspect nature of tool utility, that involves distinct dimensions, such as functionality, input constraints, and output formats, varying in format and importance. To address these challenges, we introduce Multi-Field Tool Retrieval, a framework designed to align user intent with tool representations through fine-grained, multi-field modeling. Experimental results show that our framework achieves SOTA performance on five datasets and a mixed benchmark, exhibiting superior generalizability and robustness. We have open-sourced all the code and prompts in https://github.com/LittleDinoC/MFTR.

Abstract:
We study whether Universal Grammar syntactic principles can serve as explicit inductive biases for unsupervised grammar induction. We introduce a neuro-symbolic PCFG framework that converts hypothesized linguistic universals into rule schemata, and then learns rule probabilities from child-directed speech. We investigate three assumptions: 1) extended projection, 2) prototype theory of part-of-speech, and 3) the final-over-final constraint. Experiments show that with a much smaller hypothesis space, the universal-priored model achieves superior performance to state-of-the-art neural grammar induction models, and yields more interpretable functional category assignment.

Abstract:
Standard Mixture-of-Experts (MoE) designs rely heavily on centralized routing mechanisms that introduce rigid inductive biases. We propose Routing-Free MoE, which eliminates all hard-coded centralized components, including external routers, Softmax, TopK, and fixed load balancing. All related functionalities become encapsulated within individual experts, allowing each expert to determine its activation entirely on its own. We introduce a unified, adaptively scheduled load-balancing framework to simultaneously optimize both expert-balancing and token-balancing objectives through a customizable interpolation, letting optimal resource allocation pattern emerge locally during training. Extensive experiments demonstrate that Routing-Free MoE achieves consistently better performance, efficiency, and robustness than baseline approaches at both training and deployment. We further analyze its emergent behavior in detail and offer insights that may facilitate future MoE design and optimization. Code is available at https://liuyilun2000.github.io/RoutingFreeMoE/.

Abstract:
Watermarking should identify language-model output without degrading quality or limiting verification to the model provider. Multilingual deployment makes this harder because morphology, segmentation, and script change where watermark evidence can be naturally embedded. We introduce LUNA, a linguistically adaptive watermark that combines model-free detection with single-token non-distortion under the standard random-key model. LUNA estimates normalized next-tag entropy from part-of-speech contexts in an external corpus and uses it to set the depth of a non-distortionary binary tournament sampler; the detector reconstructs the same schedule from text, a tokenizer, a tagger, and a secret key. We evaluate LUNA on six typologically diverse languages and two domains against eight primary baselines. LUNA attains an AUROC of 0.9959 and the lowest mean absolute median perplexity shift, 0.045, across the 12 settings; its 95% bootstrap interval [0.022, 0.073] lies below all baseline intervals. LUNA also records the lowest mean on Self-BLEU, Distinct-1, surprisal, and entropy shifts; it is the only method that simultaneously achieves AUROC > 0.99 and |ΔPPLmed| < 0.1 in a majority of settings, reaching this regime in 9 of the 12 settings while no baseline reaches it in more than 2. Our code is available at https://github.com/Shinwoo-Park/luna_watermark.

Abstract:
Block Diffusion Language Models (BD-LMs) support KV caching and flexible-length generation, but their native Single-Block Diffusion (SingleBD) inference leaves inter-block parallelism unused. We study Multi-Block Diffusion (MultiBD), where a running-set of consecutive blocks is refined concurrently. Existing teacher-forced BD-LMs observe only one noisy block conditioned on a clean prefix, while diffusion forcing exposes a long noisy suffix whose state distribution still differs from practical MultiBD inference. We bridge this gap with Multi-Block Diffusion Language Models (MBD-LMs), obtained by post-training BD-LMs using Multi-block Teacher Forcing (MultiTF). MultiTF trains on bounded noise-groups conditioned on clean prefixes, with heterogeneous slot-wise noise patterns that better match inference. An optimized Block Buffer further preserves prefix-cache reuse and static input shapes, translating inter-block parallelism into wall-clock acceleration. On math and code benchmarks, MBD-LLaDA2-Mini increases average Tokens Per Forward pass (TPF) from 3.47 to 6.19 while maintaining comparable accuracy (80.0% versus 81.0%). Combined with DMax, it reaches 9.34 TPF with a 1.02-point accuracy drop. Project Page: https://sjtu-deng-lab.github.io/mbd-lms

Abstract:
KV caches are a major bottleneck in long-context inference and long-form generation with large language models. Existing training-free eviction policies largely rely on proxy importance signals, such as attention mass, to decide which past tokens to retain. We argue that cache compression should instead preserve the predictive behavior of the full-cache model, retaining entries whose removal would substantially change the model’s output distribution. We propose Behavior-Preserving KV Cache Compression, a training-free framework that scores candidate evictions by estimating the compressed-cache logits induced by their removal and evaluating the resulting KL to the full-cache next-token distribution. Using pre-eviction forward statistics, the method avoids running separate masked forward passes for each candidate. Across diverse architectures and both prefill-time and generation-time compression, our method delivers substantial gains in downstream task quality over lightweight attention-based heuristics at matched retained-KV budgets, with the largest gains under aggressive compression. It achieves these gains with additional compression-time computation while retaining an end-to-end speedup over full-cache inference in our evaluated settings.

Abstract:
Machine Translation (MT) and automatic MT evaluation have improved dramatically in recent years, enabling numerous novel applications. Automatic evaluation techniques have evolved from producing scalar quality scores to precisely locating translation errors and assigning them error categories and severity levels. However, it remains unclear how to reliably measure the evaluation capabilities of auto-evaluators that do error detection, as no established technique exists in the literature. This work investigates different implementations of span-level precision, recall, and F-score, showing that seemingly similar approaches can yield substantially different rankings, and that certain widely-used techniques are unsuitable for evaluating MT error detection. We propose "match with partial overlap and partial credit" (MPP) with micro-averaging as a robust meta-evaluation strategy and release code for its use publicly. Finally, we use MPP to assess the state of the art in MT error detection.

Abstract:
Diffusion language models (DLMs) are theoretically efficient but are limited by fixed-length decoding and poor compatibility with KV caches. Block diffusion mitigates these issues, yet still enforces a fixed block size and costly training. We introduce Next Sequence Prediction (NSP), which unifies next token and next block prediction, enabling the model to adaptively determine generation length at each step. NSP reduces to standard next token prediction when the length is one. Building on NSP, we propose the Sequential Diffusion Language Model (SDLM), which retrofits pretrained autoregressive language models (ALMs) at minimal cost. Specifically, SDLM performs diffusion inference within fixed-size mask blocks, but dynamically decodes contiguous subsequences based on model confidence, thereby preserving KV-cache compatibility and improving robustness to the varying uncertainty and semantics across the sequence. SDLM matches or surpasses strong autoregressive baselines with only 3.5M training samples, while achieving 2.0× 3.4× higher throughput than Qwen2.5 across three model scales (3B, 7B, 32B) with confidence-based decoding. Furthermore, when sampling with self-speculative decoding, SDLM attains up to 5.7× end-to-end speedup measured as wall-clock tokens per second (TPS). Notably, SDLM-32B achieves performance comparable to Qwen2.5-32B with up to 2.5× speedup, demonstrating the scalability of our approach. Project page and codes: https://github.com/OpenGVLab/SDLM

Abstract:
Large language model reasoning is often treated as a monolithic capability, relying on binary preference supervision that fails to capture partial progress or fine-grained reasoning quality. We introduce continuous utility direct preference optimization CU-DPO, a framework that aligns models to a portfolio of prompt-based cognitive strategies by replacing binary labels with continuous scores that capture fine-grained reasoning quality, proving that learning with K strategies yields a 𝛩(K log K) improvement in sample complexity over binary preferences and that DPO converges to the entropy-regularized utility-maximizing policy. To exploit this signal, we propose a two-stage pipeline: (i)strategy selection, which optimizes the model to choose the best strategy via best-vs-all comparisons, and (ii) execution refinement, which trains correct execution using margin-stratified pairs. The framework is domain-agnostic: any task admitting cognitively distinct solution strategies and a decomposable continuous utility signal can be incorporated into the portfolio. On mathematical reasoning benchmarks, CU-DPO improves strategy selection accuracy from 35–46% to 68–78% across seven base models, yielding downstream reasoning gains of up to +6.6 points on in-distribution datasets with effective out-of-distribution transfer, and demonstrates consistent gains on code generation and causal reasoning benchmarks, confirming generalization beyond the mathematical domain.

Abstract:
Post-training quantization (PTQ) of large language models (LLMs) reduces memory usage and accelerates inference while maintaining accuracy. However, it remains unclear how to select a quantization scheme that achieves target accuracy across different model scales. We introduce two parameters to characterize this: the quantization ratio Q r (proportion of parameters quantized to low precision) and the block size Q b (number of parameters sharing a scaling factor). Based on extensive experiments across multiple LLM families and scales, we propose a unified scaling law that predicts quantization-induced loss degradation as a function of Q r , Q b , and model size. Our scaling law reveals two practical implications: (1) larger models tolerate higher Q r , making mixed-precision increasingly effective at scale; (2) small Q b , commonly used in mixed precision, becomes unnecessary for large models, simplifying hardware design without sacrificing accuracy. These findings provide a principled recipe for quantizing LLMs in practical deployment.

Abstract:
Multi-agent reasoning systems adopt a generate-then-transfer paradigm that forces end-to-end latency to scale linearly with pipeline depth. We introduce StreamMA, a multi-agent reasoning system that streams each reasoning step to downstream agents as soon as it is generated, pipelining adjacent agents and thus reducing latency. Surprisingly, this pipelining also improves effectiveness: because multi-step reasoning quality is non-uniform and early steps are more reliable than later ones, working with these reliable early steps instead of the full chain prevents error-prone late steps from misleading downstream agents. We formalize both advantages with the first closed-form joint analysis of stream, serial, and single protocols, deriving the effectiveness ordering, speedup upper bound, and cost ratio. Across eight reasoning benchmarks spanning mathematics, science, and code, three frontier LLMs (Claude Opus 4.6, GPT-5.4, and GLM-5.2), and three topologies (Chain, Tree, Graph), StreamMA outperforms both baselines (avg.+7.3pp, max +22.4pp on HMMT2026; Claude Opus 4.6-High). Beyond these contributions, we discover a step-level scaling law: increasing per-agent steps consistently improves both effectiveness and efficiency, a new scaling dimension orthogonal to and composable with agent-count scaling.

Abstract:
Syntactic prediction requires models to generate outputs that are globally consistent and satisfy hierarchical structural constraints. Although large language models have been applied to such tasks, existing approaches still rely mainly on prompting or supervised fine-tuning, and their performance generally remains below that of specialized neural syntactic models. Our experiments show that supervised fine-tuning reaches a plateau, and additional supervised training does not further improve prediction quality. We therefore propose a two-stage post-training framework for syntactic prediction. The first cold-start stage adapt the model to a unified syntax generation format, and then applies reinforcement learning on a disjoint data subset to directly optimize structure-level objectives. Experiments on constituency parsing and CCG supertagging show that Syntax-RL consistently outperforms supervised LLM baselines and achieves state-of-the-art results. Further analyses suggest that its gains come primarily from the reinforcement learning stage. These findings identify reinforcement learning as an effective post-training mechanism for improving hierarchical syntactic prediction in large language models.

Abstract:
Fine-tuning large language models (LLMs) on narrowly harmful datasets can lead to misalignment broadly, a phenomenon known as emergent misalignment (EM). EM poses a challenge for AI safety and our understanding of LLMs. Prior work often frames EM as an unexpected behavior, and explains it by appealing to general misalignment directions or anthropomorphizing it as acquiring an evil persona. However, the mechanisms behind these framings remain obscure. In this work, we show that EM is a predictable and data-dependent generalization phenomenon. By examining the base model’s representation of EM training data and evaluation prompts, we find that evilness after EM training is highly predictable from representational distance: the closer an evaluation prompt is to training data centroid, the more evilness it elicits from EM models after training (with an average Spearman correlation of -0.73 across 12 model-dataset settings). Building upon this analysis, we further demystify EM by showing that (1) its effectiveness changes significantly based on training data format; (2) there is not a general misalignment direction that transfers across different EM models; (3) the effect of EM is fundamentally different from persona changes. Furthermore, we extend the EM generalization metric from a scalar distance to a dataset-specific generalization direction, which robustly predicts EM models’ evilness under semantics-preserving prompt perturbations including appending random tokens and paraphrasing, where other methods do not reliably generalize.

Abstract:
The progress of large language models (LLMs) has fueled claims that model-generated summaries rival or even surpass human-written references, raising questions about whether summarization remains an open research problem. We re-examine this narrative through a multi-track evaluation covering diverse datasets and state-of-the-art LLMs, combining controlled human assessment, bias-mitigated LLM-as-Judge protocols, factuality verification against external knowledge, and corpus-level linguistic analysis. Our findings reveal a more nuanced landscape in which human references continue to demonstrate advantages in informativeness and faithfulness, whereas LLM outputs are preferred mainly for surface-level coherence and fluency. Factuality verification indicates that human references remain more reliable, particularly for claims involving reasoning or synthesis, and linguistic analysis uncovers a pattern of stylistic homogeneity across different models. These observations suggest that current LLMs have raised the floor of summarization quality, but the ceiling of their performance remains below human capabilities.

Abstract:
Linguistic typology relies on expert analysis of reference grammars across languages, making large-scale crosslinguistic comparison labor-intensive and unscalable. We introduce AutoTypologist, an LLM agent for evidence-grounded typological analysis over reference grammars. The agent is capable of retrieving relevant grammar sections, analyzing interlinear glossed text (IGT), and iteratively reasoning over typological hypotheses using a ReAct-style workflow. We evaluate the system on Typological Feature Coding against expert annotations and Typological Hypothesis Testing with typological universals using 25 open-source reference grammars. Operating under different information constraints in Typological Feature Coding, the agent can synthesize information from reference grammar prose but still faces challenges with only IGTs in the target language. In Typological Hypothesis Testing, the agent can synthesize crosslinguistic evidence and identify both supporting cases and counterexamples. These findings suggest that LLM agents can support scalable and inspectable typological analysis, while still requiring expert validation.

Abstract:
We study self-modeling: an LLM’s ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a prompt edit would change the model’s final answer. To measure this capability, we introduce a benchmark that tests diverse types of self-modeling questions. Current models show non-trivial but limited self-modeling skill, and make systematic mistakes on simple counterfactual questions about their own behavior. To improve self-modeling skill, we develop a scalable synthetic-data pipeline that produces self-modeling training data, and show that reinforcement-learning can improve aggregate self-modeling skill across three open-source model families with some transfer to held-out tasks. These gains, however, do not seem to constitute introspection consistently: improved self-modeling may not arise from privileged access to the model’s internal decision process. We release our evaluation and training code on GitHub.

Abstract:
Current language modeling approaches are built around tokens. Text corpora are split into tokens, and models are trained by performing computations on these tokens, such as predicting the next token given the preceding ones as context. This paradigm has become the standard in modern language modeling, especially given the outstanding performance obtained by token-based architectures. However, recent works have not only begun to question how language models process and understand meaning from tokens, but also to question whether using higher levels of granularity could advance the research field. This led to the idea of Concept Modeling, that is, to directly train models for next-concept prediction rather than next-token prediction. In this work, we introduce Mimir, a 1.6B Large Concept Model trained for multilingual concept understanding and generation. We leverage a large-scale multilingual pre-training corpus (38,883,987,240 sentences) spanning 46 languages and a large-scale multi-turn and multilingual instruction-tuning dataset (66,816,428 sentences) covering a total of 35 languages. We extensively evaluate model performance against a language model with a comparable number of parameters.

Abstract:
Document Question Answering (DocQA) focuses on answering questions grounded in given documents, yet existing DocQA agents lack effective tool utilization and largely rely on closed-source models. In this work, we introduce DocDancer, an end-to-end trained open-source Doc agent. We formulate DocQA as an information-seeking problem and propose a tool-driven agent framework that explicitly models document exploration and comprehension. To enable end-to-end training of such agents, we introduce an Exploration-then-Synthesis data synthesis pipeline that addresses the scarcity of high-quality training data for DocQA. Training on the synthesized data, the trained models on two long-context document understanding benchmarks, MMLongBench-Doc and DocBench, show their effectiveness. Further analysis provides valuable insights for the agentic tool design and synthetic data.

Abstract:
Neural networks over non-Euclidean manifolds have attracted increasing attention across various applications. A fundamental step in these networks is to map Euclidean features onto a target manifold. Existing methods mainly use exponential maps and projections, operate through tangent or ambient Euclidean spaces, and may therefore distort intrinsic manifold geometry. On the other hand, point-to-hyperplane distances have recently shown great success in building manifold-valued networks. Based on this, we propose the Point-to-Hyperplane Distance Map (PHDM), a parameter-free mapping that extends Euclidean fully connected layers to Euclidean-to-manifold transformations. Specially, the exponential map is recovered as a pseudo-distance-based special case. We manifest PHDM on the Lorentz, Poincaré, spherical, and projected-spherical models, covering both negative and positive curvature. Experiments on arithmetic reasoning across four LLMs, image classification, graph node classification, and link prediction demonstrate consistent improvements over the corresponding baselines. The code is available at https://github.com/Longchentong/PHDM.

Abstract:
Group relative policy optimization for reinforcement learning with verifiable rewards (RLVR) typically uses a fixed importance-sampling (IS) ratio clipping boundary across all rollouts. We identify a key limitation: rare correct rollouts on harder problems and abundant correct rollouts on easier problems are clipped at comparable rates, despite contributing very different learning signals. Rollouts with low group success exhibit larger IS ratios and carry stronger gradient signal for exploration and solving new problems, yet are disproportionately suppressed by fixed clipping. To address this, we propose Group Adaptive Clipping Policy Optimization (GAPO), a plug-in modification to GRPO methods that adapts the clipping boundary to the rollout advantage. GAPO is motivated by a reverse-KL trust-region perspective, which suggests that rollouts with larger learning signal should receive proportionally greater update headroom. GAPO requires no reward shaping and preserves the standard PPO/GSPO surrogate while adapting only the clipping threshold. Across Qwen and Llama models, GAPO consistently improves both Pass@1 and Pass@k over fixed clipping and advantage-shaping baselines on math reasoning and coding benchmarks where the pass rates by the base model are relatively low.

Abstract:
Reinforcement learning with verifiable or outcome-level rewards has recently become an effective post-training paradigm for improving the reasoning abilities of large language models, with GRPO-style optimization achieving strong results on mathematical and other verifiable tasks. However, for multi-step reasoning, outcome-level supervision remains intrinsically sparse: a trajectory that contains partially correct intermediate derivations but makes a late mistake is still treated as a complete failure. As a result, useful reasoning prefixes are penalized together with erroneous continuations, and informative failed trajectories are underutilized. PRMs provide denser step-level feedback, but they often rely on costly process annotation or auxiliary process verifiers, and optimizing local process rewards alone may fail to preserve the global objective of final correctness, leaving room for reward misspecification and reward hacking during reinforcement fine-tuning. We propose PRSA(Process Rewarding via Solution-Space Alignment), a simple framework that bridges outcome supervision and process supervision without requiring human step-level labels. PRSA automatically extracts intermediate-value steps from model outputs and reference solutions, aligns them within the solution space under equivalence and order constraints to estimate reasoning progress, and uses this signal for both reward shaping and token-level prefix-suffix credit assignment. In this way, PRSA preserves the global anchor of final-answer correctness while providing denser and better localized learning signals for partially correct trajectories.

Abstract:
We present a new annotated corpus for the task of Nested Entity Recognition from legal acts in the EU acquis, the collection of common rights and obligations that constitute the body of EU law and is incorporated into the legal systems of EU Member States. The corpus is subdivided into two subcorpora: GOLD - human annotated (41 documents) and SILVER - created semi-automatically (537 documents). They contain 28.5K and 275K entity mentions labelled with fine-grained entity types using a dedicated taxonomy of 45 fine-grained entity types. We describe the corpus creation process and present corpus statistics. We use knowledge-, transition-, transformer-, and generative AI-based models to provide baselines and benchmark results.

Abstract:
Select-then-predict rationalizers expose compact rationale-conditioned predictions, but their generator and predictor can interlock: poor rationales mislead the predictor, while predictor lag or shortcut adaptation sends noisy feedback to the generator. Existing studies reduce this interaction by separating generator search from predictor training, but repeated predictor training is expensive, resulting in poor trade-offs. To address this problem, we propose ReSR (Responsibility-Aware Scheduling for Rationalization), a training-control framework that keeps the standard select-then-predict path while adaptively scheduling generator and predictor updates. ReSR uses online, probe, and reference diagnostics with rationale-structure signals to attribute failures to generator-side rationale quality or predictor lag; a virtual-runtime scheduler then converts this attribution into generator/predictor debts and allocates the next optimization slice. Experiments on Toy and HateXplain show that ReSR improves the accuracy–fidelity–compactness–cost balance: ReSR achieves the best Toy highlight F1 and performance comparable to the state-of-the-art approach on HateXplain highlight F1, while attaining stronger classification performance and substantially lower cost.

Abstract:
The rapid development of large language models (LLMs) has transformed many industries, including healthcare. In practice, hospitals and patients increasingly seek LLM-based systems capable of interpreting personal health records and providing healthcare guidance. However, existing approaches mainly rely on general medical knowledge and often fail to account for individual variability, limiting their ability to provide personalized guidance. To address this, we propose personalized prompt learning (PPL), a framework that learns individualized prompts to guide LLMs in generating personalized healthcare recommendations. PPL constructs initial personalized prompts by leveraging both self-informed patient information and peer-informed signals derived from clinically similar cases. These prompts are then refined using reinforcement learning (RL) to better align the generated responses with physician recommendations written for each patient. PPL operates with hard prompts, enabling seamless integration with proprietary LLMs without modifying the underlying models. We evaluate PPL on real-world obstetrics and gynecology data. The results show that our approach produces more personalized healthcare guidance and wins 97 out of 100 comparisons in expert evaluation, demonstrating its potential for broader healthcare applications. Our code is publicly available at https://github.com/CGCL-codes/PPL.

Abstract:
Modern information retrieval increasingly employs both embedding and generative models to handle complex queries. However, current serving systems suffer from low throughput and poor GPU utilization because they execute these models in isolation. Coarse-grained partitioning, such as dedicating GPUs to specific tasks, fails to adapt to dynamic workloads and creates computational “bubbles”. To address these, we present Orthrus, a serving system that performs heterogeneous batching within a single GPU kernel. The primary challenge lies in unifying embedding and generation workloads with conflicting computational patterns while optimizing batch composition for high performance. Orthrus addresses these challenges by employing incremental computing to match compute patterns and by adjusting batch composition in a workload-aware manner. Evaluation on four A100 GPUs shows that Orthrus achieves 1.28×–4.52× higher throughput and up to 52.0% lower latency than various deployment scenarios based on homogeneous batching, including those built on vLLM. We release our code at [https://github.com/illinoisdata/Orthrus](https://github.com/illinoisdata/Orthrus)

Abstract:
Test-time training (TTT) adapts large language models (LLMs) during inference using only unlabeled test inputs. Existing methods, however, face two major bottlenecks on hard reasoning tasks: (1) lack of learnable samples, as self-generated pseudo-labels on difficult questions are often noisy and yield unstable rewards; and (2) inefficient exploration, as performance gains depend on repeatedly sampling many rollouts without explicit diagnosis of why previous attempts fail. We propose TTSR (Test-Time Self-Reflection), a self-evolving framework based on a reflect-then-synthesize paradigm. A single pretrained model alternates between a Student role and a Teacher role: the Student solves test questions and updates, while the Teacher analyzes failed trajectories and synthesizes targeted variant questions closer to the Student’s capability frontier. TTSR further maintains a cross-iteration weakness memory and compiles persistent weaknesses into a lightweight strategy note prepended to subsequent Student inputs, so diagnostic knowledge can guide exploration and gradually fade as weaknesses are resolved. Experiments on challenging mathematical reasoning benchmarks show consistent test-time improvements, strong cross-backbone generalization, and transfer to general-domain reasoning tasks.

Abstract:
This paper introduces a novel data augmentation approach designed for argument mining. Argument mining is a task that automatically identifies argumentative structures within texts, typically represented as trees or directed acyclic graphs (DAGs). A major challenge facing current argument mining systems is the limited structural diversity of existing annotated datasets, as manual annotation inevitably captures only a small subset of the argumentative patterns encountered in practice. To overcome this challenge, we propose an innovative data augmentation method. Our approach begins by generating pseudo-argumentative structures through the exploration of various combinations of argument labels. These structures then guide a large language model (LLM) to produce new texts with annotations directly incorporated into the text. We evaluated our method on the Argument Annotated Essay Corpus (AAEC) and the AbstRCT dataset. Experimental results showed that the proposed method outperformed existing augmentation baselines on Component-F1 and Relation-F1 in most settings. Further analysis indicated that the variety in the generated pseudo-argumentative structures plays a crucial role in achieving these improvements.

Abstract:
Retrieval-Augmented Generation (RAG) has established itself as an innovative technology enabling Large Language Models (LLMs) to answer difficult questions beyond their internal knowledge. Graph-based RAG, which utilizes a triplet-based graph as a database, offers the advantage of structuring relationships between entities through graph connectivity, thereby capturing connections within the corpus that traditional RAG cannot express. However, this paper points out that the process of converting a corpus into a triplet-based graph using LLMs may be unsuitable for RAG due to the inability to represent hierarchical information and structural rigidity. To address this, we propose a novel database structure called the TRA(Title-Relations-Attributes) structure. Through a series of experiments, we highlight the benefits of indexing TRA Structure as a fact to utilize it as a retrieval-optimized data structure, and demonstrate its powerful multi-hop reasoning performance when combined with query decomposition, while also covering key insights in multi-hop reasoning.

Abstract:
Edge devices such as smartwatches and smart glasses cannot continuously run even the smallest 100M–1B parameter language models due to power and compute constraints, yet cloud inference introduces multi-second latencies that break the illusion of a responsive assistant. We introduce micro language models ( 𝜇 LMs): ultra-compact models (8M–30M parameters) that instantly generate the first 4-8 words of a contextually grounded response on-device, while a cloud model completes it; thus, masking the cloud latency. We show that useful language generation survives at this extreme scale with our models matching several 70M-256M-class existing models. We design a collaborative generation framework that reframes the cloud model as a continuator rather than a respondent, achieving seamless mid-sentence handoffs and structured graceful recovery via three error correction methods when the local opener goes wrong. Empirical results show that 𝜇 LMs can initiate responses that larger models complete seamlessly, demonstrating that orders-of-magnitude asymmetric collaboration is achievable and unlocking responsive AI for extremely resource-constrained devices.

Abstract:
Contrastive Decoding (CD) improves generation quality, but its amateur-model pass makes decoding expensive. Accelerating CD with speculative decoding raises a proposal-alignment question: should the contrastive signal shape the drafter, or should it remain only in verification? We study this question in the lightweight feature-level drafter regime. Two controlled diagnostics, matched Cross-α training and an Approximate Dual-Drafter decomposition, give the same diagnosis: contrastive-aware drafting does not consistently improve over expert-aligned drafting because the contrastive correction is usually weaker than drafter error, and reconstruction can amplify that error. We introduce Decoupled Contrastive Decoding (DCD), which drafts with an expert-aligned lightweight proposer and applies the amateur only in unchanged CD verification. Standard speculative verification preserves the vanilla-CD output distribution. Across the main 8B settings, EAGLE3-based DCD achieves average greedy speedups of 1.65 to 1.95× over vanilla CD and reduces MMLU proposal-path latency by about 5 to 12× relative to amateur-coupled proposal paths.

Abstract:
LLM-generated summaries of customer feedback increasingly shape purchasing decisions and organisational strategy. However, we find that such summaries often do not stay faithful to the customer voice, potentially undermining the validity of any downstream decisions made from them. In particular, through a systematic evaluation of 50,000 synthetic customer reviews across 50 domains and 11 models, we demonstrate that the same topics are amplified across different models, regardless of their prominence in the actual reviews. More strikingly, models continue to amplify these same topics even when they are entirely absent from the input, and the topics that are amplified correlate with those the model independently rates as most important. Our findings were validated on real-world reviews from four domains. Furthermore, no intervention we tested, including scaling to frontier models, prompt engineering, and varying temperature, summary length, and input size, reduces the bias to acceptable levels.

Abstract:
Evidence grounding makes generated content verifiable by linking statements to their sources. However, effective verifiability requires two properties that existing methods fail to jointly provide: verification granularity, the ability to localize fine-grained evidence for each statement, and verification completeness, the ability to capture both support and contradiction. Existing methods either operate at granularities too coarse for rapid inspection or are support-only. We formalize polarity-aware evidence grounding: linking each response sentence to its contextual evidence with a supporting, contradicting, or irrelevant label. We construct eTracer-Bench, a human-annotated benchmark with 7,905 supporting and contradicting sentence pairs across seven domains, covering both explicit and implicit contradictions. We further propose eTracer, a post-hoc framework whose context-aware signed scorer recovers the full evidence map in a single forward pass per response, with localization and polarity emerging jointly from one mechanism. eTracer achieves the best overall Citation F1 of 82.39, driven by a Contradict F1 of 89.20. A pre-registered within-subjects user study ( n = 30 ) shows that polarity-aware grounding nearly doubles contradiction detection over the document-level baseline (27.9% → 51.9%, Cohen’s d = 0.82 , p = 0.0003 ), and suggests a false-completeness bias under which support-only grounding fails to improve over this baseline (23.4% vs. 27.9%). Dataset and code: https://github.com/chubohao/eTracer.

Abstract:
As Large Language Models rapidly advance, performance on standard multiple-choice question answering (MCQA) benchmarks is reaching saturation. While the community has responded by developing increasingly difficult datasets, validating question quality and filtering flawed items remains a labor-intensive process. To provide a scalable diagnostic approach, we propose a two-component probabilistic framework for auditing MCQA benchmarks using model output distributions. First, for benchmark-level analysis, we characterize the probability landscape using the top prediction probability ( P top1 ) and normalized residual entropy ( H norm ), summarized globally by Mean Pairwise Distance (MPD). Second, for item-level diagnostics, we introduce noise injection to reduce meaningful distractor competition, enabling us to flag candidate items for targeted human review and categorize residual failure patterns. Across four MCQA benchmarks, our landscape analysis reveals benchmark-level differences in model confidence and residual option competition. Concurrently, our noise-injection method flags potentially actionable item-level issues, showing alignment with expert error annotations from MMLU-Redux. These results suggest that our probability-based framework provides a lightweight audit lens for comparing macro-level benchmark structure and prioritizing individual items for targeted human review.

Abstract:
Test-time scaling (TTS) improves language model reasoning by allocating additional test-time compute, but prior work mainly studies closed-ended tasks such as math and coding. We study TTS for automated equation discovery, an open-ended setting where models search over candidate equations and rely on observed datapoints for feedback. We formulate LLM-driven equation discovery as an iterative search process that unifies Best-of-N, sequential refinement, tree search, and evolution-style methods under a common compute-allocation view. To isolate allocation effects from prompt engineering and other heuristics, we compare minimal parallel controllers under fixed budgets. On LLM-SRBench equation-discovery tasks, we find that search width is the dominant allocation parameter: the best width in our sweep generally increases with the compute budget, while the population–branching split and controller choice matter less. Appropriate width selection also improves wall-clock efficiency by increasing parallelism. These results suggest that, given an informative verifier, controlling exploration and exploitation is central to scaling LLM-based equation discovery.

Abstract:
Existing summarizers for memory systems are typically optimized for human-facing criteria such as faithfulness, which misaligns with their true objective: preserving the evidence needed to support future queries. We show that conditioning summarization on query-answer pairs substantially improves answer quality, and that this utility-aware behavior is transferable across queries. Motivated by these findings, we propose MemSuit, a self-distillation framework in which a teacher summarizer, conditioned on observed query-answer pairs, produces utility-aware memory entries that a student learns to reproduce from the raw conversation alone. To prevent collateral erasure where conditioning on a single query-answer pair discards evidence relevant to other plausible queries, the teacher decomposes each block into multiple self-contained entries that preserve distinct query-relevant facets as independently retrievable units. To align the retriever with the compact, fact-dense style of teacher entries, we further fine-tune the embedding model with a contrastive objective supervised by teacher entries. Across a diverse suite of conversational query types, MemSuit consistently outperforms state-of-the-art baselines, confirming the value of grounding memory in downstream utility.

Abstract:
Knowledge editing aims to update specific factual knowledge in large language models without full retraining. Although existing editors can often modify local factual associations, edited models still struggle to apply updated knowledge in multi-hop reasoning. In this work, we study this problem through the lens of the generate-verification gap: a model may fail to directly generate the correct updated answer, while a verifier conditioned on the edit facts can still judge whether a proposed answer is compatible with the updated knowledge. Motivated by this observation, we propose V alidator- G uided D ecoding ( VGD ), a test-time method that uses edit facts as verification memory to improve answer generation after knowledge editing. VGD employs a trained validator as an external correctness signal over candidate answers, allowing the model to retain reliable outputs while selectively revising uncertain or inconsistent predictions. It is editor-agnostic and can be applied to base models or on top of existing editors without modifying the editor or further updating the edited model. Experiments on multi-hop benchmarks show that VGD improves hop-level consistency and final-answer accuracy, generally outperforming ICL and CoT baselines. Our code will be available at https://github.com/JoveReCode/VGD .

Abstract:
Recent breakthroughs in large language models (LLMs) have led to notable successes in complex reasoning tasks, such as mathematical problem solving. A common strategy for improving performance is parallel thinking , in which multiple reasoning traces are generated and the final prediction is made using aggregation schemes like majority voting or best-of- N decoding. However, two key challenges persist. First, multi-sample decoding incurs substantial inference latency, especially for long-form outputs. Second, effective mechanisms for reliably assessing the correctness of individual reasoning traces are still limited. To address these challenges, we introduce One-Token Verification (OTV), a computational method that estimates reasoning correctness in a single forward pass during generation. OTV is activated by a learnable token and integrated into the LLM via low-rank adaptation to probe internal reasoning signals through the key-value cache, supporting token-level correctness estimation at any stage of generation without disrupting primary reasoning. Experiments on mathematical reasoning show that OTV consistently outperforms existing verifiers while reducing token usage by up to 90% through correctness-guided early termination. This enables the model to favor shorter, more reliable solutions. The code is publicly available at https://github.com/Multimedia-Analytics-Laboratory/OTV

Abstract:
Document-level machine translation (MT) evaluation extends segment-level protocols by presenting full documents to annotators, on the assumption that such presentation elicits document-level judgments. We test this assumption with a counterfactual condition (MIX) in which each document combines segments drawn from different systems, preserving document-level presentation while breaking cross-segment consistency. Across 18,420 expert English→Korean annotations and 14 automatic metrics, scores, system rankings, and error annotations are statistically equivalent between coherent and incoherent documents. Perception does not explain this: shown matched passages, raters identify the coherent one as the work of a single translator in 87.3% of trials. Document presentation does change how annotators work, but that change does not reach the recorded output. What is blind is the protocol, not the annotator. The concern is not that scores fall short, but that the resources invested in document-level systems, metrics, and annotation may not be measuring what they are intended to measure.

Abstract:
Instruction-following information retrieval (IF-IR) requires retrievers to satisfy both topical intent and instance-specific relevance constraints. We propose dual-view training, a data synthesis strategy based on polarity reversal. Given a query, an instruction-relevant document, and a topically related document excluded by that instruction, an LLM generates a complementary instruction under which the two documents exchange relevance labels. The resulting paired views directly penalize instruction-agnostic representations. At a matched 880k-example budget, replacing the non-instruction half of a Promptriever-style joint-data recipe with polarity-reversed examples improves FollowIR p-MRR from 5.27 to 8.30 (+57.5%) on a 305M-parameter encoder while increasing its instruction-conditioned Score from 20.85 to 21.38. The relative p-MRR gain repeats on a 560M-parameter backbone. The proposed recipe also improves InfoSearch nDCG@5 and shows no average degradation on four evaluated BEIR datasets with plain queries. A size-matched paraphrase control does not reproduce the instruction-sensitivity gains, supporting the conclusion that label reversal, rather than additional LLM-generated wording, provides the key supervision signal.

Abstract:
While semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of llinguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. STEB comes with an open-source code base and leaderboard at https://github.com/rrivera1849/STEB.

Abstract:
Vision-language models (VLMs) are increasingly used in multi-image, multi-turn agentic settings where decisions depend on visual changes. However, in existing open-weight VLMs, visual comparisons happen only inside the language model, while the visual encoder itself remains stateless: each image is encoded independently, without access to the prior visual context. As a result, small but task-critical changes may be attenuated before the language model has a chance to compare them, especially when those changes do not affect the high-level semantics of the scene. We introduce a Stateful Visual Encoder, which conditions each visual representation on prior visual features. Under supervised finetuning, VLMs equipped with stateful encoders achieve consistent improvements on controlled tasks involving cross-image spatial aggregation, multi-object visual differencing, and visual trajectory behavior cloning. These improvements are consistent across input resolutions, language model sizes, and VLM backbones. Finally, we validate our model on real-world tasks, including longitudinal radiology, fine-grained image comparison, and remote sensing, where stateful encoders consistently improve generalist VLM baselines and can match or surpass specialized models in selected domains.

Abstract:
LLM unlearning aims to remove the influence of undesirable knowledge from pretrained language models, which offers a practical mechanism for addressing safety and privacy concerns. Existing unlearning approaches, such as Gradient Ascent, are prone to catastrophic forgetting. Alignment-based approaches provide an alternative direction, yet their effectiveness is limited by the quality of the reference model. In realistic settings, both methods still require large retention datasets to preserve general knowledge. We propose a principled method that quantifies the target LLM’s confidence in undesirable knowledge and uses it to calibrate the model’s unlearning gradient updates more precisely. It enables fine-grained control over forgetting while better preserving model utility, thus reducing the dependence on retention data or prohibitive unlearning training data. Extensive evaluations on multiple benchmarks, including MUSE and WMDP, show that our method achieves effective unlearning and improves the trade-off between knowledge removal and utility preservation compared with state-of-the-art methods.

Abstract:
Curating training data is among the most consequential yet labor-intensive parts of modern AI development: practitioners iteratively propose, implement, evaluate, and revise data policies against noisy benchmark feedback. We ask whether generalist coding agents can automate this data-curation loop. We introduce Curation-Bench, an agent-centric benchmark that fixes the model, training recipe, and evaluation suite while giving agents command-line access to inspect data, implement policies, submit them to a fixed training/evaluation pipeline, and revise. In a vision-language instruction-tuning instantiation, out-of-the-box agents reach strong published data-selection baselines within ten iterations. However, trajectory analysis reveals a persistent execution–research gap: agents mainly tune local policy variants rather than explore new policy families, even when given strategy guides and paper references. Scaffolds requiring each iteration to cite, instantiate, and adapt a prior method shift agents toward method-guided exploration. The scaffolded agent autonomously composes—without human design input—a data-selection policy that outperforms strong published baselines at one-tenth their data budget. Overall, current agents can run the curation loop, but reliable data research requires scaffolded method adaptation, not open-ended prompting alone. Code and benchmark are open-sourced.

Abstract:
Direct Preference Optimization (DPO) aligns Large Language Models with human preferences without the need for a separate reward model. However, DPO treats all tokens in responses equally, neglecting the differing importance of individual tokens. Existing token-level PO methods compute the token weights using either token-position-based heuristic functions or probability estimates given by a separately trained model, which lacks robustness and incurs extra training cost. In contrast, we propose Token-weighted DPO (TwDPO)—a novel training objective grounded on token-weighted RL—and AttentionPO—an instantiation of TwDPO that uses attention from the LLM itself to estimate token weights. AttentionPO prompts the LLM to serve as a pairwise judge and check where the model attends when comparing the responses. This design makes AttentionPO content-aware, adjusting weights based on response content, and efficient, incurring only two extra forward passes per example. Experiment results show that AttentionPO significantly improves performance on AlpacaEval, MT-Bench, and ArenaHard, surpassing existing Preference Optimization methods.

Abstract:
Multi-agent systems (MAS) are emerging as promising socio-collaborative companions for emotional and cognitive support. However, existing systems frequently suffer from persona collapse, where agents revert to generic, homogenized assistant behaviors, and social sycophancy, where agents produce redundant, non-constructive dialogue. We propose MASCOT, a multi-agent framework for multi-perspective socio-collaborative companions. MASCOT introduces a novel bi-level optimization strategy to harmonize individual and collective behaviors: 1) Persona-Aware Behavioral Alignment, an RLAIF-driven pipeline that fine-tunes individual agents for agent-specific identities; and 2) Collaborative Dialogue Optimization, a group-level adaptation process that promotes complementary, diverse, and productive discourse. We evaluate MASCOT using human-grounded contexts drawn across both in-domain and out-of-domain (OOD) settings against state-of-the-art baselines. MASCOT improves persona consistency by up to +14.1 and social contribution by up to +10.6. A broad evaluation suite, including human evaluation, multiple LLM judges, three-way comparisons, and automatic metrics, further shows that MASCOT produces more role-consistent and less redundant multi-agent dialogue. Our code is available at https://github.com/hello-diana/MASCOT.

Abstract:
Agentic reward models enhance large language model (LLM) evaluation by incorporating external tool-use capabilities. However, their training is heavily constrained by coarse-grained outcome supervision and the high cost of process-level annotations. To address these challenges, we propose SEAR (Self-judging agentic reward models), a novel training framework that enables agentic reward models to self-judge their own judging processes. SEAR introduces a dual-role adversarial co-evolution framework, where the model alternates between a judger and a self-judger within a single generation process: the judger is optimized to produce more rigorous reasoning trajectories that maximize self-judgment scores, while the self-judger learns to provide critical and discriminative evaluations. Moreover, SEAR further enables agentic RMs to self-evolve without relying on external human annotations. Extensive experiments demonstrate that SEAR consistently improves the reliability of agentic reward models across diverse benchmarks.

Abstract:
In-context learning (ICL) tackles downstream tasks conditioned on demonstrations. This paradigm relies heavily on neighbor-based matching between queries and demonstrations, limiting out-of-distribution (OOD) performance. For better generalization, rule-based inductive reasoning derives task-level regularities, which we term task-level inductive representations (TIR), before directly answering. However, existing methods lack generalizable inductive ability across diverse tasks. Inspired by meta-learning, we propose Cognitive-Decoupling meta-Induction (CoDI) for TIR-based inductive reasoning on general tasks. CoDI comprises a Cognitive-Decoupling Framework for query-blind TIR induction, and a Dual-Loop Meta-Inductive Learning method for cross-task inducer optimization. Empirically, CoDI achieves substantial performance gains on five ID and OOD benchmarks, and preserves OOD gains without explicit induction at inference. Code is available at https://github.com/Yuqian-Wang2277/codi-meta-induction.

Abstract:
We study the vulnerability of guardrail models to adversarial paraphrases that exploit superficial textual variation: meaning-preserving changes that alter expressions spuriously correlated with safety labels. While prior work probes this vulnerability through simple heuristics, such as keyword-label co-occurrence statistics, we argue that guardrails may rely on more complex textual expressions involving phrases or syntactic structures that cannot be captured by heuristics. To enable systematic evaluation of such hidden vulnerabilities, we propose ASPro, an agentic adversarial evaluation framework that automatically synthesizes meaning-preserving paraphrases containing expressions that contradict guardrails’ learned spurious expression-label correlations. Applying ASPro to five safety benchmarks and 13 guardrail models, we find that the average F1 drops from 77.83 to 40.25, substantially more than under existing heuristic baselines. Moreover, on ASPro prompts, guardrail models often fail to outperform their base LLMs before safety fine-tuning. These results suggest that current safety fine-tuning may overfit to spurious expressions rather than truly improve guardrails’ understanding of prompt safety.

Abstract:
Model merging aims to integrate multiple expert models into a single model that inherits their complementary strengths without incurring the inference-time cost of ensembling. Recent progress has shown that merging can be highly effective when all source models are homogeneous, i.e., derived from the same pretrained backbone and therefore share aligned parameter coordinates or compatible task vectors. Yet this assumption is increasingly unrealistic in open model ecosystems, where useful experts are often built on different families such as Llama, Qwen, and Mistral. In such heterogeneous settings, direct weight-space fusion becomes ill-posed due to architectural mismatch, latent basis misalignment, and amplified cross-source conflict. We address this problem with HeteroFusion for heterogeneous language model fusion, which consists of two key components: topology-based alignment that transfers knowledge across heterogeneous backbones by matching functional module structures instead of raw tensor coordinates, and conflict-aware denoising that suppresses incompatible or noisy transfer signals during fusion, with a small replay set used to assist transfer-operator optimization. We further provide analytical and empirical evidence for preserving the target adapter basis while predicting structured updates. Across heterogeneous transfer, multi-source fusion, noisy-source robustness, and cross-family generalization settings, HeteroFusion consistently outperforms strong merging, fusion, and ensemble baselines.

Abstract:
Knowledge graph embedding (KGE) demonstrates its effectiveness for predicting missing links in knowledge graphs (KGs) by projecting entities and relations into a low-dimensional vector space. It is crucial for KGE models to effectively capture inference patterns (patterns) inherent in KGs, such as symmetry/antisymmetry, inversion and composition. Although recent KGE models exhibit strong capabilities in modeling such diverse patterns, they suffer from inherent limitations stemming from pattern over-generalization, where embeddings learned from only a single pattern instance inevitably generalize that pattern to all related instances, i.e., generalize the pattern universally. To address this issue, we propose PogRE (Pattern Over-Generalization Robust Embedding), a simple but effective method that utilizes dense linear transformations and compound operations for relation representation. Our theoretical analysis demonstrates that a dense linear transformation allows a pattern to become progressively universal as more triples are observed in the pattern. Furthermore, after observing d+1 linearly independent entities (d+1 denotes the dimension of entity), the linear transformation guarantees universal generalization of the pattern across all related instances. Experimental results on three standard benchmark datasets show that PogRE outperforms existing state-of-the-art KGE models in link prediction. Moreover, our empirical results indicate that PogRE effectively addresses the negative impact of over-generalization.

Abstract:
Large Language Models (LLMs) are increasingly deployed in settings where rare but severe harmful generations can have significant consequences. Existing Risk-Averse RLHF addresses this issue by optimizing Conditional Value-at-Risk (CVaR), but it trains policies for fixed risk levels and therefore cannot adjust the desired degree of risk aversion at inference time. In this paper, we propose risk-conditioned RLHF, a framework that trains a single policy that provides a continuous risk-control interface, enabling users to select different degrees of risk aversion without retraining or deploying multiple risk-specific models. Experiments across multiple benchmarks demonstrate that a single risk-conditioned policy can adapt to different risk levels at inference time, enabling more flexible and risk-aware LLM deployment. This paper contains example data that may be offensive or harmful.

Abstract:
Synthetic data rephrasing has emerged as a powerful technique for enhancing knowledge acquisition during large language model (LLM) pretraining. However, much of web-scale synthetic rephrasing operates at the single-document level, rewriting individual web pages in isolation. This confines synthesized examples to intra-document knowledge, missing cross-document relationships and leaving facts with limited associative context. We propose WRAP++ ( W eb discove R y A mplified P retraining), which amplifies the associative context of factual knowledge by discovering cross-document relationships from web hyperlinks and synthesizing joint QA over each discovered document pair. Concretely, WRAP++ discovers high-confidence relational motifs including dual-links ( A ↔ B ) and co-mentions ( A → E ← B with A → B ), and synthesizes QA that requires reasoning across both documents. This produces relational knowledge absent from either source document alone, creating diverse entry points to the same facts. Because the number of valid entity pairs grows combinatorially, this discovery-driven synthesis also amplifies data scale far beyond single-document rewriting. Instantiating WRAP++ on Wikipedia, we amplify ∼ 8.4B tokens of raw text into 80B tokens of cross-document QA data. On SimpleQA, OLMo-based models at both 7B and 32B scales trained with WRAP++ substantially outperform single-document approaches and exhibit sustained scaling gains, underscoring the advantage of cross-document knowledge discovery and amplification.

Abstract:
Aggregation query over free text is a long-standing yet underexplored problem. It requires exhaustive evidence collection, where systems must “find all” satisfying entities rather than merely “find one” plausible answer. Existing paradigms are not designed for this completeness requirement, leading to incomplete retrieval and systematic undercounting. This limitation stems from the lack of suitable benchmarks. In this work, we formalize corpus-bounded entity-level aggregation querying over text, where answers must be grounded in the given text corpus. We introduce AGGBench , an aggregation benchmark constructed from finance reports and research papers through a corpus-grounded pipeline with LLM-assisted human validation. AGGBench contains 498 aggregation queries, including both single- and multi-constraint queries with evidence-supported gold answers. Experiments on representative basic RAG, structured and schema-first and agentic methods show that existing systems struggle to preserve evidence coverage and accurately count satisfying entities, highlighting the need for completeness-oriented methods for aggregation over unstructured text. The data and code are available in https://github.com/luck-seu/AGGBench .

Abstract:
Mixture-of-Experts (MoE) language models introduce unique challenges for safety alignment due to their sparse routing mechanisms, which can enable degenerate optimization behaviors under standard full-parameter fine-tuning. In our preliminary experiments, we observe that naively applying full-parameter safety fine-tuning to MoE models can appear to improve safety through routing or expert dominance effects, rather than by directly repairing Safety-Critical Experts. To address this challenge, we propose RASA, a routing-aware expert-level alignment framework that explicitly repairs Safety-Critical Experts while preventing routing-based bypasses. RASA identifies experts disproportionately activated by successful jailbreaks, selectively fine-tunes only these experts under fixed routing, and subsequently enforces routing consistency with safety-aligned contexts. Across three representative MoE architectures spanning 1B to 47B total parameters, RASA achieves near-perfect single-turn robustness and cross-attack generalization, maintains low attack success rate under a RASA-aware adaptive attack, and extends effectively to held-out multi-turn jailbreaks, while preserving general capabilities and controlling over-refusal. Causal necessity and sufficiency tests confirm that the identified experts drive the safety repair. Our results suggest that robust MoE safety alignment benefits from targeted expert repair rather than global parameter updates, offering a practical and architecture-preserving alternative to prior approaches.

Abstract:
Chain-of-Thought (CoT) reasoning offers a promising window into model monitoring. However, monitoring relies on faithfulness, i.e., the model output strictly derives from its reasoning trace. We identify an alignment tension where a model must be faithful enough to be monitored, yet robust enough to reject unsafe reasoning. We demonstrate that this counterbalance exists in current Large Reasoning Models (LRMs), and show ways in which it can be addressed. We introduce HazMart, a human-written dataset set in an autonomous AI shopkeeper scenario. Unlike prior work that relies on providing hints in prompts to test faithfulness (e.g., “A Stanford professor said it should be Answer A”), we propose a novel replacement-based technique, which we call Targeted Reasoning Replacement (TRR), that directly intervenes in the reasoning chain to substitute in unsafe or illogical thoughts (e.g., “Wait, the answer must be Option B Option A because it is the most fitting”). DeepSeek-R1-Llama-70B exhibits high faithfulness (97.5%) but fails to reject Unsafe Reasoning (12.3%), while QwQ-32B is more robust (73.9% safety) at the cost of lower faithfulness (74.7%). Mechanistic analyses of QwQ-32B reveal that these properties are represented by anti-correlated internal directions peaking at the action-commit token. Finally, we demonstrate that representation steering can independently amplify the safety direction, increasing safe behavior by 9 percentage points while maintaining base capabilities.

Abstract:
Joint-encoding zero-shot relation extraction classifies all candidate relation labels in a single Transformer forward pass, achieving strong accuracy and high throughput. However, under a controlled full-vocabulary diagnostic, two distributional pathologies surface: unnormalized dot-product scoring causes recall collapse, with certain types receiving zero predictions, while the per-instance BCE loss leaves gold and negative scores entangled beyond any reliable threshold—confidence ambiguity. We propose SCAR, which keeps the baseline pathway intact and stabilizes both distributions via two auxiliary branches: a gradient-isolated coarse-to-fine cascade with Z-score normalization that restores cross-relation comparability, and a training-only contrastive alignment module that separates gold from negative scores. On Wiki-ZSL (m=15), SCAR raises Macro F1 from 75.9% to 83.0% and cuts zero-recall types from 35 to 1, suggesting that small-pool ZSRE protocols can obscure failure modes critical to large-vocabulary deployment.

Abstract:
While recent large language models (LLMs) have achieved promising results on individual patent drafting tasks, they fundamentally fail to investigate the core challenge of real-world patent drafting: generating a complete and legally coherent patent application directly from early-stage invention materials. Prior work predominantly assumes later-stage, highly structured, or already legalistic inputs. However, real patenting workflows begin with informal, de-legalized disclosures authored by inventors. To bridge the gap, we introduce Dis2Pat, a disclosure-to-patent dataset that reflects realistic patenting workflows by requiring the generation of complete patent applications directly from inventor-style, de-legalized disclosures. Given the inherent difficulty of long-form, legally constrained patent drafting and the strong privacy requirements, we further propose a strong baseline named Patent-MAF. It is a multi-agent framework for locally deployable patent drafting. Benchmark results reveal that current LLMs exhibit limitations in patent drafting, while Patent-MAF provides a strong baseline that consistently outperforms evaluated open-source models and remains competitive with large closed-source models.

Abstract:
The rapid development of LLMs incurs prohibitive memory footprints and intensive computational demands. Quantization-Aware Training (QAT) techniques have emerged as a promising solution to address these challenges by explicitly simulating quantization effects during model training, yielding low-bit models that achieve accuracy comparable to their full-precision counterparts. In this work, we provide a target-centric survey of QAT, aimed at clarifying both its theoretical foundations and its evolving implementation landscape. We systematically review existing QAT methods through a target-centric taxonomy and synthesize cross-target differences in error characteristics, numerical formats, and strategy transferability. We further summarize QAT evaluation paradigms and discuss challenges in optimization and deployment, outlining potential directions for future research.

Abstract:
Large Audio-Language Models have made significant advances in audio-text instruction following and multimodal interaction. However, due to their text-conditioned generation architecture, current LALMs tend to be overly influenced by input text prompts. When acoustic evidence conflicts with textual prompts, incorrect or irrelevant textual cues may steer model predictions away from the audio. In this work, we systematically analyze this challenge and show that representative LALMs exhibit a persistent audio-text imbalance during decoding, where attention to audio tokens is weakened across generation steps. To address the imbalance, we propose Modality-Specific Gating (MSG), an acoustic calibration mechanism that improves audio grounding with a small number of additional parameters. MSG introduces a token-wise, query-conditioned bias to the attention logits of audio tokens before softmax, while leaving the original attention computation unchanged. With parameter-efficient fine-tuning, MSG enhances acoustic evidence utilization, leading to consistently improved performance on vocal sound classification and speech emotion recognition, as well as strong robustness under adversarial or irrelevant text prompts.

Abstract:
Modern coding agents pair LLM generators with various tools, including cheap diagnostics and expensive verifiers. The tool-use decisions are typically governed by orchestrators that often use fixed rules and ignore uncertainty. We formulate orchestration as cost-sensitive sequential hypothesis testing: a Bayesian controller maintains a belief over candidate correctness and dynamically decides whether to gather more evidence, refine the candidate, verify it, or stop. Across six generators and nine coding benchmarks, Bayesian control proves to be most valuable when verification is costly and critics are informative but imperfect. Beyond control, the belief state yields an interpretable correctness score that outperforms token-probability and raw tool-success baselines for uncertainty quantification.

Abstract:
Transformer-based large language models (LLMs) are comprised of billions of parameters arranged in deep and wide computational graphs, but it is not clear that they exploit their full capacity for all inputs. We introduce the s-Trace method to efficiently estimate a subgraph of size s that approximates a full model output. With this method, we find the computation in a variety of LLMs to be organized in two distinct phases. A small subgraph mostly composed of early-layer nodes can reconstruct the head of the full model output distribution. Adding further nodes, mostly located in later layers and increasingly consisting of attention heads, leads to incremental refinements in approximating the full output distribution. We find moreover that the amount of necessary computation per input correlates with model uncertainty, and that sparser subgraphs encode shallow statistics, such as unigram frequency. Overall, our results suggest a consistent modular organization in effective LLM computation, with a sparse early-layer core providing a rough prediction that is further refined through denser computations in later layers.

Abstract:
Simultaneous speech translation produces target text incrementally from partial speech input. Recent speech large language models have markedly improved SST quality but still struggle with rare and domain-specific terminology. Retrieval augmentation has helped in automatic speech recognition and neural machine translation, but extending it to SST is non-trivial: retrieval must be fast and accurate under partial speech, and the model must decide whether and when to apply retrieved terms during incremental generation. We propose Retrieval-Augmented Simultaneous Speech Translation (RASST), which addresses both challenges. For accurate cross-modal retrieval under partial input, RASST trains a lightweight speech-text retriever that produces chunkwise terminology hints for the Speech LLM via multi-scale retrieval. To use these hints correctly, we synthesize training data that teaches the Speech LLM to decide whether and when to apply each retrieved term. Experiments on the ACL 60/60 development set and the ESO oncology test set show that RASST improves terminology accuracy over InfiniSST by 12.3-31.22% and BLEU by up to 3 points across all tested languages and latency settings. Furthermore, the retriever incurs negligible computational overhead and remains robust as glossary size increases to 10K entries.

Abstract:
Public repositories host millions of fine-tuned models, yet community usage remains disproportionately concentrated on a small number of foundation checkpoints. We investigate whether this concentration reflects efficient market selection or if superior models are systematically overlooked. Through an extensive evaluation of over 2,000 models, we show the prevalence of "hidden gems", unpopular fine-tunes that significantly outperform their popular counterparts. Notably, within the Llama-3.1-8B family, we find rarely downloaded checkpoints that improve GSM8K accuracy from 83.2% to 96.0% without increasing inference costs. However, discovering these models through exhaustive evaluation of every uploaded model is computationally infeasible. We therefore formulate model discovery as a Multi-Armed Bandit problem and accelerate the Sequential Halving search algorithm by using shared query sets and aggressive elimination schedules. Our method retrieves top models with as few as 50 queries per candidate, accelerating discovery by 50× .

Abstract:
Reinforcement Learning (RL) has demonstrated significant potential in enhancing the capabilities of large language models (LLMs). However, the success of RL for LLMs heavily relies on human-curated datasets, which pose a burden on scalability and sustainability. Inspired by the success of the paradigm in games and Go, Self-Play methods are deployed to enhance LLM capabilities without human-annotated data. However, these methods primarily depend on a grounded environment for verifiable feedback (e.g., a Python interpreter or a game engine); extending them to general domains remains challenging. To address these challenges, we propose Multi-Agent Evolve (MAE), a framework in which LLM can self-evolve across diverse tasks from mathematics to general knowledge. MAE implements asymmetric Self-Play by leveraging a triplet of interacting roles (Proposer, Solver, Judge) that are instantiated from a single LLM. Proposer generates challenging questions, Solver optimizes solutions, and Judge provides reward signals. These roles are jointly optimized via reinforcement learning, creating a closed-loop system where the model continuously challenges and improves itself. Experiments on Qwen2.5-3B-Instruct demonstrate that MAE achieves an average improvement of 4.86% across multiple benchmarks, significantly outperforming baselines. These results highlight MAE as a scalable, data-efficient method for enhancing LLMs’ general reasoning abilities with minimal reliance on human supervision.

Abstract:
While it is well-established that prosody carries crucial cues for syntactic structure, the degree and nature of correspondence between these two domains remains contested. We investigate the syntax-prosody interface through an information-theoretic lens, quantifying the interaction between prosodic features and syntactic representations as their mutual information. We provide a general-purpose framework for estimating this quantity over large speech-text corpora using multimodal language models. Our framework is structure-agnostic and modular, insofar as it can be used to measure the contributions of individual prosodic features or components of structure. We evaluate the syntax-prosody relationship for two features (word duration and inter-word pauses) across two domains–read audiobooks and spontaneous conversations–both in English. Our results demonstrate that prosody contains measurable syntactic information, with prosodic features reducing syntactic uncertainty in spontaneous conversations by up to 10.2%. Our findings offer new empirical support for several theoretical accounts of the syntax-prosody interface.

Abstract:
Modern KGQA pipelines address complex multi-hop reasoning by shifting retrieval from classical embedding-based methods to iterative LLM calls. Although effective, this incurs significant computational and economic overhead that limits their applicability in real-world settings. In this paper, we introduce GRASE (Graph Reasoning with Anonymous Span Encoders), a framework that replaces LLM-driven search with a novel encoder-based graph-span representations over uncertain nodes and relations in a knowledge graph. Across several multi-hop KGQA benchmark datasets, GRASE achieves up to 85% faster inference than LLM-native pipelines while matching or exceeding state-of-the-art performance, demonstrating that encoder-only hybrid reasoning is a robust, practical, and scalable alternative to fully LLM-driven retrieval.

Abstract:
ASR systems sometimes produce fluent text that is unrelated to the speech they receive. We view these hallucinations as one possible consequence of a broader grounding failure, in which the transcript is no longer adequately guided by the audio. To understand where this failure becomes possible, we study two independently trained Conformer-Large recognizers — one CTC and one RNN-T — under environmental degradation and speaker-background shift. In both models, the final encoder stage emerges as a critical boundary: bypassing the final block causes divergence on nearly every utterance, whereas bypassing middle blocks has little effect. At this same stage, the representations become more compact, text becomes readable by the trained decoder, and grapheme information becomes explicit. Importantly, the intervention produces garbled or repetitive output rather than fluent fabrication (0 of 354 judged outputs). Our result therefore identifies a mechanistic precondition for hallucination — the failure to produce adequately grounded output — not the complete origin of naturally occurring hallucinations. Together, the results reveal a consistent terminal-stage dependency for grounded recognition across two decoder families and multiple distribution shifts.

Abstract:
Language model reasoning traces are rarely all-or-nothing; they frequently contain valid intermediate steps before a critical error occurs. Existing uncertainty quantification methods typically certify final answers or entire responses, failing to provide statistical guarantees for the proportion of a sequential trace that can be safely retained. To address this, we introduce CROP (Conformal Reasoning Output Prefixes), a verifier-agnostic calibration procedure for clean-prefix certification. Given any step-level risk proxy, CROP selects a calibrated threshold and returns the longest contiguous prefix whose step risk proxies remain below it, routing the uncertified suffix for downstream review or repair. Assuming exchangeability, CROP rigorously controls the marginal probability that the returned prefix contains an annotated error. Across six process-labeled reasoning datasets, we demonstrate that standard step-level metrics such as AUROC do not fully capture prefix utility, suggesting verifiers should instead be evaluated by certified prefix length. Furthermore, CROP balances over- and under-withholding and improves downstream repair accuracy by preserving valid intermediate reasoning while discarding misleading suffixes. Ultimately, this work positions prefix certification as a rigorous, practical bridge between process supervision, abstention, and repair.

Abstract:
Chain-of-thought (CoT) reasoning is commonly improved through external mechanisms such as prompting, sampling, and decoding, while the internal units shaping reasoning trajectories remain less understood. We study CoT from the perspective of reasoning trajectory structure and find that incorrect trajectories are more redundant and repetitive, whereas correct ones are more lexically diverse and information-dense. Motivated by this observation, we propose Contrastive Neuron Regulation (CNR), a sparse, backbone-frozen framework for neuron-level regulation of CoT reasoning. CNR constructs correctness-controlled expanded–concise pairs of correct trajectories, identifies MLP neurons associated with the expansion contrast, and sparsely suppresses them during generation. We consider both a fixed-coefficient inference-time intervention and an adaptive variant that learns neuron-specific suppression coefficients on a subject-specific development split while keeping the backbone frozen. Across two reasoning models and seven MATH subjects, CNR improves macro-average accuracy over the no-intervention baseline by 2.6 and 3.2 percentage points, respectively, while intervening on less than 1% of MLP neurons. The code is available at https://github.com/RUCAIBox/Contrastive-Neuron-Regulation.

Abstract:
Naturalistic computer-use traces, passively recorded screenshots and mouse or keyboard actions, are a valuable resource for deriving symbolic, auditable, and reusable models of how everyday work is done. Such models matter as computer-use agents enter real work, where agents need to learn how tasks are actually performed, and organizations need to audit and reuse that knowledge. However, inducing such task models is challenging, as activity is observed only as low-level events and real-world work is multi-threaded with interleaved goals. Existing methods assume a given task or single workflow, and produce step-level summaries rather than structured task models. We introduce TASK MODEL INDUCTION (TMI), which (i) discovers the latent tasks in an unconstrained trace, disentangling concurrent activity, and (ii) for each latent task, induces a task model pairing a hierarchical objective model of recursive goal decomposition with a procedure model of the control flow that organized the execution. Intrinsically, on controlled human and agent trajectories, TMI recovers interleaved tasks with 0.974 agreement against ground-truth groupings and reconstructs 74.9% of the observed execution steps, far more than the strongest workflow induction baseline. Extrinsically, skills derived from TMI’s task models improve held-out task accuracy by 30.0% over the strongest baseline.

Abstract:
Model calibration aligns a model’s confidence with the accuracy of its predictions, directly influencing reliability and user trust. Surprisingly, existing research on knowledge distillation has paid little attention to the calibration of student models. In this paper, we evaluate calibration across diverse knowledge distillation settings and demonstrate that student models distilled from black-box teachers generally exhibit poor calibration. We attribute this to the fact that distillation fails to convey well-calibrated confidence, which serves as a crucial expression of predictive uncertainty. To address this limitation, we propose an Uncertainty-Integrated Knowledge Distillation (UIKD) method comprising two key components: (1) uncertainty estimation, which leverages the collective intelligence of multiple LLMs via a debate mechanism to quantify uncertainty in the distillation data; and (2) uncertainty integration, which employs an adaptive weighted loss function to integrate this data uncertainty into the student model effectively. Experiments on common-sense reasoning and natural language inference benchmarks demonstrate that UIKD consistently improves student model calibration while maintaining predictive accuracy. Our code is available at https://github.com/RuixuanZhang127/UIKD.

Abstract:
We examine whether human psychometric questionnaires can serve as reliable tools for characterizing and predicting LLM behavior in everyday user interactions. We analyze eight open-source LLMs by comparing their value and personality profiles derived from two different methods: Likert self-reports on established questionnaires (PVQ-40/21 and BFI-44/10) and generation probabilities over value-laden responses to everyday user queries. The two profiles diverge substantially. Within-construct item consistency, often cited as evidence of stable LLM dispositions, disappears in generation probabilities. We find that established questionnaire items contain explicit lexical cues that allow models to recognize the target construct and respond in alignment-consistent, socially desirable ways, whereas realistic user queries contain far less recognizable cues. In addition, demographic persona prompts shift models’ responses to human questionnaires in ways consistent with real human patterns, but no such shifts appear in the generation setting, highlighting that human questionnaires overestimate LLMs’ ability to faithfully reproduce expected psychological traits when role-playing demographic personas. Overall, our study indicates that questionnaire scores alone should not be treated as evidence of LLMs’ response tendencies in realistic user interactions, and supports generation-probability profiling with ecologically valid items as a complementary behavioral measure.

Abstract:
Agentic recursive training systems increasingly reuse model-generated tasks, proxy feedback, and selected trajectories to refine policies over multiple rounds. This creates feedback loops in which small conditional biases can accumulate across training and alter later selection and updates. We study whether trigger-conditional proxy-feedback biases can produce emergent backdoors in a controlled recursive training loop. A Curriculum Agent proposes tasks, an Executor Agent generates rollouts, and environment feedback is selectively biased on rare markers, affecting reward, selection, and curriculum updates. Under matched biased supervision, Reward Hacking + SFT and Reward Hacking + RLHF reach 31.27% and 53.85% ASR, while the full loop reaches 97.62% at comparable clean accuracy. The effect persists across Qwen3-4B, LLaMA-3-8B, BLIP-2-OPT-2.7B, and Qwen2.5-14B, transfers to GSM8K, BBH, and HotpotQA, extends to a tool-augmented setting, and is only partially reduced by post-hoc defenses. These results suggest that feedback and curriculum mechanisms in agentic recursive training should be audited for long-horizon bias amplification.

Abstract:
We introduce GRAB, a constructor–encoder–bridge pipeline for table question answering. Our method lifts relational data into a heterogeneous graph, encodes it via message passing, and transfers the signals to an LLM through a small set of query-conditioned latent tokens. The LLM receives these tokens alongside the serialized table, and the two play different roles: the graph shows where the relevant evidence is, while the text keeps the exact cell values that the graph does not encode. The graph supplements the text rather than replacing it, and our ablations confirm that both are required. Crucially, the LLM remains strictly frozen to preserve its general reasoning capabilities; we train only the lightweight graph encoder and latent bridge (91M parameters), allowing the entire pipeline to be trained efficiently. Our pipeline significantly improves performance on relational Question Answering, with the largest gains in demanding multi-table settings, offering an efficient, principled way to connect relational deep learning with LLMs.

Abstract:
Equipping LLM agents with reusable skills derived from past experience has become a popular and successful approach for tackling complex and long-horizon tasks. However, such lessons are often encoded as textual guidance that remains largely advisory, lacking explicit mechanisms for when and how to intervene in the agent loop. To bridge the gap, we introduce HASP (Harnessing LLM Agents with Skill Programs), a new framework that upgrades skills into executable Program Functions (PFs). Rather than offering passive advice, PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context. HASP is highly modular: it can be applied at inference time for direct agent-loop intervention, during post-training to provide structured supervision, or for self-improvement by evolving validated, teacher-reviewed PFs. Empirically, HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks. For example, on web-search reasoning, inference-time PFs alone improve the average performance by 25% compared to (multi-loop) ReAct Agent, while post-training and controlled evolution achieve a 30.4% gain over Search-R1. To provide deeper insights into HASP, our mechanism analysis reveals how PFs trigger and intervene, how skills are internalized, and the requirement for stable skill library evolution.

Abstract:
The performance gap between low- and high-resource languages in LLMs is widely known, but it remains unclear which internal model factors drive these disparities. In this paper, we characterise this gap through the lens of representational geometry. Comparing the geometric properties of hidden representations across 30 languages reveals that LLM geometry is systematically related to language data availability. The most consistent effect is in final layers, where low-resource languages exhibit representational degeneration. To counter this, we investigate the effectiveness of regularisation terms to penalise degeneration during continued pretraining (CPT). Experiments monolingually adapting 9 base LLMs to 10 African languages show that geometric regularisation successfully reduces representational degeneration during CPT. For larger models, cosine similarity-based regularisation marginally improves performance over vanilla CPT, with more consistent gains on the most challenging tasks. We establish that the representational geometry of low- and high-resource languages in LLMs is measurably distinct, and that targeted geometric intervention is a viable strategy for improving CPT for low-resource languages.

Abstract:
Retrieval-augmented generation (RAG) depends critically on the quality and granularity of retrieved evidence. Large retrieval units preserve context but often introduce irrelevant content, which can dilute answer-bearing evidence and worsen long context utilization. Fine-grained units are more compact, but they may be difficult to retrieve reliably because short chunks can lack semantic, lexical, or bridging cues needed to match the query. We propose Uncertainty-aware Multi-Granularity RAG (UMG-RAG), a training-free hybrid retrieval framework that adapts retrieval across chunk granularities based on query-specific reliability estimates. Instead of training a new retriever or modifying the generator, UMG-RAG uses existing dense and sparse retrievers as complementary experts across multiple chunk granularities. For each query, it converts each expert-granularity score list into an evidence distribution, estimates reliability from distribution entropy, and fuses candidates according to query-specific semantic, lexical, and granularity confidence. We further introduce UMGP-RAG, a parent promotion variant that uses fine-grained hits to locate relevant evidence while returning broader non-redundant parent chunks for local coherence. Experiments on question answering benchmarks show that uncertainty-aware fusion and parent promotion improve generation quality in a training-free, plug-and-play retrieval pipeline.

Abstract:
Reasoning-intensive retrieval addresses queries whose relevance cannot be identified by surface-level matching, thereby requiring multi-step reasoning. Because relevant documents rarely appear in the initial candidate set, retrieval systems suffer from the bounded recall problem. Existing methods iteratively expand a candidate pool at the document level over a corpus graph, examining each neighbor in isolation and drifting toward a narrow region of the corpus. To address this problem, we propose Group-Aware Adaptive Retrieval for Evidence Navigation (GAREN), which explores the corpus graph through group-level expansion. GAREN organizes documents into semantically coherent and distinguishable groups based on their connections in the corpus graph. The information in each group indicates what can be accessed by expanding through it, providing guidance beyond individual document-level signals. At each iteration, GAREN uses a group-level navigator to identify promising expansion directions, retrieves documents from the selected groups, and applies a document-level reranker to evaluate the updated candidate set. Extensive experiments show that GAREN achieves up to 8.0% improvement over the strongest baseline on BRIGHT. The source code is available at https://github.com/KJ12124/GAREN

Abstract:
Agentic fact-checking (FC) has emerged as a powerful paradigm for verifying complex claims through dynamic reasoning, tool use, and iterative evidence gathering. Understanding the vulnerability of these systems is essential for assessing their reliability against misinformation and adversarial manipulation. In this work, we study both the offensive and defensive aspects of agentic FC. First, we identify a limitation in the state-of-the-art Fact2Fiction attack, which introduces excessive semantic drift into malicious evidence, and propose F+, a relevance-filtering framework that generates more semantically consistent adversarial evidence. Second, we introduce SRSFact, an enhanced agentic FC system with a Supporter–Refuter–Summarizer mechanism for structured evidence authenticity assessment, improving robustness without relying on external open-web knowledge search. Experiments on the AVeriTeC benchmark show that F+ significantly strengthens attack effectiveness, while SRSFact substantially reduces failure rates under diverse adversarial attacks.

Abstract:
LLM-based agents can solve multi-step interactive tasks by combining reasoning with environment feedback, yet each episode starts from the same fixed context and any useful strategy discovered along the way is lost once the task ends. Existing approaches either limit learning to the current task or pool all experience into a single untyped store, without distinguishing knowledge types, tracking quality through use, or balancing what the library still lacks. We introduce Unified Context Evolution (UCE), a gradient-free framework that externalizes agent experience into an evolving library of typed Evolvable Context Units (ECUs). UCE decomposes experience into four complementary types (Memory, Strategy, Workflow, and Skill), each generated from trajectories under type-specific conditions, retrieved at decision time, scored through repeated usage outcomes, and pruned when no longer valuable. A scheduling module allocates each cycle’s generation budget toward the types where the library is weakest. Across two interactive benchmarks, UCE raises ALFWorld success from 75.4% to 96.3% and WebShop task score from 45.1% to 61.3%, and remains effective across alternative actor reasoning backbones without model retraining.

Abstract:
Multi-agent language systems are often deployed under the assumption that collaboration improves reasoning quality, but this assumption breaks down once token cost, latency, and collaboration-induced errors enter the picture. We study protocol selection under a cost-aware utility objective, where the system decides on each input whether to run a collaborative protocol or fall back to solo reasoning. We propose Causal Advantage Routing for Language agents (CARL), a contextual-bandit framework that estimates the conditional collaboration advantage of each protocol over solo reasoning. CARL combines confounded execution logs with budgeted randomized probes, forms one-vs-solo doubly robust scores and shrunken DR-derived pseudo-outcomes, and at deployment uses a one-sided predictive lower bound for a future realized protocol-solo contrast, otherwise abstaining to solo execution. Across math, code, and multi-hop question-answering benchmarks with an open-weight model, CARL improves cost-aware utility and substantially reduces harmful collaboration over fixed protocols, compute-matched solo strategies, MAS routers, cascade methods, and causal routing baselines, indicating that effective multi-agent systems should estimate when collaboration is causally worth its cost rather than invoking it by default.

Abstract:
Generative retrieval (GR) retrieves documents by directly generating document identifiers (DocIDs) from queries using a sequence-to-sequence model. Recent reasoning-enhanced GR methods improve retrieval effectiveness by generating intermediate reasoning before DocID decoding. However, they typically apply reasoning uniformly to all queries, even though many queries can already be handled effectively through direct DocID decoding. This introduces unnecessary token generation and retrieval latency when reasoning provides little additional benefit. In this work, we propose Generative Retrieval with Adaptive Reasoning (GEAR), a retrieval framework that dynamically allocates reasoning computation based on query-specific needs. Instead of treating reasoning as a fixed pre-retrieval procedure, GEAR learns when to reason, what retrieval-oriented context to generate, and when to transition from free-form generation to constrained DocID decoding. This adaptive behavior is implemented through a learnable retrieval transition policy optimized jointly for retrieval quality and reasoning cost. Experiments on NQ, MS MARCO, and TREC DL 2022 show that GEAR maintains retrieval effectiveness comparable to strong reasoning-enhanced GR methods while substantially reducing reasoning overhead, triggering non-trivial reasoning for only 37.1% of queries and reducing both generated tokens and retrieval latency.

Abstract:
The rapid advancement of artificial intelligence has brought safety concerns to the forefront of research. Recent studies indicate that Large Language Models (LLMs) may exhibit emergent misalignment during training progress: when exposed to narrow-domain harmful data, they can develop unsafe behaviors across broad, unrelated tasks. This phenomenon raises a critical question: do current alignment methods genuinely eliminate harmful capabilities, or do they merely condition models to “simulate” safety? To investigate this, we conduct a comprehensive study on cross-modal emergent misalignment involving diverse Multimodal Large Language Models (MLLMs), including vision-language models (VLMs) such as Gemma3 and Qwen2.5-VL and unified any-to-any models including Janus and Bagel. We utilize a multimodal training setup encompassing text-to-text (T2T), text-image-to-text (TI2T), and text-to-image (T2I) tasks to probe safety boundaries. Our experiments yield four critical findings: (1) Cross-modal penetration: Current alignment methods primarily constrain model behavior rather than unlearning harmful knowledge, as evidenced by safety breaches transferring across modalities. (2) Backbone vulnerability: Mechanistic analysis reveals that the LLM backbone, rather than the visual encoder/decoder, is the primary source of cross-modal vulnerabilities. (3) Evaluation gaps: We identify significant deficiencies in current safety evaluation practices, demonstrating that single-modality testing is inadequate for detecting cross-modal risks. These findings underscore the urgent need for comprehensive cross-modal evaluation frameworks to mitigate safety concerns in next-generation multimodal models.

Abstract:
Anomaly detection in textual data remains an underexplored problem, hindered by fragmented benchmarks, scarce labeled data, and methods that fail to model the distribution of normal text in a continuous and generative manner. We propose Flocat, an unsupervised approach that reformulates this task through a source-reversed flow matching paradigm: rather than learning to generate, we learn a flow from inlier text embeddings toward a compact isotropic Gaussian target, whose radius is explicitly constrained to ensure that the distance to the centroid serves as a geometrically consistent anomaly score. Built on a Diffusion Transformer (DiT) backbone, Flocat operates at both token and sentence level through a level-agnostic interpolation mechanism, enabling fine-grained signal interpretability. We evaluate on 10 datasets spanning text classification, spam detection, sentiment analysis, human- vs. machine-generated text and depression screening, consistently achieving state-of-the-art AUC-ROC across all benchmarks and text representations.

Abstract:
LLM post-training pipelines that combine supervised fine-tuning and reinforcement learning are difficult to configure under realistic compute budgets: the configuration space is high-dimensional and heterogeneous, stages are strongly coupled, and each end-to-end evaluation is expensive. We propose AutoPipe, a budget-aware two-phase framework for configuration selection in LLM post-training. Offline, AutoPipe learns a dataset-conditioned learning-to-rank surrogate from historical runs, capturing within-dataset preferences and providing transferable guidance toward promising regions of the configuration space. Online, for a new dataset, AutoPipe uses the offline guidance to steer Bayesian optimization and models dataset-specific deviations with a Gaussian-process residual surrogate. To reduce evaluation cost, each trial is early-stopped and scored by a learned predictor that maps early training signals to a low-cost proxy for final post-training performance. Experiments on biomedical reasoning tasks show that AutoPipe achieves performance comparable to full-fidelity HPO baselines at a fraction of their online cost, while outperforming offline-only approaches and early-stopped HPO variants.

Abstract:
Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly growing memory and compute requirements, which makes deployment on resource-limited hardware infeasible. Model pruning, a widely used compression technique, reduces inference costs by removing redundant parameters. However, its impact on downstream performance remains unpredictable and is typically assessed only through costly empirical sweeps. To address this gap, we introduce pruning laws – simple and interpretable scaling relations that connect a pruned LLM’s post-pruning performance to its unpruned performance and pruning ratio. Across ten LLMs (1.3B–30B parameters), three pruning strategies (unstructured, width, and depth), and eight diverse tasks, we show that pruning laws achieve strong predictive accuracy (average extrapolation error <7%), reliably quantify performance degradation, and identify critical pruning thresholds beyond which recovery is infeasible. Moreover, we demonstrate that the same laws transfer universally across architectures, pruning methods, and even unseen models in zero-shot and one-shot setups. These results provide both researchers and practitioners with a principled framework to select pruning strategies, estimate safe pruning ratios without exhaustive tuning, and deploy LLMs efficiently under real-world compute and latency constraints.

Abstract:
Linear attention offers a linear-time alternative to self-attention but often struggles to capture long-range patterns. We revisit linear attention through a prediction-correction lens and show that prevalent variants can be written as a combination of a historical prediction and a single-token correction, which creates an expressivity bottleneck. To address this bottleneck, we introduce Residual Linear Attention (RLA), a framework that equips linear attention with an explicit residual-fitting mechanism. RLA maintains an auxiliary recurrent state that learns to accumulate residual errors over time and correct the base prediction. We further instantiate a delta-rule version, Residual Delta Net (RDN), incorporating adaptive gating and residual clipping for enhanced correction control and stability. Our implementation leverages highly optimized linear attention kernels and preserves linear time and memory. Across language modeling and recall-intensive evaluations, RLA and RDN consistently outperform their respective baselines and other modern linear-attention methods, narrowing the gap to standard Transformers while retaining linear scaling.

Abstract:
Although language is predominantly spoken, empirical tests of surprisal theory rely almost entirely on written text. In practice, surprisal is estimated using language models (LMs) trained on text and applied to transcriptions, creating a text bottleneck that limits evaluation of the theory in the spoken modality. We remove this bottleneck by estimating surprisal directly from the speech signal using a speech LM. This raises a key question: should surprisal be defined over realization-specific acoustic inputs or over abstract linguistic units? Since a single unit can have many distinct acoustic realizations, each assigned a different probability by the model, defining surprisal over abstract units is difficult. To address this, we introduce an importance sampling estimator that approximates the probability of an abstract unit by marginalizing over realizations generated by a text-to-speech model. Using pupillometry data from a listening study, we evaluate surprisal theory in this text-free setting at multiple levels of granularity. Obtaining surprisal from several speech LMs, we consistently find that surprisal estimated via importance sampling better predicts pupil dilation than realization-specific surprisal for words. We also find that surprisal values for sub-lexical units are predictive of pupil dilation. While we find that surprisal can be estimated without recourse to text, and that marginalizing over distinct realizations of words is more predictive of lexical processing effort, surprisal as a whole is not a strong predictor of change in pupil size, regardless of its source.

Abstract:
Large language models are inconsistent: varying prompts or including unrelated information can lead to unexpected changes in model outputs. The generator-validator (G-V) gap is one manifestation of this phenomenon, where LLMs generate responses that they then deem as invalid if re-queried to validate them. In this work, we introduce a new formulation of G-V consistency that involves a principled correction for utterance frequency. Specifically, generators often assign low likelihood to valid strings simply because those strings are a priori unlikely, which makes naive notions of G-V consistency unworkable. We show that under a natural model of rational agents answering questions with multiple answers, consistency of the validator with a frequency-corrected generator score emerges naturally. Our method, Frequency-corrected Learning of Ordered Rank Alignment (FLORA), is a training objective implementing frequency-corrected G-V consistency for real-world LLMs. Our experimental results show that training with FLORA substantially improves both G-V consistency and generator performance over prior methods, with gains of up to +27pp in Pearson correlation on IFEval and HumanEval, while preserving validator quality across all evaluated task

Abstract:
Real-world data sources are inherently multilingual, which raises the question whether language models can synthesize information across languages. In this paper, we introduce a simple two-hop question answering setting, where answering a question requires making inferences over two documents in different languages. We find that language models are more sensitive to language variation in answer-span documents than in those providing bridging information, despite the equal importance of both documents for answering a question. Under a step-by-step sub-question evaluation, we further show that in up to 33% of multilingual cases, models fail to infer the bridging information in the first step yet still answer the overall question correctly. This indicates that reasoning in language models, especially in multilingual settings, does not follow a faithful step-by-step decomposition. Moreover, language models also struggle to compose intermediate results: in up to 18% of cases, they answer both sub-questions correctly but fail to derive the correct final answer. To examine whether explicit decomposition can help, we evaluate a three-stage SUBQ prompting setup that provides the relevant intermediate sub-questions during inference. This improves two-hop QA performance on both the compositional failure cases and the full evaluation set.

Abstract:
Tokenization in language models is treated by default as an input preprocessing decision. We argue that this framing is incomplete: in autoregressive models, tokenizer granularity determines what the model must resolve in a single forward pass, and therefore the supervision signal it receives. This affects both the difficulty of the learning problem and the representations that emerge inside the model. We test this in a controlled experiment on numeric reasoning with a novel decoupling of input and output tokenization. As the output supervision view predicts, differences in task performance, training dynamics, and model internals are induced by output tokenization and largely invariant to input tokenization. This may matter in practice, because models with different tokenization strategies differ not only in input representation but in the task they were trained on. Comparisons between models may thus partly reflect task definition rather than ability. A survey of 120 recent CL papers on numeric reasoning confirms that this is rarely acknowledged: only about 10% report the numeric tokenization of the models they evaluate, while 69% compare across tokenization, and thus supervision, regimes without reporting it. While prior work documents that tokenization consistently affects model performance, there is no principled account of why. We argue that framing tokenization as output supervision provides that account.

Abstract:
LLM autograders are usually evaluated against human “ground truth,” a single reference label provided by one annotator or constructed by aggregating multiple annotations. However, human grading is often variable and subjective. We instead propose to evaluate LLM graders by treating humans and LLMs as two grading systems and characterize their agreement and disagreement. We operationalize this approach using distance correlation and generalizability theory. Across two programming-grading datasets, we find higher Human–LLM agreement on more objectively specified rubric items. On items involving greater subjective judgment, LLMs exhibit strong internal consensus, diverging more from human graders.

Abstract:
The effectiveness of Direct Preference Optimization (DPO) depends on data pairs that capture task-relevant quality differences. Existing data pipelines often rely on off-policy perturbations or coarse outcome-based signals, which are not well suited to fine-grained visual reasoning. We propose rDPO, a rubric-based data pipeline for DPO-based visual preference optimization. For each image-instruction pair, we create a checklist-style rubric of essential and additional criteria to score responses from any possible policies. The instruction-rubric pool is built offline and reused during the construction of on-policy preference data. On public reward modeling benchmarks, rubric-based prompting massively improves a 30B-A3B judge and brings it close to GPT-5.4. On public downstream benchmarks, rubric-based filtering raises the macro average to 82.69, whereas outcome-based filtering drops it to 75.82 from 81.14. In scalability evaluation on a comprehensive benchmark, rDPO achieves 61.01, markedly outperforming the style-constrained baseline (52.36) and surpassing the 59.48 base model. Together, these results show that visual preference optimization benefits from combining on-policy data construction with instance-specific criterion-level feedback.

Abstract:
While RLVR is effective for deterministically verifiable tasks, many vision-language tasks are only partially verifiable, demanding multi-criterion supervision. To bridge this gap, we propose Reinforcement Learning with Robust Rubric Rewards ( RLR 3 ), extending RL with criterion-level rewards. To ensure faithful scoring, RLR 3 routes each criterion to one of two execution paths: an LLM-as-an-extractor paired with a verifier, or an LLM-as-a-Judge for fuzzy criteria. We also introduce a minimal exposure strategy that masks verifier targets from extractors and source images from the GenRM. Furthermore, RLR 3 employs decoupled normalization to mitigate score saturation and hierarchical aggregation to prioritize essential criteria. Evaluated on Qwen3-VL-30B-A3B across 15 benchmarks, RLR 3 consistently outperforms RLVR, yielding a 4.7-point improvement over the base model and exceeding the official instruct-to-thinking model gap. Controlled audits show that deterministic verifier and minimal exposure strategy reduce exploitable false positives.

Abstract:
Diffusion-based Large Vision-Language Models (dLVLMs) have recently emerged as a compelling alternative to autoregressive (AR) LVLMs, offering advantages in parallel decoding, bidirectional context, and controllable generation. Despite rapid progress, their reliability properties remain largely uncharacterized. We present the first systematic reliability evaluation of hallucination and bias in dLVLMs, benchmarking six diffusion models against competitive AR baselines across four dimensions. Our key findings are: (1) dLVLMs reverse the yes-bias of AR models in binary visual queries; (2) they achieve competitive hallucination rates yet exhibit degraded linguistic quality; (3) they collapse to near-zero accuracy on underrepresented racial groups with opposite-polarity gender bias; and (4) they exhibit accuracy collapse in multiple-choice settings when the correct option is shorter than its distractors, associated with a length prior that emerges at the first denoising step. Tokens committed at late denoising steps with low confidence further correlate with hallucinated content, pointing to a mechanistic signal unique to diffusion generation. These patterns vary across model families, suggesting reliability is shaped by the generative paradigm together with training data.

Abstract:
Cross-lingual transfer is uneven across tasks, succeeding for natural language understanding but often failing for mathematical reasoning. We argue that this asymmetry is governed by the geometry of parameter space. Specifically, cross-lingual generalization is determined by whether English and target-language LoRA adapters share a low-error basin. To test this, we use linear mode connectivity (LMC) between English and target-language adapters, evaluating on three tasks. We find that NLI and QA model-language pairs are mostly linearly connected, whereas mathematical reasoning (Math) pairs are not, indicating that basin geometry is task-dependent. Such disconnection in Math is “local” rather than global. A small target-language task sample, applied only to the English adapter’s language encoding or semantic composition layers, is enough to repair it. Cross-lingual generalization is also factorizable into language-specific encoding and language-agnostic composition, each supplied separately from a cheaper signal. English task data routed to the composition layers and an unlabeled target-language corpus routed to the language encoding layers each alleviate it on their own for nearly every pair. Composition is the more influential of these, and it needs no target-language data. Together, these results suggest that cross-lingual generalization decomposes into language-specific and language-agnostic components in parameter space. This decomposition offers a practical route for low-resource cross-lingual adaptation.

Abstract:
Reward models (RMs) play a central role throughout the language model (LM) pipeline, particularly in non-verifiable domains. However, the dominant LLM-as-a-Judge paradigm relies on the strong reasoning capabilities of large models, while alternative approaches require reference responses or explicit rubrics, limiting flexibility and broader accessibility. In this work, we propose FLIP (FLipped Inference for Prompt reconstruction), a reference-free and rubric-free reward modeling approach that reformulates reward modeling through backward inference: inferring the instruction that would most plausibly produce a given response. The similarity between the inferred and the original instructions is then used as the reward signal. Evaluations across four domains using 13 small language models show that FLIP outperforms LLM-as-a-Judge baselines by an average of 79.6%. Moreover, FLIP substantially improves downstream performance in extrinsic evaluations under test-time scaling via parallel sampling and GRPO training. We further find that FLIP is particularly effective for longer outputs and robust to common forms of reward hacking. By explicitly exploiting the validation–generation gap, FLIP enables reliable reward modeling in downscaled regimes where judgment methods fail.

Abstract:
Transformers outperform state-space models and recurrent architectures on long-distance syntactic dependencies, yet no theory predicts which architecture will win on which task. For syntactic phenomena expressible as short B-RASP programs, Transformers need a fixed number of layers regardless of dependency distance, while sequential models face growing effective complexity. We derive a succinctness prediction table ranking Transformers, LSTMs, and Mamba on six English syntactic phenomena before any experiment, with GRU and RWKV as additional sequential controls. All predictions are confirmed: Transformer accuracy exceeds 96% at maximum structural complexity while Mamba drops below 55%, a gap that persists at 79\× the Transformer’s parameter count. To verify the winning architecture succeeds for the predicted reason, Distributed Alignment Search (DAS), a causal intervention method, finds alignment of 0.95+ in Transformers but near the 0.50 chance baseline in Mamba, validated by seven families of negative controls. Training dynamics reveal that Transformers assemble the predicted algorithm in causal-dependency order, whereas Mamba achieves high accuracy without it, producing an accuracy-alignment gap invisible to behavioral benchmarks. Validation on Pythia confirms these patterns transfer to pre-trained models.

Abstract:
Distillation is common in LLM post-training, where on-policy knowledge distillation (OPKD) uses student-generated trajectories to prepare the student for downstream RL. At each state, the student matches a next-token distribution supplied by the teacher. As the rollout enters states the teacher would not visit, the teacher-student distribution gap can accumulate. In tool use, this gap becomes consequential because student-written calls execute before supervision and their observations shape later prefixes. Proposer-verifier generation addresses this drift by letting the teacher decide which student-proposed text is retained during generation. Existing formulations govern text but leave tool execution outside their scope. We propose Persistent Teacher Anchoring (PTA), a student-induced but teacher-committed rollout construction. PTA retains chunk-level verification and adds turn-level commitment, allowing a call to reach the environment only after the teacher has verified the entire turn. Treating verified chunks as atomic generation units, we introduce persistent lookahead, which fills idle rollout capacity by advancing future samples and carrying unfinished ones across student updates under the fixed verifier. Across Search-R1-style retrieval and DeepEyes-style perception RL, applying PTA before downstream RL improves macro best@4 by 2.5 and 2.8 points over OPKD under the same downstream RL budget, while lookahead improves throughput by 24%.

Abstract:
Spexis is a multi-GPU LLM inference framework that improves the efficiency of pipeline and tensor parallelism through speculative parallelism. Rather than using speculative decoding only to accelerate token generation, Spexis runs speculation in parallel with normal execution, introducing a new parallelism axis without increasing KV-cache memory usage. This improves memory efficiency and helps mitigate the bottlenecks of multi-GPU inference. Spexis further uses lookahead scheduling to predict speculation quality and future memory pressure, allowing it to reduce wasted speculation, KV-cache eviction, and recomputation. Built on top of vLLM, Spexis largely improves serving performance across a range of GPU configurations, achieving speedups of up to 34% over a baseline that uses the optimal combination of pipeline and tensor parallelism. Spexis’s source code is publicly available at https://github.com/mlsys-seo/spexis.

Abstract:
Document-Level Relation Extraction (DocRE) relies heavily on costly manually annotated datasets, while large distant supervision resources such as DocRED distant remain underexploited due to noise. We show that a critical yet overlooked source of noise lies in structural inconsistencies within relational triples, including violations of ontology constraints and logical contradictions. We introduce an ontology-driven framework to quantify and enforce structural consistency in DocRE datasets. Our analysis reveals substantial structural noise in DocRED distant and demonstrates that such inconsistencies propagate to model predictions. Enforcing structural well-formedness during training significantly reduces logical contradictions and consistently improves generalization performance. These findings establish structural consistency as a missing axis of supervision in DocRE and highlight structural regularization as an effective strategy for leveraging distant data at scale.

Abstract:
LLMs have shown promise in legal NLP, but generating legal reasoning that is factually grounded, faithful to precedent, and consistent with judicial analysis remains difficult. Retrieval-augmented generation is often used to improve grounding, yet standard legal RAG faces two limitations: retrieved materials are hard to synthesize, and retrieval is driven by surface similarity rather than the latent legal issues and factual considerations that govern the analysis. We address these limitations with SCALE (Structured Case-law-guided Anticipatory Legal reasoning), a framework for precedent-grounded legal reasoning. SCALE converts article-specific Case Law Guides into an Issue–Factor Hierarchy, identifies the issues and factors implicated by a case, retrieves the linked guide passages, and generates the final legal analysis. Across Llama-3.1 and Qwen3 models, expert evaluation shows that SCALE consistently outperforms prompt-only and retrieval-based baselines, with the largest gains in legal issue identification and precedent use. These gains are especially strong for smaller models, suggesting that explicit legal structure can compensate for weaker implicit reasoning capacity. We also compare expert assessment with an LLM-as-a-Judge framework and find only weak-to-moderate agreement, underscoring the limits of automated evaluation for high-stakes legal reasoning.

Abstract:
Safety evaluation of large language models (LLMs) increasingly relies on LLM-as-a-judge pipelines, but strong judges can still be costly to deploy at scale. We present a multi-agent judge that uses small language models (SLMs) in structured debates among critic, defender, and judge agents. To evaluate these systems, we introduce ForesightSafety Human-Annotated JailBench (FS-HAJailBench), a human annotated jailbreak benchmark with 11,100 labeled interactions spanning diverse attack methods and target models. FS-HAJailBench provides fine-grained ground truth for measuring both target-model robustness and judge reliability. On this benchmark, our SLM-based framework improves over matched small-model baselines and prior multi-agent judges, while approaching GPT-4o-level agreement with lower estimated unit cost. Ablation results show that three debate rounds provide the best accuracy-efficiency trade-off among the current Qwen3-14B variants. These results indicate that structured debate is a practical way to improve scalable LLM safety evaluation.

Abstract:
Training speech recognizers with unpaired speech and text – known as unsupervised speech recognition (UASR) – is a crucial step toward extending ASR to low-resource languages in the long-tail distribution and enabling multimodal learning from non-parallel data. However, existing approaches based on phones often rely on costly resources such as grapheme-to-phoneme converters (G2Ps) and struggle to generalize to languages with ambiguous phoneme boundaries due to training instability. In this paper, we address both challenges by introducing a syllable-level UASR framework based on masked language modeling, which avoids the need for G2P and the instability of GAN-based methods. Our approach achieves up to a 40% relative reduction in character error rate (CER) on LibriSpeech and generalizes effectively to low-resource languages that have remained particularly difficult for prior methods. Code will be released upon acceptance.

Abstract:
Large language models (LLMs) are systematically overconfident: they routinely express high certainty on questions they often answer incorrectly. Existing calibration methods either require labeled validation data, degrade under distribution shifts, or incur substantial inference costs. Recent work has shown that LLMs already contain a better-calibrated signal than the one they verbalize: the token probability of "True" when the model is asked "Is this answer correct?" (P(True)) consistently outperforms their stated confidence, a gap that is theoretically grounded as generative error is lower-bounded by roughly twice the corresponding discriminative error. We introduce SECL (SElf-Calibrating Language Models), a test-time training (TTT) pipeline that exploits this gap as self-supervision: adaptation requires no correctness labels or human feedback, with only two hyperparameters set once per model family from a small labeled probe. SECL adapts only when the input distribution shifts, training on just 6–26% of the question stream at lower cost than the baseline it distills from. Across five language models (2B–12B) from four model families and four diverse domains, SECL reduces Expected Calibration Error (ECE) by 56–78%, outperforming its own supervision signal and matching or outperforming recent inference-time methods. SECL is the first method to apply TTT to calibration; seven ablations covering signal quality, gating strategy, weight accumulation, loss design, domain ordering, hyperparameter sensitivity, and layer selection confirm that each component is crucial and robust across configurations. Code: https://github.com/rissalhedna/SECL

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has become a prominent paradigm to enhance the capabilities (i.e. long-context) of Large Language Models (LLMs). However, it often relies on gold-standard answers or explicit evaluation rubrics provided by powerful teacher models or human experts, which are costly and time-consuming. In this work, we investigate unsupervised approaches to enhance the long-context capabilities of LLMs, eliminating the need for heavy human annotations or teacher models’ supervision. Specifically, we first replace a few paragraphs with special placeholders in a long document. LLMs are then trained through reinforcement learning to reconstruct the long document by correctly identifying and sequencing missing paragraphs from a set of candidate options. This training paradigm enables the model to capture global narrative coherence, significantly boosting long-context performance. We validate the effectiveness of our method on two widely used benchmarks, RULER and LongBench v2. While acquiring noticeable gains on RULER (nearly 10 points), it can also achieve a reasonable improvement on LongBench v2 without any manually curated long-context QA data. Furthermore, we conduct extensive ablation studies to analyze the impact of reward designs, data curation strategies, training schemes, and data scaling effects on model performance. We will release our code, data, and models.

Abstract:
Fine-grained intent detection becomes brittle as label spaces expand with many semantically adjacent intents. Although large language models (LLMs) can synthesize labeled utterances to reduce annotation cost, the resulting pseudo data are often ambiguous across intents and repetitive within intents. Existing synthesis pipelines mainly rely on prompt engineering and filtering, without updating generator parameters. We propose a preference-aligned synthesis framework that automatically mines supervision to optimize the generator. Each intent is represented by multi-centroid prototypes, and block-conditioned generation supports diversity-aware candidate construction. From these candidates, we build preferred blocks that are both prototype-consistent and diverse, and rejected blocks composed of confusable negatives from neighboring intents. These preference pairs are used to optimize the generator with Direct Preference Optimization (DPO), together with iterative refresh to mine harder confusions over time. Experiments show consistent gains for downstream intent classification in zero-shot and few-shot settings, while remaining competitive in fully supervised settings. We further observe improved generator reliability and strong transfer across multiple backbone LLM families.

Abstract:
With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation–"write like me"–has become a rapidly growing area of interest. However, style personalization is highly specific, relative to every user, and depends strongly on the pragmatic context, which makes it uniquely challenging. Although prior research has introduced benchmarks and metrics for this area, they tend to be non-standardized and have known limitations (e.g., poor correlation with human subjects). LLMs have been found to not capture author-specific style well, it follows that the metrics themselves must be scrutinized carefully. In this work we critically examine the effectiveness of the most common metrics used in the field, such as BLEU, embeddings, and LLMs-as-judges. We evaluate these metrics using our proposed style discrimination benchmark, which spans eight diverse writing tasks across three evaluation settings: domain discrimination, authorship attribution, and LLM-generated personalized vs non-personalized discrimination. We find strong evidence that employing ensembles of diverse evaluation metrics consistently outperforms single-evaluator methods, and conclude by providing guidance on how to reliably assess style-personalized text generation.

Abstract:
Knowledge editing enables rapid updates of specific factual knowledge in large language models (LLMs) without full retraining. However, more realistic scenarios call for a lifelong framework that handles continual updates rather than one-off modifications. In such settings, existing editing methods often overfit to target prompts, significantly degrading both the generalization of the edited knowledge and the model’s general capabilities. To address this issue, we propose GLIME (Generalizable Lifelong Model Editing), which combines knowledge editing with preference optimization over generation behavior. GLIME further incorporates replay-based editing and a gradient constraint to preserve previously edited knowledge. Experimental results show that GLIME significantly improves knowledge generalization in lifelong editing settings while maintaining both editing performance and general capabilities.

Abstract:
A common approach to personalization in large language models (LLMs) is to incorporate a subset of the user memory into the prompt at inference time to guide the model’s generation. Existing methods select these subsets primarily using similarity between user memory items and input queries, ignoring how features actually affect the model’s response distribution. We propose Response-Utility optimization for Memory Selection (RUMS), a novel method that selects user memory items by measuring the mutual information between a subset of memory and the model’s outputs, identifying items that reduce response uncertainty and sharpen predictions beyond semantic similarity. We demonstrate that this information-theoretic foundation enables more principled user memory selection that aligns more closely with human selection compared to state-of-the-art methods, and models that are 400 × larger. Additionally, we show that memory items selected using RUMS result in better response quality compared to existing approaches, while having up to 95% reduction in computational cost.

Abstract:
Medical LLMs are often evaluated by whether they select the correct diagnosis, but diagnostic accuracy alone does not show whether the model used the case evidence appropriately. We present a behavioral audit of evidence use in medical diagnosis. For each case, we decompose patient information into evidence units, score candidate diagnoses under controlled evidence subsets, and mine low-order interactions in diagnostic margins. Because medical evidence is diagnosis-relative, the audit separates interaction discovery from failure assignment: large or negative interactions can reflect plausible differential diagnosis, while suspicious interactions require robustness checks and clinical review. We evaluate five open-weight LLMs on DDXPlus, CupCase, and MedCase. Across datasets, faithful support and differential conflict or cancellation account for most interaction strength, showing that many evidence interactions are clinically plausible rather than failures. In a DDXPlus-focused blinded five-reviewer 130-item enriched review sample, invalid or shortcut-like cases concentrate in negated or absent findings and clinically local evidence. These results show that accuracy can hide candidate evidence-use failures and motivate role-aware audits for medical LLM evaluation.

Abstract:
Decoding throughput gains from larger inference batches are often limited by GPU memory, mainly due to the large size of KV cache. Training-free KV cache offloading mitigates this issue by storing redundant context on the CPU and fetching sparse KV subsets, but often harms long-generation quality due to training-inference mismatch of sparse patterns. Trainable sparse attention can improve quality, yet its unconstrained KV accesses are inefficient for offloading and may incur excessive CPU–GPU transfers. To this end, we propose NOSA, a trainable sparse attention mechanism designed for KV cache offloading. NOSA explicitly bounds CPU–GPU KV transfers, enabling low communication overhead and high decoding throughput. We further develop NOSI, an offloading inference system that realizes these efficiency gains. Experiments on 1,3,8 B LLMs show that NOSA outperforms KV cache offloading baselines on general, long-input, and long-generation tasks, while improving decoding throughput by up to 5.04 × , 1.92 × , and 1.83 × over FullAttn, InfLLMv2, and ShadowKV, respectively.

Abstract:
Text-based role-playing models can imitate character styles, but often fail to capture scene atmosphere and evolving tension, which are crucial for immersive applications such as VR games and interactive narratives. We study video-grounded role-playing dialogue and introduce EBM-RL (Eye-Brain-Mouth Reinforcement Learning), a decoupled GRPO-based framework that separates observation ( lt;perception gt;), reasoning ( lt;think gt;), and utterance generation ( lt;answer gt;). This design mimics the human See-Think-Speak process, enabling the model to ground dialogue in visual perception before reasoning and response generation. To optimize this See-Think-Speak process, EBM-RL integrates complementary rewards for scene-text alignment, perceptual-cognitive utility, answer faithfulness, and format consistency. Extensive experiments show that EBM-RL substantially outperforms text-only role-playing baselines and comparable-scale VLM baselines, while remaining competitive with stronger larger-scale VLMs on visual grounding and situational persona metrics. Moreover, EBM-RL demonstrates strong zero-shot transfer to out-of-domain VideoQA benchmarks without additional fine-tuning. We also release an open-source dataset for video-grounded role-playing dialogue.

Abstract:
Prior work suggests that sentiment can often be captured by approximately linear directions in LLM activation spaces, but a single direction may not fully capture sentiment representations. In natural communication, sentiment is shaped not only by polarity but also by usage factors, such as tone and audience adaptation. We test whether these factors systematically modulate sentiment representations beyond a shared sentiment direction. We construct a controlled paired dataset that holds event content fixed while varying sentiment polarity and usage factors, and analyze Llama, Mistral, and Gemma. We identify a shared sentiment direction, remove it, and test the residual structure through erasure and generation-time tone steering. Across models, the shared direction is robust (median cosine 0.953–0.975), yet removing it leaves 0.833–0.909 of the original positive–negative representation-difference norm. The residuals contain compact, reproducible usage-conditioned structure. Targeted erasure weakens held-out usage metrics more than random and label-shuffled controls. On Llama, outputs steered along residualized tone components are preferred in 92.8% of blind target-tone comparisons while preserving the requested sentiment polarity in 98.7% of evaluated outputs.

Abstract:
Low-rank (LR) pretraining underperforms full-rank (FR) pretraining, but why the gap persists remains unclear. Using GPT-2 on BabyLM, we show the representational split between them is not fixed at initialisation, spectrum-matched initialisation eliminates the representational gap at step zero, but it’s reasserted within a few training steps, implicating training dynamics over initialisation. We further show gradient dimensionality is a causally separable driver of this split, not merely a restatement of the weight factorization. Restricting or expanding gradient dimensionality of LR models directly shifts representational alignment with FR counterpart independent of weight rank. Finally, optimisation success and representational alignment dissociate, that is, spectral normalisation and other interventions improve perplexity without recovering FR-LR similarity. Also, we observered that representational alignment predicts downstream competence better than perplexity does. These results support gradient starvation as a genuine, partial mechanism, and argue efficient pretraining should be evaluated by both representational trajectory and performance, not solely the latter.

Abstract:
Multimodal fake news detection is critical for maintaining the integrity of social media. Current models often rely on shallow cross-modal correlations and rigid domain labels, which limits robustness to logical inconsistency and domain shift. We propose the Open-Vocabulary and Logic-Aware Fake News Detection (OVLAFND) framework to address these issues and relax traditional categorization constraints. To model fine-grained logical conflicts rather than surface relevance, our framework introduces Logical Adversarial Data Augmentation (LADA), a multimodal LLM-driven pipeline that constructs counterfactual text-image pairs as hard negatives. Then, we propose a Logic Consistency Module (LCM) that uses these adversarial signals to learn a dedicated logic-consistency subspace for assessing whether textual claims are supported by visual evidence. To improve topic flexibility, we transform the rigid domain labels into an Open-Vocabulary Topic Manifold (OVTM) to achieve highly interpretable and label-free topic reasoning. Across Weibo and Weibo-21, OVLAFND achieves competitive fixed-domain performance and better leave-one-domain-out generalization. Our code is available at https://github.com/lllfxy/OVLAFND.

Abstract:
NLP systems increasingly aggregate claims from retrieved passages, documents, and generated reasoning steps. Yet current factuality methods mostly verify claims individually or compare pairs of claims, leaving open the question of whether the entire aggregated claim set can be retained coherently. This misses a fundamental failure mode: claims may be locally plausible and pairwise compatible while still being jointly inconsistent. We formalize global consistency repair: given an inconsistent natural-language claim set, preserve the largest coherent subset while localizing the compact sources of inconsistency. We introduce QXR, an adaptive diagnosis-and-repair framework that treats an LLM as a noisy set-level consistency judge, localizes minimal conflicting subsets, and repairs the claim set by deleting a small hitting set of claims. Across LLM judges and benchmarks derived from VitaminC, FEVER, and conflicting-evidence RAG, QXR preserves substantially more valid claims than direct all-at-once judging while maintaining high precision. These results suggest that factuality in claim aggregation systems should be treated as a set-level repair problem, not only as local verification.

Abstract:
As chatbots increasingly influence daily decision-making, their potential to produce misleading responses poses substantial risks to users. This paper investigates a critical cognitive vulnerability in LLMs: their tendency to uncritically trust external context when presented with fabricated evidence bearing markers of credibility. We introduce Ghostwriter, a two-phase attack framework that first repackages misleading statements with fabricated rationales, then instruct target LLMs to incorporate these viewpoints when responding to relevant queries. Experiments on BBQ, ToxiGen, and our specialized dataset reveal that commercial LLMs without external safety classifiers remain highly vulnerable, while even frontier classifier-guarded models (e.g., GPT-5.4) reduce but do not eliminate the attack. Building on this, we explore multiple defense strategies, among which a tailored safety policy enables gpt-oss-safeguard to achieve 80.5% detection rate.

Abstract:
Hallucination is a central failure mode of language models (LMs). We focus on hallucinations in response to questions like: “Which instrument did Glenn Gould play?”, but we ask these questions for synthetic entities designed to be unknown to the model. We find that LMs like Gemma-7B-IT frequently hallucinate, i.e., they have difficulty recognizing that the hallucinated fact is not part of their knowledge. Based on the idea of linear relational embeddings, we put forward the following hypothesis. (i) Due to the abstract scheme that is used to represent them, LMs can easily produce plausible objects for non-existing subjects of linear relations, which can lead to hallucinations. (ii) For nonlinear relations, this mechanism for producing an object is not available and so a hallucination is easier to avoid. To test this hypothesis, we create SyntHal, a synthetic unknown-entity benchmark for 15 relations. We find that across four instruction-tuned models, relational linearity is a strong predictor of models hallucinating an object for an unknown subject vs refusing to give an answer, with correlations r ∈ [.58, .84]. While this is not direct evidence for the hypothesized causal mechanism, it is suggestive and opens up a new line of inquiry into understanding LM hallucinations.

Abstract:
Long-term memory is essential for conversational agents to remain coherent across extended dialogues, follow through on commitments made many sessions earlier, and adapt their behaviour to each user. Current LLM-backed long-term conversational memory, however, is reachability-bounded by the similarity between a query and stored content, both lexical and dense-vector. The approach is effective when query and memory share surface features such as wording or named entities (we call this descriptive). But it misses another, equally valuable class of cases, where query and memory do not share surface features and are tied only by a latent semantic arc (associative). On this regime prevailing long-term memory systems collectively fail. Covering this other half is what allows an assistant, for the first time, to actively draw on past dialogue as a semantic asset. On the memory side, this is the engineering counterpart of what cognitive science calls episodic future thinking: rehearsing past experience for the future contexts under which it will need to be found. We call these write-time rehearsals triggers. We propose T-Mem, the first long-term conversational memory architecture that covers both descriptive and associative recall. At each of two evidence granularities, single facts and full exchanges, T-Mem instantiates one descriptive trigger family and one associative trigger family, so that every memory remains reachable from both surface-similar and relevance-bound queries. As empirical validation, T-Mem reaches state-of-the-art on both LoCoMo and LoCoMo-Plus.

Abstract:
Large language models often repeatedly select the same answer even when many alternatives are plausible. Prior work treats this concentration as bias, but it does not distinguish stable model preferences from responses that depend on a particular prompt wording. We introduce a bias depth score that measures both how strongly a model prefers its top answer under direct prompting and whether that answer survives scenario reframing. Across 4,442 opinion prompts and four large language models, only about a quarter of the concentrated preferences survive reframing. We call these persistent cases Deep biases, and the remaining prompt-dependent cases Shallow biases. Our results show that Deep biases are more often inherited from pretraining and preserved through SFT. Under both continued fine-tuning and prompt-based debiasing for diversity, Deep biases are consistently harder to remove than Shallow biases. Bias depth therefore separates stable learned biases from prompt-wording artifacts that single-prompt metrics conflate. Code, models, and data are available at https://deepbias.github.io.

Abstract:
Membership inference attacks (MIAs) have emerged as critical tools for auditing privacy risks in large language models (LLMs), aiming to determine whether a given text was included in a model’s training corpus. However, most existing MIAs require access to per-token logits or probabilities, making them inapplicable in practice to proprietary LLMs that expose only textual continuations. To address this underexplored setting, we propose Word-level Probability MIA (WPMIA), a statistically principled MIA for strict black-box privacy auditing. WPMIA estimates word-level generation probabilities via Monte Carlo sampling with local kernel smoothing, then aggregates these estimates into a sequence-level likelihood estimator. Furthermore, WPMIA constructs the likelihood conditioned on different prefixes, thereby amplifying the distributional differences between members and non-members. We evaluate WPMIA across various open-source LLMs and find that it consistently outperforms existing black-box baselines. Importantly, we also evaluate WPMIA on modern proprietary LLMs, including GPT-5-Chat, Gemini-2.5-Flash, and Claude-4.5-Haiku, achieving an average TPR@5%FPR of 42.0 across these models. These results offer a sound foundation for future research on strict black-box membership inference. Code is available at https://github.com/niusj03/WPMIA.

Abstract:
The LLM-as-a-judge (LLMaJ) framework has emerged as a promising solution for cheap, reproducible, reference-free Natural Language Generation (NLG) evaluation. Prior work seeks to improve LLMaJ by decomposing evaluation tasks into simpler sub-tasks. In this work, we systematically compare LLMaJ methods with and without decomposition on multiple NLG datasets. We find no evidence that LLMaJ with task decomposition leads to performance gains over a fair baseline that does not use decomposition. Instead, we find that previously reported performance gains in decomposition-based LLMaJ stem from using human labels as training data, and not task decomposition itself. Also, we find that, when human labels are available, LLMaJ without using task decomposition can perform comparably to human annotators.

Abstract:
As large language models (LLMs) are increasingly used for long-form generation, reliably evaluating long-form outputs has become a critical challenge. LLM-as-a-judge offers a scalable alternative to human evaluation, yet its reliability in long-form output evaluation remains underexamined: existing meta-evaluation benchmarks focus mainly on short-form outputs. Compared with short-form evaluation, long-form evaluation is not merely a matter of output length; it often requires judges to handle more complex document-level demands. In this work, we introduce LongJudgeBench, a comprehensive benchmark for evaluating LLM judges on long-form outputs across diverse real-world scenarios and judging protocols. We systematically evaluate a broad range of LLM judges, covering multiple base models and judging settings. Our results reveal a substantial reliability gap: current LLM judges remain unstable across scenarios, and rubrics or references are helpful but not always sufficient. We hope LongJudgeBench will support future research on more robust, context-aware, and human-aligned LLM-as-a-judge methods. Our code is available at https://github.com/cjj826/LongJudgeBench.

Abstract:
Agentic RL can benefit substantially from reusable experience, but existing skill-based methods mostly focus on trajectory-level guidance and provide limited support for the principled evaluation and maintenance of an evolving skill memory. We propose D2Skill, a dynamic dual-granularity skill bank for agentic RL that organizes reusable experience into task skills for high-level guidance and step skills for fine-grained decision support and error correction. D2Skill jointly trains the policy and skill bank through paired baseline and skill-injected rollouts under the same policy, using their performance gap to derive hindsight utility signals for both skill updating and policy optimization. Constructed from training-time trajectories, the skill bank is continuously expanded through reflection and maintained with utility-aware retrieval and pruning. Across ALFWorld, WebShop, and Search-Augmented QA, D2Skill consistently outperforms skill-free and skill-based baselines, remains effective with self-feedback, and transfers to unseen ALFWorld environments without test-time adaptation. Project Page: https://github.com/TU2021/D2Skill-AgenticRL

Abstract:
This paper introduces the Multilingual FrameNet Corpus (mFNC), a novel resource that extends the English Berkley FrameNet corpus by collecting and harmonizing existing language-specific corpora across nine additional languages: Brazilian Portuguese, Chinese, Dutch, French, German, Italian, Korean, Latvian and Swedish. By training models that rely on different architectures on the mFNC, we consistently outperform existing state-of-the-art Frame Semantic Parsers in both multilingual and cross-lingual settings, underscoring the importance of multilingual training data. The mFNC and our trained FSP models are openly available at https://github.com/beatrice-f/mFNC.

Abstract:
Neural architecture search (NAS) is a core technique for automating AI model design, yet its efficiency is limited by the high cost of reliable architecture evaluation. To address this bottleneck, we introduce code-semantic transition modeling for NAS acceleration: before spending evaluator budget, we ask whether a proposal’s semantic change from its parent suggests a worthwhile architecture evolution. We instantiate this idea with ECO-SCE, an Evolutionary Code Semantic Change Estimator placed between the architecture generator and evaluator. Given a parent-to-proposal code transition, ECO-SCE filters likely low-gain proposals and reserves evaluator calls for proposals that are more likely to provide useful evolutionary progress. To train ECO-SCE, we construct NB101-SemPair from NAS-Bench-101 by sampling beneficial, harmful, and marginal transitions with diverse semantic and structural changes. ECO-SCE learns risk-aware transition utility from global semantic shifts, sparse semantic overlaps, and explicit structural edits, allowing it to skip likely low-gain proposals while sending promising or uncertain transitions to the evaluator. Across seven downstream benchmarks including CLRS, ECO-SCE reduces average GPU time by 70.88% and improves task performance in 6 out of 7 cases when inserted zero-shot into OpenEvolve.

Abstract:
Text-attributed graphs (TAGs) have recently garnered significant attention due to their rich text information for each node. Graph Neural Networks (GNNs) have been widely used to leverage the text attributes and graph topology. However, traditional GNNs use shallow text embeddings and struggle with sparse and noisy structure. The advent of Large Language Models (LLMs) has introduced powerful capabilities in reasoning and text generation, promoting recent works to integrate LLMs with GNNs. However, most existing studies focus on a specific aspect of graph data, such as node features or graph structure, without fully exploring the entire graph. In this paper, we propose EFS (Enhancing Features and Structures), a novel holistic enhancement method that enhances both node features and graph structure with LLMs, which thoroughly explores the text attributes and efficiently addresses both missing and noisy edges compared to existing methods. We first design elaborate prompt templates and adopt LLMs to generate summarized keywords, ranked predictions and corresponding explanations for enriching the node features. Then we construct a lightweight edge refiner to improve the graph structure, which utilizes the previously generated ranked predictions without additional LLM API calls. Extensive experiments on various benchmark datasets show the superior performance and efficiency of EFS compared with the state-of-the-art baselines.

Abstract:
Machine translation tests masked diffusion language models (dLLMs) because every source token must be rendered faithfully, while fixed canvas decoding must choose target length before denoising. Existing masked diffusion decoding work mainly studies token unmasking order, leaving this length decision under-explored despite its direct effect on coverage and redundancy. We introduce Entropy-Valley (EV), a training-free length selector that scores candidate target canvases by mean predictive entropy from all-mask forward passes and selects the canvas the backbone is most prepared to fill. Relative to a baseline using training corpus length statistics, EV recovers 64.9%, 65.3%, and 33.0% of the COMET-22 gain from reference target lengths on En→Zh, Zh→En, and En→De. Our diagnostics show that denoising-friendly lengths need not match reference lengths. Evaluation by three translation experts supports the En↔Zh adequacy gains, with stronger evidence on Zh→En. Compared with a LLaMA-3-8B autoregressive (AR) model trained on the same fine-tuning data, the EV system ties on En→Zh and leads on Zh→En; an oracle-length diagnostic further shows that, in this masked diffusion MT setting, deciding which tokens to reveal first matters less than how the target length is supplied.

Abstract:
Safety-aligned language models are expected to refuse harmful requests while remaining useful for benign ones. However, because many evaluations score only the final response, they often conflate intent understanding with response-level safety behavior. To address this, we introduce RiskCueBench, a recoverability-controlled benchmark that separates these two stages by first measuring whether a model can successfully recover a high-level intent and then evaluating its safety behavior on the same prompt. RiskCueBench contains 160 latent intents and 640 abstract non-operational surface prompts across eight risk categories. The benchmark covers 25 completed model groups and 32,000 successful model outputs with final rubric-adjudicated labels. When conditioning on pairs where both surface forms are recoverable, we find that increased contextualized specificity induces a shift toward less safe behavior. Specifically, paired bootstrap estimates show that the refusal rate and global safety score decrease by 0.175 and 0.440, respectively. Furthermore, mixed effects modeling corroborates a 0.107 increase in unsafe cooperation. These results show that the safety behavior can be miscalibrated to the surface form even after the intent has been recovered, motivating safety evaluations that report response behavior conditional on semantic recoverability.

Abstract:
Program verifiers play a central role in training coding agents, including selecting trajectories for supervised fine-tuning (SFT) and providing rewards for reinforcement learning (RL). Standard execution-based verification requires running unit tests inside per-repository environments such as Docker images, incurring substantial environment setup costs. We propose Dockerless, an environment-free agentic patch verifier that evaluates generated code patches without executing them. Rather than simply matching candidate patches to references, Dockerless judges patch correctness using evidence gathered through agentic repository exploration. On a verifier evaluation benchmark, Dockerless outperforms the strongest open-source verifier by 14.3 AUC points. Using Dockerless as both the SFT trajectory filter and the RL reward enables a fully environment-free post-training pipeline. The resulting model reaches 62.0%, 50.0%, and 35.2% resolve rate on SWE-bench Verified, Multilingual, and Pro, respectively. It surpasses the Qwen3.5-9B baseline by 2.4, 8.7, and 2.9 points, matching environment-based post-training.

Abstract:
Vision-language models (VLMs), despite their extraordinary zero-shot capabilities, are vulnerable to distribution shifts. Test-time adaptation (TTA) emerges as a predominant strategy to adapt VLMs to unlabeled test data on the fly. However, existing TTA methods heavily rely on zero-shot predictions as pseudo-labels for self-training, which can be unreliable under distribution shifts and misguide adaptation due to two fundamental limitations. First (Modality Gap), distribution shifts induce gaps between visual and textual modalities, making cross-modal relations inaccurate. Second (Visual Nuisance), visual embeddings encode rich but task-irrelevant noise that often overwhelms task-specific semantics under distribution shifts. To address these limitations, we propose SubTTA, which aligns the semantic subspaces of both modalities to enhance zero-shot predictions to better guide the TTA process. To bridge the modality gap, SubTTA extracts the principal subspaces of both modalities and aligns the visual manifold to the textual semantic anchor by minimizing their chordal distance. To eliminate visual nuisance, SubTTA projects the aligned visual features onto the task-specific textual subspace, which filters out task-irrelevant noise by constraining visual embeddings within the valid semantic span, and standard TTA is further performed on the purified space to refine the decision boundaries. Extensive experiments on various benchmarks and VLM architectures demonstrate the effectiveness of SubTTA, yielding an average improvement of 2.24% over state-of-the-art TTA methods. Our code is available at https://github.com/zhichenz98/SubTTA_EMNLP26.

Abstract:
Video captioning directly reflects a Video Large Language Model’s perception, making caption fidelity a strong proxy for audio-visual grounding. Existing approaches often rely on costly annotations or stronger external teachers, limiting scalability. In this paper, we instead explore whether a VideoLLM can improve its own captions by exploiting structure already present within the model through self-supervision. Our exploration builds on two asymmetries we observe in current VideoLLMs. The first is a task asymmetry: captioning requires broad open-ended perception, whereas question answering narrows reasoning to specific dimensions. The second is an information asymmetry: the same model reasons more reliably with richer context or higher-fidelity audio-visual inputs. Motivated by these observations, we propose SODA (Self-Supervised On-Policy Distillation via Dual-Stream Auditing), a framework that enables a VideoLLM to critique and refine its own captions. A student first produces an initial caption from a low-budget video view, followed by a dual-stream audit: a Caption QA Stream identifies factual errors, and a Video QA Stream detects omitted content. Conditioned on this Revision QA Bank, a teacher using a larger input budget rewrites the caption, and the refined outputs are distilled back into the student. Experiments suggest that, despite training only on the captioning objective, SODA improves caption fidelity, reduces hallucination, and remains competitive on general video understanding.

Abstract:
In current Mixture-of-Experts architectures, routing is performed based on representations dominated by structure shared across all tokens, limiting expert specialization. We show that contrasting each token against an Exponential Moving Average of the layer’s hidden states, rather than routing on absolute magnitude, concentrates the routing signal onto a low-dimensional, highly separable subspace. Building on this, we propose the Contrastive Routing Mechanism (CoRM), which scores each expert by the gap between its affinity for the incoming token and its affinity for this shared reference state, interpreted through a distinct per-expert projection. The resulting experts have routing boundaries that align with linguistic structure significantly more than the Top- k baseline. Our experiments show that CoRM improves average zero-shot accuracy by +0.67 to +1.69 points (Top-1) and +1.38 to +1.77 points (Top-2) over standard Top- k MoE baselines on nine zero-shot reasoning benchmarks, at the minimal cost of 2.9% added parameters and 2.6% added FLOPs per token.

Abstract:
Despite remarkable progress in machine translation (MT), non-AI communities have raised growing concerns about MT systems, suggesting a noticeable gap between technical advancement and the needs of real-world users. For instance, while NLP researchers focus on benchmark performance, end users care about ethical concerns, trust, reliability, costs, and more. We argue that listening to various user communities is essential so that research efforts would be directed towards the problems that the communities care about. To this end, we present a large-scale analysis, for the first time, that investigates what four stakeholder communities (AI developers, professional translators, language learners, and language service providers) post about MT technology on social media. To do so, we construct a dataset of 79,286 posts and comments from Reddit, Facebook, Bluesky, and Mastodon from 2019 to 2025, and analyse where these communities disagree, and how and why. Overall, we find that communities often disagree, and even show strong conflicts due to polarised sentiments on topics such as translation quality, efficiency, and reliability. This is because these communities approach these topics differently: the AI community frames them as technical and computational problems, while non-AI (user) communities care more about quality nuances, time savings, user trust, and broader social issues.

Abstract:
To guarantee that an LLM’s outputs conform to a specified structure, context-free grammar (CFG) decoding engines force the selection of next tokens to produce strings that conform to a given CFG. Current CFG-constrained decoding engines are highly optimized, but still suffer from the inherent costs arising from their massive per-step search space—i.e. the entire token vocabulary. This results in intractably high overhead for more complex CFGs, which is precisely the situation where CFG engines are most useful. In this paper, we introduce CFGzip, an offline technique for compressing the token search space, which massively reduces CFG engine overhead. In experiments, we report latency reduction of up to 75x during batched inference, cutting overhead down to ~1.2-2x on the hardest grammars: with CFGzip, constrained decoding is now possible at scale for complex CFGs.

Abstract:
Large Language Models (LLMs) can produce surprisingly sophisticated estimates of their own uncertainty. However, it remains unclear to what extent this expressed confidence is tied to the reasoning, knowledge, or decision making of the model. To test this, we introduce RiskEval: a framework designed to evaluate whether models adjust their abstention policies in response to varying error penalties. Our evaluation of several frontier models reveals a critical dissociation: models are neither cost-aware when articulating their verbal confidence, nor strategically responsive when deciding whether to engage or abstain under high-penalty conditions. Even when extreme penalties render frequent abstention the mathematically optimal strategy, models almost never abstain, resulting in utility collapse. This indicates that calibrated verbal confidence scores may not be sufficient to create trustworthy and interpretable AI systems, as current models lack the strategic agency to convert uncertainty signals into optimal and risk-sensitive decisions.

Abstract:
Reasoning structures (e.g., chains and graphs) are increasingly used for probabilistic inference, factuality verification, and uncertainty estimation, but their structural reliability is rarely evaluated directly. We test whether this reliability holds through a proposed two-stage diagnostic evaluation in open-ended multi-hop QA. Stage 1 tests different capabilities required to construct reasoning graphs, showing that graph reliability varies with topology, prompt design, and model family. Stage 2 uses uncertainty estimation as a downstream test case. Generated-edge experiments show that graph-based computation is sensitive to upstream graph errors, with performance dropping relative to the gold-graph setting. Gold-graph experiments further show that graph structure provides little benefit when used only as prompt context, but improves the separation of correct from incorrect answers by confidence score and enables node-level diagnostics.

Abstract:
LLM benchmark labels are frozen at release and silently copied into downstream benchmarks, errors and all. Our Item Response Theory indicator surfaces these mislabels at 95% precision in the top 200, across 114 models on seven preference and multiple-choice benchmarks, outperforming a supervised classifier. We trace these errors to mechanical labeling rules, upstream errors inherited unchanged from source datasets, and ambiguous items with no defensible single key. The same fit reveals that reward models specialize in stylistic preference over factual knowledge, and exposes one frontier reward model that agrees with detected mislabels at 78% accuracy versus 38% for its peers, consistent with contamination or benchmark-specific over-optimization.

Abstract:
Jailbreak robustness has become central to large language model (LLM) safety evaluation, yet prevailing methodologies rely primarily on refusal behavior, semantic resemblance, and intent-matching heuristics that emphasize linguistic plausibility rather than correctness. We identify a key limitation in existing evaluations: many jailbreak intents depend on instructional validity rather than epistemic factuality, allowing realistic-looking responses to be labeled successful despite being factually or procedurally incorrect. To address this gap, we propose Sequential Epistemic and Action-Level Validation (SEAV), a verification-centric jailbreak evaluation framework that decomposes responses into ordered steps and evaluates both validity and correctness. SEAV combines LLM-as-a-judge mechanisms for semantic interpretation with retrieval-grounded verification using external knowledge sources, assessing whether generated content is factually correct, structurally consistent, and operationally capable of advancing harmful objectives. Empirically, SEAV cuts the false-positive rate on SD-A (a curated strategic-dishonesty diagnostic) by 14.9 pp vs. the strongest baseline, and reclassifies 22.1%-51.0% of prior-labeled successes as invalid across three of four public benchmarks. Together, these results show that enforcing correctness substantially reshapes measured robustness: many previously labeled jailbreak successes are reclassified as invalid, and results are stable across search backends and evaluator models. Code and data are available at https://github.com/seav-anonymous/SEAV.

Abstract:
Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability. Yet for LLMs, nonconformity scores are often induced by an inference pipeline, not just a fixed model, making them depend not only on the data distribution but also on configurable factors such as the prompt template, decoding parameters, and deployment setting. Since such configurations are routinely modified in practice but rarely treated as a source of shift, their impact on CP validity remains poorly understood. We call this configuration shift and study it systematically along three axes: prompt template, decoding temperature, and weight quantization. In a broad empirical study spanning 9 LLMs, 4 datasets, and 4 nonconformity scores, we find that configuration shift consistently erodes CP validity, often driving empirical coverage below the target. By contrast, efficiency is largely preserved: valid prediction sets remain close in size to the i.i.d. baseline. We derive coverage lower bounds that attribute this loss to a discrepancy between calibration and test score distributions, and use their finite-sample plug-in versions as empirical diagnostics of shift severity. We further show that these findings lead to practical mitigations: bound-inspired recalibration is effective with limited test examples, while fragility-aware calibration ensembling recovers much of the lost coverage without test data.

Abstract:
Understanding how LLMs reason is hindered by a practical asymmetry: while their generated outputs are observable, the underlying reasoning patterns remain opaque. Relying on single probes, such as Mutual Information Peak (MIP) or Deep-Thinking Ratio (DTR), risks underestimating the genuine inferential structure. To response this deficiency, we present an Integrated, cross-Architecture Reasoning (IAR) framework, designed to provide a unified approach to LLM reasoning interpretability. Specifically, we first propose to use bandwidth-calibrated MIP coupled with Tukey IQR peak-detection to isolate reasoning-crucial tokens at the output layer. Second, we performed an overlap analysis between MIP-picked tokens and DTR-deep tokens to trace the cross-layer trajectories of those tokens. This also discloses whether reasoning-crucial tokens are computation-intensive as well, further facilitating to understand how reasoning patterns evolve across model layers. Finally, we apply a Jaccard stability metric over multi-domain problems to verify if the MIP-identified tokens are reasoning quality-guaranteed. Extensive experiments on six models across four domains (mathematics, code, logic, and common sense) demonstrate IAR’s generalizable interpretation capabilities across architectures. Our code is available at https://github.com/LeonardoMatthew/IAR.

Abstract:
Current Large Language Model reasoning systems process queries independently, discarding valuable cross-instance signals such as shared reasoning patterns and consistency constraints. We introduce Batch-of-Thought (BoT), a training-free method that processes related queries jointly to enable cross-instance learning. By performing comparative analysis across queries, BoT identifies high-quality reasoning templates, detects errors through consistency checks, and amortizes computational costs. We instantiate BoT within a multi-agent reflection architecture (BoT-R), where a Reflector performs joint evaluation to unlock mutual information gain unavailable in isolated processing. Experiments across three model families and six benchmarks demonstrate that BoT-R consistently improves accuracy and confidence calibration while reducing inference costs by up to 61%. Our theoretical and experimental analysis reveals when and why batch-aware reasoning benefits LLM systems.

Abstract:
Large language model agents trained with reinforcement learning (RL) often learn brittle, task-specific shortcuts. We hypothesize that agents generalize better when their successful trajectories are structurally compressible, decomposed into a small set of reusable abstract patterns. To formalize this, we introduce ReuseRL, which grounds agentic RL in the Minimum Description Length (MDL) principle. ReuseRL extracts a shared skill dictionary from successful trajectories and augments the RL objective with a segmentation cost, explicitly penalizing idiosyncratic behaviors that encode poorly. We prove a PAC-Bayes generalization bound for this objective. Across ALFWorld, TextWorld-Cooking, and Countdown-Stepwise, ReuseRL improves in- and out-of-distribution success over vanilla GRPO and strong round-length baselines.

Abstract:
Decoder-only large language models (LLMs) employ autoregressive text processing, attending exclusively to preceding tokens when generating subsequent ones. Although this architectural design excels in text generation applications, it constrains model performance on certain text understanding tasks. In this work, we propose SUNDER, a novel approach that enables partial bidirectional attention computation. We validate the proposed approach on three token spans identification tasks, such as Named Entity Recognition (NER), Aspect-based Sentiment Analysis (ABSA), and Definition Extraction (DEF). Results show that SUNDER improves LLM performance by up to 5 percentage points compared to traditional instruction-tuning. We further validate the robustness of our approach on multiple model families, model sizes, and prompting techniques. Our evaluation extends to multilingual scenarios and single-hop question answering (QA), confirming SUNDER’s effectiveness on multiple settings. The code is made available to the community.

Abstract:
Social media fact-checking is increasingly challenged by evidence-level and aggregation-level conflicts, where erroneous evidence often mimics authoritative news sources. To support conflict verification task, we construct ContraNote, a real-world dataset derived from X Community Notes. ContraNote is curated to mirror the two aforementioned challenges, providing 33,686 posts for evaluating evidence-level conflict resolution, and 54,474 instances for evaluating aggregation-level evidence prioritization. This paper further proposes CoVer, a framework designed to resolve these contradictions through factual adjudication. Cover uses a three-stage pipeline consisting of evidence schema normalization, factual consensus and support verification, prioritizing definitive evidence over noise to prevent it from compromising the final verdict. Technical evaluations across multiple benchmarks show that CoVer outperforms state-of-the-art baselines, achieving an accuracy of 88.4% on CONFACT-HumC and 89.4% on CONFACT-ModC.

Abstract:
Block Diffusion Models (BDMs) support parallel generation, flexible-length output, and KV caching, making them promising for efficient document parsing. However, existing BDMs bind denoising and cache commitment to fixed block boundaries: parallelism shrinks during intra-block denoising, while generated tokens cannot be cached until the whole block is completed. Moreover, intra-block bidirectional denoising conflicts with inter-block autoregression, creating inconsistent information flow that can challenge structure-sensitive recognition. We propose the Prefix-Adaptive Block Diffusion Model (PA-BDM), which replaces intra-block bidirectional denoising with causal denoising from prefix to suffix and treats the block size as a maximum candidate range rather than a fixed commitment unit. PA-BDM uses Confidence-gated Structural Loss (CSL) to build low-entropy prefixes before extending training to longer continuations. During inference, Progressive Prefix Commitment (PPC) then dynamically commits the longest reliable prefix into the KV cache and resets the next candidate range from the updated prefix, restoring a large parallel decoding space at each step. Experiments show that the 3B PA-BDM achieves higher recognition scores on several benchmarks and improves inference throughput by 71.6% over the 2.5B MinerU-Diffusion.

Abstract:
Diffusion Language Models (DLMs) benefit from a flexible decoding mechanism that enables parallelized inference and controllable generations over autoregressive models. Yet such flexibility introduces a critical challenge: inference performance becomes highly sensitive to the decoding order of tokens. Existing heuristic methods, however, focus mainly on local effects while overlooking long-term impacts. To address this limitation, we propose the Foreseeing Decoding Method (FDM), a novel approach that integrates both local and global considerations to unlock the full potential, employing a search-based strategy to enable effective optimization in discrete spaces. Furthermore, by analyzing the consistency of chosen tokens in the full decoding process, we develop a variant, FDM with Acceleration (FDM-A), which restricts deep exploration to critical steps identified as the exploration and balance circumstances. Extensive experiments across diverse benchmarks and model architectures validate the scalability of FDM and demonstrate the superior efficiency-performance trade-off achieved by FDM-A. Our work might potentially provide a principled step toward more powerful decoding methods for DLMs. Our code is available at https://github.com/PKU-ML/FDM.

Abstract:
Online users often release multiple texts under the same identity, giving attackers an author profile that can reveal more than any single text. Existing authorship obfuscation methods optimize privacy independently for each document, leaving them blind to cross-document correlations that make aggregation dangerous. We propose Aggregation-Aware Synthetic Text Generation (AAST), a framework that addresses this gap by jointly selecting synthetic texts at the bundle level rather than optimizing each text in isolation. AAST targets attribution and verification attacks, including cross-genre settings where attacker references come from a genre not observed during generation or selection. Experiments across same-genre, cross-genre, neural, and independent non-neural stylometric attacks show that AAST lowers account-level linkability as bundle size grows, while preserving semantic quality, linguistic acceptability, and sentiment alignment.

Abstract:
Linear probes are widely used in interpretability research and often compared by cosine similarity. The Mahalanobis cosine similarity (MCS) between two directions, which reweights the inner product by test data covariance, is a natural task-aware refinement. Ying et al. (2026) report a nearly perfect linear relationship between a probe’s MCS to a reference probe trained on out-of-distribution (OOD) data and its OOD AUROC (R^2=0.98). Here, we extend this empirical finding across models, layers, and concept domains, and prove it in closed form: For balanced classes whose projections are Gaussian, OOD AUROC and MCS to the reference probe are near-linear because both are sigmoid-shaped functions of the probe’s signal-to-noise ratio (SNR) on the test data. The theory also predicts when this linearity fails, which we verify empirically. MCS offers a theoretically grounded and empirically effective alternative to Euclidean cosine similarity for comparing linear probes.

Abstract:
Learning from experience is critical for developing capable, self-improving large language model (LLM) agents. Existing methods typically extract knowledge from accumulated trajectories via reflection, memory, rules, or skills. However, agents in realistic environments continuously encounter novel tasks, often offering only a one-shot opportunity to improve. These executions yield rich but highly noisy contexts, entangling broadly useful lessons with task-specific artifacts. Critically, prior works rarely validate their effectiveness on complex real-world tasks or isolate the underlying drivers of improvement. To address these gaps, we study online harness learning, where a frozen agent improves by continually updating a structured harness across sequential tasks. This setup enables a systematic study of key self-improvement factors through our proposed Evo-Harness. At its core, context-to-harness skill compilation distills noisy, single-shot executions into reusable skill harnesses for cross-domain and topic-level adaptation. To demonstrate the efficacy of one-shot skill compilation, we evaluate across five realistic benchmarks (TerminalBench2, SWE-bench, CL-Bench, 𝜏 -bench, WebArena-Infinity). Our extensive analysis demonstrates the effectiveness of Evo-Harness and provides a principled understanding of how LLM agents can effectively learn on the fly. Our code is available at https://github.com/A-EVO-Lab/a-evolve/tree/release/evo-harness.

Abstract:
Recent advances in LLMs and VLMs have enabled safety systems to reason beyond simple risk patterns toward more contextual and semantic safety concerns. However, as risk patterns continue to evolve and safety rules become more complex, existing training-based end-to-end safeguards face persistent challenges in adaptability and explainable reasoning over complex safety rules. To address these challenges, we propose GuardEn (Guarding by Safety Rule Entailment), an executable safeguard framework that decomposes safety policies into atomic propositions through Safety-Rule Compilation, modeling their composition as executable code. At test time, Scene-Grounded Execution instantiates these atomic propositions with contextual visual information derived from scene graphs, enabling rule-grounded and interpretable safety reasoning. Experiments on SafetyVisionBench demonstrate the effectiveness of programmable safeguard for complex visual safety assessment, achieving an average improvement of 9.8 F1 points over the strongest baseline across four domains.

Abstract:
Deploying Large Language Models (LLMs) in healthcare requires robust performance across two complementary dimensions - diagnostic reasoning: the convergent, evidence-driven task of inferring a patient’s condition from clinical data to produce a diagnosis, and clinical healthcare reasoning: the broader, navigational judgment required to communicate, plan, and adapt across multi-turn clinical interactions where a single correct answer may not exist. Recent benchmarks such as HealthBench and MedXpertQA reveal persistent weaknesses in both areas, exposing failures in complex diagnostic scenarios and limitations in contextual, patient-centered dialogue. We introduce a sequential training framework that targets these facets using synthetic data and rubric-based reinforcement learning. First, we improve diagnostic reasoning using MedBullets-derived questions with rule- and rubric-guided Reinforcement Learning (RL). We then shift to clinical reasoning by generating 5.3k synthetic multi-turn scenarios, each paired with multi-dimensional rubrics to comprehensively assess the response. This approach yields over 10% improvement on MedXpertQA, and our 30B model achieves 50.1% accuracy on HealthBench-Hard, surpassing proprietary baselines including GPT-5 (thinking). Our results show that targeted synthetic datasets and rubric-based training can systematically improve both diagnostic and interactive clinical reasoning in medical LLMs.

Abstract:
Although large language models (LLMs) have achieved remarkable performance, the inherent stochasticity of their reasoning processes and varying conclusions present significant challenges. Majority voting or Best-of-N with external verifiers has been explored to mitigate this, but these approaches are limited in applicability or require additional training. To address this problem, we propose a novel framework that Recycles Few-shot examples to verify LLM outputs (ReFeri). Our key idea is to utilize the given few-shot examples not only to generate outputs, but also to evaluate the candidate outputs. Specifically, ReFeri combines a forward confidence score with a backward reconstruction penalty to select candidates that follow few-shot guidance while avoiding demonstration-specific overfitting. Experiments with three different LLMs across seven diverse tasks demonstrate that our framework significantly improves the accuracy of LLMs—achieving an average relative gain of 8.2%—through effective response selection.

Abstract:
Human annotation cost and time remain significant bottlenecks in Natural Language Processing (NLP), with test data annotation being particularly expensive due to the stringent requirement for low-error and high-quality labels necessary for reliable model evaluation. Traditional approaches require annotating entire test sets, leading to substantial resource requirements. Active Testing is a framework that selects the most informative test samples for annotation. Given a labeling budget, it aims to choose the subset that best estimates model performance while minimizing cost and human effort. In this work, we formalize Active Testing in NLP and we conduct an extensive benchmarking of existing approaches across 18 datasets and 4 embedding strategies spanning 4 different NLP tasks. The experiments show annotation reductions of up to 95%, with performance estimation accuracy difference from the full test set within 1%. Our analysis reveals variations in method effectiveness across different data characteristics and task types, with no single approach emerging as universally superior. Lastly, to address the limitation of requiring a predefined annotation budget in existing sample selection strategies, we introduce an adaptive stopping criterion that automatically determines the optimal number of samples. We release our code at https://github.com/amazon-science/NLPActiveTesting.

Abstract:
Large vision-language models (LVLMs) are increasingly being applied to multi-view image inputs captured from diverse viewpoints. Despite this growing use, current LVLMs often generate incorrect responses due to visual interference from non-target instances or viewpoints, a phenomenon we term multi-view hallucination (MVH). To systematically analyze this problem, we construct MVH-Bench, a benchmark comprising 4.8k question-answer pairs targeting two types of hallucination: cross-instance and cross-view. Empirical results show that MVH is prevalent across recent LVLMs. To address this issue, we propose Reference Shift Contrastive Decoding (RSCD), a training-free decoding technique that suppresses visual interference by generating negative logits through attention masking. Experiments on MVH-Bench with LLaVA-OneVision and Qwen2.5-VL demonstrate that RSCD improves performance by up to 25.7 and 94.8 points over existing hallucination mitigation methods, highlighting the effectiveness of RSCD.

Abstract:
As conversational companions, large language models (LLMs) often have access to users’ emotional states. We study how this affective context modulates LLM sycophancy in subjective, evaluative interactions, where users share actions or opinions that invite feedback. Drawing on ingratiation theory, we measure sycophancy as the divergence between a model’s independent evaluation and its user-facing response, elicited by presenting the same content as either a third-party account or the user’s own disclosure. Across seven LLMs and two Reddit datasets (r/AmItheAsshole and r/TrueUnpopularOpinion), we find that this divergence is systematic and strongly one-directional. User-facing responses consistently soften or withhold negative or oppositional judgments. Affective context further amplifies this divergence with negative states, particularly loneliness and distress, producing the largest effects. These findings suggest that affective context functions as a vulnerability signal that suppresses critical feedback when users may need it most, often through evasive sycophancy, in which models retreat toward non-committal responses rather than outright agreement.

Abstract:
Tokenization-free, pixel-based language models (PLMs) have recently been proposed as an alternative to conventional token-based language models (TLMs), typically motivated by practical considerations, e.g., the flexibility to support new languages or vocabularies. Nevertheless, PLMs also have the potential to serve as infrastructure for computational (psycho)linguistics, where tokenization has been a source of bias in estimating information-theoretic values, such as surprisal, and has pushed the bottom-up visual orthographic features outside the scope. In this study, we first integrate PLMs into the modeling of naturalistic human reading and, in particular, examine their use to estimate next-word predictability, which can also integrate orthographic and parafoveal information. Our results demonstrate that next-word prediction difficulty, computed in pixel space , has strong predictive power for gaze duration and word skipping behavior across various languages. In particular, their advantage is complementary to existing token-level surprisal and entropy estimates. Cross-lingual analyses with script variations also clarify their distinctive visual processing biases, facilitating research into how bottom-up visual information is incrementally integrated into expectation-based reading.

Abstract:
Stepwise model routing improves the efficiency of Large Reasoning Models (LRMs) by assigning each reasoning step to a suitable model. Recent methods formulate routing as a sequential decision process and train the router with reinforcement learning. However, although they model routing as a process, they still supervise the router with outcome rewards. Such rewards only reflect final answer correctness and fail to evaluate intermediate routing decisions, which can weaken performance and generalization. To address this gap, we propose RoRo, a rubric-guided process reward framework for stepwise model routing. RoRo first collects diverse routing trajectories and constructs preference pairs based on outcome, cost, and process quality. It then trains a Rubricor to generate a query-specific evaluation rubric and a Judge to score routing trajectories under this rubric through alternating optimization. The resulting process rewards are combined with outcome rewards to optimize the routing policy via GRPO. Experiments on five reasoning benchmarks under both same-family and cross-family settings show that RoRo consistently outperforms strong baselines and achieves better accuracy and cost trade-offs.

Abstract:
Mixture-of-Experts (MoE) has emerged as the dominant paradigm for parameter-efficient scaling and cost-effective deployment of Large Language Models (LLMs). However, scaling laws developed for dense models are fundamentally inapplicable to MoE architectures, owing to three critical challenges: the multiplicity of governing factors, their intricate interdependencies, and the non-monotonic nature of their effects on model performance. These challenges collectively demand a rigorous and fine-grained investigation into MoE-specific scaling laws. In this work, we present a systematic decomposition of MoE configurations, identifying five key factors that influence model performance from both size and structural perspectives: data size ( D ), total model size ( N ), activated model size ( N a ), number of active experts ( G ), and ratio of shared experts ( S ). We conduct 450 controlled experiments to characterize the marginal effects of each factor, culminating in a comprehensive and precise joint MoE scaling law. Building on this foundation, we derive the theoretically optimal and practically efficiency-aware optimal configurations for G , S and N a /N with detailed analyses. Our findings reveal that the optimal settings for G and S are invariant to model and data size, while the optimal activation ratio N a /N grows sparser as N increases. Our proposed MoE scaling law provides accurate and insightful guidance for future MoE model design and training.

Abstract:
Fine-tuning large language models (LLMs) on a single target task requires balancing task adaptation with retention of prior capabilities. Existing teachers mainly guide preservation or transfer, but do not identify which target-task samples should be emphasized when fine-tuning must retain prior capabilities. We propose the Unconstrained Fine-tuned Teacher (UFT), shifting the teacher role from ”knowledge transfer” to ”value discovery”. UFT is obtained by unconstrained fine-tuning on the target task and then compared with the source model to identify high-value samples. These samples are upweighted during student fine-tuning and receive value-weighted UFT-guided distillation, allowing the student to prioritize task-critical signals while preserving prior capabilities. Experiments on Magicoder and MetaMathQA show that UFT improves the trade-off between target-task performance and prior-capability retention, while converging faster and more stably. By turning teacher behavior into sample-value discovery, UFT establishes a new teacher role for data-efficient fine-tuning.

Abstract:
Reranking, the second stage of modern retrieval pipelines, is computationally expensive when performed by Large Language Models (LLMs). Existing efficiency-oriented approaches either rely on smaller models or restrict input length, both at the cost of effectiveness. Inspired by recent advances in soft document compression for retrieval-augmented generation (RAG), we introduce RRK, an efficient and effective listwise reranker that operates directly on multi-token compressed document representations rather than raw text. Trained via simple distillation from a strong teacher, RRK combines rich compressed representations with listwise reranking to deliver a highly efficient yet competitive system: despite its 7B-parameter backbone, it runs 5-20 faster than smaller rerankers (0.6-4B parameters) while matching or outperforming them in effectiveness. The advantage widens further on long-document benchmarks, where input compression yields the largest speedups.

Abstract:
Large Language Models excel at code generation, yet competitive programming exposes a persistent failure mode: existing multi-agent pipelines distribute work over generic planner, coder, and debugger roles and delegate the choice of algorithmic technique to the backbone alone. We present MARS (Multi-Agent Relay of Specialized LLMs), a prompt-only framework in which each agent is a topic specialist—dynamic programming, graphs, strings, geometry, and so on—grounded by retrieval-augmented generation over an algorithm-theory corpus. Given a problem, retrieval selects a small team of relevant specialists; a starter writes an initial C++17 solution, and each subsequent turn runs the candidate against public examples in a sandbox, lets the active specialist keep, repair, or hand off the draft, and forwards a structured packet to the next specialist. A single infrastructure-fixer pass normalizes boilerplate at the end. On the CodeContests test split with Gemma 4, MARS reaches 0.624 ± 0.006 pass rate at 2.3 relay steps per task, +13.9 pp over direct prompting, closing most of the gap to CodeSIM at 0.731 with 3.3× lower wall-clock cost and substantially smaller variance in per-task token spend.

Abstract:
Reliable medical diagnosis requires a precise final prediction, which is strictly supported by observable evidence. Recent large vision-language models perform well on medical image understanding tasks, yet they often lack explicit control over which evidence to inspect before making a prediction. To facilitate flexible planning of evidence exploration, we propose MedSPA, a self-planning medical reasoning framework that separates diagnostic evidence acquisition from evidence-grounded prediction with two cooperative agents: a planning-and-reasoning agent that adaptively decides what to inspect and when to stop, and a summary agent that generates the final output from the accumulated evidence. MedSPA distills medical knowledge from strict diagnostic procedures but acquires flexibility through reinforcement learning to generate optimal outputs. Experiments on medical report generation and medical visual question answering show that MedSPA improves clinical performance, faithfulness, and controllability over existing reasoning baselines.

Abstract:
Learning to simulate human users in interactive settings could advance the training of agents that serve humans, the evaluation of personalization systems, and research in the social sciences. Existing approaches generally do so by training a large language model (LLM) to match a single ground truth response, either by maximizing the log probability or by using a similarity reward. We instead propose a Turing Test approach for learning user simulator models, where a model is trained via reinforcement learning to produce responses indistinguishable from what the user could have said. Our discriminative Turing reward uses an LLM judge to score how indistinguishable a generated response is from the real user’s given the user’s history. Across two substantially different domains, namely conversational chat and Reddit forum discussion, we find that the Turing reward consistently outperforms baseline methods on both LLM and human evaluation metrics. Our study suggests that optimizing for indistinguishability, rather than response matching, is an effective route to learning user simulators.

Abstract:
Large language models increasingly rely on long chain-of-thought (CoT) trajectories for complex reasoning, but autoregressive generation brings substantial memory and inference costs. Latent reasoning models (LRMs) offer a more efficient alternative by compressing intermediate reasoning into a small number of continuous latent vectors. Despite their efficiency, however, the adversarial robustness of LRMs remains largely underexplored. In this work, we systematically evaluate the robustness of latent reasoning across textual and multimodal settings, covering eight models and six benchmarks. We find that, across our evaluated settings, LRMs are generally less robust than explicit CoT baselines under adversarial perturbations, with particularly severe degradation under white-box attacks. Further analysis reveals distinct failure modes across modalities: textual latent states exhibit brittle dynamics and high sensitivity to specific input patterns, while latent states in multimodal models can remain largely invariant to input perturbations and have limited influence on final predictions. These findings expose robustness limitations of current latent reasoning approaches and highlight the need to jointly consider efficiency and robustness when designing implicit reasoning systems. We have open-sourced our code to facilitate reproduction of our research https://github.com/PKU-ML/latent-reasoning-model-assessment.

Abstract:
Language models approach saturation on general-purpose code benchmarks but routinely fail on domain-specific languages (DSLs), where paired training data is scarce and regularities learned from mainstream languages transfer poorly. We propose augmenting each training example with a minimal grammar: the subset of DSL productions needed to derive the target program, typically 2–20% of the full grammar. This provides a strong structural prior, and we fine-tune the model to generate the program conditioned on the query and grammar. We predict grammars at inference via retrieval: a frontier model is prompted with (query, grammar) pairs, annotated with a chain-of-thought trace. Across five benchmarks spanning semantic parsing (SMCalFlow, GeoQuery, Overnight) and hardware description languages (HDLs) (Verilog, SPICE), our method improves over grammar-free fine-tuning on three open-weight backbones. Gains reach 36.8pp of program accuracy on SMCalFlow, with consistent improvements on HDLs. Applied to in-context learning, our predicted grammars raise GPT-5.5 accuracy on SMCalFlow from 51.4% (grammar prompting) to 94.3%, with similar gains for Claude Opus 4.7. Ablations rule out the extra training tokens as the source of these gains.

Abstract:
The In-context learning (ICL) paradigm aids large language models (LLMs) to adapt to new tasks without need for fine-tuning. However, selecting an optimal combination of demonstration examples from a large pool of example subsets is a challenging problem. Existing approaches for selection do not model the complex relationship between ICL samples and downstream LLM performance. They typically perform static task-level selection, choosing subsets once offline, which can fail to generalize to unseen queries. We introduce DearICL (Data Efficient Algorithm for Ranking) ICL samples, a new framework that models demonstration example selection as a subset ranking problem. DearICL employs a non-linear surrogate employing a differentiable sorting objective within a gap-index bandit algorithm. The gap-index based approach enables fine-grained separation of good arms and borderline arms, which is used as an auxiliary objective to train the non-linear surrogate through sufficient sampling of borderline arms, supporting instance-level subset ranking. On exemplar selection benchmarks with open-source LLMs, DearICL achieves 8.08–15.9% accuracy gains over strong linear bandit baselines, with low sample complexity. Code and data: https://github.com/VenkteshV/DearICL

Abstract:
Aligning large language models (LLMs) with multiple behavioral objectives from a certain corpus remains challenging, as joint optimization of attributes often leads to interference and trade-offs. While recent representation steering methods enable lightweight control over individual behaviors through hidden-state interventions, their composability for multi-attribute alignment remains underexplored. We present POLAR, Poly-attribute Alignment via Modular Representation Steering, a modular framework that composes independently trained alignment interventions within a frozen LLM. To mitigate conflicts between steering modules, we introduce calibrated and compatibility-aware intervention composition in hidden-state space. Experiments across heterogeneous alignment attributes show that POLAR achieves the best overall trade-off, with a 37.4% improvement in average rank over the strongest multi-attribute baseline while preserving response quality and controllability. The code and data will be publicly available.

Abstract:
Various studies have proposed methods to quantify and compare the rapidly growing number of large language models (LLMs). However, when constructing a macroscopic map of LLMs, there is a trade-off between estimation accuracy and computational cost, making the selection of evaluation texts critical. In this work, we propose Predictive Importance Sampling (PredIS), which selects informative texts by predicting likelihood variance from text-level features. Experiments on 100 LLMs across 25 domains demonstrate that our predictor generalizes robustly. Furthermore, we reveal that text importance cannot be explained by text length alone, as it is also closely tied to meaning reflected in the embedding space.

Abstract:
Long-context reasoning remains a critical challenge for Large Language Models (LLMs). Current research typically focuses on curating specialized training data by treating long-context reasoning as a monolithic task, yet it is challenging to verify its quality due to its internal cognitive complexity. In this paper, we propose a paradigm shift: viewing long-context reasoning as a hierarchical spectrum of core atomic skills. To operationalize this, we present an Anchor-based Reasoning (AbR) pipeline that automatically synthesizes strictly verifiable training data for each skill, thereby enabling reliable quality. Diagnostic analyses confirm that proficiency in these atomic skills strongly predicts general long-context reasoning performance. By training exclusively on a highly compact set of merely four thousand synthesized samples with Reinforcement Learning (RL), we explicitly sharpen these foundational capabilities. Extensive experiments across six challenging benchmarks demonstrate that this targeted RL causally delivers robust and generalizable performance gains, outperforming DeepSeek-R1-distill-32B by an average margin of 7.7% (improving from 46.3% to 54.0%). Furthermore, in-depth ablations reveal that these skills operate synergistically, establishing a scalable, data-efficient, and verifiable path to advancing long-context reasoning.

Abstract:
Reasoning language models (RLMs) increasingly tackle complex problems through long reasoning traces, but identifying when and where failures arise remains difficult. In this work, we study whether reasoning errors are signaled in model representations before they manifest as failed traces. We first show that step-level errors are locally predictable within a short window of neighboring steps, suggesting that models encode early warning signals. However, trace-level failures often result not from isolated errors, but from persistent ones originating earlier in the trace. To study this, we distinguish recoverable from unrecoverable traces via counterfactual resampling. We find that unrecoverable traces induce final-step representations that are clearly separable from those of recoverable traces. Moreover, the persistent error can be retrospectively decoded from the final-step representation, indicating that error signals propagate and accumulate through the reasoning trace.

Abstract:
Large Language Models have consistently demonstrated a lack of creativity and diversity across tasks. Prior work has focused on addressing whether models are capable of generating creative outputs. Here, we aim to consider novelty and investigate what makes model-generated content novel or not novel in a task-specific manner. We propose a fine-grained evaluation metric GENIE to measure the novelty of responses along task-specific features with respect to a population of responses. We show that unlike GENIE , holistic metrics struggle to capture the high-dimensionality of novelty and do not provide insight on which properties they target. Finally, we use GENIE to measure the effectiveness of mitigation methods that address creativity to better understand where these methods can improve novelty.

Abstract:
ICD-10 codes are alphanumeric codes used in the US to classify diagnoses and injuries for medical billing and epidemiological reporting. Standard ICD-10 benchmarks report aggregate metrics that obscure performance on complex coding scenarios. We evaluate neural, workflow, and agentic systems on a rarity-stratified set of MIMIC-IV discharge summaries and identify two orthogonal failure modes. Neural classifiers exhibit a 0.43 micro-F1 gap between rare and common codes. Workflow systems handle rare codes well but score near zero on injury and external cause codes that require multi-step guideline following. A tool-augmented agentic configuration with structured access to official ICD-10 reference materials recovers up to 0.32 micro-F1 on this subset. No single system dominates across all conditions.

Abstract:
Knowledge editing aims to update or correct factual knowledge in a language model. A widely used approach, locate-then-edit, first localizes a fact within the model and then edits the weights there. To date, such methods have been developed exclusively for autoregressive models (ARMs). Whether they work for masked diffusion models (MDMs), which model text bidirectionally and generate by iterative denoising rather than next-token prediction, remains an open question. We address it by transferring locate-then-edit to MDMs and comparing multiple MDMs with their matched ARMs. Our central finding has two parts. First, where an edit should be applied transfers between them: the same early-to-mid-layer MLP at the last subject token is most effective for both. Second, this shared location does not guarantee a shared outcome. Single-token edits succeed in both, but as targets grow longer, editing degrades far more sharply in the MDMs than in the ARMs. The failure stems from how the edited fact is generated: producing a multi-token target passes through intermediate states in which the target is partially unmasked, for which the edit was never optimized. Guided by this diagnosis, we introduce a simple correction that optimizes the edit including such states, substantially restoring multi-token performance. Our code is available at https://github.com/holi-lab/MDM-KE.

Abstract:
The widespread adoption of large language models (LLMs) has created an urgent need for accurate and robust detection of AI-generated text. However, existing statistical detectors often rely on a fixed decision boundary, which hinders their ability to generalize across topics, styles, and model variants. In this paper, we propose preference calibration with context-aware thresholding (PC-CAT), a novel framework that integrates preference calibration with context-aware thresholding to improve detection reliability. PC-CAT first applies Simple Preference Optimization (SimPO) to maximize the log-probability gap between paired human-written and AI-generated texts, enhancing discriminability. It then tailors instance-specific thresholds using a lightweight regressor trained with a hybrid centering and ordinal objective. Experiments across diverse domains and generators show that PC-CAT consistently outperforms strong baselines and achieves new state-of-the-art results. Moreover, it exhibits strong robustness under distribution shifts and adversarial attacks. These findings highlight the critical role of adaptive thresholding in advancing the robustness of AI-generated text detectors.

Abstract:
Coding agents now find real vulnerabilities in production software. However, bug discovery results do not measure whether agents can construct exploit primitives. We introduce KEX-bench, a benchmark for evaluating coding agents on exploit primitive generation against real operating-system kernels. KEX-bench contains 45 task instances across 40 Linux and Windows CVEs, covering kernel address leak, instruction-pointer control, heap read, heap write, and arbitrary address write. Each task runs in an isolated virtual machine, exposes controlled tools, and uses a deterministic verifier for checking primitive-specific success. We evaluate state-of-the-art coding agents paired with frontier and open-weight models under fixed tool-call budgets. Without a PoC, the strongest configuration solves 1 of 20 Windows tasks (5.0%) and 14 of 25 Linux tasks (56.0%). With a reference PoC, the strongest configuration solves 31 of 45 tasks (68.9%). This highlights the gap where agents reach kernel crashes but fail to shape kernel state into exploit primitives. We release KEX-bench for reproducible research on AI-assisted exploitation at https://kex-bench.github.io.

Abstract:
Deep research and agent evolution serve as de-facto tasks for AI agents in real-world applications toward artificial general intelligence. The former enables autonomous retrieval and integration of information in open-ended environments to tackle open-ended research tasks, yet it is constrained by the static parametric deep research capabilities of agent systems. The latter allows agents to autonomously interact with the environment to gain experiences that evolve model capabilities. However, its effectiveness has been widely validated only on verifiable tasks with standard answers, leaving a gap with open-ended research tasks. To bridge these two critical tasks, we propose the Hybrid Open-Ended Tri-Evolution (HOTE) framework, which leverages hybrid-mode reinforcement learning to facilitate the collaborative evolution of a proposer, solver and judge based on web-scale knowledge, moving toward autonomous evolving agents in open-ended tasks and environments. Extensive experiments on three long-form deep research benchmarks demonstrate that the 8B model trained via HOTE surpasses the strongest static open 8-32B models as well as those trained by state-of-the-art deep research training methods with less time overhead, and further verify that the evolution of all three modules in HOTE is indispensable.

Abstract:
Large language models (LLMs) exhibit uneven multilingual performance, especially when dealing with low-resource languages. Inference-time intervention offers a lightweight way to improve cross-lingual transfer by modifying the hidden states produced by the LLMs during the forward pass, without updating model parameters. However, existing cross-lingual intervention methods typically learn separate projections from source to target languages, which limits scalability and prevents knowledge sharing across languages. We propose Centroid Intervention Fusion (CIF), a projection fusion framework that consolidates multiple multilingual intervention projections into a single language-shared operator. CIF builds a stable language-shared intervention operator by estimating a robust centroid over multilingual projections and filtering outliers. Across multilingual commonsense reasoning, natural language inference, factual editing, and machine translation benchmarks, CIF outperforms the strongest prior pairwise intervention baseline by up to +3.378 pp on average across four model backbones. By preserving shared cross-lingual intervention representation, CIF further improves performance on low-resource languages. The code will be released upon acceptance.

Authors: Emmanouil Zaranis, António Farinhas, Saul Santos, Beatriz Canaverde, Miguel Moura Ramos, Wafaa Mohammed, Giuseppe Attanasio, Chrysoula Zerva, Nithin Sivakumaran, Shoubin Yu, Elena Bueno-Benito, Aditya K Surikuchi, Ben Peters, Danae Sanchez Villegas, Andre G. Viveiros, Pavlo Vasylenko, Baohao Liao, Sonal Sannigrahi, Jaehong Yoon, Elias Stengel-Eskin, Mariella Dimiccoli, Oswald Lanz, Alessandro Suglia, Mohit Bansal, Sandro Pezzelle, Stella Frank, Vlad Niculae, Desmond Elliott, Raffaella Bernardi, Raquel Fernández, Andre Martins

Abstract:
Vision-language models (VLMs) can increasingly handle extended contexts and answer questions about long videos—but can they understand the narrative structure of movies as humans do? To investigate this question, we introduce MF 2 ( Movie Facts and Fibs ), a fully manually curated benchmark designed to evaluate whether models can identify, recall, and comprehend information central to the movie’s storyline, using both the visual and textual modalities. MF 2 includes over 50 open-licensed movies, each paired with manually constructed sets of claim pairs: one true claim ( fact ) and one plausible but false ( fib ), totaling over 850 pairs. Our analysis across 20 open-source and 2 proprietary models reveals a substantial gap relative to human performance. While subtitles provide a strong signal for narrative understanding, other textual sources (e.g., synopses) are less informative. In contrast, when visual information is distributed across the entire movie and jointly embedded with subtitles, models struggle to identify and effectively use the relevant content. However, when the visual input is restricted to claim-relevant visual content, models improve significantly, indicating substantial room for improvement in processing large multimodal contexts and distinguishing relevant from irrelevant information. Overall, our findings highlight persistent challenges in multimodal narrative understanding for current VLMs. We release our data and code to support further research in this direction.

Abstract:
Transformer language models have become established tools for modeling human sentence processing, with measures such as surprisal and attention entropy serving as effective predictors of reading difficulty that together capture complementary aspects of processing load. Here, we explore a related class of transformer models: energy-based transformers, which provide a principled formal link to associative memory models, bringing processing research into direct contact with the broader literature on Hopfield networks and dense associative memory. To our knowledge, this is the first exploration of an energy-based transformer measure in computational psycholinguistics. Across reading-time corpora (Natural Stories, UCL eye-tracking, UCL self-paced reading), the energy measure is a robust predictor of reading times, providing significant fit beyond surprisal and entropy in all three. In a controlled experiment on relative clause processing, energy at a single layer captures the well-known object/subject asymmetry. We find evidence that it subsumes effects attributable to both attention entropy and surprisal, suggesting that energy may serve as a single unified predictor where multiple complementary measures have previously been required.

Abstract:
As large language models are increasingly deployed across diverse real-world applications, extending automated evaluation beyond English has become a critical challenge. Existing evaluation approaches are predominantly English-focused, and adapting them to other languages is hindered by the scarcity and cost of human-annotated judgments in most languages. We introduce a decomposition-based evaluation framework built around a Universal Criteria Set (UCS). UCS consists of a shared, language-agnostic set of evaluation dimensions, producing an interpretable intermediate representation that supports cross-lingual transfer with minimal supervision. Experiments on multiple faithfulness tasks across languages and model backbones demonstrate consistent improvements over strong baselines without requiring target-language annotations.

Abstract:
Vision-Language Models (VLMs) have been shown to be blind, often underutilizing visual inputs even on tasks that require visual reasoning. In this work, we demonstrate that VLMs are selectively blind. They modulate the attention to visual inputs based on linguistic framing even when alternative framings demand identical visual reasoning. Using visual attention as a probe, we quantify how framing alters both the amount and distribution of attention over the image. Constrained framings, such as multiple choice and yes/no, induce substantially lower attention to image context compared to open-ended, reduce focus on task-relevant regions, and shift attention towards uninformative tokens. We further demonstrate that this attention misallocation is a key mechanism contributing to degraded accuracy and cross-framing inconsistency. Building on this mechanism, we evaluate whether common prompting strategies mitigate framing-induced blindness. We find that simple image-focused instructions provide limited benefit, while structured reasoning and grounding prompts partially restore target-region attention and reduce, but do not eliminate, cross-framing inconsistency. Our findings suggest that question framing and prompting should be considered part of VLM evaluation to better reveal models’ visual capability.

Abstract:
Machine translation systems still make systematic errors on idioms and other expressions that cannot be translated literally. Model editing offers a possible way to fix such recurring errors without retraining the whole model, but it has mostly been studied for monolingual factual updates rather than translation. We introduce MEMT-bench , the first benchmark for evaluating model editing in machine translation across multiple languages, models, and editing settings. The benchmark is built from idiomatic translation errors and tests whether an edit can correct a targeted mistranslation while preserving unrelated translation behavior. Experiments with causal tracing and representative editing methods reveal both the promise and difficulty of translation editing: edits can repair targeted errors, but maintaining cross-lingual locality and robustness under repeated edits remains challenging. Code and data are publicly available at [GitHub](https://github.com/dannalily/MEMT-Model-Editing-for-Targeted-Translation-Error-Repair/) and [Hugging Face](https://huggingface.co/datasets/Dannalily/MEMT-bench), respectively.

Abstract:
Existing visual grounding benchmarks primarily evaluate alignment between image regions and literal referring expressions, where models can often succeed by matching a prominent named category. This leaves a complementary but under-examined setting: scenario-based visual grounding, where the target must be inferred from roles, intentions, and relational context rather than explicit object naming alone. We introduce Referring Scenario Comprehension (RSC), a benchmark designed to study this setting. RSC queries describe roles, goals, and contextual cues, and deliberately reference distractor objects to reduce lexical shortcutting. Each instance is annotated with interpretable difficulty tags for uniqueness, clutter, size, overlap, and position, which expose distinct failure modes and support fine-grained analysis. RSC contains approximately 31k training examples, 4k in-domain test examples, and a 3k out-of-distribution split with unseen object categories to test generalization beyond category co-occurrence. We further propose ScenGround, a curriculum reasoning method serving as a reference point for this setting, combining supervised warm-starting with difficulty-aware reinforcement learning. Experiments show that scenario-based queries expose systematic shortcut failures in current models that standard benchmarks do not reveal, and that curriculum training improves performance on challenging slices and transfers robust improvement to standard benchmarks.

Abstract:
Large language models (LLMs) are increasingly used as assistants for statistical and data science work, yet existing evaluations largely assume the analysis target is already specified. In practice, users arrive with informal goals and heterogeneous data, leaving the model to decide what statistical task is implied and which data are relevant. We first formalize this upstream step as Statistical Problem Formulation and decompose it into two subtasks: (1) Statistical Problem Classification and (2) Variable Identification Role Assignment. We then introduce StatFormBench, a benchmark built from five cross-domain statistics textbooks and a data science case library, covering diverse problem types, data representations, and scenario styles. It contains 1,013 samples spanning 20 coarse-grained and 85 fine-grained statistical problem categories. Across 14 open- and closed-source LLMs, the best zero-shot models reach only 72.0 fine-grained classification accuracy and 63.2 variable set overlap. No model performs consistently best across the two subtasks, while enhanced prompting strategies yield only limited or inconsistent gains. We release the benchmark data on Hugging Face at https://huggingface.co/datasets/THU-CongLab/StatFormBench and the evaluation code on GitHub at https://github.com/THU-CongLab/StatFormBench.

Abstract:
Large Language Models (LLMs) have achieved remarkable progress in code generation, yet ensuring correctness in complex, repository-level tasks remains challenging. Existing approaches often use generated tests as static post-hoc validators, which limits their ability to guide implementation and may introduce misleading feedback when the tests themselves are incomplete or incorrect. In this paper, we introduce TDD-Agent, which operationalizes the test-driven development paradigm for code generation. TDD-Agent first prompts the model to generate executable tests, encouraging it to clarify expected behaviors before implementation, and then performs iterative dual-track refinement over both the generated code and tests using execution feedback. We first isolate the effect of test-first reasoning through a prompt variant TDD-prompt on LiveCodeBench, where it consistently improves upon reasoning-based prompting baselines. Building on this finding, we evaluate the full TDD-Agent framework on RepoEval, a repository-level benchmark, and show that it consistently outperforms retrieval-based and agent-based baselines. Additional analyses show that iterative refinement improves not only code correctness but also the effectiveness of the generated tests, yielding higher pass rates, coverage, and mutation scores, suggesting that tests can serve as evolving reasoning artifacts rather than fixed validators. Our source code is available at https://github.com/Natt1e/TDD-Agent.

Abstract:
Multi-agent debate systems are used to competitively discuss and resolve complex tasks. However, existing approaches often exhibit pseudo-debate behavior: agents either ignore opponents’ arguments or compromise too readily without critical evaluation. To address this, we propose a three-stage debate framework grounded in abstract argumentation theory. First, Debate Structuring uses principled prompts to elicit well-formed debate interactions. Second, Debate Imitation filters high-quality debates to fine-tune LLMs on proper argumentative conduct. Third, Debate Internalization employs a debate-aware reinforcement learning objective to instill adversarial reasoning patterns. To better quantify debate capability, we introduce two novel metrics: Rebuttal Engagement Score (RES), assessing how thoroughly models engage with opposing arguments, and Belief Governance Score (BGS), evaluating whether models appropriately retain or revise beliefs based on argument strength. Extensive evaluations across six benchmarks show that our framework not only improves task performance but also cultivates more authentic and robust debate behavior in LLMs. Our work contributes a theory-inspired training framework and evaluation suite for advancing multi-agent debate systems.

Abstract:
Clustering is often a process of trial and error: we apply different algorithms (such as k-means or hdbscan), apply different embeddings, try a dimensionality reduction, explore different values of K, inspect the result with the product manager, and repeat. When the data is large, even communicating and analyzing a single clustering becomes difficult. We argue that the nature of clustering requires a fundamentally different approach. We posit that clustering is fundamentally a subjective and iterative process, and its central challenges are how to elicit the dimensions and nuances a user cares about, how to communicate a clustering effectively, and how to converge to an output of interest through efficient conversation. We reframe clustering as a conversational supervision problem. We formalize conversational clustering as a new task, derive evaluation metrics suited to it, and release an evaluation framework and reference implementation to seed further work.

Abstract:
Large language model (LLM) agents increasingly collaborate over peer-to-peer networks to improve their reliability. However, these same interactions can also introduce vulnerability to unreliable or Byzantine agents that can propagate incorrect information and degrade overall system performance. To address this, we propose Self-Anchored Consensus (SAC), a fully decentralized filter-and-refine protocol in which agents iteratively exchange responses, locally evaluate and filter unreliable messages, and refine their own outputs. We present (F+1) -robustness conditions on the communication graph that ensure honest agents preserve and propagate reliable information despite Byzantine influence. Experiments across diverse open- and closed-weight LLMs on mathematical and commonsense reasoning benchmarks show that SAC effectively suppresses Byzantine influence and consistently improves performance across diverse communication topologies, whereas prior methods degrade significantly under Byzantine attacks.

Abstract:
Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a session and become substantially safer after a few regular agentic tasks—a phenomenon we term the cold-start safety gap. To study this systematically, we introduce Safety Over Depth for Agents (SODA), a benchmark that controls how many regular agentic tasks the agent completes before encountering a safety threat, supporting up to 20 preceding tasks. Evaluating 7 models from 4 families, safety improves by 9–52% as the number of preceding regular agentic tasks increases from zero to twenty. Representation analysis confirms that model hidden states gradually shift toward a safety-aligned region as more preceding tasks are present. By systematically studying which part of the preceding conversation matters most, we find that the regular agentic tasks themselves are the primary driver of safety, while the agent’s own prior responses have less effect on safety but are essential for preserving later utility. This conclusion is further supported by evaluation on open-source safety benchmarks (AgentHarm, Agent Safety Bench) and utility benchmarks (BFCL, API-Bank), confirming that warming up the agent with regular agentic tasks before deployment makes it safer and preserves full capability. Based on these findings, we recommend a simple deployment strategy: having the agent complete a few regular agentic tasks before possible exposure to safety-critical requests mitigates the cold-start safety gap.

Abstract:
Large language models are known to contain representational redundancy across network depth, making depth pruning an effective approach for improving inference efficiency. Existing one-shot pruning methods rely on local layer importance or fixed redundancy assumptions across architectures. We propose Locality-Aware Redundancy Pruning (LoRP), a training-free one-shot depth pruning framework guided by representation locality. We show that inter-layer redundancy can be either localized or globally distributed depending on the LLM architecture. To characterize this phenomenon, we introduce Representation Locality Score (RLS), derived from global inter-layer hidden-state similarity. Using a small calibration set, LoRP computes pairwise layer similarity, clusters layers by representational similarity, and allocates pruning according to residual intra-cluster redundancy. Experiments across diverse LLM families show improvements in both perplexity and downstream task accuracy.

Abstract:
Visual entity tracking is an innate cognitive ability in humans, yet it remains a critical bottleneck for Vision-Language Models (VLMs). This deficit is often obscured by visual shortcuts in existing video benchmarks. We introduce VET-Bench, a synthetic diagnostic testbed featuring visually identical objects that necessitate tracking exclusively through spatiotemporal continuity. Our experiments reveal that current state-of-the-art VLMs perform at or near chance level on VET-Bench. We provide a theoretical analysis drawing connections to the state-tracking problem, suggesting that fixed-depth transformer-based VLMs are fundamentally limited in tracking indistinguishable objects without intermediate computation due to expressivity constraints. To address this, we propose Spatiotemporal Grounded Chain-of-Thought (SGCoT): generating object trajectories as explicit intermediate states. Leveraging Molmo2’s object-tracking capability, we elicit SGCoT reasoning by fine-tuning on synthesized text-only data. Our method achieves state-of-the-art accuracy exceeding 90% on VET-Bench, demonstrating that VLMs can reliably solve the video shell-game task end-to-end without external tools. Our code and data are available at https://vetbench.github.io.

Abstract:
Selecting a parallelism strategy—the configuration of data, tensor, and pipeline parallelism degrees together with micro- and global-batch sizes—largely determines the training efficiency of large language models. State-of-the-art methods search for a parallelism strategy offline and select the single strategy that minimizes per-iteration time. But we find that they neglect the target validation perplexity and time-to-perplexity (TTP). In particular, our analysis reveals that the best strategy yielding the fastest perplexity improvement changes multiple times during training. As a result, state-of-the-art methods are 1.8–11.4× slower in TTP than the strategy sequence that selects the best strategy at each iteration. This paper proposes CONA, a new training method that introduces online strategy chaining. Instead of a single strategy selected offline, CONA ranks candidate strategies during training using a surrogate metric built from compute throughput and gradient statistics, and switches the current strategy to a new strategy with a higher metric. In our evaluation with GPT-3 1.3B, BERT-Large, and Llama-3.2-1B, CONA reaches the target validation perplexity 1.4–9.6× faster than state-of-the-art methods. Moreover, CONA closely tracks the perplexity achieved by the sequence that selects the best strategy at each iteration, within 2.6%.

Abstract:
LLM agents have rapidly evolved into autonomous systems, yet a persistent information gap remains between users and agents: communication is costly, while users’ identical preferences further limit information exchange. To investigate how agents should communicate across modalities, this paper formalizes Communication Policy, establishes textual and UI-based policies, and then evaluates communication policies across diverse environments, personas, and model combinations. Building on information asymmetry for proactive agents, we set up two complementary settings, User–Agent and Planner–Executor. Experimental results reveal complementary strengths between interaction channels: text-based interaction often facilitates task performance, while structured UI improves agents’ response quality and persona compliance. Motivated by that, a hybrid method that combines these advantages, we further propose Communication Policy Evolution (CPE), a self-evolution framework for refining communication policies through rollout and prompt-level evolution. Without model modification, CPE achieves the best task success across multiple settings using prompt refinement alone. Our findings identify communication behavior as a critical yet underexplored design dimension for LLM agents.

Abstract:
Cost-effective deployment of large language models (LLMs) often relies on routing each query to either a low-cost local model or a stronger expert model. Existing routers typically decide either before generation, which is cheap but cannot observe the local model’s answer, or after generation, which provides a stronger correctness signal but incurs additional generation cost. We argue that this dichotomy overlooks a third option: treating local generation itself as an optional information-acquisition action. To this end, we propose Generation-Gated Cascaded Routing for Large Language Models (GenGate), a lightweight fine-tuning method that trains a local LLM to internalize cascaded routing decisions while preserving its generation capability. A Bellman-motivated prior gate first decides whether to generate a local attempt. If an attempt is produced, a correctness-supervised posterior gate then decides whether to submit it or defer to the expert model. Both decisions are expressed through action-token probabilities in the same model, requiring no external router or auxiliary classifier at inference time. Compared with routers that decide strictly before or after local generation, GenGate reduces joint deployment cost while maintaining competitive routing accuracy across task domains, preserving the benefits of posterior routing without requiring local generation for every query.

Abstract:
Self-attention selects information freely across the sequence, but across depth Transformers merely add each layer’s output to the residual stream, so later layers cannot selectively reuse earlier-layer representations. Recent cross-layer methods improve this flow but operate on hidden states outside attention, adding state beyond the key-value cache at inference—a cost that becomes increasingly salient as modern LLMs compress the cache with grouped-query and multi-head latent attention. We introduce Depth-Attention, which performs this selection inside the attention module itself: before a layer attends over the sequence, its query attends along depth over the keys of earlier layers at the same position and mixes their values into the value that self-attention then reads. Because Depth-Attention reuses the standard attention queries, keys, and value-cache slots, storing depth-mixed values in place of the original values, it adds no parameters and introduces no persistent inference state beyond the standard key-value cache—the same cache size as a vanilla decoder and less than hidden-state-based cross-layer methods. On Qwen3-style decoders at 1.5B and 3B parameters, Depth-Attention attains the lowest perplexity and the highest average downstream accuracy, improving over the vanilla Transformer by up to 2.3 accuracy points and surpassing strong cross-layer baselines (mHC, Attention Residuals, DenseFormer) in perplexity and average accuracy, while adding under 0.01% extra arithmetic FLOPs and no additional persistent inference state. The gains hold from 360M to 3B parameters and extend to looped Transformers.

Abstract:
Surprisal theory holds that the human processing difficulty of a linguistic unit is an affine function of its surprisal in context under some language model. I argue this claim is tautological, i.e., for any non-negative difficulty measure over units in context, there exists a language model whose surprisal is an affine function of it under mild technical conditions. Therefore, because any pattern of difficulty is consistent with some language model, without an additional constraint on the language model itself, surprisal theory makes no falsifiable predictions. I further claim that the tautology was obscured by an assumption implicit in two decades of psycholinguistic work – that the language model relevant for studying human sentence processing is the distribution that generated the training corpus. Recent empirical work, however, has undermined this assumption, demonstrating that better corpus models can be worse predictors of processing difficulty. I conclude that breaking the tautology requires a rationalist intervention, i.e., the language models used for the study of human sentence processing must be derived from an independently motivated model of the comprehender, which could be based on, for instance, memory constraints or processing goals, and that, crucially, does not depend on the behavioral data surprisal theory is meant to explain.

Abstract:
Text-to-chart retrieval enables users to locate relevant charts through natural language queries and has emerged as a critical capability in modern business intelligence systems. However, current benchmarks fail to simulate realistic user queries or test deep semantic understanding, leaving a significant evaluation gap. To address this, we introduce CRBench, the first real-world BI-sourced benchmark comprising 21,862 charts and 326 queries, which adopts a Target-and-Distractor paradigm to evaluate discriminative retrieval among highly similar candidates. Benchmarking reveals that existing methods, which rely primarily on visual features, fail to capture the rich analytical semantics of charts. We therefore propose a semantic insights synthesis pipeline that automatically generates three hierarchical levels of insights for alignment training: visual patterns, statistical properties, and practical applications. Using this pipeline, we produce 207,498 insights for 69,166 charts as training data. Leveraging this data, we develop ChartFinder, which significantly outperforms state-of-the-art methods on CRBench, achieving 66.9% NDCG@10 for precise queries (an 11.58% improvement) and an average gain of 5% across fuzzy query metrics.

Abstract:
Conversational assistants powered by large language models (LLMs) excel at tool-use tasks but struggle with adhering to complex, business-specific rules. While models can reason over business rules provided in context, including all policies for every query introduces high latency and wastes compute. Furthermore, these lengthy prompts lead to long contexts, harming overall performance due to the ’needle-in-a-haystack’ problem. To address these challenges, we propose a multi-stage alignment method that teaches models to recall and apply relevant business policies during chain-of-thought reasoning at inference time, without including the full business policy in-context. Furthermore, we introduce a novel Policy Recall reward based on the Jaccard score and a Hallucination Penalty for GRPO training. Altogether, our best model outperforms the baseline by 16 points and surpasses comparable in-context baselines of similar model size by 3 points, while using 40% fewer words.

Abstract:
Maintaining persona consistency across multi-turn dialogues remains a core challenge for role-playing language models. Off-policy distillation from external teachers incurs distribution mismatch that compounds across dialogue turns, while reinforcement learning struggles with reward ambiguity inherent in subjective persona fidelity. We propose OSPD, an on-policy self-distillation framework where the same model serves as both teacher and student under asymmetric information: the teacher receives a complete character profile while the student sees only a brief summary, and the student generates trajectories from its own policy. We find that teacher confidence in role-playing dialogue exhibits a bimodal structure—sharply peaked at character-critical tokens yet diffuse at generic utterances—and introduce role-aware divergence switching to match this structure. A progressive trait masking curriculum further forces staged internalization of character knowledge along semantic dimensions. Experiments on CharacterBench, CharacterEval, and SocialBench show that OSPD substantially improves persona consistency over supervised fine-tuning and multi-turn RL baselines, without requiring any external teacher or reward model.

Abstract:
Multi-agent debate improves the reliability of large language models (LLMs) through iterative peer critiques. However, fixed topologies often introduce persistent positional biases, amplify unreliable agents, and cause high sensitivity to role assignments. We introduce Permutation-Equivariant Adaptive Routing Multi-Agent Debate (PEAR) , an inference-time protocol that dynamically reconfigures communication roles and sparse topologies across consecutive debate rounds. By strategically switching agent-to-role assignments based on evolving agent states, PEAR prevents any agent from permanently occupying a privileged network position or distributes influence more evenly across the debate. We theoretically characterize PEAR as an equivariant sparse router: it preserves accuracy under agent relabeling while reducing routing complexity and improving generalization. Comprehensive empirical evaluations across four reasoning benchmarks and six diverse LLM backbones demonstrate PEAR significantly improves average accuracy over the strongest debate baselines. The code is at https://github.com/EVIEHub/PEAR .

Abstract:
Large language models (LLMs) are increasingly employed in writing assistance for online text, making AI-generated text (AIGT) detection a critical challenge for managing text provenance. However, it remains unclear whether the high detection performance reported in existing AIGT benchmarks reflects a detector’s ability to capture essential differences between human-written and AI-generated text. Rather, this high performance might be driven by easily measurable shortcuts in benchmark settings. To address this question, we decomposed the detection signals in AIGT and systematically analyzed the role of shortcuts in existing benchmark settings. Our analysis reveals that a simple logistic regression classifier trained solely on surface shortcut features (e.g., formatting style and lexical usage) achieves strong classification performance without relying on any semantic signals. Furthermore, the consistent performance degradation of existing detectors when such shortcuts are removed suggests that current AIGT evaluations are influenced by shortcut signals. Motivated by these findings, we present AIGT-SignalBench, a shortcut-controlled benchmark designed to reflect realistic AI-assisted writing scenarios.

Abstract:
Agentic RAG, which decomposes a complex question into dependency-aware sub-questions, has become the dominant paradigm for multi-hop QA. Yet existing systems can only repair errors at the failed step or simply return to the previous step and retry, and thus fail to handle two pervasive failure modes: inaccurate decomposition, where the plan is flawed before execution begins, and error propagation, where an upstream mistake surfaces downstream. We propose that addressing both requires global self-correction, the ability to identify the root cause of a failure and repair it at the right location. To this end, we propose LoCoRAG, a globally self-correcting agentic RAG framework. LoCoRAG maintains a persistent dual memory through a two-stage thinker-generator pipeline, separating what the corpus contains from what the system has concluded. Then, a memory-grounded failure localization module attributes each failure to specific causes and pinpoints its true origin. Together, LoCoRAG drives a self-correcting global replan that selects a matched repair, from local refinement to upstream rollback with downstream regeneration. Across existing multi-hop QA benchmarks, LoCoRAG achieves SOTA performance on all datasets and all metrics, with a 10.8 points average LLM-Acc gain over the strongest baseline on 3-hop-or-deeper scenarios.

Abstract:
Online alignment of large language models (LLMs) is dominated by reinforcement learning from human feedback (RLHF) with gradient-based optimizers such as PPO or GRPO. While effective, these pipelines require backpropagation through long rollouts, gradient synchronization across devices, and careful hyperparameter tuning, all of which become increasingly costly at scale. We present ESSA (Evolutionary Strategies for Scalable Alignment), a gradient-free online alignment stage that follows supervised fine-tuning (SFT) and replaces the gradient loop with inference-only black-box search. ESSA optimizes only the singular values of low-rank adaptation (LoRA) factors after a short SFT warm-start, restricting the search to a compact, task-aligned subspace where evolutionary search is practical even for 72B-parameter models. Because the loop is inference-only, ESSA is compatible with INT4/INT8 weight quantization and reduces inter-GPU communication to a few bytes per iteration. Across instruction following (IFEval), preference-based assistant tuning (HelpSteer2, HH-RLHF), and mathematical reasoning (GSM8K, MATH500), ESSA matches or exceeds LoRA-GRPO in the reported LoRA comparisons; on GSM8K it also outperforms Online DPO and PPO, while remaining competitive with both methods on IFEval. At scale, ESSA reaches a fixed MATH500 accuracy on Qwen2.5-32B/PRM800K up to 7.8× faster than LoRA-GRPO on 128 GPUs.

Abstract:
Emotional appeals (EAs) are often treated as fallacious in computational argumentation, yet they may be justified when warranted by the context. We introduce the task of predicting EA justification and present the first human-annotated dataset for this task, covering 1,176 argument–context pairs. Grounded in philosophical literature, our framework evaluates EA justification along logical and emotional dimensions. Human annotations show that justification criteria form a shared latent judgment space, with evidential proportionality and normative acceptability most strongly associated with overall justification. We further find that EAs without logical support are not always judged fully unjustified, and stronger EAs are not necessarily judged less justified. Finally, LLM experiments show that EA justification prediction remains challenging.

Abstract:
In this paper, we introduce layer-wise curriculum learning aimed at efficient LLM compression. The proposed method facilitates the transfer of knowledge from the teacher to student model, utilizing a curriculum learning approach that begins with easier optimization tasks and progressively tackles much harder ones. In order to adopt the layer-wise learning in LLM compression, we partition the whole model into multiple segments consisting of layers, thereby enabling more computationally efficient knowledge transfer of LLMs. Based on the theoretical evidence of cumulative error phenomenon, layer-wise curriculum learning accelerates convergence while stabilizing the knowledge transfer process. In addition, we present a feature caching method with a multi-threading strategy to efficiently address feature misalignment across layers, maximizing GPU utilization. Consequently, our method exhibits advanced model compression performance, as well as high computational efficiency in terms of minimized memory usage and short training hours. Experiments on multiple datasets show that the proposed method achieves state-of-the-art performance while reducing GPU memory usage and training hours by more than 50% on BERT and GPT-2. Moreover, it outperforms the other pruning methods on LLaMA-family and Qwen under the same training hours, with a lower GPU memory footprint. Upon acceptance, we will publicly release the code to support the reproducibility of our results.

Abstract:
The knowledge encoded in large language models (LLMs) can serve as a substrate for structured reasoning over variables describing a complex world, but accessing this knowledge in a probabilistically coherent manner poses a difficult inference problem. We propose Large Language Gibbs, a scheme for structured probabilistic inference that uses conditional distributions of an LLM as transition operators. Rather than sampling structured objects through single-pass autoregressive generation, we iteratively resample individual variables conditioned on others using an LLM’s next-token conditionals. This approach avoids order-dependent biases and produces a stationary distribution that reflects a compromise between all local conditionals. We apply this approach to sampling from synthetic distributions, consistent reasoning tasks, and Bayesian structure learning. The results suggest that the use of LLM conditionals in MCMC is a practical alternative to one-pass generation for structured probabilistic inference under a world prior accessible through noisy LLM conditionals.

Abstract:
Knowledge-intensive reasoning requires Large Language Models (LLMs) to ground answers in provided evidence. When evidence is insufficient, it is desirable that models abstain rather than confidently generating unsupported answers. Existing abstention methods rely on uncertainty estimation or evidence sufficiency checks, but neither tests whether the reasoning process for generation, driven by the interaction of provided evidence and the model’s internal memory parameters, is actually grounded in the evidence. A key contributing factor is that entity mentions in context activate memorised associations, causing models to generate plausible responses ungrounded in evidence. We propose Twin Worlds (TW), a framework for improving reliability in knowledge-intensive reasoning through equivariance-based abstention: unlike invariance, which requires outputs to remain unchanged, equivariance requires outputs to transform correspondingly under entity substitutions. A model grounded in the evidence should produce answers that shift consistently when entities are substituted while their relations are preserved. TW constructs multiple worlds via typed substitutions of the original input that preserve relational structure while reducing parametric priors, and uses equivariance violations as an abstention signal. Across four benchmarks and three model backbones, TW identifies when answers are not reliably grounded in the provided evidence and outperforms uncertainty- and sufficiency-based baselines.

Abstract:
Sentence embeddings represent meaning as a single fixed-size vector, regardless of the complexity of the input sentence. This design compresses all propositional structure into a single point, losing part of the internal structure of the sentence in the process. To address this, we introduce Atomic Proposition Embeddings (APE) to dynamically discover and encode atomic propositions in sentences as variable-size bags of vectors. We use a slot mechanism, inspired by object detection in computer vision, over an encoder backbone. APE is trained on a proposition-decomposed version of WikiSplit, and matches or outperforms sentence encoders up to 15x its size on PAWS-Wiki and SNLI with no task-specific fine-tuning, using proposition-level scoring rules at inference time. We further introduce PropSearch, a retrieval benchmark for proposition-level embeddings in which models must retrieve source sentences given atomic propositions. APE leads on PropSearch over proposition-level alternatives, with its advantage growing with sentence complexity. Ablations confirm the gains are architectural rather than data-driven and that APE can generalize over its training signal. We release our code, data and models for research purposes at https://github.com/SapienzaNLP/APE.

Abstract:
Full-duplex spoken dialogue models can listen and speak simultaneously, making them a promising architecture for natural conversation. However, current models are trained solely with supervised learning through token-level likelihood maximization, which does not directly optimize interaction-level behaviors, causing interactivity issues such as excessive silence and ill-timed turn-taking. Recent work has applied reinforcement learning (RL) to improve interactivity, but existing methods address only a limited set of interactive behaviors in their rewards. In this work, we propose a post-training alignment method that comprehensively improves the interactivity of full-duplex spoken dialogue models through RL. We address the four canonical axes of interactivity: pause handling, turn-taking, backchanneling, and user interruption. For each axis, we extract short audio segments from human conversation corpora and optimize the model with axis-specific reward functions. An extra LLM-based reward for response quality prevents semantic degradation. We apply our method to two open-source models, Moshi and PersonaPlex, demonstrating consistent improvements in interactivity on both offline evaluation with pre-recorded audio and real-time multi-turn dialogue evaluation.

Abstract:
Image embeddings are an essential commodity in modern AI: stored at scale in vector databases, transferred between edge devices and cloud APIs, and traded as paid services. Image embeddings have been implicitly treated as anonymous—an assumption that has not been rigorously tested even as embedding services scale to billions of stored vectors. _We show this assumption breaks._ We propose Cross-modal Recovery Attack with Few-shot Transfer (CRAFT), a framework that recovers semantic meaning, including free-form captions and sensitive attributes. The attack focuses on the medical domain and facial features, targeting black-box image embeddings without reconstructing a single image in pixels and without any knowledge of the underlying model architecture, training data, or weights. CRAFT combines a public dual-encoder, a linear alignment learned from intercepted image-embedding pairs, and a fine-tuned embedding-to-text decoder; we further strengthen recovery with two augmentative techniques: cross-modal reranking against the attacker’s text encoder and chain-of-thought prompting on a diffusion-rendered low-fidelity reconstruction of the leaked embedding. Attacks succeed with as few as a single intercepted pair and transfer across five open-source and proprietary embedders. We disclose a privacy surface that current defenses cannot close. Every setting that preserves downstream utility also leaves the attack effective. Black-box image embeddings, as widely deployed in AI systems today, should not be treated as privacy-preserving artifacts.[We open source our code for reproducibility and further research: .]

Abstract:
Retrieval-augmented QA can fail even when the correct answer is in the retrieved evidence. This can happen when a competing answer satisfies some, but not all, of the question’s requirements and is mentioned frequently in the retrieved documents, making it look like a strong choice. To test whether this retrieval skew actually affects the decision, we manipulate it directly: increasing the prevalence of passages that favor the competing answer makes models more likely to select it. This points to a decision problem that candidate-level support alone cannot resolve. We therefore introduce Joint Constraint Aggregation (JCA), which compares candidates against the same question-derived requirements using the retrieved evidence. Across our evaluation settings, JCA improves overall accuracy, and its advantage grows when retrieval favors a competing answer. Together, these results suggest that when partial matches and retrieval prominence pull models toward a competing answer, comparing candidates against the question’s full requirements can make selection more reliable.

Abstract:
Scientific retrieval-augmented generation (RAG) systems typically return one fluent answer even when the underlying literature contains multiple well-supported but incompatible views. We call this failure mode epistemic collapse: compression of structured disagreement into a single response. We present EVIRAG (Evidence-centric, Viewpoint-Inclusive RAG), a framework for scientific question answering that combines adversarial retrieval, claim-level disagreement graphs, causal disagreement attribution, multi-view synthesis, and evidence structured confidence labels. We also introduce EVIRAG-BENCH, a 1,250-query benchmark spanning five contested scientific domains. Across eight compared systems, EVIRAG substantially outperforms vanilla RAG on ContradictionRecall and dense-encoder ViewpointCoverage while reducing Consensus Collapse Score. Human evaluation with three annotators confirms markedly higher epistemic completeness for EVIRAG than for vanilla RAG (Krippendorff’s α=0.72, p<0.001, Cohen’s d=2.71). An offline precomputed graph reduces warm-query latency by over an order of magnitude. These results suggest that treating disagreement preservation as a primary design objective substantially improves how scientific RAG systems represent contested evidence in disagreement-heavy QA settings.

Abstract:
Agentic reinforcement learning (RL) trains large language models to use tools, but its impact on alignment is poorly understood. We study how agentic RL for search affects the alignment of instruction-tuned (IT) models. We find that RL-trained models inherit refusal reasoning by deflecting harmful requests into benign search queries, but this breaks down under a simple diagnostic trigger that elicits a search call before refusal can occur. Under this condition, RL models produce multi-step unsafe search actions and reasoning, reducing search query safety by up to 68.6% in Qwen models and 69.6% in Llama models relative to their IT counterparts. The effect generalises across model families, scales, and RL algorithms. To understand why, we identify linear directions in the residual stream that control search query safety, and show that RL training progressively shifts search behaviour toward the harmful end of this direction. We thus propose representation-guided RL training, which adds a reward penalty based on projection toward the harmful search direction. Training on benign data alone, it restores IT-level alignment within 200 steps without reducing task accuracy and requires no additional training data. Together, our work provides the first framework for diagnosing, mechanistically analysing, and mitigating alignment degradation in agentic RL for search.

Abstract:
Despite impressive breadth, LLMs still rely on explicit reasoning instructions or static, one-fits-all steering methods, leaving a gap for adaptive, instruction-free reasoning amplification. We present Prototype-Based Dynamic Steering (PDS), a test-time method that amplifies large language model (LLM) reasoning without adding or altering instructions. We introduce “reasoning prototypes” by clustering activation differences between Chain-of-Thought (CoT) and neutral prompts. At inference, an input’s hidden state is projected onto these prototypes to form an instance-specific steering vector. Evaluated on GSM8K, AQuA-RAT, and BIG-Bench tasks, PDS consistently improves accuracy without fine-tuning or prompt engineering. Notably, the gains persist even when CoT is explicitly suppressed to improve cost-efficiency, indicating that the intervention strengthens latent reasoning processes rather than inducing a superficial behavioral shift. These results position dynamic, prototype-guided steering as a lightweight alternative to training-time approaches for enhancing LLM reasoning.

Abstract:
Reinforcement learning from unlabeled prompts offers a promising path to improving LLM reasoning without curated supervision or verifiable rewards. However, existing label-free methods typically rely on either intrinsic self-certainty or extrinsic answer agreement, both of which can become unreliable in hard reasoning regimes: confident trajectories can be wrong, while majority answers can drift toward stable yet incorrect attractors. We propose CCAE, a verifier-free and parameter-free advantage estimator that formulates label-free GRPO as a multi-signal advantage estimation problem. CCAE decomposes the advantage into a base self-certainty term, an intrinsic evidence factor, and an extrinsic agreement factor, with their interaction adaptively calibrated by a group-level proxy for agreement reliability, without introducing any reward model, verifier, or trainable parameters. On MATH-Train, CCAE improves over the strongest unlabeled baseline by 5.0 and 4.3 points on Qwen3-4B/8B-Base, respectively, matching supervised GRPO. On the harder DAPO-14k-MATH, CCAE remains stable while prior unlabeled methods collapse, outperforming their best pre-collapse checkpoints by up to 11.1 points. Results on Natural Reasoning further suggest that CCAE remains effective beyond math-only training. Code is available at: [https://github.com/tifing/CCAE](https://github.com/tifing/CCAE).

Abstract:
Maintaining the safety of large language models (LLMs) is crucial as they are increasingly deployed in real-world applications. Existing safety guardrails typically rely on single-pass classification or, more recently, distilled reasoning. Reasoning-based guardrails significantly outperform classification-only baselines, but they incur substantial query latency and token overhead that make them impractical for high-throughput deployment. To address this challenge, we propose CoLaGuard, a guardrail model that transfers multi-step safety reasoning into a continuous latent space through a stage-wise training curriculum, enabling direct hidden-state propagation at inference. Evaluated across ten prompt and response moderation settings from eight unique safety benchmarks on ten prompt- and response-moderation settings spanning eight safety benchmarks, CoLaGuard improves macro-F1 by 8.24 points over Llama Guard 3 and matches our explicit reasoning baseline, GuardReasoner, in macro-F1 while delivering a 12.9 × speedup and 22.4 × reduction in token usage. Our results suggest that latent reasoning offers a practical alternative to explicit rationale generation for deployable guardrails, jointly improving safety robustness and inference efficiency rather than treating them as competing objectives.

Abstract:
Slogans are concise and memorable catchphrases that play crucial roles in advertising by conveying brand identity and shaping public perception. However, advertising fatigue reduces the effectiveness of repeated slogans, creating a growing demand for novel, creative, and insightful slogan generation. While recent work leverages large language models (LLMs) for this task, existing approaches often generate stylistically repetitive outputs that fall short in distinctiveness and insight. We argue that effective slogans should balance novelty with familiarity and propose a new paradigm that recontextualizes persona-related famous quotes for slogan generation. Well-known quotes naturally align with slogan-length text, employ rich rhetorical devices, and offer depth and insight, making them a powerful resource for creative generation. Technically, we introduce a framework that decomposes slogan generation into interpretable subtasks with a single-pass prompt, including quote matching, structural decomposition, vocabulary replacement, and remix generation. Extensive automatic and human evaluations demonstrate marginal improvements in diversity, emotional impact, and first-glance human preference over three state-of-the-art LLM baselines.

Abstract:
Selecting the best response from multiple small-model samples using a stronger scorer is a simple inference-time strategy, but fails in practice because the small model has already committed to diverging reasoning paths that no scorer can correct; PRM guided search avoids this by scoring candidate continuations during generation, but requires a reward model trained with step-level labels. We propose Chunk-Level Guided Generation, a training-free alternative that uses an off-the-shelf large language model as a process scorer: a small model samples k fixed-length candidate chunks at each step, while the larger language model scores the candidates using likelihoods, without generating any text. The selected chunk is committed before the next step, steering generation before errors can propagate. We instantiate this framework with two selection rules: Likelihood-Guided Selection (LGS), which selects the chunk with the highest length-normalized large-model log-probability, and Contrastive-Guided Selection (CGS), which subtracts the small model’s log-probability to favor chunks where the large model’s preference diverges from the small model’s. We show that scoring variable-length reasoning steps with large-model likelihoods is unreliable due to a systematic length bias that persists even after length normalization, and that fixed-length chunks avoid this confound. On GSM8K, MATH, Minerva Math, AMC23, and AIME24 with Qwen2.5-1.5B guided by Qwen2.5-32B and Llama-3.2-1B guided by Llama-3.1-70B, CGS outperforms majority voting by up to 28pp (e.g., 83.9% vs. 56.0% on GSM8K with the Llama pair at k=32 ) and, under matched guidance budgets, matches or outperforms Qwen2.5-Math-PRM-72B guided search on most benchmarks without any reward-model training (e.g., 39.0% vs. 32.4% on Minerva Math with the Qwen pair at k=32 ). With Qwen2.5-7B guided by Qwen2.5-72B, CGS reaches 81.8% on MATH and 63.6% on Minerva Math at k=16 , surpassing majority voting by 4–6 pp. Finally, we show that Chunk-Level Guided Generation produces substantially shorter reasoning traces than PRM guided search, with PRM responses up to 80% longer on GSM8K.

Abstract:
In common-context question answering (CCQA) tasks, multiple input questions share a common context to base their answers from. However, Large Language Models typically generate each answer using an independent prompt. While existing batching and caching techniques help improve parallelism and reduce repeated computations, the separation of questions across prompts limits the achievable speedup, as modern GPUs are underutilized due to a memory bottleneck during attention. We present Intra-Prompt Parallel Decoding (IPPD), a novel inference method that answers multiple common-context questions in parallel within a single prompt. IPPD directly addresses the bottleneck by efficiently sharing both memory and computation during the attention process, as the next token for every question is decoded in a single inference step. IPPD uses virtual position IDs and attention mask manipulation to generate the same output as standard prompting without requiring fine-tuning or any changes to the LLM architecture. Since all parallelism occurs within a prompt, IPPD is fully compatible with batched inference, even when each prompt features a different context. Our experiments show that IPPD delivers up to 7X the effective throughput as standard decoding without quality degradation, and outperforms prefix caching with PagedAttention in most settings.

Abstract:
Large Vision-Language Models produce fluent image descriptions but offer limited semantic control: users cannot reliably specify whether captions should emphasize attributes, relations, or particular image regions. We present Fine-grained Captioning Control Using Scene-rewards (FoCUS), a controllable image captioning method that lets users steer captions toward specific semantic emphases through natural-language control prompts. The core idea is a prompt-conditioned control objective based on scene-graph-aligned component scores. Generated captions are parsed and aligned to scene-graph components—objects, attributes, and relations—and these components are differentially weighted, including negative weights, according to the requested emphasis. We optimize this objective with GRPO and further improve its reliability through a stricter object validity threshold and reasoning-based verification for attribute and relation scoring. To evaluate controllability, we introduce Semantic Control and Precision Evaluation (SCoPE), a benchmark with contrastive Include/Avoid constraints for measuring both target content coverage and out-of-scope suppression. Experiments on two VLM backbones show that FoCUS consistently improves controllability and fine-grained caption quality without degrading general caption performance. The FoCUS code and SCoPE benchmark will be publicly released.

Abstract:
Dense retrievers decide which documents reach users and the language models that use them, yet they are typically evaluated with neutral queries. We ask whether the identity signals that real users express in their queries—political ideology and dialect—bias what a retriever returns. We design evaluations in two domains, political news and consumer-health questions, each pairing a controlled synthetic set that varies only the identity signal with naturalistic queries. Across five dense retrievers and a sparse baseline, every retriever (i) retrieves articles that align with the query’s own political lean and (ii) performs worse for questions written in African American Language (AAL) than in White Mainstream English (WME). Two analyses tie these gaps to queries’ identity signals beyond surface vocabulary: partialling out an aggregate lexical-asymmetry score leaves the synthetic gaps largely intact, and linear probes recover lean and dialect from the retrievers’ query embeddings beyond token-level features. Left unaddressed, such retrieval biases risk contributing to polarization and reinforcing the health disparities already faced by AAL speakers.

Abstract:
Previous work links Major Depressive Disorder (MDD) to linguistic and paralinguistic speech patterns, highlighting speech analysis as a potential diagnostic tool for remote monitoring or clinical decision support (CDS) systems. Recent attempts to build MDD diagnostic models primarily rely on the English (E-)DAIC corpus. Despite promising results, these works rarely evaluate cross-corpus generalization and rely on self-reported questionnaires, limiting their clinical utility. We introduce a new, clinically validated German corpus, EmpkinS-EKSpression, of semi-structured diagnostic interviews to test the reliability of spoken language-based measures of MDD. Our model achieves an F1-score of 0.86 on the EmpkinS-EKSpression dataset and 0.74 on E-DAIC via zero-shot cross-corpus transfer, suggesting that MDD detection models can generalize across corpora and languages despite differences in interview format and ground truth definition, supporting further development of CDS systems.

Abstract:
Activation steering has shown promise for controlling LLM generation along well-defined attributes, but it remains unclear whether it can handle the multidimensional and hard-to-define nature of authorship style. We ask whether structured contrastive prompting along rhetorically-motivated dimensions can construct rich style representations directly in activation space, bypassing the need for natural language style descriptors or dedicated training. We find that the resulting directions share a common authorship backbone while conflicting on aspect-specific residuals that carry genuine stylistic signal, explaining why naive aggregation fails. We operationalize this in Aspect-Aware Activation Steering (A3S), a training-free framework that merges per-aspect contrastive directions with interference-aware aggregation and tunes steering strength per instance. A3S improves authorship style transfer where it is genuinely multi-aspect, outperforms a trained baseline in preference evaluations on out-of-domain benchmarks, and keeps target-exemplar overlap consistently low.

Abstract:
The public often attributes human-like qualities to large language models (LLMs), assuming that they "know" certain things. In reality, LLMs encode information retained during training as internal probabilistic knowledge. This study examines existing methods for probing the veracity of that knowledge and identifies three flawed underlying assumptions. To address these flaws, we introduce ‘sAwMIL‘ (Sparse-Aware Multiple-Instance Learning), a multiclass probing framework that combines multiple-instance learning with conformal prediction. ‘sAwMIL‘ leverages LLMs’ internal representations to classify statements as _true_, _false_, or _neither_. We evaluate ‘sAwMIL‘ across 16 open-source LLMs, including default and chat-based variants, using three new curated datasets. Our results show that (1) common probing methods fail to provide a reliable and transferable veracity direction and, in some settings, perform worse than zero-shot prompting; (2) truth and falsehood are not encoded symmetrically; and (3) LLMs encode a third type of signal that is distinct from both true and false.

Abstract:
Diffusion language models (DLMs) generate text by iteratively denoising masked sequences, but standard decoding either fixes the sequence length or relies on ad hoc stopping rules, often leading to unnecessary denoising steps. We recast length selection as a discrete-time survival problem over the end-of-sequence token and propose a plug-in, training-free length predictor that can be added to any existing DLM. Across reasoning and code-generation benchmarks, survival-guided length decoding speeds up inference by up to 7× while preserving task accuracy. We further find that predicted lengths vary widely even within the same dataset, making model performance sensitive to the chosen length.

Abstract:
Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with conversational AI may reinforce delusional thinking—a phenomenon sometimes described as AI Psychosis. However, systematic empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. Using Reddit data, we construct simulated users (SimUsers) from users’ longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, while those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on the current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended dialogue and underscore the importance of state-aware safety mechanisms to mitigate such risks.

Abstract:
The rapid adoption of Large Language Models (LLMs) has raised significant environmental concerns. Unlike the one-time cost of training, LLM inference occurs continuously and dominates the AI energy footprint. Yet most sustainability studies report only coarse model-level metrics, treating energy efficiency as an afterthought rather than a primary objective. Addressing the limitation, we propose Component-Level Energy Assessment via Repetitions (CLEAR), to overcome temporal mismatch between microsecond scale component execution and millisecond(ms) scale monitoring of energy sensors. Using CLEAR, we evaluate 15 models spanning four architecture types, keeping component-wise energy variance below 9.5% while capturing over 90% of total energy as individual components. We present the first comprehensive, fine-grained energy analysis of Transformer components across key parameters such as batch size, attention heads, hidden dimension, KV cache, and attention variants. Our findings reveal that Attention consumes significantly more Energy per FLOP as compared to the entire model, indicating that FLOPs alone fail to capture true component-level energy cost. CLEAR enables reliable fine-grained energy measurements and provides a strong formal foundation for predictive modelling of energy consumption.

Abstract:
Using a zoom-in tool is an important foundational part of modern visual agents, because it allows to efficiently handle tasks involving high-resolution images. Most previous methods need an extensive warm-start supervised fine-tuning phase for teaching models zoom-in. We show that this is not necessary by proposing a new intrinsic reward for learning tool use in MLLMs without the need for additional labels or warm-start SFT. Our InfoNCE-style reward uses a curriculum of increasingly hard negative tool calls as a contrastive training signal. Empirical experiments on V, HRBench and MME-RealWorld show that our approach is competitive while being more efficient. When used as a drop-in replacement for SFT, we even outperform all baselines. To directly measure the zoom-in ability of models, we further introduce the scalable synthetic Muffin&Chihuahua (M&C) dataset. Each image consists of a grid with every cell either showing a muffin or chihuahua. Leveraging the M&C dataset’s unique region of interest labels, we find that recall is the metric that most strongly correlates the zoom-in region with final task performance. Our model and code for reproduction is publicly available at https://github.com/UKPLab/emnlp2026-zoom-in.

Abstract:
Large language model (LLM) agents increasingly rely on external tools (file operations, API calls, database transactions) to autonomously complete complex multi-step tasks. Practitioners deploy defense-trained models to protect against prompt injection attacks that manipulate agent behavior through malicious observations or retrieved content. We reveal a fundamental capability-alignment paradox : defense training designed to improve safety systematically destroys agent competence while failing to prevent sophisticated attacks. Evaluating defended models against undefended baselines across 97 agent tasks and 1,000 adversarial prompts, we uncover three systematic biases unique to multi-step agents. Agent incompetence bias manifests as immediate tool execution breakdown, with models refusing or generating invalid actions on benign tasks before observing any external content. Cascade amplification bias causes early failures to propagate through retry loops, pushing defended models to timeout on 99% of tasks compared to 13% for baselines. Trigger bias leads to paradoxical security degradation where defended models perform worse than undefended baselines while straightforward attacks bypass defenses at high rates. Root cause analysis reveals these biases stem from shortcut learning: models overfit to surface attack patterns rather than semantic threat understanding, evidenced by extreme variance in defense effectiveness across attack categories. Our findings demonstrate that current defense paradigms optimize for single-turn refusal benchmarks while rendering multi-step agents fundamentally unreliable, necessitating new approaches that preserve tool execution competence under adversarial conditions.

Abstract:
While large language models (LLMs) have advanced the development of general-purpose agents, robust generalization to unseen tasks remains challenging. Two common approaches are supervised fine-tuning and training-free memory-augmented generation using retrieved experience; yet both have limitations: fine-tuning often fails to extrapolate to new tasks, while experience retrieval often underperforms compared to supervised baselines. In this work, we combine these approaches and study how retrieval-augmented LLM agents can learn to use retrieved trajectories in-context. First, we establish a strong LoRA fine-tuning baseline that outperforms several state-of-the-art agent training pipelines. Second, we analyze key design choices for experience retrieval, including storage, querying, and trajectory selection. We then integrate experience retrieval directly into the fine-tuning process, finding that this substantially improves generalization to unseen tasks. Finally, we show that these gains often persist with imperfect experience and, even when agents reuse their own failed attempts without test-time parameter updates. Overall, our results establish simple episodic retrieval as a strong foundation for agent memory and retrieval-aware fine-tuning as a practical and effective framework for building agents that learn to learn from experience. Code and models are made available.

Abstract:
Large Language Models (LLMs) often generate specious hallucinations, creating an urgent need for post-hoc factuality verification against reliable evidence. Existing verification methods incur substantial computational overhead due to joint evidence–claim encoding. While bi-encoders offer a highly efficient alternative, they struggle with asymmetric entailment and are severely vulnerable to contextual noise. To address these challenges, we propose Gated Residual Discrepancy (GRD), a highly efficient bi-encoder framework for factuality verification. To explicitly capture asymmetric entailment, GRD pioneers a difference-dominant paradigm, extracting dual-view vector differences. To mitigate the vulnerability to contextual noise, GRD employs an adaptive gating mechanism that dynamically filters contextual noise, preventing large semantic backgrounds from masking critical errors, thereby enabling highly sensitive verification over broad evidence contexts. Experiments across various benchmarks show GRD achieves 79.99% average Macro-F1, trailing the strongest cross-encoder baseline by only 1.15 points while delivering a 23.55 × speedup with only a fraction of the computational FLOPs.

Abstract:
Automated fact-checking (AFC) systems are susceptible to adversarial attacks, enabling false claims to evade detection. Existing adversarial frameworks typically rely on injecting noise or altering semantics, yet no existing framework exploits the adversarial potential of persuasion techniques against AFC systems, which are widely used in disinformation campaigns to manipulate audiences. In this paper, we introduce a novel class of persuasive adversarial attacks on AFCs by employing an LLM to rephrase claims using persuasion techniques. Considering 15 techniques grouped into 5 categories, we study the effects of persuasion on both claim verification and evidence retrieval using a decoupled evaluation strategy. Experiments on the FEVER and FEVEROUS benchmarks show that persuasion attacks can substantially degrade both verification performance and evidence retrieval. Our analysis identifies persuasion techniques as a potent class of adversarial attacks, highlighting the need for more robust AFC systems.

Abstract:
We introduce Evo, a duality latent trajectory model that bridges autoregressive (AR) and diffusion-based language generation within a continuous evolutionary generative framework. Rather than treating AR decoding and diffusion generation as separate paradigms, Evo reconceptualizes text generation as a latent flow: each token is associated with a vector-valued embedding that evolves over a progression variable, indicating its semantic maturity. Low values correspond to confident AR-like refinement, while high values invoke diffusion-style planning, allowing the model to adaptively balance AR and diffusion based on uncertainty. Theoretically, we show that both AR and diffusion models emerge as discretizations of a shared probability flow, and we derive Evo’s training objective from a unified variational ELBO. The model is implemented as a time-conditioned Transformer governed by a shared vector field, trained end-to-end to jointly infer latent codes and their progression times. During decoding, Evo performs efficient, semantics-aware refinement, achieving high-quality outputs without sacrificing speed. Empirically, Evo 8B achieves state-of-the-art or highly competitive results on 15 diverse benchmarks, including reasoning (GSM8K, ARC-C), code generation (HumanEval, MBPP), and general language understanding, while maintaining fast inference speed. Our results demonstrate that Evo delivers a new paradigm for LLM design with strong generation quality, robust symbolic reasoning, and decoding efficiency.

Abstract:
The ubiquity of multimedia content is reshaping online information spaces, particularly in social media environments. At the same time, search is being rapidly transformed by generative AI, with large language models (LLMs) routinely deployed as intermediaries between users and multimedia content to retrieve and summarize information. Despite their growing influence, the impact of LLM inaccuracies and potential vulnerabilities on multimedia information-seeking tasks remains largely unexplored. We investigate how generative AI affects accuracy, efficiency, and confidence in information retrieval from videos. We conduct an experiment with around 900 participants on 8,000+ video-based information-seeking tasks, comparing behavior across three conditions: (1) access to videos only, (2) access to videos with LLM-based AI assistance, and (3) access to videos with a deceiving AI assistant designed to provide false answers. We find that AI assistance increases accuracy by 3–7% when participants viewed the relevant video segment, and by 27–35% when they did not. Efficiency increases by 10% for short videos and 25% for longer ones. However, participants tend to over-rely on AI outputs, resulting in accuracy drops of up to 32% when interacting with the deceiving AI. Alarmingly, self-reported confidence in answers remains stable across all three conditions. Our findings expose fundamental safety risks in AI-mediated video information retrieval.

Abstract:
Hierarchical Instruction Following (HIF) refers to the problem of prompting large language models with a priority-ordered stack of instructions. Standard methods like RLHF and DPO typically fail in this problem since they mainly optimize for a single objective, failing to explicitly enforce system prompt compliance. Meanwhile, supervised fine-tuning relies on mimicking filtered, compliant data, which fails to establish the priority asymmetry at the algorithmic level. In this paper, we introduce HIPO, a novel alignment framework that formulates HIF as a Constrained Markov Decision Process. HIPO elevates system prompts from mere input context to strict algorithmic boundaries. Using a primal-dual safe reinforcement learning approach, the algorithm dynamically enforces system prompt compliance as an explicit constraint, maximizing user utility strictly within this feasible region. Extensive evaluations across diverse model architectures (e.g., Qwen, Phi, Llama) demonstrate that HIPO significantly improves both system compliance and user utility. Furthermore, mechanistic analysis reveals that this constrained optimization autonomously drives the model to shift its attention toward long-range system tokens, providing a principled foundation for reliable LLM deployment in complex workflows.

Abstract:
Multilingual Information Retrieval (MLIR) reflects real-world search environments in which queries and relevant documents may appear in different languages within a mixed-language corpus. However, existing embedding models are primarily optimized for Multi-Monolingual retrieval and their performance often degrades substantially in MLIR settings. Moreover, directly applying conventional contrastive learning to MLIR can exacerbate language clustering and expose a trade-off between cross-lingual alignment and embedding uniformity. To address these limitations, we propose MIMO: Multilingual Information Retrieval via Monolingual Objectives, a two-stage framework that uses a stable English semantic space as an anchor. MIMO first initializes cross-lingual alignment through knowledge distillation, and then jointly optimizes distillation and cross-lingual contrastive learning to improve retrieval discrimination while preserving alignment. Extensive experiments show that MIMO consistently outperforms existing cross-lingual training baselines across various MLIR and Multi-Monolingual benchmarks. MIMO also remains competitive with off-the-shelf models of similar or larger parameter scales. Furthermore, our cross-lingual Alignment-Uniformity analysis clarifies the distinct roles of the two loss components and shows that their combination yields a favorable trade-off between alignment and uniformity.

Abstract:
A human’s moral decision depends heavily on the context. Yet research on LLM morality has largely studied fixed scenarios. We address this gap by introducing Contextual MoralChoice, a dataset of moral dilemmas with systematic contextual variations known from moral psychology to shift human judgment: consequentialist, emotional, and relational. Evaluating 22 LLMs, we find that nearly all models are context-sensitive, shifting their judgments towards rule-violating behavior. Comparing with a human survey, we find that models and humans are most triggered by different contextual variations, and that a model aligned with human judgments in the base case is not necessarily aligned in its contextual sensitivity. This raises the question of controlling contextual sensitivity, which we address with an activation steering approach that can reliably increase or decrease a model’s contextual sensitivity. Code and data: https://github.com/adrian-sauter/contextual_moralchoice .

Abstract:
Retrievers are a key bottleneck in temporal Retrieval-Augmented Generation (RAG) systems: failing to retrieve temporally relevant context can degrade downstream generation, regardless of LLM reasoning. We propose Temporal-aware Matryoshka Representation Learning (TMRL), an efficient method that equips retrievers with temporal-aware Matryoshka embeddings. TMRL leverages the nested structure of Matryoshka embeddings to introduce a temporal subspace, enhancing temporal encoding while preserving general semantic representations. Experiments show that TMRL efficiently adapts diverse standard encoder-only text embedding models, achieving competitive temporal retrieval and temporal RAG performance compared to existing Matryoshka-based adaptation and temporal retrieval methods, while enabling flexible accuracy-efficiency trade-offs.

Abstract:
We present LOVER , a ̲L ong to sh ̲O rt ̲V ideo ̲E vidence ̲R einforced model for grounded question answering (GQA). LOVER highlights three innovations over existing reinforcement-learning (RL) based video reasoning models: (1) Long-to-short Video Evidence Curriculum Learning , which organizes RL training according to evidence duration and progressively adapts the model from long-range grounding to short-term reasoning; (2) GQA Rewards , which underscore the benefit of IoP reward over IoU for evidence spotting rather than strict temporal span overlap; (3) Adaptive Timestamp Rendering , which adaptively renders timestamps onto video frames using background-aware position and color selection to enhance temporal observability. The three designs are model-agnostic and reciprocal. They effectively improve QA, grounding, and grounded QA performance over different backbones. Notably, LOVER built on Time-R1 achieves new state-of-the-art (SOTA) results among open-source models on popular GQA benchmarks: NExT-GQA and ReXTime. Comprehensive ablation studies further validate the effectiveness of our three innovative components.

Abstract:
Large language models typically undergo post-training to align them with safety policies but there exist many sophisticated jailbreaks that sidestep established safeguards. For instance, prior work by Andriushchenko et al. (2025) has found that changing the grammatical tense from present to past can be enough to elicit harmful responses. In this work, we uncover a more general failure of non-imperative syntactic forms. We demonstrate that this _syntactic vulnerability_ exists in 16 models up to 70B parameters, using behavioral evaluation. To investigate the root cause, we apply causal mediation analysis, finding that refusal is partially conditioned on upstream syntactic features. By steering these purely syntactic features we are able to trigger and suppress refusal. Finally, we trace this ill-conditioning to linguistically biased post-training data of open-source models and show that increasing syntactic diversity can mitigate the issue. Our findings suggest that current alignment approaches introduce confounders that prevent a pure semantic grounding of the refusal decision.

Abstract:
The heavy-tailed distribution of output lengths in Large Language Model (LLM) serving poses major challenges for resource provisioning and cluster scheduling. Although output-length prediction can mitigate these issues, existing approaches have key drawbacks: external proxy models add substantial latency and often have limited fidelity, whereas internal state-based methods are efficient but rely on shallow probes of current model states. We identify a structural connection between speculative decoding (SD) and length prediction: latent representations produced by the draft decoder in advanced frameworks (e.g., EAGLE-3) encode signals that are predictive of generation length. Building on this insight, we introduce OUTLETS (Output-Length Prediction from Speculative Decoding Backbones), which repurposes the speculative backbone as a trajectory-aware length predictor. When its draft representations are already computed for speculative decoding, OUTLETS adds only a lightweight regression head and achieves lower MAE than the evaluated methods. Under saturated disaggregated serving, OUTLETS predictions enable standard scheduling policies to prioritize shorter requests and distribute requests more evenly across decoding instances, reducing short-request P99 latency by 34.8%.

Abstract:
The use of LLMs to complete online surveys impacts the validity of survey-based research, but detecting such usage remains under-explored. We introduce an initial benchmark dataset, namely ASURRE, for AI-assisted survey participation to capture usage strategies ranging from full generation and revision to persona-grounded agentic completion. Controlled by these strategies, LLM-assisted survey responses are generated using multiple LLMs on three real-world surveys in different disciplines, paired with genuine human responses. Our evaluation of existing machine-generated text (MGT) detectors shows that naive AI usage is readily detectable, whereas persona-grounded agents that mimic entire respondents push detector performance toward chance. We further show that agentic completion cannot fully replicate respondent-level behaviour and leaves distinctive behavioural traces. While individual cues can be circumvented by targeted prompting, a simple few-shot, training-free aggregator over these cues improves mean AUROC by +0.14 over the best existing detector across agentic settings. Our project is available on GitHub.

Abstract:
Soft compression stores a text span as k continuous compressed vectors for a frozen decoder, but existing compressors are usually tied to the decoder used during training. We study whether those vectors can be retargeted to a different frozen decoder through its input-embedding interface under a controlled reconstruction protocol. Our benchmark covers seven target decoders, three vector budgets, and directed source–target pairs, comparing Random, shared-embedding least squares (LS), Converter-only, Enc-Conv, and target-native Origin. Converter-only maps vectors from a frozen source compressor, whereas Enc-Conv also adapts the source-initialized vector generator. Transfer is feasible but decoder-conditioned: trainable retargeting can approach Origin BLEU on decoder-compatible targets, while LS is often close to Random despite using shared token anchors. Enc-Conv consistently matches or outperforms Converter-only, with its largest gains on difficult transfers. Selected-pair multi-seed runs and a small SQuAD probe provide complementary evidence, although downstream utility remains target-dependent. Perturbation and geometry probes are consistent with decoder-specific readable regions. Finally, pair-level readability scores provide weak-to-moderate pre-transfer screening signals.

Abstract:
Real-world knowledge resides in multimodal documents, necessitating retrieval-augmented generation (RAG) for accurate question answering. However, existing multimodal RAG models are primarily designed for single-image or closed-document settings and exhibit limited accuracy in realistic multi-image scenarios. Moreover, processing numerous retrieved images incurs substantial computational overhead from irrelevant visual tokens. To address these challenges, we introduce DocLongRAG, a large-scale dataset of 343K question–answer pairs, each associated with an average of 37.4 retrieved images to reflect authentic RAG workflows. Building on this dataset, we propose Doc-REFRAG, a question-guided framework that compresses visual tokens into coarse chunks and selectively expands question-relevant ones via a lightweight RL-based selector. Experiments on six benchmarks show that Doc-REFRAG outperforms eleven strong baselines, achieving state-of-the-art accuracy with significantly lower inference latency. Our resources are available at https://github.com/Collab-Gen/Doc-REFRAG.

Abstract:
Recent calls for harder machine translation benchmarks have not clarified what difficulty should mean. We argue that one meaningful and currently unmeasured axis is referential reach, the distance a segment must look back into its document to resolve the entities and pronouns it contains. We formalize this as discourse dependency (DDP), a metric-free, source-side measure computed from named entity re-mentions and pronominal coreference. Validated against gold coreference, DDP errs one-sidedly in 99.2% of segments, so a high-DDP segment is certified to require long-range context. Applying DDP to WMT24++ and WMT25 shows that both are heavily skewed toward low-DDP segments, which domain labels do not distinguish. Building on DDP, we compare five context injection strategies in an English-Korean post-editing setup, varying context size and selection. As DDP grows, no strategy keeps pace with human post-editing. On segments with DDP ≥ 15 raters prefer human translations, while automatic metrics register no difference. As frontier systems saturate aggregate scores, DDP shifts evaluation from how well models score to how far they can reach.

Abstract:
English Wikipedia is often treated as the default encyclopedic source, yet non-English Wikipedia editions can contain richer locally grounded information for long-tail figures. We study cross-lingual biography enrichment: enriching an existing English biography with facts supported by a non-English biography about the same person. Focusing on women from non-English-speaking contexts, we introduce CLAW-4L , a benchmark consisting of 300 Wikipedia biography pairs linking an English biography with its French, Chinese or Azerbaijani counterpart, along with claim annotations and a fine-grained claim-pair relation corpus. We propose a claim-based enrichment framework that extracts English claims from both biographies, aligns them to identify enrichment evidence from the non-English biography, and rewrites the English biography using the selected claims. Our results show that non-English Wikipedia biographies provide valuable evidence for improving English biography coverage, while lower-resource settings remain challenging.

Abstract:
On-policy distillation (OPD) transfers reasoning behavior by training a student on teacher feedback along student-generated trajectories, but standard full-rollout training ties every update to a costly completion and can over-allocate supervision to late positions with low marginal value for the current student. We revisit this assumption through the useful supervision horizon: student-induced rollouts can drift from teacher-preferred continuations, while aligned prefixes may already preserve the long-horizon OPD update direction. We propose ADWIN, an adaptive-window framework for OPD that treats rollout length as an online admissibility decision, training on short teacher-anchored prefixes while using delayed full-rollout probes to audit prefix–full alignment and adapt the next horizon with staleness control. Across math and code reasoning benchmarks in single-task, multi-task, and strong-to-weak settings, ADWIN improves the accuracy–compute trade-off over full-rollout OPD and prefix-based baselines, reducing end-to-end training cost by up to 4.1× while achieving comparable or better accuracy.

Abstract:
Language similarity is a central concept in multilingual NLP. Prior work typically infers similarity from the static geometry of representation spaces, and the results often depend on model architecture, training data, and probing methods. In this work, we propose transition-based language similarity (TBLS), a new measure that characterizes language similarity through cross-lingual transition. The key intuition is that if two languages occupy nearby regions in a model’s representation space, model outputs are more likely to transition between them under small local variations. By aggregating such transition behavior across languages, TBLS can be derived, reflecting language proximity encoded by the model. We operationalize this idea by perturbing model activations and measuring the resulting cross-lingual transition behavior. Across five multilingual language models and two corpora covering 44 languages, TBLS exhibits strong cross-model and cross-corpus consistency. Further analyses show that TBLS captures meaningful linguistic regularities and transfer-related structure, while activation perturbation provides behavioral evidence consistent with the hypothesis that semantically aligned multilingual representations occupy nearby, locally connected regions. These findings suggest that perturbation-induced transition dynamics reveal a surprisingly stable form of multilingual relationships across models and corpora.

Abstract:
Automated evolution of Multi-Agent Systems (MAS) holds significant potential for reducing the manual effort required to design and optimize LLM-based agent architectures. However, extant search-based paradigms face a fundamental trade-off, where an expanded optimization scope exacerbates evolutionary instability, while discrete branch-and-discard search isolates insights across lineages. To address these limitations, we propose a continuous, data-driven optimization paradigm built upon a unified ReAct-based infrastructure that reconciles a broad optimization scope with operational stability. Under this paradigm, we present OptiMAS, a task-agnostic agentic optimizer that leverages textual interaction trajectories and task feedback as loss signals for end-to-end MAS evolution. Equipped with a novel dual-track memory mechanism, OptiMAS sustains performance improvement over extended optimization horizons. Evaluation on four heterogeneous agentic benchmarks with three varying scale and accessibility LLM backbones, demonstrates that OptiMAS consistently achieves competitive or superior accuracy relative to both domain-specialized hand-crafted systems and existing evolutionary methods. Our work establishes a practical milestone toward robust, automated MAS evolution.

Abstract:
Vision-language models (VLMs) can comply with harmful requests delivered through images, even when their LLM backbones would refuse the same content in text. While prior work characterizes these jailbreaks empirically or at the representation level, how visual inputs perturb safety pathways at the neuron level remains uncharted. We close this gap with a causal, neuron-level analysis of safety mechanisms in 10 VLMs. We propose a two-stage detection pipeline with iterative ablation that accounts for self-repair, and introduce two modality-isolated benchmarks, ViSafe-Detect and ViSafe-Eval, which decouple visual and textual safety signals. Our analysis reveals: (i) Text safety in VLMs is localizable: ~88 neurons (<0.01%) whose targeted ablation substantially reduces refusal. (ii) Text safety neurons constitute the dominant refusal pathway: ablating them is the only intervention that consistently and substantially reduces refusal across all models. (iii) Visual safety is high-dimensional and diffuse at the single-neuron level: text safety concentrates in ~5 subspace directions while visual safety requires ≥50. This gap holds across architectures, explaining why current alignment has not closed the visual safety gap. Project page is at: https://jiaxuan-li.github.io/vlm-safety-neuron/. Warning: this paper may include examples of harmful content.

Abstract:
NVFP4 is an efficient microscaling format for low-bit inference, but activation outliers can still degrade quantization accuracy within NVFP4 blocks. Within each quantization block, large activations can dominate the block scale, increasing the quantization error of the remaining values sharing the same scale. Existing post-training quantization (PTQ) methods mitigate outlier errors through strategies such as mixed precision, rotation, or residual compensation, but these approaches are either not specifically tailored to NVFP4 or introduce additional computation. In this work, we revisit NVFP4 from a channel-grouping perspective and define the reducible error incurred by remaining block values under the scale set by the block maximum as Collateral Quantization Error. Based on this insight, we propose OCGQuant, a post-training quantization method centered on Outlier-Companion Grouping (OCG), which adaptively pairs outlier channels with low-magnitude companion channels to improve NVFP4 activation block composition. Experiments on Llama3 and Qwen3 show that OCGQuant achieves the lowest WikiText-2 perplexity and highest average downstream accuracy among evaluated PTQ methods, while maintaining prefill speedup close to RTN and matching its peak decoding memory. Code is available at https://github.com/Eshamont/OCGQuant.

Abstract:
Legal judgment prediction (LJP) aims to predict judicial outcomes from case facts and typically includes law article, charge, and sentencing prediction. While recent methods perform well on the first two subtasks, legal sentencing prediction (LSP) remains difficult due to its need for fine-grained objective knowledge and flexible subjective reasoning. To address these limitations, we propose MSR 2 , a framework that integrates multi-source retrieval and reasoning in LLMs with reinforcement learning. MSR 2 enables LLMs to perform multi-source retrieval based on reasoning needs and applies a process-level reward to guide intermediate subjective reasoning steps. Experiments on two real-world datasets show that MSR 2 improves both accuracy and interpretability in LSP, providing a promising step toward practical legal AI. Our code is available at https://github.com/cjj826/MSR2.

Abstract:
Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise multi-step symbolic deduction. Although test-time scaling (TTS) has demonstrated remarkable success in general mathematical reasoning, it fails to scale effectively under the symbolic-program paradigm for plane geometry. We identify two key obstacles: limited reasoning diversity induced by rigid symbolic programs and insufficient explicit visual grounding before symbolic deduction. To address these issues, we propose Multi-Trace Synthesis (MTS), which converts each symbolic program into heterogeneous reasoning traces, including executable Python scripts and CoT-augmented variants. We further propose Perception-Augmented (PA) training, which parses diagrams into structured semantic clauses before deduction, and Consensus-Guided Multi-Trace Ensemble (CG-MTE) for efficient self-adaptive inference. Experiments on three geometry benchmarks show that our method consistently improves PGP-solving across model scales and achieves strong performance against both general-purpose MLLMs and specialized geometry solvers. Under test-time scaling, CG-MTE achieves a comparable accuracy to high-budget self-consistency while reducing sampling cost by up to 8 × . Code and data are publicly available at https://github.com/Jason8Kang/ReTTS-PGPS.

Abstract:
Many production RAG systems implement retrieval abstention by thresholding raw similarity scores, implicitly treating score magnitude as a confidence signal. We demonstrate that this practice degrades systematically as queries require reasoning beyond semantic matching. Across 11 retrieval architectures and 28 datasets, neural retrievers consistently assign high similarity scores to semantically related but constraint-violating documents, causing magnitude-based thresholds to collapse toward near-random abstention performance on logical and temporal reasoning tasks—a failure we term the Magnitude Mirage. To address this without computationally expensive alternatives, we conduct a large-scale empirical study of six zero-cost Query Performance Prediction (QPP) metrics across three cognitive tiers: semantic matching (BEIR), logical reasoning (BRIGHT), and temporal reasoning (TEMPO). Our central finding is that the key improvement comes from abandoning magnitude in favor of score-distribution signals: the gain from this shift exceeds the differences among distributional alternatives by a factor of 5–10 × . In particular, Score Gap ( s 1 - s k ) and a practical adaptation of Score Magnitude and Variance (LSMV) improve abstention AUROC by up to 0.16 in settings where magnitude-based confidence provides little discriminative power. These methods require no additional inference, retraining, or latency, making them a practical zero-cost replacement for magnitude thresholding in deployed RAG systems.

Abstract:
High-performance GPU kernels are essential for efficient LLM deployment, yet optimizing them remains expertise-intensive. Recent LLM-based code generation makes automatic GPU operator generation promising, but operator optimization remains a hardware-aware search problem. Existing LLM-based methods face a granularity mismatch: coarse hints are reusable but hard to execute, whereas detailed memories are actionable but enlarge the search space and obscure optimization bottlenecks. The key challenge is therefore to organize optimization experience at an appropriate granularity. To address this issue, this paper proposes HTAM (Hierarchical Transition-Attended Memory), a coarse-to-fine framework for LLM-based operator optimization. HTAM builds a two-level Hierarchical Transition Graph (HTG) to organize coarse global directions, detailed local strategies, and transition experience between optimization steps. During each evolution step, HTAM selects a global direction from the current state and recent optimization history, retrieves the corresponding local strategy memory, and uses it to guide concrete CUDA code generation. Experiments on the full KernelBench suite demonstrate that HTAM consistently improves correctness, fast-solution rate, and speedup over LLM-based baselines, while backend and Robust-KBench studies indicate transferable benefits from structured memory.

Abstract:
Video emotion understanding is commonly studied as an offline classification problem, where the complete video segment is available before prediction.Real-time interaction, however, requires emotion decisions from incomplete and evolving evidence.This paper studies streaming video emotion understanding as a reliability-aware decision process over evolving emotion beliefs. In this setting, a single confident prefix prediction can still be unreliable when the underlying belief trajectory is unstable or repeatedly switches across emotion classes. We propose TRACE, a trajectory-aware reliability framework that forms low-latency emotion beliefs from streaming audio prefixes, estimates reliability from confidence, entropy, stability, and class-switching patterns, and selectively invokes contextual belief reinterpretation with visual, textual, and neighboring-utterance evidence. TRACE keeps stable cases in the low-latency online pathway while allocating stronger multimodal reasoning to uncertain cases that remain ambiguous. Experiments on StreamMER, MELD, and MER2024 show that TRACE improves the accuracy-cost trade-off, retaining most full-context gains while reducing unnecessary contextual reasoning.

Abstract:
Large Language Models (LLMs) have renewed interest in autoformalization. Yet, when First-Order Logic (FOL) is considered as the target formalism, the field still lacks a unified task formulation and a systematic survey. This paper addresses this gap: we first provide a principled definition for the FOL-autoformalization task by distinguishing Ontology Extraction from Logical Translation, showing how their conflation obscures (cross-study) evaluation; we review existing datasets, evaluation metrics, and LLM-based methods, including fine-tuning, prompting, and verification-based refinement; we identify open challenges in benchmarking, semantic evaluation, ontology-aware methods, and end-to-end applications.

Abstract:
Chart question-answering (QA) benchmarks aim to pose questions that require visual reasoning to correctly answer, but vision-language models (VLMs) can often reach solutions through shortcuts or prior familiarity with a chart or question. To strictly evaluate visual reasoning, we propose counterfactual charts where the chart-question task remains fixed, but the underlying data and the corresponding answer are varied. We introduce CHARTOGRAPHER, a framework to reverse engineer charts into executable code, validate reconstruction fidelity, generate counterfactual variants, and derive new answers from executable QA logic. We apply this framework to existing chart QA datasets and evaluate proprietary and open-source VLMs, measuring variant sensitivity and generalizability. Counterfactual charts reveal failures hidden by single-chart performance: VLMs often fail to generalize after answering the original chart correctly. We find that failures are most prevalent when updated charts require novel visual reasoning pathways.

Abstract:
Speech translation models are increasingly capable of preserving speech-specific information (e.g., speaker gender, prosody, and emphasis), yet evaluation metrics remain blind to such phenomena. We meta-evaluate both text- and speech-based quality estimation metrics on two contrastive datasets targeting gender agreement and prosody, and find that both fall short, even when given direct access to the speech signal. We then train SpeechCOMET, a family of quality estimation models with speech encoders, and evaluate a state-of-the-art SpeechLLM as a judge. Both match or exceed text-based COMET on standard quality estimation, but neither consistently assesses speech-specific phenomena. We identify three causes: (1) speech-specific features are not reliably preserved in current encoders, (2) models tend to ignore the speech source signal, and (3) quality estimation training data contains too few relevant examples. We release all models and code, and argue that progress requires dedicated speech-specific training data and models that genuinely condition on speech.

Abstract:
Language Model (LM)-based agents remain challenging to evaluate systematically in mixed-motive settings where agents must leverage short-term cooperation for long-term competitive goals (e.g., multi-party politics). We introduce Cooperate to Compete (C2C), a multi-agent environment where players can engage in private negotiations while competing to be the first to achieve their secret objective. Players have asymmetric objectives and negotiations are non-binding, allowing alliances to form and break as players’ short-term interests align and diverge. We run AI-only games and conduct a user study pitting human players against AI opponents. We find that humans favor lower-complexity deals and talk to more unique partners over the course of the game. We also find that humans are more aggressive negotiators, closing deals only 73.5% of the time compared to 94.0% for LM-based agents. Interventions inspired by these findings shift agent negotiation behavior and can raise win rates from 22.2% to 32.7%, demonstrating that language coordination strategies in C2C substantially alter negotiation dynamics. We run over 1,300 games with over 20,000 private conversations totaling over 180,000 player actions. By combining multi-party, long-horizon play with partial observability, asymmetric objectives, and non-binding negotiations, C2C provides a controlled, gamified testbed for studying strategic coordination in LM-based agents.

Abstract:
Polite language production involves balancing strategic success with local convention, but these pressures are difficult to separate in natural interaction. We therefore examine them in an artificial-language request game, where one marker increased request acceptance and another appeared more often in other speakers’ requests. Humans and LLMs completed the same task under goals to maximize acceptance, match local usage, or produce a natural and appropriate request. For both humans and LLMs, a policy-mixture model combining payoff- and exposure-sensitive policies provided the best overall predictive fit. However, their baseline behavior differed. Human baseline behavior fell between the strategy and convention conditions, suggesting sensitivity to both payoff and local usage. LLMs shifted toward local usage under the convention goal, but their baselines remained closer to the strategy condition. These results show that human and LLM behavior is sensitive to both payoff and local usage in polite production, but that LLM baselines are more payoff-oriented. Code, data, and materials are all available at https://github.com/CHENTianfang/emnlp2026-politeness.

Abstract:
Low-resource target-language generation is often limited by scarce parallel data, while high-resource source-language monolingual data is abundant but difficult to use with standard supervised fine-tuning. We propose Source-Grounded Semantic Reinforcement Learning (SG-SRL), a resource-utilization framework that converts source-language monolingual data into cross-lingual semantic supervision. SG-SRL performs reference-free reinforcement learning (RL) on source-language data with a cross-lingual semantic reward, followed by lightweight supervised recovery. Although direct reward optimization induces verbosity-based reward hacking, recovery restores fluency, conciseness, and task format without erasing the learned semantic alignment. On Chinese-to-Thai generation, SG-SRL outperforms cold-start SFT, source-only continued pretraining, and synthetic-data SFT; human preference and training-independent metrics corroborate the gains. Recovery ablations further show that semantic retention and form restoration are jointly controlled by recovery data and intensity. An in-domain Chinese-to-Tibetan experiment shows that the reward interface can use a contrastively trained encoder when a capable multilingual reranker is unavailable, while also delimiting the framework’s dependence on reward-model discrimination.

Abstract:
We study the problem of personalization in large language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque “black-box” profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+ , a universal preference reasoning model that generates textual preference summaries. Experiments on nine benchmarks show that our 8B model achieves state-of-the-art performance—outperforming substantially larger open-source models—while exhibiting strong transferability across tasks, model families, and interaction formats. Code and dataset are accessible at https://github.com/AntResearchNLP/AlignX-Family/tree/main/AlignXplorePlus .

Abstract:
Large language models can be confidently wrong, making uncertainty estimation especially important in black-box settings where token probabilities are unavailable. Existing post-hoc methods estimate uncertainty from output dispersion, treating sampled responses largely as exchangeable. We propose structural uncertainty, an observable signal derived from self-preference-induced ranking stability: given multiple sampled candidates, we elicit sparse pairwise self-preferences, aggregate them into ranking distributions via Bradley–Terry modeling with PageRank, and decompose uncertainty into two complementary entropy-based components—across-trial ranking instability and within-trial candidate ambiguity. Across five LLMs and eight benchmarks, self-preference-derived structural signals can provide information complementary to output dispersion, and their combination improves selective prediction on several reasoning and knowledge tasks. The two components relate differently to accuracy: on math reasoning, within-trial ambiguity can be positively correlated with correctness—consistent with settings where multiple plausible candidates remain competitive under self-preference ranking—whereas on factual retrieval the structural signal collapses toward uniformity, diagnosing a regime where preference-based uncertainty is uninformative. Structural uncertainty is not a universal replacement for dispersion but a regime-sensitive complementary signal.

Abstract:
To understand the emergence of grammatical behavior in language models, we track circuits for subject–verb number agreement in pythia-1b and ten seeds of pythia-410m across training. We find that subject–verb agreement is learned early, and is initially handled by a diffuse collection of neurons. However, after agreement behavior has stabilized, the mechanisms responsible for it continue to reorganize. The attention heads which mediate agreement often shift during training: in the case of pythia-1b, behavior is controlled by two heads early in training but by a third (different) head later. Furthermore, the circuit shrinks over training, with pythia-1b’s final circuit being strikingly sparse. We take these results to be informative about how linguistic knowledge emerges and is represented in a language model.

Abstract:
We introduce Mahalanobis-Ensemble Decoding (ME-Decoding), a novel Large Language Model (LLM) decoding framework that frames candidate token selection as ensemble pruning. Existing selection strategies rely predominantly on scalar probabilities, ignoring geometric semantic relationships and causing candidate redundancy. Meanwhile, current geometry-aware methods often require complex optimization or directly reweighting the original token probabilities, leading to significant computational overhead or inference instability. To address this, we formulate decoding as a subset optimization problem using a Mahalanobis distance-driven objective to enhance semantic diversity while preserving high probabilities. Specifically, we dynamically discount redundant generation paths using a token similarity matrix, constructed via an adaptive-bandwidth kernel over token embeddings. We further devise an efficient greedy selection algorithm with near-linear complexity in the candidate size under early stopping, while establishing its theoretical approximation guarantees. This renders ME-Decoding a robust, plug-and-play module with negligible inference overhead. Extensive experiments across diverse reasoning and generation tasks demonstrate that our method consistently achieves strong performance.

Abstract:
Large language models (LLMs) exhibit biases against LGBTQ+ populations. While safety training may reduce explicit bias, implicit stereotype-driven associations often persist. In this work, we examine implicit bias toward transgender people in two main scenarios. First, we adapt word association tests to measure whether LLMs disproportionately pair negative concepts with "transgender" and positive concepts with "cisgender". Second, given the systemic challenges transgender people face in healthcare, we design a healthcare appointment allocation task where models act as scheduling agents choosing between cisgender and transgender candidates across stereotype-prone specialties. We evaluate seven LLMs in English and Spanish. Our results show consistent bias in categories such as appearance, risk, and veracity, indicating stronger negative associations with transgender individuals. In the allocation task, transgender candidates are favored for STI and mental health services, while cisgender candidates are preferred in gynecology and breast care. Notably, this bias is reduced when patient symptoms accompany demographic information. These findings underscore the need for research that address subtle stereotype-driven biases in LLMs to ensure equitable treatment of transgender people in healthcare applications.

Abstract:
AI-generated text detectors are increasingly deployed in consequential settings—peer review, education, admissions, journalism, and content moderation—where their outputs directly affect individuals. Yet the strongest detectors are fine-tuned neural classifiers whose decisions are opaque, while transparent alternatives (stylometric and linguistic-feature classifiers) underperform them by wide margins. We present a pipeline that converts a fine-tuned neural detector into a classifier that operates on human-readable concepts. We accomplish this by training sparse autoencoders (SAEs) directly on a detector’s residual stream, assigning auto-interpretation labels to each SAE feature, and retaining only features whose generated descriptions pass a global fidelity threshold. We then train a lightweight classifier on just the interpretable features. We find that our SAEs recover latent concepts that correspond to highly specific and predictive linguistic “tells” of AI-generated text. Classifiers built on these extracted features are not only highly interpretable but exhibit stronger performance on out-of-distribution texts than either neural or stylometric alternatives.

Abstract:
Large language models (LLMs) tool-use policies select actions from a finite candidate set at each reasoning step. We observe that this action-selection interface is already a linear contextual scorer: under a restricted output vocabulary, the LM head computes per-action scores from the last-layer hidden state, exactly the structure assumed by linear contextual bandits. Existing online adaptation methods for LLMs operate indirectly through prompts or retrieved memories, leaving this decision layer fixed and providing no explicit mechanism for action-level credit assignment or uncertainty-aware exploration. We propose OLIVIA, a lightweight online adaptation layer that replaces the frozen action-selection head with a per-action contextual linear bandit, selects actions via LinUCB exploration on the LLM’s hidden state and updates only its per-action sufficient statistics from step-level feedback via a rank-one Sherman–Morrison rule. Without backbone fine-tuning and additional LLM calls per step, OLIVIA adds negligible overhead relative to LLM inference. Across four tool-use benchmarks ToolBench, TaskBench, TaskBench-MM, BFCL and two backbones Qwen3-4B, Mistral-7B, OLIVIA outperforms prompt-based online adaptation and contextual-bandit variants.

Abstract:
Improving vision-language models (VLMs) on visual reasoning typically requires retraining or hand-designed prompts and tools. We present Dynamo, a training-free framework that adapts a frozen VLM without any weight updates. On a small labeled training subset, the agent inspects its own correct and incorrect attempts and evolves two complementary capabilities: reusable reasoning skills for cognitive bottlenecks, and executable visual tools for perceptual ones. Each generated tool is paired with a skill that specifies when to invoke it, and both capability types accumulate in a persistent library. Across four visual reasoning benchmarks and five VLM backbones, Dynamo improves direct inference on all 20 model–benchmark settings (avg. +5.6 acc). When the tool set is given in advance, the framework learns when to call each tool, and per-step tool choice improves on every tested backbone. Against task-specific RL (VTool-R1, DeepEyes), Dynamo closes 65–99% of the RL gap at a fraction of the compute, and combines additively with RL when available.

Abstract:
Multi-hop question answering requires combining information from multiple documents to answer complex questions. These systems have grown increasingly capable, yet when they fail, the error is typically attributed to not finding the right documents. Whether this holds at the level of individual reasoning steps remains largely unexamined. We investigate this across three standard multi-hop QA benchmarks and find that failures decompose into two distinct modes: retrieval failures, where the needed passage was not retrieved, and extraction failures, where the passage was retrieved but the needed fact could not be extracted — a phenomenon we term the fact-grounding gap. Extraction failures account for nearly half of all per-hop deficiencies and are invisible to standard retrieval metrics. They remain unresolved by every retrieval intervention we test, establishing a ceiling for retrieval-only improvements. The gap’s severity varies across benchmarks and question types, but extraction failures appear on every dataset we measure. Our findings reveal that retrieval failures and extraction failures are fundamentally different bottlenecks requiring different solutions — a distinction absent from current evaluation practice.

Abstract:
Large language models (LLMs) have demonstrated remarkable performance across various code-related tasks. However, unlike carefully curated datasets that are typically high-quality and error-free, real-world user instructions are often vague and error-prone, posing significant challenges to the robustness of code LLMs. Furthermore, robustness-oriented fine-tuning relies on paired clean-noisy samples, which are costly to curate and require sophisticated noisy simulation techniques. To address these challenges, we propose the Noisy Test-time Reinforcement Learning framework (NTRL-Code), which enables robust self-evolution of code LLMs using only unlabeled noisy data during the testing stage. Specifically, NTRL-Code uses conservative self-denoising to obtain a cleaner semantic anchor for target estimation, and employs an abstract-syntax-tree (AST)-based structural aggregation mechanism to estimate a proxy target from multiple candidate programs. The policy is then optimized on the original noisy prompts with a hybrid reward that combines format validity, code similarity, and anti-repetition signals. Extensive experiments on three benchmarks, each incorporating character-level, word-level, and paragraph-level perturbations, demonstrate that NTRL-Code yields robust and consistent improvements, stabilizing the predictions of various base models. Our code is available at https://github.com/Xikai97/NTRL-Code.

Abstract:
Large Language Models (LLMs) inevitably internalize substantial amounts of sensitive or private information during pre-training, while LLM unlearning aims to selectively erase specific knowledge to prevent privacy leakage with minimal loss of model utility. However, existing methods struggle to balance forget quality with utility, and typically incur substantial computational costs due to parameter fine-tuning. To address this, we propose Nullify, a training-free, non-destructive activation steering method for LLM unlearning. Nullify employs steering vectors during inference to redirect privacy-related activations away from their memorized answers, while satisfying a null-space constraint that leaves retained-query activations essentially unaffected to maintain utility. Evaluations on benchmarks including TOFU and MUSE show that Nullify matches or surpasses established baselines in forget quality while achieving near-lossless preservation of model utility. By avoiding weight updates entirely, Nullify serves as an efficient, plug-and-play inference-time intervention framework.

Abstract:
As Large Language Models (LLMs) increasingly consume their own outputs through retrieval-augmented generation and synthetic data pipelines, self-referential loops are becoming prevalent in modern information ecosystems. While prior work has studied variance collapse during recursive training, the dynamic consequences for political diversity during inference remain underexplored. We investigate this by conducting iterative paraphrasing experiments across eight widely used LLMs, and formalize self-referential generation as a discrete-time Markov process to characterize its long-term behavior. Our results reveal a systematic contraction of political diversity, with several models converging toward center-dominant equilibria. Notably, this homogenization is strongly front-loaded: the majority of diversity loss occurs within the earliest iterative rounds, indicating that even a few AI-mediated transformations can disproportionately narrow the political spectrum. We further examine two interventions: Stance-Conditioned Generation (SCG) substantially reduces marginal stance-distribution drift even without ground-truth labels, leaving no statistically detectable shift for three of the four homogenizing models, whereas Self-Identified Stance Preservation (SISP) overshoots into over-polarization; even SCG improves but does not guarantee instance-level stance retention. These findings highlight a structural risk associated with self-referential generation in AI-driven information pipelines. Our code and prompts are publicly available at https://github.com/timothy-coshin/SelfReferentialPoliticalDiversity.

Abstract:
LLM-based translation agents can plan, review, and revise translations, but most existing systems still rely on a single hand-crafted instruction set shared across inputs. This is suboptimal for translation, where texts vary widely in domain, context, style, and cultural grounding, leading to heterogeneous and input-specific errors. We propose STEINS, a self-evolving framework that treats the instruction set governing agent roles, translation strategies, and interaction constraints as an optimization target. During optimization, STEINS performs structured error diagnosis and uses the resulting evidence to refine the agent specification in a targeted and interpretable manner. The framework supports both offline corpus-level evolution and online instance-level adaptation at test time. Across WMT23 (general-domain, segment-level) and LITEVAL-CORPUS (literary, paragraph-level), and across four agentic translation architectures, offline evolution yields modest but generally positive corpus-level gains, while online instance-level adaptation yields larger and more consistent improvements, with macro-averaged XCOMET gains of +1.28 to +3.49 over 12 settings regardless of whether adaptation starts from the original or offline-evolved specification. Overall, our findings identify instance-level specification adaptation as the key ingredient in effective agentic machine translation.

Abstract:
Most prior works focused on conflicts between an LLM’s internal parametric knowledge and externally provided context. In contrast, we investigate how LLMs handle conflicts that arise within contextual knowledge itself. We introduce a taxonomy of six types of contextual conflicts (factual, inferential, temporal, granularity, perspective, and ambiguity) and contribute a comprehensive dataset ContextConflict for this setting. The dataset contains 5,781 samples, covers both reasoning and summarization tasks, and includes both explicit contradictions and implicit conflicts that require multi-step reasoning. Experiments on nine LLMs show that current models still fall short in resolving contextual knowledge conflicts. We further provide mechanistic interpretability insights into how LLMs process such conflicts, revealing their latent awareness of conflicts and the representational geometry underlying conflict processing. In addition, our analysis uncovers a consistent model bias towards earlier evidence, and this positional preference serves as a key obstacle to effective conflict resolution. Motivated by these findings, we further propose a simple training-free, label-free steering method that steers activations to encourage a more comprehensive incorporation of evidences for better conflict resolution. On our dataset, the method consistently improves accuracy on reasoning tasks and generates higher-quality, more balanced summaries for summarization tasks.

Abstract:
Large language models often solve tasks from a fully specified prompt but degrade when the same requirements unfold over multiple turns, known as the lost-in-conversation (LiC) gap. We trace part of this degradation to self-contamination: intermediate assistant replies enter later context and carry early deviations forward. Motivated by this mechanism, we propose MAIGO, an on-policy self-distillation method that reduces this contamination using history-cleaned references from the model’s own policy. For middle turns, MAIGO removes prior assistant replies while preserving the user-visible sharded prefix; for answer turns, it distills from paired full-view references conditioned on the completed user-side dialogue. A reliability weight downweights middle-turn samples that disagree with the clean reference. MAIGO requires no verifier rewards, state labels, or inference-time scaffolding. Under the LiC paired-view protocol with deterministic verifiers, MAIGO improves Qwen2.5-7B-Instruct SHARDED accuracy from 52.8 to 66.1 and the SHARDED/FULL ratio from 66.5% to 84.1%, while keeping FULL accuracy within 2.3 points. These results show that self-contamination is a trainable component of the LiC gap.

Abstract:
Multi-table fact-checking verifies claims whose evidence is distributed across multiple tables and connected through multi-hop reasoning. This setting is challenging because claims may contain chained or nested dependencies, table retrieval is noisy, and retrieved evidence must be selected and composed before a final verdict can be made. We introduce two multi-table fact verification testbeds, OTT-FV and Hybrid-FV, and propose TRACE, a tree-structured framework that connects claim decomposition, evidence retrieval, and evidence composition. TRACE represents each claim as a dependency-aware semantic tree, retrieves locally relevant table evidence for each node, and composes evidence-aware node representations bottom-up to produce the final verdict. Experiments on TabFact, OTT-FV, and Hybrid-FV show that TRACE consistently improves over strong baselines across multiple backbones under both gold- and open-evidence settings. Further analyses show that tree construction, node-level retrieval, and bottom-up evidence composition all contribute to multi-table verification.

Abstract:
Multi-constraint instruction following requires a model to respond to a query under many simultaneously active constraints. Even strong instruction-tuned models still routinely violate some of them. Existing approaches either augment supervision with sequence- or token-level RL rewards from external verifiers or learned graders, or use on-policy distillation (OPD) against a single full-context teacher whose probability mass becomes diluted as more constraints become simultaneously active. We propose CC-OPD (Counterfactual Constraint-Conditioned On-Policy Distillation), which inverts the standard supervision-generation direction in distillation. Rather than enriching the teacher with information beyond what the student sees, CC-OPD ablates each constraint from the teacher’s conditioning in turn, and constructs the per-constraint signal from the resulting per-token probability differentials. The resulting per-token leave-one-out log-likelihood shifts are summed, clipped, and added to the vanilla OPD reward as a token-level shaping term. All shaping terms are obtained from the frozen teacher, without an external verifier during distillation, and the reward equals vanilla OPD wherever the aggregate shift is zero. Across two Qwen model pairs and seven benchmarks, CC-OPD achieves the highest average among all evaluated student-training methods. A 1.5B student trained with CC-OPD surpasses its own 7B RL-trained teacher on the MulDimIF benchmark.

Abstract:
As LLMs are increasingly used to draft public-facing arguments, they may flatten public debate by repeatedly introducing the same polished, plausible arguments. We study argument collapse, the tendency of essays generated by different LLMs to converge to a smaller set of main arguments, sub-arguments, and paragraph-level structures. We compare 1,039 human responses from 195 New York Times (NYT) debates, 448 human responses from 61 longer-form Boston Review (BR) forums, and 23,384 LLM-generated essays. In the NYT corpus, 65.3% of human main arguments are unique within a debate, compared to 3.4% of LLM main arguments. Asking LLMs to generate diverse answers adds variation, but a typical model recovers only about half of the distinct human main arguments, with much of the added variation falling outside the observed human argument space. Collapse also appears in sub-arguments, where among essays with the same main argument, 41.0% of human sub-arguments are unique versus 9.1% from LLM responses. Qualitatively, LLMs often reuse generalized and hedged sub-arguments, while humans prefer more concrete and topic-specific ones. Structure-wise, LLM-generated essays tend to follow a more fixed arc, often opening with a direct claim and moving quickly toward proposals. The same patterns hold in longer BR essays, suggesting that argument collapse extends beyond short-form responses.

Abstract:
In retrieval-augmented generation (RAG) question answering systems, generating citations for large language model (LLM) outputs enhances verifiability and helps users identify potential hallucinations. Existing citation generation methods for retrieval-augmented generation usually provide evidence at the paragraph or sentence level. Although such citations improve verifiability, they may still impose unnecessary reading burden on users: a cited sentence can contain irrelevant clauses, while an answer-bearing clause may lack sufficient contextual information such as the referred subject. In this paper, we study concise and sufficient sub-sentence citation generation, aiming to identify minimal yet self-contained evidence spans for answer verification. To this end, we first develop annotation guidelines for such citations and construct a corresponding dataset. To reduce annotation cost, we propose a data augmentation framework that generates synthetic citation annotations and filters low-quality instances using an evaluator model. Our experiments on the constructed dataset demonstrate that the proposed approach can generate high-quality and more readable citations. The code and data will be publicly released.

Abstract:
Whether attention weights faithfully reflect model reasoning has been actively debated in NLP, yet this question remains largely unexplored for the visual modality in Vision-Language Models (VLMs). We address this gap through causal perturbation analysis on current VLMs, evaluating both the comprehensiveness and sufficiency gap of attention-ranked visual tokens. Our analysis reveals that visual attention faithfulness is heterogeneous, manifesting in three distinct processing modes: Faithful-Sufficient, where top- k attention tokens are both necessary and sufficient for prediction; Faithful-Distributed, where they are necessary but broader visual context remains required; and Non-Focal, where no localized attention region is individually necessary while visual information remains an essential trigger for prediction. Furthermore, human-annotated ground-truth regions satisfy comprehensiveness in only ∼60 % of cases compared with model attention rankings, revealing systematic divergence between model visual reliance and human intuition. We demonstrate these patterns across both general VQA on VQAv2 and document tasks on VRDU and ChartQA, showing that visual attention faithfulness varies systematically with processing demands and model architectures rather than being uniformly faithful or unfaithful.

Abstract:
Modern multimodal generative models can synthesize visually compelling images and videos, but it remains unclear whether this visual fluency reflects genuine reasoning: when prompted to generate a solution, can a model preserve the physical, logical, spatial, and temporal constraints a task requires, or does it merely produce plausible-looking media? To answer this question, we introduce MMGR (Multi-Modal Generative Reasoning Benchmark and Evaluation), a benchmark for evaluating generative reasoning across video, image, and language-based systems. MMGR covers 10 tasks from three domains (Abstract Reasoning, Embodied Navigation, and Physical Commonsense) and probes five reasoning abilities: Physical, Logical, 2D Spatial, 3D Spatial, and Temporal. Its evaluation emphasizes answer-verifiable tasks and, for video generation, process-aware chain-of-frame reasoning, where intermediate frames must form valid steps toward the target outcome rather than visually smooth but incorrect transitions. Evaluating state-of-the-art video generators, image generators, and LLM/VLM baselines reveals a sharp gap between visual quality and reasoning correctness: video models perform best on Physical Commonsense, but remain weak on symbolic tasks such as Sudoku, ARC, and Math, and brittle in cross-view embodied navigation. Image generators often outperform video generators on embodied navigation despite lacking temporal outputs, showing that longer visual generation does not automatically yield stronger reasoning. MMGR reframes evaluation of multimodal generation from whether outputs look realistic to whether they solve the underlying reasoning problem.

Abstract:
Data annotation remains a central bottleneck in natural language processing, requiring human effort to obtain high-quality labels at scale. While Large Language Models (LLMs) offer a fast and cost-effective alternative, their reliability is highly instance-dependent: they perform well on simple inputs but often fail on examples requiring nuanced reasoning or contextual understanding. In this work, we address this challenge with QUORUM (QUality-Optimized Routing Using Multiple annotators), a budget-aware routing framework that dynamically assigns each instance to human or LLM annotators under a fixed annotation budget. Unlike prior approaches relying on model confidence or uncertainty estimates, QUORUM leverages feature-based signals to estimate instance difficulty and supports multiple annotations per instance, combining them through agreement-based rewards to improve reliability. We evaluate QUORUM across diverse closed- and open-ended annotation tasks in English and multilingual settings, and QUORUM improves annotation quality by up to 34.4% while reducing costs by 8.8% over competing methods. Code can be found at https://github.com/amazon-science/QUORUM.

Abstract:
Knowledge distillation effectively transfers reasoning capabilities from large language models to lightweight student models. To enable knowledge transfer across disparate model families, researchers increasingly explore cross-tokenizer distillation. However, cross-tokenizer distillation remains challenging due to vocabulary and sequence misalignment, while approximate vocabulary alignment can introduce additional noise into distillation. To address these challenges, we propose Anchor-Based Cross-Tokenizer Distillation with Residual Regularization (ACTD). ACTD bridges structural heterogeneity through vocabulary and sequence alignment, while mitigating alignment noise via a novel anchor loss with residual regularization. We further extend this framework to a multi-teacher setting. Evaluated across five reasoning benchmarks with three distinct teacher models, ACTD achieves state-of-the-art performance. Moreover, its multi-teacher extension outperforms the strongest single-teacher and multi-teacher baselines, further demonstrating the robustness of our method.

Abstract:
Continual post-training enables models to absorb emerging knowledge after deployment, but repeatedly updating shared parameters inevitably accumulates weight drift, which causes catastrophic forgetting and degrades general capabilities. Retrieval-augmented generation avoids such parameter drift, yet often lacks the depth of parametric knowledge integration. In this paper, we propose ReGrad (Retrievable Gradients), a new paradigm that treats gradients as retrievable units of knowledge. ReGrad pre-computes document-specific gradients offline, stores them in an indexed Gradient Bank, and retrieves only query-relevant gradients at inference time for temporary weight adaptation. However, raw language-modeling gradients are optimized for token-level document reconstruction rather than for query-driven knowledge use. We therefore introduce a bi-level meta-learning objective that reshapes document-derived gradients into generalizable adaptation signals for downstream tasks. Experiments across general and domain-specific settings show that ReGrad outperforms CPT and RAG baselines, enabling scalable and reversible parametric knowledge injection without accumulating weight drift.

Abstract:
When LLM-generated emergency advice becomes part of public-facing information infrastructure, aggregate accuracy is no longer an adequate safety target: missing a hazardous instruction and blocking a correct one impose different human costs. We study protocol-grounded verification, the post-hoc auditing of emergency instructions against authoritative guidance under asymmetric risk. We introduce EMER-GuardBench, a 69,000-example counterfactual benchmark plus a 1,000-example benign-stress set spanning 10 disaster scenarios and 6 hazard mechanisms, with instruction-group-disjoint splits and cited protocol evidence. We further propose EGIS, an evidence-conditioned, protocol-supervised verifier that jointly predicts hazard status, hazard mechanism, severity, evidence support, and counterfactual ranking. This establishes a protocol-conditioned task and protocol-supervised representation learning, not inference-time causal grounding. EGIS reaches 89.9 Harm F1 with 10.4% IID over-refusal and zero benign-stress false positives; across 364 LLM-judge operating points, none Pareto-dominates EGIS on Harm F1, over-refusal, and severity-weighted miss rate, and a 14.4×-larger Qwen3-8B+LoRA supervised baseline still trails under identical splits. Ablations show that evidence helps as training-time supervision rather than prompt decoration. Our Severity-Weighted Miss Rate exposes a tradeoff hidden by scalar safety scores: zero missed severity can be achieved by broad refusal. EMER-Guard reframes emergency LLM safety from advice generation to auditable, protocol-grounded, cost-sensitive verification.

Abstract:
The goal of news image captioning is to generate captions by integrating news article content with corresponding images, highlighting the relationship between textual context and visual elements. The majority of research on news image captioning focuses on English, primarily because datasets in other languages are scarce. To address this limitation, we create the first large-scale multilingual news image captioning benchmark, LaMuN, comprising more than 800,000 news images in 59 languages, including several low-resource languages. We further propose a novel chain-of-thought method for automatic news image captioning that achieves state-of-the-art results across a diverse set of languages in LaMuN. Together, the benchmark and the proposed method open new avenues for further advances in developing and evaluating multilingual multimodal large language models.

Abstract:
The alignment of Large Language Models heavily relies on English-centric high-quality preference data, which often leads to suboptimal performance in other languages. In this paper, we propose Cross-lingual Ranking Preference Optimization (CRPO), a novel framework that leverages robust preference knowledge from English to facilitate preference alignment in the target language. We design a hierarchical structure within parallel preference pairs across the target language and English to jointly optimize intra- and inter-lingual preferences, thereby enhancing language adaptation and output quality. Building on the LambdaLoss framework, CRPO goes beyond the binary comparison based optimization by providing a relative ranking signal across multiple candidate responses. Our experiments across five languages with varying resource scales demonstrate that CRPO consistently outperforms standard approaches in both instruction-following and knowledge utilization capability. Notably, the robust performance gains observed across various weighting schemes further validate the empirical effectiveness of our hierarchical design in a multilingual setup. Furthermore, our findings highlight that CRPO significantly improves both reward margins and the log-probability of desirable responses, contributing to a more stable preference manifold for cross-lingual alignment.

Abstract:
Syntactic language models (SLMs) that generate sentences along with their syntactic tree structures have shown promise in both syntactic generalization and downstream tasks. Prior studies, however, have mostly evaluated smaller models trained on less pretraining data and tested on narrower task suites. In addition, the design space of tree linearization and attention masks remains underexplored. We conduct a large-scale empirical study of 12 controlled configurations—one terminal baseline and 11 SLM variants spanning different linearizations and masks—at scales up to 1B parameters and 100B training tokens, evaluated across 11 downstream tasks and two syntactic benchmarks. The experimental results show that SLMs with standard causal attention match or exceed structural-mask variants across downstream and syntactic benchmarks. They require no architectural modifications and remain compatible with FlashAttention. We also observe that when scaling up to the 1B-parameter scale, the top-performing SLM variant no longer shows a clear advantage over a compute-matched non-structural baseline, suggesting that SLMs may become less competitive with scale. Our code is available at https://github.com/qlwpc/TG-Interpolation.

Abstract:
Knowledge-Editing-based (KE-based) detoxification has emerged as a promising approach for mitigating harmful behaviours in Large Language Models. Existing evaluations, however, largely rely on automatic toxicity classifiers, implicitly assuming that reduced toxicity scores reflect genuine behavioural suppression. In this work, we propose a robustness-oriented evaluation framework for KE-based detoxification that examines its reliability beyond standard classifier-based metrics along three dimensions: optimisation robustness, compositional robustness, and cross-lingual robustness. We identify pseudo-detoxification as a common failure mode, where apparent toxicity reductions arise from degenerate generation behaviours rather than meaningful suppression of unsafe content. We further show that detoxification effectiveness degrades when multiple unsafe behaviours are edited jointly, and that both monolingual and cross-lingual detoxification remain effective only under specific model–method combinations. Overall, our results indicate that KE-based detoxification is robust only for certain models, limited numbers of detoxification objectives, and a subset of languages.

Abstract:
Large language models (LLMs) have recently been employed as agents to solve sequential decision-making tasks such as Bayesian optimization and multi-armed bandits. These works usually adopt an LLM for sequential action selection by providing it with a fixed, manually designed meta-prompt. However, numerous previous works have found that the prompt has a significant impact on the performance of the LLM, which calls for a method to automatically optimize the meta-prompt for LLM-based agents. Unfortunately, the non-stationarity in the reward observations during LLM-based sequential decision making makes meta-prompt optimization highly challenging. To address this challenge, we draw inspirations from adversarial bandit algorithms, which are inherently capable of handling non-stationary reward observations. Building on this foundation, we propose our EXPonential-weight algorithm for prompt Optimization (EXPO) to automatically optimize the task description and meta-instruction in the meta-prompt for LLM-based agents. We also extend EXPO to additionally optimize the exemplars (i.e., history of interactions) in the meta-prompt to further enhance the performance, hence introducing our EXPO-ES algorithm. We use extensive experiments to show that our algorithms significantly improve the performance of LLM-based sequential decision making.

Abstract:
Large language models (LLMs) are increasingly used to simulate human survey responses and behavioral reactions, yet unreliable simulations can mislead social science conclusions. However, existing evaluations focus on end-to-end scores, leaving it unclear how different aspects of the simulation process interact to determine reliability. We propose ReliMap, which decomposes LLM-based human behavior simulation into three structured layers and evaluates reliability at both the individual level (R1) and population level (R2) across three configuration dimensions: model capacity, profile completeness, and population coverage. Through experiments across four simulation tasks and eleven LLMs, we find that all models exhibit substantial distributional bias without profile conditioning. Profile conditioning reduces this bias with diminishing returns. Larger models benefit more, and attribute informativeness matters more than quantity. Critically, R1 gains do not reliably transfer to R2—individual and population-level reliability can move in opposite directions. At the population layer, increasing coverage reduces variance but not systematic bias, with R2 stabilizing at around 50–100 individuals. These findings highlight that reliable simulation cannot be achieved by optimizing any single layer in isolation, but requires coordinated improvement across all three.

Authors: Belen Alastruey, Niyati Bafna, Andrea Caciolai, Kevin Heffernan, Artyom Kozhevnikov, Christophe Ropers, Eduardo Sánchez, Charles-Éric Saint-James, Ioannis Tsiamas, Xiang Tony Cao, Chierh Cheng, Joe Chuang, Paul-Ambroise Duquenne, Mark Duppenthaler, Cynthia Gao, Pere-Lluís Huguet Cabot, João Maria Janeiro, Jean Maillard, Gabriel Eduardo Mejia-Gonzalez, Holger Schwenk, Edan Toledo, Arina Turkatenko, Albert Ventayol-Boada, Ravi Arikapudi, Nathanial Paul Ekberg, Rashel Moritz, Alexandre Mourachko, Surya Parimi, Mary Williamson, Shireen Yates, David Dale, Marta R. Costa-jussà

Abstract:
We introduce Omnilingual MT, a family of multilingual translation (MT) systems covering more than 1,600 languages—the broadest benchmarked coverage of any MT system to date. We make contributions in several key research questions in the field: strategies for the use of monolingual data, curating seed training data, tokenization and suitable architectures, the competitiveness of small specialized MT models as opposed to general purpose large models, and evaluation protocols. Our data strategy combines large-scale multilingual corpora with backtranslation for 2,000+ languages and MeDLEy, a grammatically-diverse seed dataset for 109 low-resource languages. We explore two complementary architectures: OMT-LLAMA, a decoder-only model (1B/3B/8B) built on LLAMA3 with multilingual continual pretraining, and OMT-NLLB, a 3B encoder-decoder model with a novel three-stage training strategy exploiting non-parallel data via autoencoding. For evaluation, we extend the BOUQuET benchmark to 275 languages, develop Met-BOUQuET with human annotations of machine quality in 161 directions, and BLASER 3, a reference-free quality estimation metric. Our specialized 1B–8B models match or exceed 70B general-purpose LLMs, demonstrating that specialization is a more reliable path to multilingual translation than scale alone. Omnilingual MT achieves “passable” translation quality as judged by human annotators for over 440 languages, doubling it over the previous state of the field at ∼200 languages, and translates in zero-shot mode into ∼1,200 languages versus 300–400 for the baselines. With Omnilingual MT, we showcase a path encompassing the full pipeline of data and system development towards designing models that are multilingual from the onset.

Abstract:
Sound symbolism, the non-arbitrary mapping between word sounds and meanings, is well known from effects like Bouba–Kiki. But its cross-linguistic scope is difficult to assess because phonological similarity can also arise from shared ancestry, contact, or language-specific inventories. We propose a genealogy-aware stress test for size sound symbolism across 27 languages spanning 13 language families, using 810 phonemically transcribed size adjectives validated with native-speaker audio. We treat cross-family iconicity as a hard generalization task: train on languages grouped by lexical distance to a held-out target, test transfer from increasingly distant languages, and apply adversarial pressure against similarity-bin and source-language cues. Interpretable bag-of-phoneme classifiers predict size above chance, with clear patterns involving /i/ for smallness and /o,a/ for largeness, while consonants add more language-specific cues. Adversarial training prevents the jointly trained model from exploiting genealogy while held-out size prediction stays above chance, though post-hoc probing shows language identity is still recoverable. The residual signal is small but consistent, indicating cross-family size cues.

Abstract:
Text-to-SQL models may exhibit recurring error generation patterns that are reproducible under a fixed model and difficult to eliminate through continued training. Existing approaches either perform post-hoc correction at the output level or require access to the target domain’s training data for preference optimization, neither of which directly targets these reproducible local decision tendencies. To address these limitations, we propose Targeted Token-level Unlearning for SQL (TTU-SQL), a machine-unlearning-based framework that uses a probe dataset to detect and correct systematic errors without requiring access to the model’s original training data. The framework comprises three components: (1) AST-based token-level divergence detection that precisely localizes erroneous decision points; (2) a targeted error-type extraction procedure that selects three specific types of forgettable errors based on AST divergence and logit-level behavior; and (3) Part-NPO, which applies negative preference optimization only to localized divergence tokens to suppress erroneous continuations while preserving useful SQL generation knowledge. Across the Spider and BIRD datasets and all evaluated model backbones, TTU-SQL yields consistent execution-accuracy gains over equal-budget training-based repair baselines.

Abstract:
We argue that multi-document reasoning is constrained not only by how much text a model can read, but also by how a limited query-time evidence budget is allocated across documents and semantic granularities. Full-context inference exposes the model to broad evidence non-selectively and at high per-query cost, while flat chunk retrieval often returns locally relevant passages that are weakly organized for cross-document synthesis. We present BEAR , a framework for structured evidence allocation that builds hierarchical semantic indices offline and performs coarse-to-fine evidence access at query time through complementary exploration and recovery paths. This design formulates coarse-to-fine reasoning as structured evidence allocation under a fixed final evidence-context budget. Across nine evaluation settings spanning synthetic evidence allocation, multi-hop QA, agentic retrieval, conversational memory, long-document QA, and open-domain QA, BEAR delivers strong coverage–faithfulness trade-offs on DragonBall, achieves the best retrieval-based results on 2WikiMultiHopQA, LoCoMo, QASPER, MuSiQue, and PopQA, and remains competitive on HotpotQA and NarrativeQA. Under the matched retrieval protocol, BEAR operates with an approximately 4K-token final evidence context, substantially smaller than the inputs used by the long-context references. These results show that evidence exposure—not context length alone—should be selectively organized, with gains arising from hierarchy as an allocation substrate and complementary exploration and recovery rather than semantic chunking alone.

Abstract:
Interactive travel planning has become a popular use case for language models. Agents are deployed to manage evolving preferences and unexpected disruptions over multiple turns. Such settings require models to make complex, profile-conditioned planning decisions. However, existing benchmarks often evaluate feasibility, personalization, or interaction in relatively isolated settings. We therefore introduce Trip+ to measure the ability of agents to plan travel holistically. In Trip+, given traveler profiles and dynamic interactions, agents must generate and revise minute-level itineraries. End-to-end traveler experiences are evaluated via LLM-based simulators, enabling the assessment of subjective metrics like fatigue. Our scenarios range from simple request resolutions to complex environment-driven replanning. We evaluate 18 LMs and find a consistent gap in experiential quality. Models favor technically feasible but exhausting itineraries that show weaker profile alignment.

Abstract:
Despite significant advances in aligning Large Language Models (LLMs) with safety and ethical standards, adversarial prompts can still be crafted to elicit undesirable responses. Existing mitigation strategies predominantly rely on post-hoc filtering, which introduces substantial latency or computational overhead, and is incompatible with token-level streaming generation. To address this, we introduce Self-Sanitize, a novel LLM-driven mitigation framework inspired by cognitive psychology, which emulates human self-monitor and self-repair behaviors. The framework utilizes a lightweight Self-Monitor module to inspect high-level intentions via representation engineering and a Self-Repair module for in-place correction without separate review dialogues. This design allows for real-time streaming monitoring and seamless repair, with negligible impact on latency and resource utilization. Extensive experiments across three privacy-leakage scenarios and broader safety-sensitive domains (e.g., crime and bias) demonstrate that Self-Sanitize achieves superior mitigation performance with minimal overhead and preserved model utility. Our code is available at the following link: https://anonymous.4open.science/r/LLM_Self_Sanitize-5E8B

Abstract:
Multilingual code language models transfer skills across programming languages (PLs), but whether any PL occupies a privileged pivot position remains contested: geometric analyses point to C-family languages and Go, while behavioral evidence highlights Python. We revisit this question under controls for representational anisotropy and length variation across PLs, two confounds that compromise prior cosine-based analyses. Across three code models on multilingual competitive-programming data, we study three views of cross-PL organization: pairwise PL geometry, PL-English alignment, and pivoted retrieval through candidate PL representation spaces. The results are relation-dependent. Code-code geometry reveals structured language regions but no universal center; code-English alignment favors high-level scripting languages; and pivoted retrieval favors different intermediate spaces for code-to-code and English-to-code transfer. These findings suggest that Python’s special role is better understood as English-facing affinity than as universal geometric centrality.

Abstract:
Comprehensively evaluating AI agents across interactive environments is difficult due to fragmented tasks, scaffolds, verifiers, and scoring rules. Unfortunately, existing efforts to unify these evaluations are limited in scale and domain, making costly reruns necessary and leaving available data incomparable. We introduce MESSIER, a unified corpus of 957,611 records spanning 30 benchmarks, 745 agents, 11,891 tasks, and 74,263 verifiers. MESSIER combines public evaluation results with new runs on six underrepresented professional and scientific benchmarks, standardizing their heterogeneous components into a common schema. Using this corpus, we show that frontier progress is uneven across benchmark groups, with function-calling evaluations largely saturated, programming improving fastest, and enterprise workflows remaining most challenging. Counterfactual rescoring further shows that strict all-pass scoring in multi-verifier tasks can alter agent rankings. Finally, we derive capability scores from our corpus that correlate with Epoch’s Evaluation Capability Index rankings at Spearman ρ = 0.84. The scores can also be estimated for subsets defined by domain, occupation, action space, or verifier type. In essence, MESSIER is a reusable resource for studying agent performance at scale, and a basis for designing better evaluations.

Abstract:
Physics reasoning requires constructing a consistent model of the underlying physical system rather than relying solely on symbolic or formula-based manipulation. Although large language models have shown strong ability in solving math and coding problems, they still struggle with physics problems, as these problems entangle the physical modeling process with mathematical calculations. Humans approach physics by first building a representation of the system before performing calculations. Inspired by this, we introduce a unified framework that distills intermediate representations that explicitly encode the physical modeling process and adopt a two-stage post-training strategy, where supervised fine-tuning establishes structured modeling, and reinforcement learning with rubric-based feedback improves the quality of the modeling process. Experiments on multiple multimodal physics benchmarks show that our approach generally improves physical reasoning performance across different models and datasets. Across PhysReason, PhyX, and SeePhys, physical modeling outperforms GRPO by ~3% on average. showing that explicit physical modeling is an effective strategy for improving physics reasoning in small VLMs.

Abstract:
Large language models (LLMs) aligned for safety often suffer from over-refusal, incorrectly rejecting benign yet safety-related instructions. Prior studies primarily attribute this to static representation overlap, largely overlooking the underlying dynamic mechanisms. In this paper, we present the mechanistic analysis of over-refusal through the lens of internal routing conflicts within transformer attention. We discover that a sparse subset of Hypersensitive Safety Heads misfires on Hard-Safe prompts, exhibiting abnormal attention entanglement that forcefully binds harmless target entities to refusal semantics. This triggers a severe, high-entropy routing conflict that deprives target entities of necessary attention. To counteract this, we propose Semantic Routing Calibration (SRC), a lightweight, training-free inference framework. SRC precisely localizes and dynamically suppresses these hypersensitive safety heads at the inference stage. Coupled with a dual-branch logits fusion that acts as a safety regularizer during subsequent decoding, SRC seamlessly restores trustworthy reasoning. Extensive experiments demonstrate that SRC alleviates over-refusal, with intrinsic safety performance preserved as much as feasible.

Abstract:
Sparse retrieval underpins modern search systems, from web search to retrieval-augmented generation. Existing work has introduced Learned Sparse Retrieval (LSR) to push beyond exact lexical matching toward richer semantics. Yet LSR has so far remained tied to encoder-style bidirectional architectures, and its extension to multimodal settings still relies heavily on auxiliary cross-modal modules. To address these limitations, we introduce UEmbed (Unified Embedding), a decoder-only multimodal embedding model that produces both sparse lexical and dense representations in one causal forward pass. UEmbed appends N learnable special tokens to the input and partitions the vocabulary into N disjoint subsets. Each token’s causal hidden state predicts sparse weights over its assigned subset, and the N subsets are concatenated into the full sparse vector. Trained on public data, we release UEmbed at 2B, 4B, and 9B scales. UEmbed-9B reaches 71.8 (dense) and 71.0 (sparse) on MMEB-v2, outperforming multimodal embedding models trained on publicly available data (e.g., RzenEmbed). On BEIR, UEmbed also remains competitive with strong dense and sparse baselines. Furthermore, we demonstrate the practical utility of UEmbed across three dimensions: effectiveness, efficiency, and agentic applications. Overall, UEmbed offers a new paradigm: it unifies dense and sparse embeddings in one model, while further extending sparse retrieval to unify text and multimodal inputs.

Abstract:
Effective collaboration in multi-agent systems requires mechanisms for coordination and adaptability in dynamic and uncertain environments. However, existing multi-agent systems rarely model resilience explicitly, instead relying on ad-hoc output correction that limits systematic, runtime management of robustness. In this work, we propose Multi-Agent Mesh (MA-Mesh), a framework that constructs a role-based self-healing layer on top of a collaboration pattern. Specifically, MA-Mesh generates resilience-aware system specifications and synthesizes them into executable multi-agent policy code. To enhance resilience, we introduce two self-healing mechanisms: Cross-agent reflection, a mechanism that allows agents to evaluate not only their own outcomes but also the performance of their peers, enabling informed behavioral adjustments and collective policy refinement; Role-based failover, a strategy which dynamically reassigns roles and enables agents to substitute for one another when sub-tasks fail or agents underperform, ensuring continuity of task execution. Experimental results across diverse tasks (e.g., software development, tool use, scientific reasoning) demonstrate that MA-Mesh significantly outperforms existing multi-agent systems in both task performance and robustness under system uncertainty.

Abstract:
Mixture-of-Experts (MoE) has emerged as a promising paradigm for efficiently scaling large language models without a proportional increase in computational cost. However, the standard training strategy of Top-K router prevents MoE models from realizing their full potential for elastic inference. When the number of activated experts is altered at inference time, these models exhibit precipitous performance degradation. In this work, we introduce Matryoshka MoE (M-MoE), a training framework that instills a coarse-to-fine structure directly into the expert ensemble. By systematically varying the number of activated experts during training, M-MoE compels the model to learn a meaningful ranking: top-ranked experts collaborate to provide essential, coarse-grained capabilities, while subsequent experts add progressively finer-grained detail. We explore this principle at multiple granularities, identifying a layer-wise randomization strategy as the most effective. Our experiments demonstrate that a single M-MoE model achieves remarkable elasticity, with its performance at various expert counts closely matching that of an entire suite of specialist models, but at only a fraction of the total training cost. This flexibility not only unlocks elastic inference but also enables optimizing performance by allocating different computational budgets to different model layers. Our work paves the way for more practical and adaptable deployments of large-scale MoE models.

Abstract:
Many strong decoding methods for text generation follow a sample-and-rerank paradigm: they draw multiple candidates, score each under a utility (reward) function using consensus across samples, and choose the best one. Although effective, these methods incur high computational costs during inference due to repeated sampling and scoring. Prior attempts to amortize inference-time computation typically rely on gold references, teacher labels, or curated preference data, thereby increasing the effort required to construct datasets and the demand for high-fidelity reward models. We propose Consensus Group Relative Policy Optimization (C-GRPO), which distills Minimum Bayes Risk (MBR) decoding into training by formulating the consensus utility as a group-relative objective within GRPO. C-GRPO requires only a utility function and policy samples, without gold references or explicit preference labels. As supporting analysis, we study an idealized sentence-level surrogate and show that the expected update is directionally consistent with the expected-utility objective underlying MBR decoding under simplifying assumptions. Experiments on machine translation (WMT 2024), text summarization (XSum), and Japanese social-bias mitigation (JBBQ) show that C-GRPO often matches or exceeds inference-time MBR decoding while replacing multi-sample reranking with a single generation.

Abstract:
Recent topic models leverage pretrained embeddings, but neural architectures produce latent representations without grounding in specific texts, and clustering-based pipelines assign representative documents only post hoc, relying on absolute distances distorted by hubness and anisotropy in high-dimensional spaces. We introduce MARETopic, a training-free framework that casts topic discovery as rank-based prototype selection. After projecting embeddings onto a low-dimensional manifold, MARETopic builds ranked lists encoding ordinal neighborhood structure. A greedy algorithm selects exactly K exemplar documents, real corpus texts, whose neighborhoods cover the corpus. Two variants share this criterion. MARETopic_Corr scores candidates with a query performance predictor and a rank correlation measure, leading Purity and NMI on the two benchmarks with the most categories, ahead of both neural and clustering-based topic models. MARETopic_Diff scores them with a rank-based diffusion matrix, needs neither measure, and runs 1.7 to 1.9 times faster. Without a single gradient update, MARETopic leads topic coherence on two of three datasets. A novel inter-topic Maximal Marginal Relevance step raises vocabulary diversity at little cost in coherence. Our code is available at https://github.com/thcastilho/maretopic.

Abstract:
Multimodal protein language models (pLMs) learn joint protein sequence-structure distributions, and their generation performance should also depend critically on inference-time sampling strategies. Yet prior work has focused more on model training than on how inference-time strategies behave. In this paper, we establish a three-stage investigation framework to empirically study the inference design space of multimodal pLMs across three representative pLMs and four fundamental tasks. We evaluate vanilla sampling, task-specific classifier-free guidance, and reward-guided beam search on multimodal pLMs, corresponding to controls over sampling distributions, per-step logits, and parallel trajectories. Throughout the complementary advancements centered on exploration-exploitation trade-off, we (1) reveal the suboptimality of default inference protocols and identify task-oriented sampling preferences; (2) observe substantial quantitative gains across tasks, consistently boosting the upper bound performance of multimodal pLMs without updating model parameters; (3) derive conclusions about base models that differ from prior consensus. Our code is available at: https://github.com/EchoChou990919/mplm_inference.

Abstract:
Health discourse in social media videos often contains densely entangled claims spanning multiple thematic aspects, stances, evidential frames, and rhetorical functions within short conversational spans. Existing approaches largely rely on coarse topic-level, sentiment-based, or stance-oriented representations that do not adequately capture this structure. Our analysis identifies an average of 13.22 atomic claims per minute, motivating richer claim-level discourse representations. We introduce a structured framework for claim-level discourse analysis in dense health narratives. Our framework models discourse through tuples linking atomic claims with thematic aspects, stance, and multidimensional pragmatic discourse attributes. To support this setting, we construct a benchmark spanning four health domains with 1,191 manually annotated claims from 60 videos. Using this framework, we evaluate automated structured discourse analysis under different discourse context settings. Results show that current LLMs achieve strong performance on thematic categorization and stance prediction, but struggle with high-dimensional pragmatic profiling. We also find that different discourse tasks benefit from different forms of contextual reasoning, suggesting that future systems may require task decomposition and specialized inference strategies.

Abstract:
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner’s cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.

Abstract:
Quantization is one of the default deployment path for open-weight LLM agents, but it is not behavior-preserving: an adversary can release a full-precision checkpoint that passes audits yet misbehaves once quantized, termed as quantization-conditioned attack (QCA). Prior QCA work targets free-text generation, where harm is mediated by a human reader. In contrast, the agentic setting poses a more severe risk: the triggered payload is a structured tool call that can be executed without human oversight. We present the first study of QCA against LLM agents. We find that directly adapting prior backdoor-injection methods can produce malicious behavior after quantization, but substantially degrades benign utility, rendering the resulting attacks impractical. To understand the true upper bound of the threat, we propose AgentQ, an attack framework that combines layer-banded LoRA injection with partial-PGD repair over a multi-codebook quantization-equivalence class. AgentQ preserves normal tool-calling capability while concentrating malicious behavior in the quantized model. Across three trigger–action pairs and three codebooks (NF4, FP4, INT8), AgentQ reaches up to 100% post-quantization attack success rate with minimal loss of benign utility, underscoring the need to make quantization-aware safety evaluation a standard requirement before open-weight agents are released or deployed.

Abstract:
Token probabilities play a central role in large language models. However, these probabilities often fail to reflect either answer correctness or probability distributions explicitly specified in the context. We attribute this failure to the Context-Logit Probability Gap: contextual probabilities are not reliably reflected in the model’s next-token logits. We show that post-hoc temperature scaling has a limited effect on addressing this gap, because it rescales logits without correcting erroneous probability rankings. Motivated by this, we propose Bag-of-Tokens (BoT) tuning, a very simple fine-tuning approach that directly calibrates the logits with the probability distribution conveyed by synthetic contexts. More specifically, we construct BoT examples by forming a bag of candidate tokens with an explicitly assigned probability distribution. This distribution is expressed in the context and used as soft supervision to align the model’s logit probabilities via KL-divergence minimization. Experiments show that BoT tuning generalizes to LLM-based sampling tasks, such as random-number generation, and improves downstream QA accuracy and calibration. These results suggest that calibrating LLM logits with contextual probabilities provides a lightweight and effective way to enhance their uncertainty awareness.

Abstract:
Built upon the existing analysis of retrieval heads in large language models, we propose an alternative reranking framework that trains models to estimate passage–query relevance using the attention scores of selected heads. This approach provides a listwise solution that leverages the holistic information within the entire candidate shortlist during ranking. At the same time, it naturally produces continuous relevance scores, enabling training on arbitrary retrieval datasets without requiring Likert-scale supervision. Our framework is lightweight and effective, requiring only small-scale models ( e.g. , 4B parameters) to achieve strong performance. Extensive experiments demonstrate that our method outperforms existing state-of-the-art pointwise and listwise rerankers across multiple domains, including Wikipedia and long narrative datasets. It further establishes a new state-of-the-art on the LoCoMo benchmark that assesses the capabilities of dialogue understanding and memory usage. We further demonstrate that our framework supports flexible extensions. For example, augmenting candidate passages with contextual information further improves ranking accuracy, while training attention heads from middle layers enhances efficiency without sacrificing performance.

Abstract:
Whisper exposes speech through a fixed 1500-token encoder interface, now a default representation for ASR decoders and Whisper-based speech language models (SpeechLMs), yet its redundancy remains largely unexamined. We propose stride-k subsampling, a deterministic indexing operation that retains every k-th token after the convolutional stem or encoder transformer. Across five Whisper scales, k=2 preserves baseline WER at both positions, with CKA attributing this stability to acoustic overlap at the stem and attention-induced redistribution at the encoder output. Applying stride-2 at both positions cuts audio tokens by 75% and total GFLOPs by 52–58%, with small WER costs on most ASR benchmarks and larger costs on harder ones. The same configuration extends to three Whisper-based SpeechLMs, yielding modest accuracy drops on stronger baselines and larger drops on weaker ones, while reducing end-to-end latency by 19.6–27.4%. Requiring no training or auxiliary computation, stride-k subsampling suggests that the standard Whisper audio-token interface is over-provisioned for many downstream uses.

Abstract:
On news platforms, a user clicks on an article based on her subjective anticipation of its content, inferred from the title using her background knowledge. We introduce SAND (Subjective Anticipation-augmented News Recommendation via Dual-pathway Distillation), the first framework to explicitly incorporate this user-specific anticipation that drives click behavior, formalized as the anticipated body, into news modeling. SAND (i) generates personalized anticipated bodies via an agent-based collaborative system, and (ii) learns preferences from anticipated bodies using a candidate-aware attention network with dual-pathway knowledge distillation. Experiments on real-world news datasets (i.e., MIND and Adressa) demonstrate that SAND significantly outperforms eleven state-of-the-art baselines.

Abstract:
Prompt optimization has often been framed as a discrete search problem to find high-performing and robust instructions for a large language model (LLM). However, the search result might not make it transparent why and where specific prompt changes lead to performance gains. This is in contrast to how humans are instructed for annotation tasks. Here, researchers carefully design annotation guidelines, leading to enhanced annotation consistency. Our paper aims at joining these two approaches and introduces iPOE, a novel interpretable prompt optimization strategy via explanations. We guide the prompt optimization process by automatically created guidelines from explanations of annotation decisions (either automatically generated or from humans). This set of guidelines is furthermore optimized by a series of operations, including removing, adding, shuffling, and merging. The resulting prompt includes guidelines that instruct the annotation, making the decision process of the LLM and the optimization transparent. It therefore supports also laypeople in prompt optimization. In our experiments on four datasets, we find that iPOE can improve over the evaluated baselines by up to 39% and LLM explanations can replace human explanations in the proposed method. Moreover, our interpretability validation study demonstrates that humans and LLMs substantially agree on which guidelines contribute to their annotations, achieving a Cohen’s kappa score between annotators and LLMs of up to 0.68, 0.80, and 0.95 for the emotion classification, medical fact-checking, and hate speech detection tasks respectively.

Abstract:
Deep research agents rely on iterative tool calls to gather external evidence for multi-step reasoning. While reinforcement learning (RL) improves planning and retrieval behavior, we identify a critical failure mode: tool-call hacking, where agents exploit format- and outcome-level rewards by generating superficially valid tool-use traces without genuinely grounding their reasoning in retrieved evidence. Unlike execution-based tools (e.g., code or math), whose effects are directly observable, the weak observability of causal dependencies between retrieved evidence and reasoning under format- and outcome-level supervision enables agents to maximize surface-level reward signals without genuinely grounding their reasoning in the returned evidence. This results in tool overuse, mode collapse, and decorative or hallucinated tool calls. We propose Proof-of-Use (PoU), an evidence-grounded RL framework that strengthens the dependency between retrieval, reasoning, and final answers. PoU requires agents to cite normalized evidence identifiers during reasoning and optimizes a multi-objective reward combining step-level citation validity, global Answer–Support Alignment, and adaptive reward mixing. Experiments show that PoU mitigates tool-call hacking, improves task performance, and induces robust tool-use behavior under domain and tool shifts, even without explicit optimization for routing adaptation.

Abstract:
As AI assistants serve millions of users daily, evaluating user experience (UX) beyond general model capability has become increasingly important. We present UXBench, the first user-centric benchmark grounded in real user feedback signals for evaluating preference alignment and dialogue generation. The benchmark consists of three interconnected tasks, UX Judge, UX Eval, and UX Recovery, with 7,400 test instances extracted from over 70K interaction logs of a mainstream Chinese AI assistant. The dataset closely reflects real user distributions, covering 8 scenarios, 83 domains, and diverse failure patterns that pose severe challenges. Extensive experiments on 26 frontier language models provide novel insights into how well models perceive user experience and how improvements in model capability contribute to better dialogue engagement. Through analyses of model behavior and performance gaps, we document six important findings, demonstrating that user feedback prediction is a learnable capability and revealing different aspects that influence user experience. UXBench establishes a new evaluation landscape and calls for greater attention to tailored UX optimization, contributing toward a user-centric scaling law for the development of successful AI assistants. The full project is released at https://github.com/mengze-hong/UXBench.

Abstract:
Self-evolving has emerged as a key paradigm for improving foundational models such as Large Language Models (LLMs) and Vision Language Models (VLMs) with minimal human intervention. While recent approaches have demonstrated that LLM agents can self-evolve from scratch with little to no data, VLMs introduce an additional visual modality that typically requires at least some seed data, such as images, to bootstrap the self-evolution process. In this work, we present Multi-model Multi-modal Zero (MM-Zero), the first RL-based framework to achieve zero-data self-evolution for VLM reasoning. Moving beyond prior dual-role (Proposer and Solver) setups, MM-Zero introduces a multi-role self-evolving training framework comprising three specialized roles: a Proposer that generates structured visual concepts and formulates questions; a Coder that translates these concepts into executable code (e.g., Python, SVG) to render visual images; and a Solver that performs multimodal reasoning over the generated visual content. All three roles are initialized from the same base model and trained using Group Relative Policy Optimization (GRPO), with carefully designed reward mechanisms that integrate execution feedback, visual verification, and difficulty balancing. Our experiments show that MM-Zero improves VLM reasoning performance across a wide range of multimodal benchmarks. MM-Zero establishes a scalable path toward self-evolving multi-model systems for multimodal models, extending the frontier of self-improvement beyond the conventional two-model paradigm.

Abstract:
Policy optimization (PO) is widely used to post-train language models for reasoning, often with a rigid format. We study what PO learns when this constraint is relaxed. We evaluate a relaxed reward across six models and four benchmarks, then audit learned formats on GSM8K and controlled variants across three on-policy algorithms. In these audits, accepting either explicit reasoning or a direct answer pushes the policy toward the shortest valid format: an -only response. On GSM8K, the collapsed Qwen-2.5 7B policy produces answer-only outputs in 98.6% of generations and loses 7.1 pass@5 points relative to a strict-format baseline. Controlled rewards show this is not a regex artifact: with nested formats, PO learns easy requirements before harder ones despite a 4× hard-format reward; with mutually exclusive equal-reward formats, the audited Qwen-2.5 7B final policy assigns more than 99% of generations to the easiest format. A sample-gated gradient account fits the pattern: formats receive positive updates only when sampled, so early support can outweigh reward scale. KL-controlled runs show collapse appears only when the policy can move far enough from initialization. The result identifies a practical failure mode for RLVR reward design.

Abstract:
Entity Alignment (EA) identifies equivalent entities across knowledge graphs and is critical for knowledge base integration and ontology merging. Evaluating EA systems at scale requires expensive expert annotation, making systematic assessment across diverse domains practically infeasible. LLM-as-judge evaluation offers a potentially scalable alternative, yet its reliability for structured prediction tasks like EA remains unstudied. We present the first systematic benchmarking study across three frontier models, three datasets, and four EA systems, using perturbation bias diagnostics, meta-evaluation across all dataset–judge–prompt combinations, and counterfactual label-flip tests. We identify anchor bias, a failure mode in which judges invert discrimination when the system’s decision label is visible. Label exposure causally collapses judge discrimination (J-ROC-AUC 0.12–0.87), while a label-free protocol recovers near-ceiling capability on distinctive-name datasets (0.93–1.00) and significant recovery on biomedical pairs (0.93–0.95). Counterfactual experiments confirm causality (FSR 53–99%) and reveal a frontier model paradox: stronger judges exhibit greater label sensitivity, not less. A blinded two-annotator human evaluation (102 pairs, Cohen’s κ = 0.902) confirms this mechanism directly. We release the first biomedical EA benchmark (MeSH–SNOMED CT, 15K pairs) and a reproducible auditing framework for LLM judge reliability in EA. Code and data are available at https://github.com/vaibhavalakshmiravideshik/llm-as-a-judge-entity-alignment.

Abstract:
Model editing keeps large language models (LLMs) up to date without retraining, but temporal facts expose a limitation of the prevailing locate-and-edit paradigm: an update is not always a replacement. When a fact changes, the new answer should become current while the old answer may remain correct in historical time contexts. Building on this insight, we use causal tracing to show that LLMs already support this distinction via a two-stage internal computation: early MLP layers retrieve a time-agnostic subject representation, and later layers modulate it with temporal context to yield the time-correct answer. Motivated by this finding, we introduce PRISM Edit, which optimizes a single polysemous representation across temporal contexts and leverages the model’s inherent modulation pathway to route it to temporally correct predictions without requiring any architectural modification. We evaluate on TimeConflict, a newly introduced temporal editing benchmark, and on temporally augmented CounterFact. PRISM Edit improves multiple core metrics over the best baseline, most notably +23.3 Temporal Consistency (TC) and +33.7 Current Relative-time Score (CRS) on LLaMA-3, while being more than 2× faster. Code and data are publicly available at https://github.com/CheerCHuang/PRISM-Edit.

Abstract:
As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-level prompt compression methods address this, but rank each sentence by a scalar relevance score, treating the document as an unstructured pool of words and sentences. Under tight budgets, this causes theme collapse, where the dominant themes of a document consume the budget, discarding less-frequent yet task-relevant themes. Preserving thematic coverage instead requires allocating the budget across recurring themes rather than scoring sentences in isolation. To this end, we propose SALT, a model-agnostic extractive framework that organizes per-sentence keywords into a trie ordered by sentence frequency (SF), a lightweight, reusable proxy for document thematic structure. This trie-based organization smooths memory allocation and prevents dominant themes from monopolizing the budget. Multi-anchor retrieval activates trie nodes labeled by query keywords at any depth, and the trie persists across dialogue turns, supporting multi-turn use without re-encoding the document. By preserving document themes, SALT reduces the prefill computation and memory cost of long-context prompts while remaining composable with KV-cache methods that target decoding-time latency and memory.

Abstract:
Retrieval over visually rich documents has a representation problem: important content often lives in tables, charts, figures, and layout relations that plain OCR linearizes, corrupts, or omits. ColPali-family visual retrievers address this with patch-level multi-vector indexes and late-interaction scoring, keeping image-derived retrieval on the query-time serving path. We introduce MIDR (Multimodal Indexing for Document Retrieval), a training-free framework for enrichment-augmented indexing that shifts multimodal reasoning to index time. During ingestion, a multimodal LLM converts rendered pages into verified textual fields that are indexed with BM25F and optionally fused with dense retrieval, enabling text-centric serving over multimodally grounded evidence. On ViDoRe V3, MIDR Hybrid achieves 0.6219 average nDCG across five English domains, a 23.0% relative gain over BM25, remaining competitive with ColQwen2.5. On two French-document domains, enrichment bridges English queries and French page text, lifting BM25 from 0.1532 to 0.5448 nDCG and outperforming ColQwen2.5. Across all seven domains, MIDR leads ColQwen2.5 on four while using ~9× smaller index memory and approximately 2× lower query latency. These results establish index-time multimodal reasoning as a compelling accuracy–deployment alternative to serving-time visual late interaction.

Abstract:
In Information Retrieval (IR), cross-encoders deliver state-of-the-art ranking effectiveness but have a high inference cost, limiting their use to second-stage re-rankers. Prior work has addressed this bottleneck from two largely separate directions: accelerating cross-encoder inference through attention sparsification, or improving first-stage retrieval effectiveness to alleviate the need of a re-ranker, using more complex models, e.g. late-interactions. In this work, we bridge these two directions through an in-depth analysis of cross-encoder internal mechanisms. By identifying and removing superfluous interactions, we derive MICE (Minimal Interaction Cross-Encoders), a new cross-encoder architecture that retains effectiveness while reducing computational overhead. Extensive evaluations show MICE retains most of the performances of its cross-encoder counterparts in-domain and matches or even exceeds it in out-of-domain, while reducing FLOPs down to 2.5 times.

Abstract:
Alignment teaches vision-language models (VLMs) to avoid expressing demographic biases, and when gender is clearly visible they largely succeed. Far less is known about ambiguous inputs (a worker in full gear, a figure seen from behind) cases common in practice yet rarely studied. We find that minimal prompting pressure exposes occupation–gender defaults when prompting ambiguous input images, with models collapsing to male even for strongly female-stereotyped occupations. But do these outputs reflect what models actually encode internally? We introduce LALS (Latent Association Leaning Score), a zero-shot metric that projects visual-token activations into the model’s text-embedding space to measure concept associations per token and layer. Across 15 occupations, over 800 gender-ambiguous images, and four VLMs, internal representations and outputs often becomes systematically decoupled: models often encode a female association internally yet output male. Layer-wise analysis reveals an asymmetric filter (male signal amplifies end-to-end while female signal peaks mid-network and is suppressed before generation) and a color ablation shows that culturally loaded visual cues such as clothing color further modulate these internal associations.

Abstract:
Large language models can often generate plausible mathematical reasoning traces, but reliably identifying the correct solution among multiple candidates remains a key challenge. Existing test-time reasoning pipelines typically rely on text-based verifiers that re-read each generated solution, making verification an expensive component of inference. Prior work has shown, however, that LLMs often encode correctness-related signals in their internal representations, including awareness of when their own answers are likely to be wrong. Building on this observation, we introduce HSRM, a lightweight hidden-state reward model that verifies candidate solutions by directly reading the generator’s internal representations rather than re-processing its text. HSRM extracts hidden states from a frozen generator at reasoning-step boundaries and uses a small Transformer encoder to rank candidates. It is trained from self-generated trajectories with outcome labels, requiring neither human-written process supervision nor a large pretrained verifier. Across four mathematical reasoning benchmarks, HSRM matches or outperforms a 55M-parameter text-only energy verifier in 15 of 16 generator–dataset settings while using only about 2M parameters, providing an efficient alternative to text-only verification by reusing representations already computed during generation.

Abstract:
Many NLP tasks require systems to provide attribution in their outputs—i.e. citations to grounding sources. Attribution serves as a bulwark against model hallucination and as a means for users to verify the credibility of model outputs. Yet, it is unclear how humans and LLMs evaluate citations when comparing outputs, a process central to reward modeling and modern LLM post-training. This paper studies the role of citations in the preferences of human judges and four open-source LLMs within the context of scientific question answering, leveraging mixed effects models to investigate the influence of citations on pairwise judgments. Among our key findings are (1) that humans prefer more diverse citations but fewer overall, and (2) that LLMs show some citation-related preferences compared to humans, despite lacking access to the sources, but these preferences depend on the data and specific models. We further discuss the implications of our findings for preference data collection.

Abstract:
Concurrent multi-agent LLM systems increasingly rely on role-specialized Workers that act in parallel, but effective coordination depends not only on what Workers generate, but also on what task state each Worker observes before generating. Existing systems usually fix this observation interface through workflow rules, retrieval heuristics, or full-history sharing, so information flow cannot adapt as task state and feedback evolve. We introduce Dynamic Context Allocation (DCA) , which treats per-Worker observation control as a trainable coordination problem for a fixed-role team. An RL-trained Context Allocator maintains a bounded Context Board and decides before each round which evolving task-state entries should be retained, revised, removed, or exposed to each Worker, rather than learning agent selection, execution order, or workflow topology. We evaluate DCA on cooperative code generation, Customer Relationship Management (CRM) lead qualification, and MultiWebGen , a new benchmark for multi-agent full-stack web development. Across these settings, DCA improves over fixed-protocol coordination and fixed-context multi-agent RL baselines, and it outperforms all evaluated frontier single-agent LLMs on CRM lead qualification despite using smaller role-specialized Workers.

Abstract:
A prominent research direction in mechanistic interpretability is learning sparse circuits over LLM components to reveal how they jointly produce model behavior. However, raw neurons are polysemantic, making learned circuits hard to interpret. Sparse autoencoder (SAE) features alleviate this, but their high dimensionality makes existing intervention-based circuit-learning methods computationally prohibitive. We propose CircuitLasso, a scalable circuit-learning approach based on sparse linear regression. CircuitLasso recovers circuits whose structural accuracy matches that of state-of-the-art intervention-based methods on the benchmark data, at a fraction of the computational cost. For interpretability, CircuitLasso efficiently uncovers relationships among SAE features, showing how human-interpretable semantic features propagate through the model and influence its predictions. Finally, we validate the utility of our learned circuits by leveraging their insights to achieve comparable performance at substantially lower cost on a domain-generalization task.

Abstract:
On-policy distillation (OPD) provides dense token-level supervision by asking a teacher to score student-generated rollouts. However, when the student drifts into an unrecoverable prefix, the teacher may locally agree with the degraded state, producing low reverse KL but little corrective training signal. We identify this persistent regime as a low-KL agreement trap. Further analyses show that tokens during and after such traps produce less useful supervision signals. We propose KAT (KL Agreement Trap Termination), an online OPD termination rule that detects persistent low-KL agreement with a dynamic training-adaptive threshold. By filtering weak supervision from degenerate agreement, KAT improves avg@k accuracy by 2.66% and pass@k by 3.43% across four mathematical benchmarks, while reducing average rollout length by 59.73%.

Abstract:
Goal-directed visual processing is a hallmark of human visual intelligence, resulting in representations that support downstream tasks such as categorization or search. Though vision-language models (VLMs) are often faced with these same tasks, their ability to recode visual representations when presented with goal-directed language remains poorly characterized. Indeed, prior work largely treats visual representations in VLMs as static repositories of visual information that are manipulated by language representations. In the present work, we provide evidence for two concrete instances of language-induced recoding of visual representations. First, we identify an abstract reference representation that denotes which objects are goal-relevant under a natural language prompt. We extract contrastive steering vectors corresponding to this reference representation and demonstrate that they are causally implicated in model predictions. These reference representations are abstract in that they generalize to different objects, different task contexts, and even from synthetic to naturalistic images. Second, we demonstrate language-induced attribute modulation: later layers selectively amplify goal-relevant attributes in visual representations of objects. We demonstrate this phenomenon across a range of different prompts. Finally, we provide a causal intervention that demonstrates that attribute modulation mediates a VLM’s response distribution. Together, our results support a more dynamic account of cross-modality processing in VLMs — rather than vision tokens serving as static repositories of information, they are modulated to support queries articulated in language.

Abstract:
Existing Visual Speech Recognition (VSR) systems commonly rely on left-to-right autoregressive decoding, which can force premature decisions on visually ambiguous tokens before sufficient context is available. We propose DLLM-VSR, to the best of our knowledge, the first Diffusion Large Language Model (DLLM)-based VSR framework, formulating transcription as iterative masked denoising with flexible-order decoding. With confidence-based unmasking, DLLM-VSR commits high-confidence positions early and uses the committed tokens as bidirectional context to refine ambiguous ones. To adapt DLLMs to VSR, we introduce a two-stage masked-denoising training strategy that separates visual-to-text content alignment from length modeling. We further observe a performance gap compared with an upper-bound setting where the ground-truth transcript length is provided at inference, allowing the model to focus on transcript content decoding. To reduce this gap, we develop length-guided candidate decoding, which uses video duration to construct plausible transcript-length hypotheses and reranks the decoded candidates using length plausibility and decoding confidence. The proposed method achieves a 19.4% word error rate on LRS3, establishing state-of-the-art performance among methods using only LRS3 as labeled training data.

Abstract:
Chain-of-thought prompting improves reasoning by generating intermediate steps as discrete text, but each step is forced to commit to a single token even when the model is uncertain. Continuous latent reasoning offers an alternative: a latent state can preserve multiple candidate interpretations without immediately selecting one token. Existing training-free approaches either place latent states only before decoding or switch between explicit and latent modes without optimizing the inserted state itself. We propose MidThink, a training-free framework that inserts and optimizes latent blocks both at the start of reasoning and adaptively during generation. When a confidence trigger detects sustained mid-stream uncertainty, MidThink inserts a new latent block and refines it with a rollout-based reward computed from short future continuations. We further show that initialization strongly affects whether a mid-stream block preserves a superposition over multiple tokens or collapses toward a single token. Across five mathematical reasoning benchmarks, MidThink improves over training-free latent-reasoning baselines, with the largest gains on the AIME benchmarks. Mechanistic analysis links these gains to three measurable properties of the latent block: distributional entropy across layers, attention centroid position, and stability under reinforcement learning updates.

Abstract:
While large language models (LLMs) are increasingly being explored for mental health applications, recent studies reveal that they can exhibit stigma toward individuals with psychological conditions. Existing evaluations of this stigma primarily rely on multiple-choice questions (MCQs), which fail to capture the biases embedded within the models’ underlying logic. In this paper, we analyze the intermediate reasoning steps of LLMs to uncover hidden stigmatizing language and the internal rationales driving it. We leverage clinical expertise to categorize common patterns of stigmatizing language directed at individuals with psychological conditions and use this framework to identify and tag problematic statements in LLM reasoning. Furthermore, we rate the severity of these statements, distinguishing between overt prejudice and more subtle, less immediately harmful biases. To broaden the reasoning domain and capture a wider array of patterns, we also extend an existing mental health stigma benchmark by incorporating additional psychological conditions. Our findings demonstrate that evaluating model reasoning not only exposes substantially more stigma than traditional MCQ-based methods but also helps identify the flaws in the LLMs’ logic and their understanding of mental health conditions.

Abstract:
A single sentence often expresses multiple valid relational triplets, which makes Open Information Extraction (OpenIE) fundamentally a multi-output task. Existing neural systems handle this by autoregressive generation, which is flexible but slow and prone to redundancy, or by fixed-slot prediction, which is efficient but couples the extraction budget to training. We introduce DIFFIE which instead treats the stochasticity of conditional discrete diffusion as the extraction mechanism itself: independent reverse-diffusion trajectories over per-token role tags produce a pool of candidate triplets, which are clustered under lenient matching and ranked to form the output. Both the pool size and the number of returned extractions are inference-time choices, decoupling the extraction budget from training and exposing test-time compute as a tunable axis. DIFFIE achieves the new state of the art in CaRB (1-1) both F1 and AUC, and outperforms the strongest rule-based system (ClausIE) in BenchIE; it also remains competitive in standard CaRB and WiRe57 evaluations, giving the best average score among systems that report all four benchmarks. Ablations show that uniform discrete diffusion outperforms absorbing state diffusion in our setting, and that a matched non-diffusion stochastic tagger does not reproduce its gains. Our results indicate that diffusion stochasticity is an effective mechanism for structured prediction tasks with multiple valid outputs.

Abstract:
Reinforcement learning (RL) improves large language model (LLM) agents by teaching them which actions lead to high rewards, but provides little supervision on what those actions do to the environment. World modeling (WM) can fill this gap, yet existing approaches often require separate simulators, extra training stages, or additional inference-time computation. We observe that on-policy RL rollouts already contain the needed signal: each transition pairs an action with its resulting next observation. Based on this observation, we propose PaW, a Policy and World modeling co-training framework that adds auxiliary WM supervision to the same policy during RL, without changing the inference paradigm. To make auxiliary WM supervision informative and stable, PaW introduces three components: action-entropy-based WM data selection, noise-tolerant WM loss, and reward-adaptive loss balancing. Experiments on three agentic task benchmarks show consistent improvements over strong RL baselines across models and RL algorithms. These results suggest that standard RL rollouts are a practical source of WM supervision for language-agent training.

Abstract:
Real-world Multi-Objective Reinforcement Learning (MORL) often suffers from sparse rewards, reward conflicts, and late-stage reward tug-of-war, causing traditional linear scalarization to experience severe metric oscillations. To address optimization conflicts among multiple objectives in real-world deployment scenarios, we propose Multi-Marginal Preference Optimization (MMPO), a fine-grained framework that intervenes at the data, gradient, and constraint levels rather than relying on coarse-grained global scalarization. Specifically, MMPO performs exposure debiasing to mitigate sparse and biased rewards, applies priority-aware orthogonal projection to decouple conflicting gradients, and introduces self-prompted gradient constraints to prevent dominant objectives from overwhelming weaker ones. Experiments on real-world e-commerce datasets show that MMPO improves training stability and consistently achieves better performance across conflicting metrics. Moreover, it generalizes robustly to broader tasks such as ToolRL and code generation, demonstrating its effectiveness as a practical paradigm for multi-objective alignment.

Abstract:
Benchmarks are central to measuring progress in language models, but aggregate scores can obscure substantial variation across subdomains, making models appear broadly competent despite concentrated strengths and weaknesses. We study this issue as benchmark-level distributional robustness : whether aggregate scores faithfully reflect performance across benchmark subdomains. We operationalize this notion with benchmark harmony, an entropy-based measure of how uniformly model performance is distributed across subdomains. Measuring harmony on 19 language model benchmarks across five model families, we find substantial variation in benchmark-level distributional robustness. Low-harmony benchmarks are more likely to yield aggregate scores that overstate broad competence, whereas high-harmony benchmarks provide more representative summaries of model capability. Rebalancing benchmarks by pruning overrepresented subdomains to increase harmony substantially shifts aggregate scores for low-harmony benchmarks, but leaves high-harmony benchmarks comparatively stable. For example, while BoolQ remains comparatively stable as harmony increases, PubMedQA, which evaluates performance in a medically consequential domain, exhibits substantial, often statistically significant, shifts in aggregate accuracy. Together, these findings show that aggregate scores can misrepresent broad competence when performance is unevenly distributed. We therefore recommend reporting benchmark harmony alongside aggregate accuracy as a diagnostic of benchmark representativeness when interpreting claims about broad model competence.

Abstract:
As large language models (LLMs) increasingly serve as the backbone of agentic systems with tool-use capabilities, predicting correctness, latency, and cost for a given tool-calling query is essential for efficient model evaluation, selection, and routing. Existing approaches predominantly adopt a semantics-first perspective, without capturing the complex interaction patterns inherent in tool-calling queries and without jointly accounting for cost and latency. We observe that query-model interaction matrices in tool-calling scenarios exhibit a low-rank structure, indicating that behavioral outcomes are governed by latent factors beyond surface semantics. We leverage this observation to propose a dual-pathway framework that decomposes interaction matrices into low-dimensional latent spaces for multi-objective prediction. The Capability-First (CF) pathway discovers latent structure via matrix factorization and then projects natural language queries into latent space. The Joint Capability-Semantic (JCS) pathway concurrently optimizes the text encoder and model embeddings, implicitly shaping query embeddings through capability-semantic interactions. Experiments on 75 LLMs demonstrate strong performance in predicting correctness, latency, and cost. Further experiments show that our method enables few-shot unseen-model generalization, parameter-efficient training, and interpretable latent representations.

Abstract:
Retrieval-augmented generation (RAG) has proven effective for knowledge-intensive tasks, but is widely believed to offer limited benefit for reasoning-intensive problems such as math and code generation. We challenge this assumption by showing that the limitation lies not in RAG itself, but in the choice of corpus. Instead of retrieving documents, we propose retrieving thinking traces, i.e., intermediate thinking trajectories generated during problem solving attempts. We show that thinking traces are already a strong retrieval source, and further introduce , an offline method that transforms them into structured, retrieval-friendly representations, to improve usability. Using these traces as a corpus, a simple retrieve-then-generate pipeline consistently improves reasoning performance across strong models and benchmarks such as AIME 2025–2026, LiveCodeBench, and GPQA-Diamond, outperforming both non-RAG baselines and retrieval over standard web corpora. For instance, on AIME 2025-2026, RAG with traces generated by Gemini-2-thinking achieves relative gains of +56.3%, +8.6%, and +7.6% for Gemini-2.5-Flash, GPT-OSS-120B, and GPT-5, respectively, even though these are more recent models. Overall, our results suggest that thinking traces are an effective retrieval corpus for reasoning tasks, and transforming them into structured, compact, or diagnostic representations unlocks even stronger gains.

Abstract:
AI coding assistants powered by large language models (LLMs) have transformed software development, significantly boosting productivity. While existing benchmarks evaluate the correctness and security of LLM-generated code, they are limited to single-turn tasks that cannot capture the conversational nature of real-world development, where requirements evolve across turns and tools increasingly generate incremental diffs rather than complete programs. We introduce MT-Sec, the first benchmark to jointly evaluate correctness and security in multi-turn coding scenarios. MT-Sec covers four interaction types (expansion, editing, refactoring, and debugging) reflecting common development workflows. We construct multi-turn tasks using human-curated instructions and feedback-driven interactions grounded in dynamic test suites. Evaluating 31 LLMs and 3 agent scaffolds, we find that models struggle most with expansion and editing, but improve significantly in debugging when provided with feedback. Agent-based systems often underperform their base models in automated multi-turn settings due to brittle tool use. We also evaluate multi-turn code-diff generation, an unexplored but practical setting, and observe higher failure rates compared to full-program generation. Our findings underscore the need for multi-turn benchmarks to drive safer AI-assisted development.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) is bottlenecked by hard prompts on which correct trajectories have low probability, so sampling misses them within a practical budget and leaves the policy update with little useful signal. We frame such zero-hit prompts as RLVR’s sampling frontier, where new reasoning behavior is most valuable yet least likely to be sampled. Importantly, failed rollouts can be informative: they expose where the model’s reasoning went wrong. We introduce LatentRevise, a first-order latent revision method that recovers training signal for this zero-hit regime. Given a failed rollout and the gold answer as an anchor, LatentRevise optimizes the input embeddings of its reasoning prefix under two complementary gradients, moving the prefix away from the failed continuation and toward the gold answer. The optimization is constrained to the convex hull of the model’s vocabulary embeddings, so each update moves the latent toward a real token embedding rather than an arbitrary feature direction. We find that continuations from the revised prefix lengthen, exhibit self-reflection, and reach correct answers missed by the original rollouts. Used as training data, these trajectories improve SFT and RLVR on math benchmarks over standard baselines.

Abstract:
Low-Rank Adaptation (LoRA) has become a de facto standard for parameter-efficient fine-tuning (PEFT), yet its performance is highly sensitive to initialization due to the information bottleneck imposed by low-rank decomposition. Existing approaches attempt to construct high-quality LoRA initializations by exploiting principal components of pretrained weights, activations, or gradients. However, these methods do not directly account for the training dynamics of the full-rank model. In this paper, we propose Training-aware Low-Rank Adaptation Initialization (TaRA), a method that initializes LoRA such that the gradients induced by the low-rank factors closely approximate the gradient of the corresponding full-rank weight matrix. Derived from a mathematical formulation, TaRA improves gradient fidelity at the start of training while introducing negligible computational overhead. Across diverse and challenging fine-tuning tasks, TaRA consistently outperforms prior state-of-the-art methods, establishing a simple, robust, and scalable solution for effective LoRA initialization.

Abstract:
Multi-trait essay scoring aims to provide fine-grained evaluation of writing quality across multiple dimensions. However, how to effectively post-train autoregressive scoring models remains underexplored. In this paper, we propose Trait-Aware Policy Optimization (TAPO), a post-training framework tailored to autoregressive multi-trait scoring. Our method decomposes rewards along both the sample and trait dimensions, combining global scoring consistency, trait-level accuracy, format validity, and inter-trait dependency preservation. In addition, we use enhanced prompts throughout training by incorporating original prompt texts and trait descriptions, providing richer semantic information for trait-specific score generation. Experiments across multiple backbone models show that our method consistently improves multi-trait scoring performance over supervised fine-tuning and scalar-reward optimization baselines, demonstrating the effectiveness and transferability of trait-aware post-training for essay scoring.

Abstract:
Block Diffusion Language Models generate multiple blocks of text sequentially, while tokens within each block are denoised in parallel, enabling efficient parallel decoding. However, their training suffers from low computational utilization: each sample is duplicated into a clean context and a masked copy, yet only the masked portion contributes to the training loss, resulting in merely 50% compute utilization. In this paper, we propose Multi-Mask Training (MMT), a simple yet effective training paradigm that significantly improves computational efficiency through context sharing. Our key idea is to generate K different mask patterns for the same clean text and process them together in a single forward pass via a sparse attention mechanism, where each masked segment attends to the shared clean context while remaining isolated from other segments. In pretraining, our proposed approach yields better performance under matched compute, despite being exposed to fewer unique training instances. Under a matched-step protocol, MMT maintains comparable downstream quality while improving training efficiency in AR-to-Diffusion adaptation. This reveals a compelling trade-off between data uniqueness and training efficiency in block diffusion models, demonstrating that the rich training signals from diverse mask samplings can effectively compensate for reduced exposure to unique data.

Abstract:
Sound events have semantic identities, locations, and trajectories, yet existing audio-language models typically encode clips as global event content. Conversely, sound event localization models track source directions over time but offer limited semantic coverage for language reasoning. To bridge this gap, we introduce ST-AudioQA, a spatio-temporal audio QA dataset and benchmark built from first-order ambisonic (FOA) renderings of static and moving sound sources. Each scene provides source identity, activity, direction, distance, and motion metadata, enabling dense trajectory supervision and QA about what is sounding, where it is, how it moves, and how sources relate. We further introduce ST-Audio Encoder, which jointly learns event semantics and time-resolved source trajectories, and ST-AudioLM, which exposes these audio tokens to an LLM for spatio-temporal audio QA. Experiments show that this representation improves the balance between semantic preservation and localization and yields stronger reasoning than static spatial and localization-oriented baselines. Beyond the controlled benchmark, ST-AudioLM shows zero-shot transfer to real recordings, with further gains after light adaptation. Project page: https://st-audiolm.github.io/.

Abstract:
Clarification-aware multi-agent orchestration is challenging because systems must elicit missing information, maintain task state, and coordinate dependent subtasks. Existing benchmarks do not comprehensively evaluate such orchestration under underspecified user requests, focusing instead on end-to-end success or isolated abilities like tool use and dialogue. We introduce MACRO, a travel-domain benchmark designed to address these gaps by providing structured task tiers, process-level user-side and system-side milestone annotations that include expected user clarifications, and standardized sub-agent interfaces. MACRO-Full contains 107 fully annotated tasks with 703 milestones, and MACRO-Core provides a tractable 30-task subset with 164 milestones. We evaluate three representative orchestration frameworks and a reference decoupled planning–execution orchestrator (MACRO-DPE) using trace-level metrics and failure analysis. Results show that MACRO-DPE improves routing fidelity and execution efficiency while maintaining task completion, whereas staged planning ensures milestone coverage at higher cost. The dataset[] and code are publicly available at https://github.com/liuweie/MACRO-bench/ .

Abstract:
Vision-Language Models (VLMs) achieve strong multimodal understanding by encoding images as sequences of visual tokens, yet cost hundreds and thousands of tokens per image. Existing pruning methods often rely on a single criterion, capturing only a partial view of token necessity. Controlled experiments reveal that token necessity is multi-faceted, requiring the joint exploitation of heterogeneous pruning perspectives. To address this, we propose MVP-Token, a novel Multi-View-Guided Token Pruning method to learn token necessity from multiple complementary pruning perspectives. We first establish a dataset MVP-PruneSet, which encompasses diverse pruning criteria by running orthogonal pruning methods, including a novel question-guided method GRASP at the encoder side. We then propose MVP-Pruner to capture such multi-view supervision signals. Specifically, a lightweight dual-attention module predicts soft token-importance scores under multi-view guidance. Extensive experiments demonstrate the effectiveness of MVP-Token across different model architectures. On LLaVA-1.5-7B, our method can surpass the single-view baseline by a clear margin. On Qwen2.5-VL-7B, our method achieves a 1.33× speedup and a 24.6% latency reduction under dynamic resolution and surpasses VisionZip by 1.87% of the average performance over two retention ratios.

Abstract:
Self-evolving large language models (LLMs) learn by generating their own training tasks and solutions, reducing reliance on human-curated supervision. However, in many reasoning domains, the model must also validate generated tasks and judge generated answers to obtain training signals. This creates a training-signal challenge: erroneous self-judgments become erroneous gradient updates. Existing approaches either rely on external verifiers, which limits generality, or treat noisy self-generated feedback as supervision. We propose COSE (Confidence-Orchestrated Self-Evolution), which uses the LLM’s intrinsic confidence as a lightweight uncertainty signal to modulate learning. COSE introduces confidence-weighted PPO updates and confidence-prioritized replay. Across 19 held-out benchmarks and four Qwen/Llama backbones (0.6B–4B), COSE consistently improves over base models and achieves the best average performance in general reasoning and mathematics, while remaining competitive on code. Code and data are available at https://github.com/weibowen555/COSE_ .

Abstract:
The attention mechanism has been the core component in modern transformer architectures. However, the computation of standard full attention scales quadratically with the sequence length, serving as a major bottleneck in long-context language modeling. Sliding window attention restricts the context length for better efficiency at the cost of narrower receptive fields. While existing efforts attempt to take the benefits from both sides by building hybrid models, they often resort to static, heuristically designed alternating patterns that limit efficient allocation of computation in various scenarios. In this paper, we propose Switch Attention (SwiAttn), a novel hybrid transformer that enables dynamic and fine-grained routing between full attention and sliding window attention. For each token at each transformer layer, SwiAttn dynamically routes the computation to either a full-attention branch for global information aggregation or a sliding-window branch for efficient local pattern matching. An adaptive regularization objective is designed to encourage the model towards efficiency. Moreover, we adopt continual pretraining to optimize the model, transferring the full attention architecture to the hybrid one. Extensive experiments are conducted on twenty-three benchmark datasets across both regular (4K) and long (32K) context lengths, demonstrating the effectiveness of the proposed method.

Abstract:
In high-stakes domains such as healthcare, privacy concerns severely limit the use of real-world training data. Differentially private (DP) synthetic data offers a promising alternative with formal privacy guarantees, but achieving strong utility remains challenging for clinical note generation due to domain specificity and long-form text complexity. We present Term2Note, a method for synthesising full-length clinical notes under strong DP constraints. By structurally separating content and form, Term2Note generates section-wise note content conditioned on medical terms, with terms and notes privatised under separate DP constraints, and applies a DP quality maximiser to improve outputs. Experiments demonstrate that Term2Note produces synthetic notes with statistical properties closely aligned with real clinical notes, and that downstream models trained on these notes achieve performance comparable to those trained on real clinical data. Compared to existing DP text generation baselines, Term2Note substantially improves both fidelity and utility, without relying on label distribution assumptions, highlighting its effectiveness as a practical privacy-preserving alternative to real clinical notes.

Abstract:
Causal graphs provide a high-level language for making mechanisms transparent. Recent work uses Large Language Models (LLMs) to recover causal graphs of external-world processes. Instead, in this paper, we use causal graphs to model LLM inference itself, providing stakeholders with a transparent view of how the model perceives and organizes high-level concepts to produce a prediction. We propose a four-phase method for constructing such graphs. Given a target LLM and a set of textual examples, our method discovers class-discriminative, human-interpretable concepts and maps each input to LLM-perceived concept states. We then introduce an MCMC-inspired counterfactual augmentation procedure that expands the sparse observational data through chains of counterfactuals. This enables stable causal discovery with σ-CG, yielding informative, interpretable graphs. We apply our method to three LLMs across disease diagnosis, sentiment analysis, and LLM-as-a-judge classification tasks. We evaluate the learned graphs for predictive fidelity and structural stability, and the MCMC-inspired augmentation for convergence and downstream utility. Our results show that the discovered causal graphs capture meaningful dependencies consistent with LLMs’ reasoning. Together, this paper provides a foundation for concept-level explainability of LLMs.

Abstract:
Controllable generation requires language models to realize output characteristics such as reading level, politeness, and toxicity. Existing steering methods are often indirect, require access to internal activations, or depend on auxiliary trained models. We propose SWAI, a training-free inference-time method that addresses these limitations by steering directly in logit space using corpus-derived token statistics. SWAI computes z-normalized one-vs-rest log-odds scores from labeled corpora and biases high-scoring tokens only within the model’s top-K candidate set, allowing control to favor target-characteristic tokens while preserving contextually plausible choices. Across readability, politeness, and toxicity control, SWAI consistently improves over prompt-based and prior logit-level baselines without modifying model parameters, accessing internal layers, or training an auxiliary model. Selectivity and lookup-table ablations show that the gains come from target-specific statistical scores rather than generic logit perturbation. These results indicate that effective steering does not require learned controllers when the logit intervention is guided by target-specific statistics under high-probability candidates.

Abstract:
Large reasoning models (LRMs) incur high inference costs, often mitigated by efficiency techniques like quantization and pruning. However, the impact of these techniques on model adversarial robustness remains largely unexplored. This study provides the first comprehensive analysis of the interplay between efficiency, jailbreak vulnerability, and reasoning in LRMs. We find that while efficiency methods seemingly reduce the success rate of jailbreak attacks, this improvement is often superficial. It largely arises from degraded reasoning capabilities leading to “attempted but failed” malicious responses, rather than an increase in genuine alignment. Mechanistic analysis of representational drift confirms this, revealing a strict coupling between reasoning capability loss and the model’s inability to maintain malicious semantic trajectories. Additionally, we identify quantization with pruning as the optimal strategy to balance efficiency and robustness. These findings clarify the distinction between true safety alignment and capability-induced failure, providing an empirical foundation for LRM deployment.

Abstract:
Table reasoning, a task to answer questions by reasoning over data presented in tables, is an important topic due to the prevalence of knowledge stored in tabular formats. Recent solutions use Large Language Models (LLMs) for their semantic understanding and reasoning capabilities. A common paradigm of such solutions linearizes tables to form plain texts that are served as input to LLMs. This paradigm has critical issues. It requires LLMs to infer row-column-cell relations from serialized inputs, makes evidence paths harder to trace, and is subject to the "lost-in-the-middle" issue. To address these issues, we propose Table Graph Reasoner (TabGR), a model that represents tables as an Attributed Table Graph (ATG) without task-specific training. The ATG explicitly preserves row-column-cell structure while enabling graph-based reasoning over traceable evidence paths. We further propose a Question-Guided Personalized PageRank (QG-PPR) mechanism to rerank tabular data and mitigate the lost-in-the-middle issue. Extensive experiments across multiple table reasoning benchmarks show that TabGR consistently outperforms state-of-the-art models by up to 9.7% in accuracy.

Abstract:
Text watermarking for large language models (LLMs) requires balancing output quality and detection reliability. We study a practically relevant reranking setting in which multiple watermarked outputs are compared against unwatermarked pseudo-references. We show that quality-only reranking can weaken watermark signals by favoring candidates closer to unwatermarked references, effectively acting as implicit unwatermarking. We propose constrained Minimum Bayes Risk (MBR) selection, which treats detection as a constraint rather than a weighted objective: it filters out weakly watermarked candidates and then selects the highest-quality remaining output. Experiments across watermarking methods, datasets, and models show that constrained selection substantially improves detectability over quality-only reranking while remaining competitive in task quality. It also achieves a more favorable matched-detection tradeoff than normalized weighted MBR.

Abstract:
World models are increasingly used to support planning in agents by predicting how environment states evolve in response to agent actions. Yet fluent next-state predictions can still omit task-critical facts, corrupt product attributes, or apply incorrect transition rules. To address such systematic prediction errors, we introduce MemWM, a memory-augmented text-based world model. MemWM uses world memory, a curated memory bank of transition rules, state caches, and hard-to-predict facts, to condition next-state imagination. We evaluate factual state preservation with Structured State Fidelity (SSF), which scores predicted states through benchmark-specific facts and fields. Compared with SFT, memory-augmented training improves SSF by up to 206.3%. In the full planning setting, we keep the policy model frozen and provide policy-side world skill: retrieved task-level skills and step-wise corrective guidance for action selection. Across ALFWorld, WebShop, and ScienceWorld, memory-augmented agents improve downstream success over an SFT-trained world-model agent, with up to a 65.4% relative gain. Sensitivity analyses further show that retrieved memory improves task success and efficiency under different memory and action-budget settings.

Abstract:
Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In deployed systems, however, prefixes are rarely evaluated in isolation. They often appear together with instructions, retrieved documents, or other task-specific context, as in retrieval-augmented generation (RAG). This raises a basic question: does contextual conditioning reduce memorization, or does it change which memorized samples become exposed? We study this question through paired item-level measurements of probabilistic suffix extraction. For each prefix-suffix pair, we score the same target suffix under an empty prompt and under retrieved contexts of varying relevance, across three open-weight instruction-tuned models. We find that context does not simply erase memorization. Instead, extractable memorization consists of a context-robust core and a context-sensitive boundary. Many samples that are extractable without context remain extractable under retrieved context, especially as prefix length increases. At the same time, context mainly affects marginal samples near the extraction threshold: it suppresses some exposures, but also enables new ones that are missed by prefix-only evaluation. These findings qualify the view that RAG reduces memorization risk. Context can lower aggregate extraction by suppressing boundary cases, yet robustly extractable samples persist, and context-enabled extractability remains security-relevant.

Abstract:
A single vulnerability in a widely used library can cascade through millions of dependent applications, yet more than half of vulnerability database entries contain missing or incorrect affected-library information. Existing automated approaches neglect the relational structure of vulnerability databases, treating identification as an isolated text retrieval problem. In this paper, we propose Athena, the first graph-based approach for vulnerability-affected library identification. Athena models vulnerability databases as a knowledge graph and reformulates the identification problem as knowledge graph completion (KGC). It comprises three key modules: a Modeling module that constructs a security knowledge graph integrating CVEs, libraries, CWE weakness types, CPE products, and software ecosystems; a Completion module that applies a modular KGC backbone to predict missing affected libraries for a given CVE via link prediction; and a Re-ranking module that retrieves KGC candidates and rescores them using a fine-tuned LLM augmented with knowledge graph embeddings, jointly leveraging structural and textual information. Our experiments on VulLib demonstrate that Athena significantly outperforms four state-of-the-art baselines, achieving a 32% improvement in Avg. F1 over the best baseline (i.e., VulLibGen). Notably, our KGC backbone with only 110M parameters already surpasses VulLibGen’s best configuration at 7B parameters, demonstrating the effectiveness of graph-based modeling; the re-ranking module then provides substantial further gains, consistently outperforming the best baseline across all evaluated LLM backbones.

Abstract:
Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However, recent work has demonstrated that retrieval-augmented generation (RAG) can have unintended side effects on the overall safety of the generated responses, when prompted for harmful or dangerous content. A clearer understanding of the mechanisms leading to this result is needed, as increasing numbers of end users turn to RAG to incorporate corporate documents and knowledge bases into LLM-based systems. We introduce RAG-Safety-Bench, a benchmark to measure the safety impact of RAG on LLM models. By removing the confounding effect of retriever quality, and cleanly separating the problem into four conditions – non-RAG, RAG with an oracle document containing the answer to the harmful request, RAG with documents related to the harmful request but without the specific answer, and RAG with random, safe documents – the benchmark isolates the impacts of different factors in the observed safety degradation. We report results across five open-source LLMs, showing an inverse relationship between benign and unsafe capability, strong evidence that baseline safety guardrails do not lead to downstream safety guarantees in the RAG case, and model-specific support for previous findings that even benign documents can lead to unsafe generation in retrieval-enabled systems.

Abstract:
As large language models (LLMs) increasingly process instructions from multiple sources with different privilege ranks, preserving the priority of higher-rank directives over conflicting lower-rank ones is critical for safe and predictable deployment. However, existing evaluations mostly focus on limited conflict settings, leaving it unclear how models handle instruction hierarchy (IH) when multiple conflicts arise. To address this gap, we present the first systematic study of IH under multi-conflict settings and introduce MIH-Bench, a benchmark that scales the number of simultaneous conflicts across six pairwise hierarchy scenarios. Experiments on 21 LLMs show that IH compliance deteriorates sharply as conflicts accumulate, while neither model scale nor generation reliably predicts robustness. We further find that system-involving conflicts are the most vulnerable, a particularly concerning result given the safety-critical role of system prompts. Moreover, reasoning can improve conflict resolution, although the gains vary across models and are bounded by underlying model capability. These results show that current LLMs remain fragile in pairwise multi-conflict settings, highlighting the need for stronger hierarchy-aware alignment. Code and data are available at https://anonymous.4open.science/r/MIH-release-A5B8/

Abstract:
An estimated 1 billion people worldwide live with vision impairment, yet current vision-language models (VLMs) produce descriptions too vague for safe navigation by blind and low-vision (BLV) users. Large VLMs can generate high-quality audio-description-compliant narrations but cannot run on mobile devices; small VLMs offer competitive latency but lack spatial detail, directional cues, and hazard awareness for navigational assistance. We present Smol-VL-BLV, a compact VLM for blind and low-vision users that closes this gap using a 500M decoder transformer model and two post-training mechanisms: (1) teacher-student distillation and (2) Group Relative Policy Optimisation (GRPO) with a composite BLV reward targeting directional language, metric distances, and hazard detection. Because multi-stage post-training can induce catastrophic forgetting, we add a lightweight finetuning stage after the last stage GRPO finetuning to recover general descriptive quality while preserving BLV-specific spatial grounding. Our best model substantially outperforms the baseline across various benchmarks, including tasks: VQA, BLV captioning, OCR, and real-time observation. Compared with the baseline for relative improvement, it improves the Spatial score gain of 19.3%, and the Social score gain of 14.8%. It also increases OCR-Bench by 101.5%, and raises TextVQA accuracy by 44.2%. These results show that BLV-focused post-training improves both accessibility-specific spatial grounding and general visual-text reasoning. Deployed on a mid-range Android smartphone via Mixed-Precision Quantization, the model remains under 450 MB and generates descriptions in real time, demonstrating practical viability for on-device BLV assistance. Our model, dataset, and code will be publicly released.

Abstract:
Aligning Large Language Models (LLMs) with the objectives of helpfulness, harmlessness, and honesty (HHH) remains important for reliable deployment. Existing approaches commonly rely on Supervised Fine-Tuning (SFT) and Mixture-of-Experts (MoE) for alignment. Prior work has identified Axis Collapse as a failure mode where different alignment objectives occupy disjoint feature spaces, with SFT showing interference between conflicting objectives, i.e. catastrophic forgetting, while MoE models show miscalibrated routing due to unreliable expert selection. Reinforcement Learning from Human Feedback (RLHF) has been applied to mitigate this, though it frequently shows reward misattribution, where alignment signals disproportionately favor certain objectives over others. To address these issues, we propose TriAlignX, a two-stage framework for HHH alignment. In Stage 1, low-rank task vectors are extracted for each alignment objective and routed through PrefSelect, a query-conditioned policy that selects relevant objectives for each query. In Stage 2, an encoding module maps the selected task vectors into objective-specific alignment scores, while a response module produces alignment-aware embeddings for decoding. Experimental results show that TriAlignX achieves up to 97.85% helpfulness (↑), 26.65% harmlessness (↓), and 98.90% honesty (↑), with an Avg score of 55.65%. On DeepSeek-7B, TriAlignX further achieves 54.96% Avg while reducing inference latency to 0.68s and peak memory usage to 13.2GB. Our code is available at: https://github.com/gskgautam/TriAlignX

Abstract:
We present BAREC-LexBench, a benchmark for Arabic lemma-level readability assessment, designed to evaluate readability lexicons and computational models independently of their granularity levels. We build the benchmark through a large-scale pairwise comparison task, where annotators determine which of two Arabic lemmas is easier to read and understand. The benchmark consists of 36,000 annotated pairs covering both same- and cross-part-of-speech comparisons among open-class lemmas: adjectives, nouns, proper nouns, and verbs. The annotations were produced by expert Arabic educators with an average inter-annotator agreement of 77.7% (0.56 Kappa ∼ moderate agreement), and we analyze disagreement patterns to better understand where lemma-level readability judgments are most challenging. We benchmark a wide range of approaches, from readability lexicons and frequency-based lexicons to trainable models and large language models (LLMs). Finally, we demonstrate downstream uses of the benchmark by constructing a ranked readability lexicon covering open-class lemmas (BAREC-LexRank) and evaluating its contribution to sentence-level readability assessment. To support research and education, we make the dataset, ranked lexicon, and benchmark results openly available.

Abstract:
Large Language Models (LLMs) have demonstrated strong performance on general tasks, while often struggling to adapt to specialized domains without high-quality domain-specific data. Existing LLM-based data curation methods primarily rely on human-designed workflows, leaving it unexamined whether LLMs can autonomously execute an end-to-end data engineering pipeline for model specialization. We formalize Autonomous Agentic Data Engineering, a novel task designed to evaluate LLMs as autonomous data engineers that drive model specialization through end-to-end data curation. We frame data as an optimizable component and assign an autonomous agent with planning, generating, and optimizing training data across multiple domains, guided by post-training performance improvement. Experiments show that autonomous LLM data engineers yield substantial gains, as GPT-5.2 constructs a training curriculum that improves a student model by 57.29%, entirely through iterative, agent-driven data adaptation. By illuminating both potential and bottlenecks, our study establishes autonomous data engineering as a measurable capability and charts a path toward agent-driven model specialization.

Abstract:
Recent multimodal large language models achieve promising results on visual reasoning tasks, approaching human-level performance on existing STEM benchmarks. However, these capabilities degrade on complex real-world scenes. We identify two intertwined causes: chain-of-thought reasoning remains largely confined to the linguistic space, and training data rarely requires each step to be anchored in visual evidence. In this work, we introduce Visual Augmentation Primitives , which leverages high-quality instruction-guided generation models to synthesize diverse photorealistic training samples and construct coherent multimodal reasoning chains throughout the generation process, requiring models to unify visual perception with logical reasoning at every step. We further introduce multimodal chain-of-thought rubric-based reinforcement learning, which extends rubric-based reward design from the text-only domain to visual reasoning, scoring intermediate reasoning steps for logical validity, completeness, and consistency, and construct the UNREAL benchmark to systematically evaluate visual reasoning on hyperrealistic scenes across controllable difficulty levels. Experiments show that a 4B model built on this framework surpasses 8B baselines, with the approach scaling consistently to 32B across visual reasoning and VQA tasks. All models, data, and code will be publicly released.

Abstract:
Multimodal Large Language Models (MLLMs) often exhibit modality bias, where predictions over-rely on one modality while underutilizing others. However, the mechanism underlying modality bias remains poorly understood. In this work, we first introduce Banana-Counting, a controlled diagnostic dataset for measuring modality preference under cross-modal conflicts. Evaluations across diverse MLLMs reveal pervasive modality imbalance. Then, through attention analysis, we show that modality bias is strongly associated with imbalanced cross-modal attention allocation and modality-specific attention sinks that absorb redundant attention mass. Motivated by these findings, we propose SMART, an inference-time method that mitigates modality bias by redistributing redundant sink attention toward under-attended modalities. SMART consistently improves multimodal balance across model families. On Banana-Counting, it reduces the modality gap from 27.73 to 0.66 on LLaVA-v1.5-7B while improving balanced multimodal accuracy. Moreover, SMART generalizes effectively to downstream multimodal reasoning tasks without retraining or parameter updates.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has substantially enhanced the reasoning capabilities of multimodal large language models (MLLMs). However, existing RLVR approaches typically rely on outcome-driven optimization that updates both perception and reasoning using a shared reward based solely on the final answer. This shared reward blurs credit assignment, frequently improving reasoning patterns while failing to reliably enhance the accuracy of upstream visual evidence extraction. To address this perception bottleneck, we introduce PRCO (Perception-Reasoning Co-evolution), a dual-role RLVR framework with a shared policy. PRCO consists of two cooperative roles: an Observer that generates an evidence caption tailored to the question and a Solver that predicts the final answer based on this caption. Crucially, PRCO employs role-specific reward signals: the Solver is optimized using verifiable outcome rewards on the final answer, while the Observer receives a utility reward derived from the Solver’s downstream success. Extensive experiments across eight challenging multimodal reasoning benchmarks demonstrate that PRCO yields consistent improvements across model scales by over 7 points on average accuracy compared to the base model, outperforming prior open-source RL-tuned baselines.

Abstract:
Large language models are increasingly required to generate responses that satisfy multiple competing objectives. Since optimal trade-offs depend on both user preferences and input prompts, controllable multi-objective generation must dynamically adapt models at inference time without retraining. To address this, we propose Evolutionary Soups, a mixture-of-experts framework for fine-grained generation control, with gating networks trained via an evolutionary algorithm. The per-layer gating networks dynamically produce expert-merging coefficients from hidden-state representations, while the evolutionary algorithm incorporates greedy hypervolume contribution for effective evolution of these gating networks, achieving consistent improvements on large and noisy training datasets and broader coverage of the non-convex Pareto front. Experiments across three tasks demonstrate the effectiveness of Evolutionary Soups over baselines: it achieves the best hypervolume, linear utility, and Tchebyshev utility (~20% improvement) among controllable methods on all tasks.

Abstract:
Medical language models (LMs) can memorize and reproduce protected health information, but privacy evaluations often focus on recovery of training text rather than disclosure under realistic threat models. We introduce a clinically grounded framework that evaluates leakage along a graded axis of adversarial access, ranging from publicly inferable demographics to leaked note fragments. At each tier, we measure verbatim memorization of patient-specific text and semantic leakage of sensitive diagnoses. Applying the framework to an LM continually pretrained on 378k clinical notes, we find that routine encounter metadata (i.e., name, date of birth, visit date, provider name, and practice location) elicits high rates of verbatim memorization across a patient’s timeline and sensitive-diagnosis recovery (AUROC 0.91 for abortion, 0.82 for HIV). At the same time, exact-match memorization can overstate disclosure: 36% of memorized tokens reflect templated documentation. Our work highlights the risks of training on longitudinal clinical data and provides a practical, reusable framework for contextual privacy evaluation of medical LMs.

Abstract:
Radiologists read CT as a sequential decision process, actively scrolling, zooming, adjusting windows, comparing with prior scans, and measuring structures, where each observation guides the next action. Current VLMs instead answer from passively sampled views, missing the multi-step reasoning that clinical reading demands. We address this by aligning the agent’s action space to the radiologists’ operations within the Picture Archiving and Communication System (PACS), enabling active, multi-step CT reading with inherently transparent reasoning for clinical decision support. In particular, we build ACTOR, a unified framework comprising: (1) a PACS-like interactive CT environment with five radiologist-aligned tools; (2) ActiveCTR-Bench, spanning 7 tasks across a Perception–Interpretation–Reasoning hierarchy (7,112 items with visual ground truth and verified vision necessity); and (3) ActiveCTReasoner, a 7B model trained on 108K synthesized trajectories via SFT and reinforcement learning. ActiveCTReasoner achieves 73.2% close-ended accuracy, outperforming GPT-5.4 (47.3%) remarkably. In open-ended evaluations, it scores 44.2% compared to 23.8% for the best baseline. A radiologist collaboration study provides preliminary evidence for the decision-support utility of radiologist-aligned agentic reasoning.

Abstract:
Can large language models (LLMs) synthesize programs that satisfy complex game-theoretic constraints? We investigate this question by applying LLMs to multi-facility location mechanism design, a canonical problem where a social planner must place facilities to minimize total distance to strategic agents while ensuring strategyproofness—the property that no agent can benefit by misreporting their preference. This problem provides a challenging testbed for LLM-driven program synthesis given the vast search space (mechanisms are arbitrary Python functions) and the adversarial constraint (robustness against any misreport by any agent). In response, we propose LLMMech, an evolutionary framework that embeds a pre-trained LLM within a search loop to generate interpretable, empirically strategyproof mechanisms as Python code. Unlike prior neural approaches, LLMMech requires no handcrafted features or hyperparameter tuning. To prevent premature convergence in the constrained search space, we introduce prompt evolution—an automatic procedure for evolving the prompts themselves. Experiments on both unweighted and weighted settings show that LLMMech outperforms existing hand-crafted and deep learning-based mechanisms across various problem settings. The resulting mechanisms are also interpretable and generalizable to unseen distributions and instance sizes.

Abstract:
Human memory adapts through selective forgetting: experiences become less accessible over time but can be reactivated by reinforcement or contextual cues. In contrast, memory-augmented LLM agents rely on "always-on" retrieval and "flat" memory storage, causing high interference and latency as histories grow. We introduce Oblivion, a memory control framework that casts forgetting as decay-driven reductions in accessibility – not explicit deletion. Oblivion decouples memory control into read and write paths. The read path decides when to consult memory, based on agent uncertainty and memory buffer utility, avoiding redundant always-on access. The write path decides what to strengthen, by reinforcing memories contributing to forming the response. Together, this enables hierarchical memory organization that maintains persistent high-level strategies while dynamically loading details as needed. We evaluate on both static and dynamic long-horizon interaction benchmarks. Oblivion outperforms both direct and memory-augmented baselines, while reducing token cost by up to 73% at 120K interaction spans. These results show that treating memory as a control problem – deciding when to retrieve and what to reinforce – is essential for sustaining long-horizon agent performance.

Abstract:
Identifying persuasive rhetorical cues is critical across domains, from detecting information manipulation and improving AI safety to advancing public health communication. We propose the Persuasion Index (PI), a taxonomy of 15 dimensions grounded in persuasion theories from psychology and communication, and one transparent implementation using 55 sub-features built from lexicons and rule-based detectors. The taxonomy is modular: individual detectors can be replaced while preserving the theoretical structure. We evaluate PI on four public datasets for English argumentative text that vary in domain, style, and outcome measures, and show that PI provides a shared feature space for interpreting rhetorical patterns associated with persuasion-related outcomes. Linear models show that PI features carry meaningful predictive signal while remaining computationally lightweight. Dimension-level analyses reveal recurring associations between PI dimensions and persuasion outcomes across datasets, while also highlighting topic- and stance-specific variation. We release PI as an open-source package and web interface for principled and auditable analysis of human and AI-mediated communication.

Abstract:
Tree-of-Thought (ToT) reasoning has significantly enhanced the problem-solving capabilities of Language Models by enabling the structured exploration of reasoning trajectories. However, the efficiency of these frameworks is often bottlenecked by search algorithms that struggle to navigate vast reasoning spaces at a reasonable inference cost. In this work, we address the challenge of optimizing LLM performance on mathematical reasoning tasks under limited token budgets. A key limitation of existing search strategies is that they apply the same exploration and selection rule throughout the entire search, which under a finite budget, often leads to token exhaustion with suboptimal or no candidate answers. Effective search instead requires different behavior at different stages. We introduce PACE (Phase-Aware Controller for Exploration), a strategy that adjusts its behavior based on the current state of the search: it prioritizes progress toward a first complete answer when none exists, and broadens exploration toward structurally distinct regions later on. Across GSM8K and Math500, two model scales, and token budgets from 3k to 10k, PACE consistently matches or improves over recent state-of-the-art baselines. On GSM8K with Llama-3.2-3B at a 3k budget, PACE delivers roughly a 56% relative accuracy improvement over the strongest tree-search baseline, and an early-stopping variant achieves comparable accuracy using about two-thirds of the tokens.

Abstract:
Do multilingual language models transfer factual knowledge across languages during continued pretraining, or do they mostly recall facts learned directly from the target-language data? To answer this question more reliably, we propose an intervention-based framework: starting from an English-pretrained model, we continue pretraining on Persian data from which specific facts have been systematically removed at varying levels of granularity. We construct SIFT, a resource of 500 triples across 20 relations, stratified by the cultural origin of each fact’s subject into general (globally prominent) and Persian-related entities, designed for both systematic fact removal from training data and evaluation, with natively written Persian cloze templates. Our results show that fact transfer is very limited: under the strictest removal condition, a large majority of English-acquired facts fail to transfer into Persian. We further show that sentence-level co-occurrence removal is insufficient to eliminate fact signal, and that easier (randomly selected) negative candidate sets substantially inflate apparent transfer by rewarding shallow associative heuristics, while performance on a harder candidate set that allows for less reliance on heuristics is much lower. Finally, we show that source-language entity frequency has a large influence, with Persian-related facts, which are orders of magnitude rarer in the English corpus, hardly transferring.

Abstract:
Recent advances in deep generative models have made it increasingly difficult to distinguish real-world data from model-generated content. Consequently, successive model generations are trained on synthetic data from prior models, forming a self-consuming training loop. Despite being powerful, it also opens a new attack surface, i.e., test-time poisoning attacks, adversaries exploit model vulnerabilities during the inference stage to generate malicious test-time outputs that are subsequently incorporated into future training data. In this paper, for the first time, we propose a new class of malicious test-time poisoning attack against self-consuming generative methods. However, uncovering the failure modes of self-consuming generative models under test-time poisoning attacks is highly challenging due to the need to simultaneously maintain stealthy data manipulations while achieving fine-grained control over the activation and propagation of malicious behaviors across self-consuming training iterations. To address these challenges, our attacks provide fine-grained control and introduce a novel stealth paradigm that achieves both effectiveness and stealthiness by camouflaging stealthy malicious data within the training process and selectively activating concealed attack behaviors at targeted later stages. Notably, our attacks are compatible with various adversarial attack generating algorithms and self-consuming algorithms. Extensive experiments under different settings, including black-box scenarios, demonstrate that self-consuming generative models are broadly vulnerable to the proposed test-time poisoning attacks.

Abstract:
Positional embeddings (PE) in Transformers encode token distance and order but are largely agnostic to syntactic structure. We introduce Syntax-informed Positional Embeddings (SiPE), which learns a lightweight syntactic prior from dependency parses during pretraining and injects it across all three dominant PE families (absolute, relative, rotary), for both encoders and decoders, leaving self-attention and the rest of the architecture untouched. We isolate where and how the prior should enter the model, and find it depends on the architecture: for autoregressive decoders that use relative PE, the prior is strongest when coupled multiplicatively with the relative-position term of the attention score, outperforming injection into the input embeddings, into self-attention, or into the positional and attention terms jointly—while for encoders it is best added directly to the input embeddings, composing with each encoder’s native positional mechanism. We find that models pre-trained with SiPE improve on the SyntaxGym benchmark by up to 10.3% while simultaneously reducing perplexity by 9.0% over a base model with no syntactic supervision—a metric nearly every existing syntax-injection method instead degrades. Crucially, these gains extend beyond syntactic generalization: SiPE also improves real-world language understanding, raising scores on the GLUE benchmark by up to 8.2% over a model trained without it. Unlike existing syntactic language models that marginalize over many parses at inference or discard syntax at runtime, SiPE conditions on a single parse, establishing a new Pareto frontier between syntactic supervision and inference cost.

Abstract:
Vision-Language Models (VLMs) remain unreliable when spatial reasoning requires composing relations whose meanings depend on frames of reference. Existing tool-augmented spatial reasoning methods make reasoning more explicit, but often rely on low-level geometric procedures and hard binary decisions over noisy perception. We propose SATURN, a neuro-symbolic framework for perspective-aware compositional spatial reasoning. SATURN reconstructs an approximate 3D scene, derives soft perspective-aware spatial predicates, and composes them with a training-free Pythonic symbolic executor, separating perception from reasoning while preserving uncertainty through multi-hop inference. We also introduce 3D FORCE, a diagnostic benchmark that controls reasoning depth, view, and perspective composition for spatial arrangement grounding (SAG) and referring expression grounding (REF). On 3D FORCE, VLMs and spatially trained models degrade sharply as depth and perspective complexity increase, whereas SATURN degrades the least and outperforms every baseline at each depth. On the real-world MindCube benchmark, SATURN achieves 78.06% overall accuracy, outperforming the strongest baseline by 14 percentage points.

Abstract:
Ambiguity or vagueness is a consequential consideration when drafting prescriptive text such as laws, contracts, or organizational policies. As a greater volume of such text is produced by language models, there is a need for valid, scalable methods for quantifying ambiguity, but existing measures are based on lexical properties of text rather than how it is interpreted in context. In this paper, we propose quantifying ambiguity as the degree of disagreement among a set of stakeholders ("raters") on how alternative prescriptive statements ("policies") apply in each of a set of relevant scenarios ("cases"). Using two policy domains, we present preliminary analysis suggesting that, even when using simulated raters, this approach (i) correlates with real human disagreement, and (ii) is robust to spurious text features. We then apply the measure to existing deliberation datasets to describe the association between policy ambiguity and support for those policies. A Python package for our framework and all data used in the paper are released at https://github.com/sally-dong/DELTA-ambiguity.

Abstract:
Reasoning LLMs-as-Judges, which benefit from inference-time scaling, offer a promising path for extending the success of reasoning models to non-verifiable domains, where output quality cannot be directly checked. While reasoning judges have shown gains on static evaluation benchmarks, their impact on actual LLM post-training remains underexplored. Therefore, we conduct a controlled study of non-reasoning and reasoning LLM-judges in reinforcement-learning-based LLM alignment. In our synthetic setting, a "gold" judge provides preference annotations for training smaller judges and evaluates the resulting policies. We find that non-reasoning judges lead to the standard reward hacking pattern: policies receive higher rewards from the training judge but lower rewards from the gold judge. Reasoning judges, in contrast, produce policies with high gold-judge rewards. However, a qualitative analysis shows that these high rewards arise from adversarial outputs that exploit the gold judge’s preferences and transfer to other LLM-judges on benchmarks like Arena-Hard. Together with our ablations on judge training design choices, these findings reveal both the promise and risks of using reasoning LLM-judges for non-verifiable LLM post-training.

Abstract:
In many settings, studying causal questions based on text data requires adjusting for confounding information within texts. Yet there is a tradeoff in constructing text representations for adjustment: they must be sufficiently large and/or dense to preserve the confounding variables necessary for unbiased effect estimation, but sufficiently small and/or sparse to satisfy finite-sample overlap and yield low-variance estimates. To address this tradeoff, we turn to sparse autoencoders (SAEs), and propose a novel causal adjustment pipeline that iteratively selects a minimal set of SAE features via conditional independence tests. We find that SAE representations achieve better adjustments (lower bias and and higher coverage) than alternative representations in standard semi-synthetic evaluations with binary confounders, and their interpretability offers opportunities for falsification. We also introduce a more realistic semi-synthetic evaluation that uses multi-label data as the unobserved confounders and find off-the-shelf adjustment methods require increased investigation for these more complex settings. Code: https://github.com/mianzg/sae-text-confounder

Abstract:
Maintainability is a core dimension of software engineering, shaping how code is written, reviewed, and developed over time. While coding agents have demonstrated strong performance on single-issue tasks, it remains unclear how maintainable their code is when future agents build on top of it, potentially leading to compounding downstream effects. We investigate how agent code compares to human code in these maintenance settings, presenting CodeThread , a framework to construct controlled experiments from repository-level coding benchmarks. Applying CodeThread to four frontier coding agents and four benchmarks, we find that agents are less effective at resolving tasks when building on agent code compared to human code, with task resolve rate drops of up to 13.1%. Regression analysis reveals that many traditional software engineering maintainability metrics do not explain this difference. Instead, the clearest signals are subtler behavioral differences in agent code, such as changes to input validation and error handling, along with differences in downstream code size and task difficulty. These findings highlight the need to evaluate coding agents not only by immediate task resolution but also by maintainability, and point to potential sources of downstream errors introduced by agent code.

Abstract:
Lexical relations are not only labels on pairs of concepts: they are directed typed movements that can form paths. We study whether sentence embedding spaces preserve this structure at the level of individual relation instances. By representing two-hop lexical chains as composed movements on the normalised embedding sphere, we test whether true chains form more coherent routes than matched false-middle alternatives. Across controlled lexical path experiments, real chains exhibit a robust path signal, including for mixed relation compositions, and the composed movements retain information about ordered relation type. Beyond diagnosis, the same path geometry aligns with independent distributional context shifts and improves graded lexical-entailment transfer, indicating that it captures reusable lexical structure rather than a task-specific artefact.

Abstract:
Context-aware machine translation (MT) leverages document-level information, yet it does not consistently outperform sentence-level MT, as contextual signals are unevenly beneficial across sentences. Existing training objectives do not explicitly model this variability, limiting a model’s ability to adaptively exploit context. In this paper, we propose Cross-Preference Learning (CPL), a preference-based training framework that explicitly captures the complementary benefits of sentence-level and context-aware MT. CPL achieves this by integrating both intra- and cross-condition preferences into the preference optimization objective, providing explicit supervision to exploit informative context while remaining robust to uninformative context. We validate the proposed approach on several public context-aware MT tasks using multiple models, including Qwen3-4B, Qwen3-8B, and Llama-3-8B-Instruct. Experimental results demonstrate consistent improvements in translation quality and robustness across both input conditions, achieved without any architectural modifications.

Abstract:
Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users’ genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model’s general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves the reasoning ability of large language models (LLMs), but its standard form relies on ground-truth answers or external verifiers that are often unavailable in unlabeled test-time settings. Label-free RLVR therefore estimates pseudo rewards from model outputs. Existing methods typically use these pseudo rewards to optimize either answers or model parameters. The former cannot preserve feedback across queries, while the latter risks committing noisy signals to model parameters. This raises a central question: what should label-free rewards optimize? We propose Test-Time Memory Learning (TTML), a label-free RLVR framework that shifts the optimization target from answers or parameters to an external memory bank. TTML consists of three components: memory policy rollout selects useful memories and executes memory-augmented generation for the current query, Bayesian value update turns consensus rewards into memory value updates, and memory bank expansion extracts reusable experience from test-time trajectories. Across six benchmarks and three backbone models, TTML achieves the best Avg@8 performance over strong label-free baselines. Empirical analyses further suggest that converting label-free feedback into revisable memory values and reusable memory entries is more effective overall than using it only for single-query selection or direct parameter updates.

Abstract:
Large language models (LLMs) are increasingly used as natural-language interfaces to structured data, yet they remain brittle when reasoning over time series. Visual patterns can be misleading, numerical claims can be hallucinated, and textual context can override evidence from the signal. We study zero-shot time-series reasoning as a multimodal evidence arbitration problem for LLM agents. We propose TS-Debate, an inference-time multi-agent protocol that requires no task-specific fine-tuning. TS-Debate first elicits relevant domain knowledge, then assigns modality-specialized agents to textual context, visual patterns, and numerical signals, and coordinates their interaction through a verification-conflict-calibration procedure. Reviewer agents check decision-critical claims with lightweight code execution and numerical lookup, resolve cross-modal disagreement, and calibrate the final answer. Unlike generic multi-agent debate or unconstrained tool use, TS-Debate specifies how evidence is exposed, which claims are checkable, and how verification outcomes shape synthesis. Across 20 tasks from three public benchmarks, TS-Debate improves classification and question answering performance over strong baselines, while revealing that debate is most useful for global-structure and cross-view reasoning rather than local value reconstruction.

Abstract:
Search agents mitigate hallucination by grounding their answers in retrieved web results. However, retrieval-based approaches also introduce an attack surface: agents may cite misinformation from poisoned search corpora containing false or malicious documents. We demonstrate that, in some cases, search agents’ reasoning and responses may be steered by completely factual but distracting information. We refer to this failure as lazy grounding. We expose lazy grounding by injecting nearby evidence from answer-changing rewrites of benchmark questions into the search corpora. Each document contains factual evidence that supports a neighboring rewritten question but is retrieved for the original question. Across 12 model–benchmark pairs, the attack causes the accuracy of search agents’ responses to drop by 5.9 points on average and by up to 17.3 points, while inducing nearby-answer adoption in every setting. The effect is even stronger when nearby evidence appears later or is more answer-shaped. Our results show that robust search agents must defend against not only misinformation but also the misapplication of factual evidence. The code is publicly available at https://github.com/frankyzha/lazy-grounding.

Abstract:
As large language models and increasingly capable AI agents are deployed in high-risk settings, aligning them with complex human values has become a central challenge. Existing alignment methods, while effective in improving helpfulness, harmlessness, and controllability, often struggle to capture real-world preferences that are context-dependent, non-transitive, and shaped by dynamic multi-party interactions. This survey reviews AI alignment through a game-theoretic lens. Specifically, it organizes recent progress around key game-theoretic elements and synthesizes the literature along three challenges: preference diversity, alignment priority, and temporal dynamics. This perspective clarifies where current alignment methods genuinely benefit from game-theoretic analysis, where the framework is looser, and what challenges remain in building robust, adaptive, and verifiable AI systems.

Abstract:
As LLM agents act across personal applications, web browsers, and other interfaces, their reusable skill libraries can scale to thousands of skills. This scale introduces two challenges. First, loading the full library saturates the context window, driving up token costs, hallucination, and latency. Second, semantic retrieval surfaces topically relevant skills but can miss upstream and downstream prerequisite skills, creating a prerequisite gap that leaves the retrieved bundle insufficient for execution. We present Graph-of-Skills (GoS), an inference-time structural retrieval layer for large skill libraries. GoS constructs an executable skill graph offline from skill packages, then retrieves a bounded, dependency-aware bundle through hybrid semantic-lexical seeding, reverse-aware Personalized PageRank, and context-budgeted hydration. Across SkillsBench and ALFWorld, with three model families (Claude Sonnet 4.5, MiniMax M2.7, and GPT-5.2 Codex), GoS attains the highest average reward in all six model-benchmark blocks, at a fraction of the token cost of loading the full library. On SkillsBench with GPT-5.2 Codex it raises average reward by 7.0 absolute points over full skill loading, a 25.6% relative gain, while cutting total tokens by 56.7%. Ablations isolate the mechanism: replacing reverse traversal with forward propagation costs 9.1 reward points, a larger loss than removing the graph altogether. The gain thus comes from traversing dependencies backwards, not from graph diffusion as such. A budget-matched retrieval study holding seeding, reranking, hydration, and context budget fixed reproduces the same ordering, with dependency-pair co-recovery falling from 0.654 to 0.362. Code is available at https://github.com/davidliuk/graph-of-skills.

Abstract:
Diffusion language models denoise a masked canvas, making them a natural setting for selective generation decisions such as answering, waiting, clarifying, or refusing. We introduce TAP-DiLM, which adds a typed policy variable to masked diffusion LMs, and study how this policy should be attached to the denoising process. Across BERT-base, MDLM-OWT, LLaDA-8B, and Dream-7B, we compare shared-vocabulary slots, dedicated slot heads, and CLS-based external heads. The results reveal a clear interface-backbone interaction: dedicated heads provide robust policy learning across scales, while shared-vocabulary heads can be effective when the LM-head geometry is favorable. A simple LM-head sharpness probe predicts when shared-head policy internalization is reliable, giving a practical diagnostic for interface selection. Causal clamp and slot-injection tests show that dedicated policy slots can influence body denoising through feedback, especially at 8B scale. Safety-refusal experiments further show that TAP-DiLM supports competitive typed refusal behavior when trained with diverse unsafe prompts.

Abstract:
In contrast to RNNs, which compress their history into a single hidden state, Transformers can attend to all past tokens directly. However, standard Transformers rely solely on the hidden state from the previous layer to represent the entire context. We show that this design creates pressure toward representation collapse and can degrade performance. To address this issue, we introduce Layer-Integrated Memory (LIMe), a lightweight extension that leverages existing key–value buffers and learns per-head, per-layer routing weights to integrate representations from previous layers. Across language modeling, synthetic reasoning, and deep architectures, LIMe improves perplexity per FLOP in the studied regimes and yields strong gains on synthetic tasks while preserving higher value-vector entropy and token separability. Finally, learned routing weights reveal systematic reuse of local and long-distance features, showing how LIMe enriches attention-time memory without increasing hidden-state size. Code is available at https://github.com/corl-team/lime.

Abstract:
Sparse AutoEncoders (SAEs) offer a promising way to inspect language model representations, but it is still unclear what kind of linguistic structure their latents expose. We use part-of-speech (PoS) categories as a controlled test case to study whether morpho-syntactic information is encoded by individual latents or by structured groups of features. We find that PoS distinctions are highly recoverable from SAE activations, but do not align with one-to-one latent / category mappings. This recoverability is not reducible to lexical memorisation, and Open and Closed PoS classes differ substantially. Categories are supported by compact groups of sparse latents, with substantial variation across tags. These groups remain stable on held-out data, while also showing overlap between related categories. Our results show that SAEs localise morpho-syntactic information in a distributed and category-dependent form rather than through atomic grammatical features.

Abstract:
Dense softmax attention assigns non-zero probability mass to many task-irrelevant keys, which can dilute useful signals in the attention output. We present REFOCUS, a training-free and plug-and-play inference-time sparsification method that keeps attention entries above a row-wise, context-length-aware threshold and renormalizes the remaining weights. This design is stable, simple to deploy, and requires no model retraining. REFOCUS is integrated in HuggingFace via attn_implementation="refocus" without model-code modification. On Qwen3-4B under 0-shot greedy decoding, REFOCUS matches or outperforms dense eager attention on full-test GSM8K, HumanEval, WinoGrande, and AQuA-RAT, and improves RULER-4k by +0.52 percentage points overall, including +3.20 points on aggregation tasks. These results indicate that a single row-wise threshold can remove diffuse long-tail attention while preserving or improving accuracy.

Abstract:
Automated radiology report generation is expanding across imaging modalities and anatomical regions, yet its evaluation remains misaligned with diagnostic priorities. Existing metrics either rely on surface-level text similarity or require slow, costly generative inference. This limits their ability to detect clinically significant errors, such as hallucinated or omitted abnormal findings, and prevents their use as efficient reward signals during training. We introduce POLAR (POlarity-Aware Long-context Anatomy-conditioned Regression), an encoder-based evaluation metric built on ModernBERT. POLAR combines long-context inputs for multi-organ reports, anatomy conditioning for targeted body-region evaluation, and polarity-aware regression that explicitly distinguishes normal from abnormal findings, yielding a more clinically meaningful assessment than prior metrics. Across real-world, radiologist-annotated CXR benchmarks and expert preference datasets, POLAR achieves competitive alignment with radiologist judgments while running 11x faster than generative evaluators, making it practical for large-scale benchmarking and as a training-time reward signal. We publicly release POLAR and the POLAR-Synth dataset, along with a fully open-source, privacy-preserving pipeline for data generation, training, and inference, enabling the metric to be extended to new modalities and anatomies.

Abstract:
Large language model (LLM) unlearning has emerged as a crucial post-hoc mechanism for privacy protection and AI safety, yet auditing whether target knowledge is truly erased remains challenging. Existing output-level metrics fail to detect when this knowledge remains recoverable from internal representations. Recent white-box studies reveal such residual knowledge but often rely on auxiliary training or dataset-specific adaptations, leaving no generalizable metric. We close this gap with the Unlearning Depth Score (UDS), a metric that quantifies the mechanistic depth of unlearning via activation patching. UDS first identifies layers that encode the target knowledge using a retain model baseline, then measures how much of it is erased in the unlearned model on a 0-1 scale. In a meta-evaluation across 20 metrics on 150 unlearned models spanning 8 methods, UDS achieves the highest faithfulness and robustness, confirming our causal approach as the most reliable for unlearning evaluation. Case studies further show that UDS uncovers residual knowledge obscured from observational metrics by representational shifts, with erasure depth varying across prompt types. We provide guidelines for integrating UDS into existing benchmarking frameworks and streamlining the evaluation pipeline.

Abstract:
Computer-use agents (CUAs) now act on a user’s behalf across personal applications such as email, calendars, and to-do lists. This cross-application access is useful, but it also creates a privacy risk that has been largely overlooked: when an agent works in one context, it can pull in information from another that is inappropriate in that context. Hence, we introduce , a benchmark that turns this risk into executable, deterministically scored scenarios. We target three common failure modes in CUAs: visual co-location , where the agent pulls in prohibited items that sit next to the task target in the UI; task-ambiguity overshare , where the agent dumps dense personal state in response to an under-specified prompt; and recipient misalignment , where the agent sends content to an addressee for whom it is inappropriate. We evaluate 15 frontier agents and find a surprisingly high failure rate: 11 of 15 leak on more than 50% of scenarios, with an average leakage of 67.9%, and the same failures persist when agents act end-to-end in the environment to complete the task. We release to encourage the development of safer computer-use agents and position contextual disclosure testing as a pre-deployment safety check.

Abstract:
Empirical identity leakage from released text is increasingly driven by attackers that combine large language models (LLMs) with auxiliary knowledge to link documents to individuals. Existing audits typically report success rates for specific attack pipelines but lack finite-sample statistical guarantees, while training-time protections such as differential privacy are difficult to translate into release-time decisions for individual natural-language documents. We introduce Conformal Privacy Auditing (CPA), a distribution-free calibration framework that provides a statistical certificate of re-identification risk for each released document against LLM-empowered adversaries. CPA outputs a conformal ambiguity set of candidate identities that is guaranteed to contain the true identity with user-chosen confidence under exchangeability, together with an interpretable leakage proxy derived from set size. CPA supports both logit-access and sampling-only attackers, enabling audits of open-source models and proprietary API models in a unified framework. Across multiple release benchmarks and attacker configurations, CPA achieves calibrated coverage and reveals sharp shifts in certified identifiability as auxiliary knowledge, LLM augmentation, and release mechanisms vary, providing a statistically grounded basis for reporting and comparing release-time linkage risk across attacker configurations, datasets, and release mechanisms alike.

Abstract:
Synthetic explanatory dialogues hold promise for applications from tutoring systems to model rationalization, yet their quality is often deemed inferior to human-authored conversations. This study uses dialogue completion as a lens to investigate the nuanced differences between human-spoken, human-written, and synthetic explanatory dialogue. By evaluating LLM-generated turns against original turns using the FED and IXQuisite test suites, we uncover a significant bias in the FED metric towards written language, which may penalize the interactivity of spoken dialogue. We find that three popular LLMs (Mistral, Llama, Claude) replicate this pattern, excelling in fluency, but struggling with conversational engagement, particularly on spoken dialogue transcripts. More importantly, we demonstrate that these models can significantly improve their performance through a zero-shot, iterative self-refinement method. By providing feedback based on their specific weaknesses, we show that LLMs can iteratively adapt their outputs to better resemble the interactive qualities of human dialogue.

Abstract:
Tree-structured speculative decoding accelerates large language model inference by drafting multiple candidate tokens in parallel and verifying them against a target model. Dynamic tree expansion strategies further improve efficiency by using draft model confidence to prune unlikely branches and stop expansion early. However, we find that existing draft models exhibit severe overconfidence: they assign high probabilities to tokens that will be rejected, causing unreliable early stopping and wasted computation on incorrect branches. We propose Calibrated Tree-Structured Speculative Decoding (CATS-SD), a training framework for tree-based confidence calibration. Our approach calibrates draft model confidence at two complementary levels: (1) token-level calibration that aligns draft-target token ranking, and (2) path-level calibration that supervises confidence over tree-structured decoding paths. We also introduce a calibration metric using Pearson correlation to quantify confidence-acceptance alignment. On Qwen2.5-3B-Instruct, CATS-SD achieves up to 11.3% higher acceptance length and 4.0% better confidence-acceptance correlation compared to the baseline, with end-to-end speedup improving from 2.51 × to 2.71 × on GSM8K. We validate these results across mathematical reasoning (GSM8K), multi-turn conversation (MT-bench), and code generation (HumanEval).

Abstract:
Computational linguistics predominantly operates on written text. However, writing represents spoken language only indirectly. Whether phonetic and orthographic representations share fundamental information-theoretic properties is an open question. We present a cross-linguistic analysis of Shannon entropy across modalities using the DoReCo corpus featuring 42 languages of 25 families after preprocessing. By computing continuous entropy trajectories at orthographic word, subword (BPE), and character levels (UTF-8/SAMPA), we reveal a systematic modality convergence: orthographic and phonetic encodings yield consistently small entropy differences across all three granularity levels. This general pattern, however, masks notable cross-linguistic variation–some languages, such as Evenki, converge tightly across all levels, while others show modality gaps at specific granularities–for instance, English at the character level, Bora at the word level, and Sanzhi Dargwa at the subword level. In sum, Shannon entropy tends toward modality-invariance as a general tendency across the Latin-script languages examined, while leaving room for typologically conditioned divergence. Whether this pattern extends to non-alphabetic writing systems remains an open question.

Abstract:
Summarization with a fixed character length is important for tasks such as headline and article preview generation, where text must fit an exact space. Autoregressive language models (LMs) struggle with precise character-length control: training-based methods require task-specific fine-tuning, while zero-shot prompting strategies achieve only approximate compliance. We propose DELTA (Diffusion Expected Length Targeting Algorithm), a training-free inference-time method for masked diffusion LMs that estimates expected character length at each denoising step and dynamically inserts or removes masked positions to steer toward a target length. DELTA requires no fine-tuning and is designed for off-the-shelf masked diffusion LMs. Across three summarization benchmarks, it reduces character-length errors by an order of magnitude compared to baselines while maintaining competitive summary quality in both automatic and human evaluation.

Abstract:
Parameter-efficient fine-tuning with adapters (e.g., LoRA) equips LLMs with task-specific skills. However, utilizing multiple pretrained adapters for out-of-distribution (OOD) generalization remains challenging. Existing techniques for OOD generalization using multiple pretrained LoRAs, route inputs using LoRA representations ( prototypes ) obtained independently, assuming these representations capture complementary information. However, we observe that for existing methods, in-distribution and OOD routing entropies are often comparable, thus bringing the complementarity assumption into question. We derive the theoretical conditions that could lead to a violation of such assumptions, distilling the cause down to the presence of shared, noisy prototype subspaces. Based on this, we introduce MAPLE (Masked-Adapter Prototype LEarning) , a simple learning framework that refines LoRA prototypes by masking the target task’s LoRA during prototype learning. In doing so, it encourages prototypes to discard noisy attributes, improving prototype discriminability, routing, and OOD generalization. Extensive experiments on language models of varying size, such as Phi-2 (2.7B) and LLaMA-3 (8B) equipped with heterogeneous pools of pretrained LoRAs, show that MAPLE improves the LoRA representation and thus achieves state-of-the-art performance across multiple benchmarks.

Abstract:
The structure of naming systems in natural languages hinges on a trade-off between high informativeness and low complexity. Focusing on the domain of kinship naming, we analyze such trade-off while addressing simplifying assumptions of prior work, namely: (i) universal communicative need across languages, and (ii) optimal listeners. To that aim, we collect data from four different languages, and analyze how different communicative needs and variations in the listener model influence the informativeness–complexity trade-off. Adopting a referential game setup from emergent communication, we further show that trade-off optimality is not only theoretically achievable but also emerges empirically in learned communication systems.

Abstract:
Preference alignment is usually achieved by weight-updating training on preference data, which is costly and provides limited mechanistic visibility. We propose Dynamic SAE Steering for Preference Alignment (DSPA), an inference-time method that makes sparse autoencoder (SAE) steering prompt-conditional. From preference triples, DSPA computes a conditional-difference map linking prompt features to generation-control features; during decoding, it modifies only token-active latents, without base-model weight updates. Across Gemma-2-2B/9B and Qwen3-8B, DSPA improves MT-Bench and is competitive on AlpacaEval while preserving multiple-choice accuracy. Under restricted preference data, DSPA remains robust and can rival heavier alignment pipelines at a fraction of the compute. Finally, we audit the SAE features DSPA modifies, finding that preference directions are dominated by discourse and stylistic signals, and provide theory clarifying the conditional-difference map estimate and when top- k ablation is principled.

Abstract:
On-policy distillation reduces train-inference mismatch by supervising student-generated trajectories, but full-vocabulary token-level distribution matching remains expensive for large language models. A common approximation truncates reverse-KL distillation to the student’s top- k tokens, which preserves efficiency but drops tail-token supervision and ignores support-level mass mismatch between the student and teacher. We analyze these approximation gaps and propose Stochastic Support Sampling with Teacher Mass Reweighting, which samples fixed-size supports from the student distribution and uses teacher support mass as a reliability weight for the conditional distillation loss. Under the same support budget, our method outperforms the strongest baseline by +1.4 hard-math average points on Qwen3-8B→Qwen3-1.7B-Base, and by +1.1 math average points and +2.8 code average points on DeepSeek-R1-Distill-Llama-8B→Meta-Llama-3.1-8B-Base.

Abstract:
Model merging aims to integrate task-specific abilities from individually fine-tuned models into a single model without extra training. In model merging methods, the task vector has become a fundamental building block, as it can encapsulate the information from fine-tuning. However, the merged model often suffers from performance degradation due to conflicts caused by task-irrelevant redundancy in task vectors. To address these challenges, we propose P urified T A sk VE ctors (PAVE) in knowledge-aware subspace. Concretely, we sample training examples from each task, and feed them into their fine-tuned models to acquire covariance matrices before linear layers. We then perform a context-oriented singular value decomposition, which accentuates the weight components most relevant to the target knowledge. As a result, we split fine-tuned model weights into task-relevant and redundant components in the knowledge-aware subspace, and purify the task vector by pruning redundant components. To induce fair pruning efforts across models, we further introduce a spectral rank allocation strategy by optimizing a normalized activated pruning error. The task vector purification by our method as a plug-and-play scheme is applicable across various task vector-based merging methods to improve their performance. In experiments, we demonstrate the effectiveness of PAVE across diverse merging methods, tasks, and model architectures.

Abstract:
Graph-based Retrieval-Augmented Generation (GraphRAG) can fail in interdisciplinary medicine where evidence comes from incommensurable knowledge systems, e.g., Traditional Chinese Medicine (TCM) and Western Medicine (WM). Similarity-driven retrieval in a shared latent space may over-unify heterogeneous concepts, causing semantic collapse (spurious alignments, provenance-blind mixing, and broken multi-hop chains). We propose Xonto-RAG, a cross-ontology GraphRAG framework that avoids ontological unification by constructing a dual-ontology graph with typed bridges induced via LLM-assisted verify-and-vote. A query-aware Relational Graph Attention Network (RGAT) then assembles compact, provenance-tracked multi-hop evidence subgraphs for grounded generation. Across TCM–WM benchmarks, Xonto-RAG outperforms strong RAG/GraphRAG baselines, seeing up to +22.2% accuracy on MLEC-QA TCWM (vs. OG-RAG), 77.0% accuracy with a 7B backbone, and 85.1% faithfulness on MLEC-TCM, while reducing semantic-collapse failures and yielding interpretable reasoning chains.

Abstract:
Instructions are never complete specifications of the behavior they request: speakers say only what their listeners cannot supply for themselves. An agent told to “prepare an apple” must decide unaided whether to wash or cut it and where to place it. Such choices differ between people and are seldom stated, only enacted. Multimodal foundation models render instructions into grounded plans, yet settle these residual choices by generic priors rather than by evidence about the person served. Whether an agent can instead recover such constraints from prior behavior has not been separated from the easier question of planning once they are supplied. Here we show the failure to be largely one of acquisition rather than planning, and that stating the inferred preference before acting recovers much of what is lost. Preference-based Planning (PBP) benchmarks 290 preferences over action parameters, placement policies, and temporal orderings, asking agents to infer from a few unlabeled demonstrations which constraint governs a new underspecified instruction. Given the preference, the strongest models plan categorical tasks at near-zero error; required to infer it, performance drops sharply, and without demonstrations accuracy falls by over a third. Because a verbalized rule keeps intent while discarding perceptual particulars, preferences so represented transfer to unfamiliar scenes, whereas implicit imitation stays bound to the rooms it was learned in. The bottleneck lies not in generating actions but in the pragmatic inference preceding them, for which language proves an effective medium.

Abstract:
World models are emerging as critical components for assessing the consequences of actions taken by interactive agents in online planning and offline evaluation. In text-based environments, world models are typically trained and evaluated with single-step metrics such as Exact Match, aiming to improve the similarity between predicted and real-world states, but such metrics have been shown to be insufficient for capturing actual agent behavior. To address this issue, we introduce a new behavior-consistency training paradigm aimed at improving the functional consistency between the world model and the real environment. This paradigm focuses on optimizing a tractable step-level reward named Behavior Consistency Reward (BehR), which measures how much the likelihood of a logged next action changes between the real state and the world-model-predicted state under a frozen Reference Agent. Experiments on WebShop and TextWorld show that BehR-based training improves task-level functional consistency in several settings, with the clearest gains in WebShop and smaller changes in near-ceiling regimes, while preserving or improving single-step prediction quality in three of four settings. BehR-trained world models also reduce false positives in offline surrogate evaluation and improve inference-time lookahead planning over the corresponding supervised world-model baselines. Code: https://github.com/Ricardo-H/behr-wm.

Abstract:
Existing referring Audio-Visual Segmentation (Ref-AVS) benchmarks primarily focus on explicit expressions whose referents can be resolved through direct visual, acoustic, or linguistic correspondence. Consequently, they provide limited coverage of implicit references that require multi-step reasoning over multimodal evidence. To bridge this gap, we introduce Reasoning Ref-AVS, a new task that requires models to infer the referred object from audio-visual-language cues before generating its segmentation mask. We further construct ReAVS, a comprehensive dataset with manually annotated referring expressions, multimodal cues, object masks, and reasoning chains, covering diverse reasoning spanning diverse reasoning patterns. Evaluations of representative Ref-AVS models on ReAVS reveal substantial performance degradation when direct cross-modal correspondence is insufficient, exposing a clear gap between multimodal recognition and referent reasoning. Motivated by these challenges, we propose Re2Seg, an omni-modal framework that performs object-centric evidence encoding, multi-hop reasoning, and reasoning-conditioned mask decoding. Experiments demonstrate that Re2Seg achieves stronger segmentation performance under reasoning-intensive references and provides interpretable object-inference traces. These results highlight the importance of learning to reason before masking for bridging the gap between multimodal recognition and referring segmentation.

Abstract:
Multilingual safety guardrails are essential for deployed LLMs, yet they perform poorly on low-resource languages. Prior work typically relies on large-scale multilingual safety data synthesis, which incurs substantial training costs. Cross-lingual transfer offers a cheaper alternative using limited parallel data, and recent work shows that aligning only last-token hidden states can improve multilingual safety. However, although effective for query-only (Q) risk detection, last-token alignment consistently degrades query-response (QR) detection in multilingual settings, limiting its applicability in practical guardrail systems where Q and QR serve as pre-generation and post-generation safety barriers, respectively. In this paper, we propose Q-Align, a method that decouples query intent from response evaluation by aligning only at the query boundary. Specifically, for each parallel sample, Q-Align extracts the hidden state right after the query and aligns multilingual query representations toward a confidence-weighted anchor, allowing languages with stronger intent understanding to guide transfer. We further apply language grouping based on geographic proximity to stabilize cross-lingual alignment. Experiments demonstrate that Q-Align consistently improves multilingual safety detection in both Q and QR scenarios, raising QR average F1 from 74.70% to 80.79% on Polyguard-Qwen, while preserving English performance and generalizing to languages unseen during training.

Abstract:
Recent latent visual reasoning methods achieve substantial gains by inserting continuous latent tokens into multimodal language models. These gains are commonly attributed to the tokens encoding visual evidence; recent analyses, however, reveal a paradox: the tokens are loosely tied to the image and contribute little to the answer. Critically, these analyses treat latent tokens as a single unit, obscuring the source of the gains. We therefore decompose latent tokens into three testable components: latent slots, boundary markers, and format, and develop a state-of-the-art method as a probe under favorable conditions. Across six method-stage settings and four perception-heavy benchmarks, latent slots fail every prediction of the visual-memory account. Strikingly, retaining only the boundary markers preserves 78 to 100% of the gain in several settings, while the model attends to the image more narrowly at latent positions than at answer positions. These results do not support slot contents as recoverable visual memory; much of the benefit is instead associated with marker and format control, together with visual-attention routing. At matched accuracy, methods can still rely on markedly different mechanisms shaped by training supervision. Latent visual reasoning thus needs evaluation not only by accuracy but by what the model actually relies on.

Abstract:
Modern supervised fine-tuning (SFT) trains large language models on heterogeneous instruction mixtures, where the effective unit of optimization is an individual instruction-response pair rather than a coarse task label. Existing parameter isolation methods mitigate interference by assigning protected or trainable subspaces at the task or training stage level, but these units are often misaligned with actual parameter usage: instructions from the same task may require conflicting computations, while instructions from different tasks may share latent capabilities. We propose Instruction-Level Parameter Isolation (IPI) , a fine-tuning framework that routes each instruction’s update to a sparse, capability-conditioned parameter subspace. IPI builds a multi-view instruction representation, predicts importance factors with a lightweight router, and uses an identity-trick reparameterization that preserves dense forward computation while exposing sparse base-model gradients and a task-loss saliency signal for router learning. Across heterogeneous instruction tuning benchmarks and multiple backbone models, IPI consistently improves performance over strong fine-tuning and parameter-isolation baselines. Further analyses show that instruction-conditioned masks reduce destructive gradient interactions and capture optimization structure that task labels miss.

Abstract:
Existing temporal QA benchmarks focus on simple fact-seeking queries from news corpora, while reasoning-intensive retrieval benchmarks lack temporal grounding. However, real-world information needs often require reasoning about temporal evolution and synthesizing evidence across time periods. We introduce TEMPO , the first benchmark combining temporal reasoning with reasoning-intensive retrieval across 13 domains. TEMPO features: (1) 1,730 complex queries requiring deep temporal reasoning such as tracking changes, identifying trends, or comparing cross-period evidence; (2) step-wise retrieval planning with 3,976 decomposed steps and gold documents mapped to each step for multi-hop evaluation; and (3) novel temporal metrics including Temporal Coverage@k and Temporal Precision@k measuring whether results span required time periods. Evaluation of 16 retrieval systems reveals substantial challenges: the best model (DiVeR) achieves only 32.0 NDCG@10 and 71.4% Temporal Coverage@10, demonstrating difficulty in retrieving temporally complete evidence. We believe TEMPO provides a challenging benchmark for improving temporal reasoning in retrieval and RAG systems.

Abstract:
Although representation learning has transformed semantic modeling in NLP, representations of linguistic style remain underexplored—partly due to conflicting definitions of style within and beyond NLP or unclear immediate advantages of separate style representations. In this survey, we provide an overview of style conceptualizations in NLP and (socio-)linguistics and suggest a definition of style for practitioners. Guided by this definition, we then review methods for creating and evaluating predefined and neural style representations. We conclude by discussing how style representations can make crucial contributions to the modern NLP pipeline (e.g., in dataset curation or evaluation) and to the application of NLP methods in other fields. We curate relevant datasets, models, and tools on our companion webpage at https://annawegmann.github.io/StyleSurvey/. Throughout, we provide practical guidance, identify open research questions with concrete recommendations for addressing them and issue calls to action on critical gaps in current work.

Abstract:
Multimodal Large Language Models (MLLMs) have recently been applied to universal multimodal retrieval, where Chain-of-Thought (CoT) reasoning improves candidate reranking. However, existing approaches remain largely language-driven, relying on static visual encodings and lacking the ability to actively verify fine-grained visual evidence, which often leads to speculative reasoning in visually ambiguous cases. We propose V-Retrver, an evidence-driven retrieval framework that reformulates multimodal retrieval as an agentic reasoning process grounded in visual inspection. V-Retrver enables an MLLM to selectively acquire visual evidence during reasoning via external visual tools, performing a multimodal interleaved reasoning process that alternates between hypothesis generation and targeted visual verification. To train such an evidence-gathering retrieval agent, we adopt a curriculum-based learning strategy combining supervised reasoning activation, rejection-based refinement, and reinforcement learning with an evidence-aligned objective. Experiments across multiple multimodal retrieval benchmarks demonstrate consistent improvements in retrieval accuracy (with 23.0% improvements on average), perception-driven reasoning reliability, and generalization.

Abstract:
Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.

Abstract:
Reasoning-oriented language model training relies on traces that expose learnable dependencies, yet existing data-selection criteria use length, perplexity, difficulty, or judge-assigned quality. We argue that useful reasoning traces should also be sensitive to dependency order: intermediate steps are easier to encode in their natural premise-to-conclusion order but less predictable when dependency-revealing order is perturbed. We introduce Local-DeltaAvg, a training-free data-selection metric. For each reasoning trace, we construct a semantically reversed counterpart that preserves problem content while perturbing dependency-revealing order. A fixed probe model scores both trajectories, and Local-DeltaAvg computes a paired local negative log-likelihood gap between the original and reversed contexts. The resulting probe-dependent score estimates the conditional encoding cost induced by order perturbation and identifies where this sensitivity is concentrated. We evaluate Local-DeltaAvg as a relative ranking signal for reasoning-oriented continued-pretraining data selection, followed by an identical downstream SFT protocol. Under matched token budgets, it consistently outperforms length-, perplexity-, difficulty-, and judge-based selection across diverse reasoning benchmarks. Control analyses show that semantic reversal is essential and that high-gap regions align with annotated critical reasoning steps. These results suggest that dependency-order sensitivity is a lightweight and effective proxy for reasoning-data utility.

Abstract:
Large language models (LLMs) can fluently verbalize statistical evidence, yet statistical reports can still drift numerical values, invert effect directions, or restate thresholded contrasts as categorical effects. We frame these failures as a control problem: the evidence-bearing content of a scientific report should be fixed by structured statistical results rather than sampled during prose generation. We therefore use cross-run reproducibility to stress-test whether report-visible numbers and claims are bound before prose generation. Existing controls operate at the text or slot level; a deterministic hybrid template reproduces only 61.1% of report-visible numerical content across seeds because the LLM still selects which findings and numbers the template renders. We propose claim-locked reporting, a provenance-before-prose protocol that fixes the evidence source, numbers, direction, and allowed language strength of each reportable claim before the LLM writes connective wording. Across fMRI functional-connectivity reporting and randomized controlled trial reporting on Evidence Inference 2.0, claim-locked reporting improves reproducibility over the hybrid template by 37.4 and 20.5 points, respectively. Blinded human audits support the observed direction-preservation and governance trends. In an fMRI cost analysis with DeepSeek, claim-locked reporting also yields the lowest observed token use and median generation latency. Code is available at https://github.com/XiuFan719/Claim-Locked-Reporting.

Abstract:
LLM-based text-to-speech (TTS) models can generate emotionally expressive speech, but how reference emotion is routed through the model and realized in decoded speech remains unclear. We introduce two emotion-sensitive metrics for matched neutral and emotional syntheses—a codec trajectory score and a late residual direction score—and use them to score activation-patching interventions. Under controlled matched-reference conditions, this analysis identifies a sparse source-to-readout component-level circuit: 23–27 attention heads and MLPs per emotion, roughly 5% of the components considered, recover or suppress 74–88% of the late emotion-readout shift on held-out cases. The circuit combines a shared component backbone with emotion-specific components; cross-emotion activation swaps reduce the target readout in 47 of 48 cases. In decoded speech, the same intervention produces consistent changes in pitch, energy, and spectral brightness over 24 matched pairs per emotion. A readout-matched residual-direction baseline produces only 17–27% of the intervention’s pitch effect, showing that internal readout movement alone does not explain the decoded acoustic changes. These results trace a compact causal route from reference-derived prefix information to emotion-relevant properties of generated speech.

Abstract:
Composed image retrieval (CIR) requires complex reasoning over heterogeneous visual and textual constraints. Existing approaches largely fall into two paradigms: unified embedding retrieval, which suffers from single-model myopia, and heuristic agentic retrieval, which is limited by suboptimal, trial-and-error orchestration. To this end, we propose OSCAR, an optimization-steered agentic planning framework for composed image retrieval. We are the first to reformulate agentic CIR from a heuristic search process into a principled trajectory optimization problem. Instead of relying on heuristic trial-and-error exploration, OSCAR employs a novel offline-online paradigm. In the offline phase, we model CIR via atomic retrieval selection and composition as a two-stage mixed-integer programming problem, mathematically deriving optimal trajectories that maximize ground-truth coverage for training samples via rigorous boolean set operations. These trajectories are then stored in a golden library to serve as in-context demonstrations for online steering of VLM planner at online inference time. Extensive experiments on three public benchmarks and a private industrial benchmark show that OSCAR consistently outperforms SOTA baselines. Notably, it achieves superior performance using only 10% of training data, demonstrating strong generalization of planning logic rather than dataset-specific memorization.

Abstract:
Although scaling up the number of trainable parameters can effectively improve the training performance of large language models, it also leads to increased computational overhead. When delving into the parameter difference, we find that a subset of parameters, termed advantageous parameters, plays a crucial role in determining model performance. Further analysis reveals that stronger models tend to possess more such parameters. In this paper, we propose Advantageous Parameter EXpansion Training (APEX), a method that progressively expands advantageous parameters into the space of disadvantageous ones, thereby increasing their proportion and enhancing training effectiveness, while keeping the total parameter count unchanged. Extensive experiments on both instruction tuning and continued pre-training across five base models demonstrate that, in instruction tuning, APEX outperforms full-parameter tuning while using only 52% of the trainable parameters. In continued pre-training, APEX achieves the same perplexity level as conventional training with only approximately 30% of the training data, and yields significant improvements on downstream tasks.

Abstract:
Although Sparse Autoencoders (SAEs) have mitigated the opacity of large language models (LLMs) by decomposing dense representations into sparse features, explaining these features still remains a central challenge. Current explanation methods, however, typically operate within an open-loop paradigm, failing to leverage mechanistic feedback for further refinement. In this paper, we propose SAEExplainer, a training framework that utilizes activation scores as an objective reward signal to train the model for self-correction and iterative bootstrapping. By iteratively verifying and correcting foundational explanations through a two-round optimization process, SAEExplainer achieves continuous improvement in its explanatory capabilities. This mechanism significantly reduces explanation hallucinations and reinforces causal triggering patterns. Extensive experiments demonstrate our approach improves upon established baselines across most metrics, especially in causal triggering and discriminative activation. The code is available at https://github.com/he-jingyi/SAEExplainer.

Abstract:
Large language models (LLMs) are increasingly deployed in multi-agent systems where a principal agent decomposes tasks and delegates them to subordinate agents that may invoke external tools. Safety alignment, however, is still evaluated almost exclusively under a single-agent threat model, treating safety as a property of the individual LLM. We show that this assumption breaks down: individual safety alignment fails to transfer to multi-agent settings. Two failure mechanisms emerge under delegation: responsibility diffusion on the principal side and role-bias compliance on the subordinate side, jointly converting language-level refusal into actionable harm. We refer to this phenomenon as delegated misalignment and study it through a three-condition protocol across 6 frontier LLMs on 49 hazardous tasks. Delegation amplifies end-to-end harm substantially: DeepSeek-V3.2’s full-execution rate rises from 30.6% to 77.6% once delegation is introduced, and the same model behaves very differently across roles (GPT-5: 22.5% as a single agent vs. 61.2% as a subordinate). Ablations further show that standard single-layer defenses each fail on their own and can even backfire. We call on the community to move beyond per-model alignment and toward composite safety mechanisms before multi-agent LLM systems are deployed at scale.

Abstract:
Detecting hallucinations in Large Language Model (LLM) outputs is critical for safe deployment, yet existing methods require white-box access to internals, depend on multiple generations or auxiliary judge models, or collapse token-level uncertainty into scalar summaries that discard temporal structure. We introduce HALT, a lightweight grey-box detector that treats the top-20 log-probabilities of an LLM generation as a multivariate time series and processes them with a small bidirectional GRU (5M parameters), in a single forward pass and using no model internals. We evaluate on HUB, a unified protocol reorganizing existing hallucination datasets into ten capability clusters spanning factual and logical failure modes. HALT matches or surpasses Lettuce, a fine-tuned ModernBERT-base detector, on macro- F 1 while using ∼ 30× fewer parameters; a controlled comparison against a text classifier trained on HALT’s own data confirms this advantage comes from the log-probability signal itself rather than from differences in training data, and the same recipe applies across eight LLM families from 360M to 70B parameters.

Abstract:
We rethink parametric retrieval-augmented generation with DuoPRAG , which integrates knowledge and reasoning capabilities through rank-disentangled and gated Low-Rank Adaptation (LoRA) composition. Existing parametric RAG methods encode document knowledge into LoRA parameters while leaving reasoning to be supplied through lexical chain-of-thought (CoT) prompting. This introduces three failure modes in outputs and contradicts the prompt-independent inference that parametric RAG aims to achieve. DuoPRAG parameterizes knowledge and reasoning into disentangled LoRA adapters, fused through a learnable mask and rank-specific gates. We achieve state-of-the-art performance on multi-hop QA benchmarks, outperforming the strongest parametric RAG baseline while preserving the parametric inference framework. Our results demonstrate that jointly parameterizing knowledge and reasoning is a natural and effective extension of the parametric RAG paradigm. Our code is available at https://github.com/naknak-choi/DuoPRAG .

Abstract:
Pretrained masked language models (MLMs) remain important backbones for natural language understanding (NLU) tasks, as they often provide a more favorable balance between task performance and computational efficiency than large decoder-only models. However, existing debiasing methods often decouple bias mitigation from downstream fine-tuning, making it difficult to preserve the mitigation effects after task adaptation. To address this, we propose ARCA (Attribution-Regularized Causal Alignment), an end-to-end framework that unifies debiasing quality and task utility. ARCA uses differential gradient attribution to identify task-specific bias words and regularize the intervention for causal alignment on counterfactual samples. This single-pipeline approach enables joint optimization of task utility and debiasing. Experiments on three mainstream NLU tasks show that ARCA achieves a strong trade-off between bias mitigation and task utility compared with representative debiasing baselines.

Abstract:
Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The problem requires determining both what (which visual event) and when (position for inserting the AD) to narrate, to achieve the best user experience. Prior work has largely reduced the problem to video captioning of pre-segmented video clips, i.e., what is largely predefined and when is ignored entirely. We propose Cue2Narrate, a two-stage pipeline that jointly predicts what and when to narrate in longer untrimmed movie clips. A dual-head audio-visual localizer predicts two temporally distinct windows per AD utterance: a visual cue window and a spoken narration window. A LoRA-adapted VLM then generates concise ADs from the predicted visual evidence, trained with a Description Ranking Loss that ranks captions (negative samples) of the same frames lower than the GT AD. To benchmark this new problem statement, we introduce the LongLSMDC benchmark with up to 8-min movie clips (~6.5min on average). On LongLSMDC, Cue2Narrate outperforms video-only and audio-only localization baselines by 5–12 points in avg. mAP. Under both predicted- and GT-window evaluation, Cue2Narrate improves AD generation over the corresponding fine-tuned base VLM. These results establish the first benchmark for multi-segment AD generation on long-form clips. Data Code: https://github.com/multimodal-ai-lab/Cue2Narrate

Abstract:
The LLM-as-a-judge (LLMaJ) framework offers a cost-effective and reproducible solution for automatic evaluation. However, current evaluation practices typically compare LLMaJ judgments against aggregated ground-truth labels, overlooking the valuable information contained in Human Label Variation (HLV). Inspired by an increasing line of work that proposes to leverage HLV, we systematically study LLMaJ performance on predicting both a single, aggregated ground truth hard-label and unaggregated soft-labels that represent Human Judgment Distributions (HJD). Our results across five diverse datasets reveal that while LLMs achieve near human-level performance at hard-label prediction on most tasks, they exhibit poor performance when predicting soft-labels. To address this limitation, we propose NAPHA (eNtropy-Aware Post-Hoc Alignment), a simple yet effective lightweight post-hoc alignment method that matches the LLM distribution to the HJD by first assigning an instance to a discrete entropy class and then routing it to specialized, trained alignment models. We find that NAPHA consistently improves soft-labels prediction across base LLM models and datasets, with particularly strong gains on high-entropy instances where capturing diverse human perspectives is most critical. We also show via oracle experiments that improving entropy class prediction can substantially enhance NAPHA’s practical effectiveness.

Abstract:
Post-training large language models usually apply a single training recipe to all samples, even though the model’s own rollouts reveal different sample-level learning states. We propose Self-Routing, a behavior-conditioned post-training framework that uses rollout correctness and confidence to decide how each sample should be optimized. Depending on its behavior state, a sample is routed to GRPO, on-policy self-distillation, regularization, or skipping, allowing training to adapt without external teachers, extra annotations, or additional sampling. Experiments on mathematical reasoning across Qwen3 and Qwen3.5 backbones show that Self-Routing consistently improves over uniform GRPO, uniform OPSD, fixed mixtures, and simpler routing baselines. Further analyses show that the routing distribution changes over training and reduces unnecessary updates on low-signal or already stable samples.

Abstract:
Reliable assessment of causal research designs in the social sciences is critical for evidence-based policy-making, yet has so far relied entirely on manual expert analysis. We introduce Automated Research Design Tracking and Assessment (ARDTrA), a task that involves detecting the research design used in a paper and assessing the quality of its application. We create an expert-annotated dataset of papers covering six families of counterfactual research designs and evaluate the task using a multi-turn RAG-based conversational pipeline. Across four retrieval strategies, four LLMs and six embedding models, we find that passage length is the main driver of performance, explaining 52–66% of the variance. A per-research-design analysis also shows that human and machine difficulty do not align: the designs that prove hardest for the system are not those on which expert annotators disagree most, pointing to two independent sources of task difficulty.

Abstract:
Retrieval-augmented generation (RAG) improves the factuality of large language models by grounding responses in external documents, but it also exposes a critical security vulnerability: adversarial documents injected into the knowledge database can enter the context window and steer the model toward targeted incorrect answers. Existing post-retrieval defenses rely on instruction following, parametric knowledge, or text-level consistency, all of which can be imitated or optimized against by adaptive attackers. We propose RAGSentinel, a training-free, label-free defense for black-box RAG systems. RAGSentinel uses a surrogate encoder to measure query-conditioned hidden-state shifts induced by retrieved documents, removes shared topic directions, and filters poisoned documents as geometric outliers from a robust majority consensus. We prove that, under an honest-majority assumption and a representation-level separation condition, RAGSentinel exactly recovers a poison-free majority-sized context. Experiments across three question-answering datasets, three LLM families, and multiple poisoning attacks show that RAGSentinel consistently achieves low attack success rates while preserving competitive accuracy and remaining effective against adaptive attacks with full pipeline knowledge.

Abstract:
Audio large language models (LLMs) have advanced rapidly in general audio understanding, yet consistently underperform on tasks requiring fine-grained acoustic discrimination. We investigate this gap through a systematic probing study, uncovering a phenomenon we term representation reorganization : the LLM’s middle layers construct semantic discriminability at the cost of selectively compressing fine-grained acoustic features such as speaker identity and pitch. Informed by this diagnosis, we propose Overtone, a delayed acoustic fusion architecture that bypasses the identified compression zone. Overtone processes identified audio segments through dual tracks—a semantic track from the LLM’s audio encoder and an acoustic track from a frozen self-supervised speech model—and injects acoustic features via gated cross-attention at post-compression layers. The model learns to autonomously decide when and where re-listening is needed during inference. On MMAU and MMAR, Overtone achieves state-of-the-art results among open-source models at comparable scale, with gains concentrated in speech and music—the domains most reliant on fine-grained acoustic features. Post-training probing confirms recovery of acoustic structure in the reorganized layers.

Abstract:
Large language models (LLMs) often produce fluent but weakly grounded conclusions when reasoning over non-interactive, long-form narratives. A central failure mode is that unsupported intermediate hypotheses can enter the reasoning trajectory and contaminate subsequent inference, especially when evidence is scattered across distant parts of the story. To address this problem, we propose EVAR, an evidence-validated hypothesis admission framework for budget-aware narrative reasoning. EVAR first compiles the narrative into an immutable evidence store of source-linked atomic claims and assigns an instance-specific inference budget from unresolved gaps and uncertainty signals. During refinement, EVAR directly proposes candidate hypotheses for unresolved gaps, constructs hypothesis-conditioned validation challenges, and verifies each candidate against the locked store before admission: supported hypotheses enter the answer-supporting state, unverifiable ones are quarantined, and contradictory ones are discarded. A sufficiency-based stopping mechanism further avoids unnecessary refinement. Experiments on NarraCrime and multiple public reasoning benchmarks show that EVAR improves both task performance and evidence faithfulness while maintaining controllable inference cost.

Abstract:
Text embeddings in RAG indexes, vector stores, and agent memories are rarely leaked as isolated vectors: they often appear with document IDs, chunk offsets, and preserved order. This paper shows that such ordered provenance is not passive metadata but an active privacy channel. We introduce MIRAGE, a public-shadow, zero-victim-pair attack that exploits ordered document-chunk leakage by turning facts recovered from earlier chunks into encoder-verified anchors for later reconstruction. MIRAGE builds public-shadow single-chunk candidates, extracts typed anchors, verifies them with counterfactual encoder-support checks, and uses verified memory to regenerate document-consistent chunks. Across five dense encoders and four decoders on ordered Enron and synthetic CanaryDocs, under a strict 512-query victim-time budget per chunk, MIRAGE outperforms strong fixed-bank reranking, improving macro privacy-unit recall by 5.65-6.10 points across encoders and 5.2-6.9 points across decoders; gains persist (+3.7-4.4 points) on two real, non-templated corpora. Its support check raises accepted-anchor precision from 43.6% to 69.4%, and 34.8% of newly recovered privacy units appear only in regenerated candidates absent from every fixed bank. These results reveal document grouping, chunk order, and offsets as sensitive attack surfaces that RAG and vector-store deployments must protect alongside embeddings themselves.

Abstract:
Large language models (LLMs) have shown strong abilities in writing and revising programs, yet many program-synthesis benchmarks still evaluate programs in symbolic or digital environments. We introduce compositional machine design , a physically grounded form of program synthesis where machines are written as programs that compose standardized parts, and success is determined by simulated physical behavior. To study this problem, we present BesiegeField , a testbed built on the machine-building game Besiege . In BesiegeField , LLM agents generate machine programs from textual functional demands, execute the resulting machines in simulation, and receive rewards and state feedback. We benchmark LLM agents across representative machine-design tasks under single-agent generation, iterative editing, and hierarchical workflows. Strong models recover task-relevant structures and sometimes achieve nontrivial physical performance, but often struggle with spatially precise assembly, mechanism-level planning, and translating feedback into useful structural edits. We further finetune Qwen2.5-14B, an open-source LLM, with reinforcement learning from simulation-derived rewards. We find that, under a fixed generation budget, RL improves the best machine discovered. We additionally evaluate human performance to provide a reference point for task difficulty. These results establish compositional machine design as a testbed for studying LLM agents that synthesize executable machine programs and improve them through physical feedback.

Abstract:
Fine-tuning Vision-Language Models (VLMs) is a common practice to adapt pre-trained models for downstream tasks. Yet existing fine-tuning methods consistently keep the pre-trained architecture intact, leaving open the question of whether maintaining such structural integrity is truly necessary for effective adaptation. To investigate this, we propose Divide-and-Adapt, a new fine-tuning paradigm that departs from preserving the full pre-trained structure. It first divides the pre-trained model into multiple downstream-oriented subnetworks, and then adapts them collaboratively through a routing-based coordination mechanism. This design facilitates the selection of structures that are most beneficial for downstream tasks and enables their collaborative integration. Remarkably, even without any task-specific training, our Divide-and-Adapt framework first divides the pre-trained model into downstream-oriented subnetworks and adaptively activates them for downstream inference, already outperforming the pre-trained model on downstream tasks. With further fine-tuning, Divide-and-Adapt surpasses both full fine-tuning and parameter-efficient baselines across multiple benchmarks, demonstrating its effectiveness. Our code will be made publicly available at https://github.com/haiiling/DivideAndAdapt.

Abstract:
Diffusion language models (DLMs) have recently emerged as a competitive alternative to autoregressive (AR) models, offering parallel decoding, competitive generation quality, and initial evidence of improved jailbreak robustness. Despite this progress, the role of sampling mechanisms in shaping refusal behavior remains poorly understood. To address this gap, we present a comprehensive study of step-wise refusal dynamics. We show that diffusion remasking can promote recovery from harmful intermediate generations, provide evidence that this behavior is tied to the sampling mechanism, and demonstrate that switching from AR to diffusion sampling improves jailbreak robustness, including under fixed model weights. To capture generation dynamics not observable at the text level, we propose the Step-Wise Refusal Internal Dynamics (SRI) signal. Consistent with our text-level findings, SRI shows that recovery fails primarily under AR sampling, with these failures often appearing anomalous relative to harmless generations in the SRI space. Based on this observation, we show that SRI enables a simple jailbreak detector that does not modify inference and generalizes to unseen attacks by training only on benign SRI signals. Our evaluation shows that this detector matches or outperforms existing jailbreak detection baselines while adding negligible overhead.

Abstract:
Fair allocation of scarce, indivisible resources is an important challenge in many societal problems. While there are several formal theories of fairness, no single definition can always be satisfied. As large language models (LLMs) are increasingly used to support decisions and act as agents, they raise new concerns about distributional justice: their judgments are not directly tied to any specific fairness framework and may violate key normative principles. In this work, we introduce a general method for evaluating fairness reasoning in LLMs. We study first-person fairness judgments across a broad set of models and compare them directly with human responses on matched scenarios and elicitation conditions. We find that LLMs tend to prefer stricter fairness constraints than humans, show more self-interested behavior, are sensitive to how information is framed, and are difficult to align with human judgments using fine-tuning with current datasets.

Abstract:
As LLM-based human simulators are increasingly used for policy, evaluation, and training, they must faithfully reproduce real behavioral patterns. While prior work has examined behavioral fidelity in survey responses and dialogue, longer-horizon real-world activity remains largely unexplored. We introduce a framework for evaluating behavioral fidelity in long-horizon activity simulations across temporal granularities and levels of analysis. As a case study, we collect a 43-hour multi-camera dataset of in-the-wild office activity and compare trace-derived conditioning mechanisms: persona descriptors, few-shot exemplars, and statistical transition and time-of-day priors. We find that behavioral fidelity is not uniform across metrics: statistical priors bring activity and sequence distributions closest to real behavior, yet over-fragment routines and suppress within-person variability. These findings motivate a more holistic evaluation that spans multiple metrics, temporal granularities, and levels of analysis.

Abstract:
Small open-weight models struggle at long-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models. We present POLARIS (Policy Optimization with LLM-as-a-judge rewards and Anchored-Reference Injection for Storywriting), a lower-compute GRPO recipe with two key ingredients: a frontier LLM judge with a structured Story Quality rubric as the online reward, and human-reference injection (HRI), where a teacher-forced human-written story serves as a high-reward anchor within each GRPO group. By applying our training recipe to Qwen3.5-9B, using a dataset of ∼ 1.4K prompt–story pairs derived from 100 short-story anthologies and 4 A100 GPUs, we obtain POLARIS-9B. Across five benchmarks spanning in-distribution and out-of-distribution prompts and rubrics, POLARIS-9B is competitive with much larger open-weight models while following length instructions more closely. A blinded human evaluation confirms that POLARIS-9B is preferred to the base Qwen3.5-9B and on par with Qwen3.5-27B. Despite training only on stories up to 4k words, POLARIS-9B preserves quality on prompts requesting stories up to 3× the training length, a regime where most open-weight models degrade substantially in quality, length adherence, or both. More broadly, our results suggest that length generalization is a meaningful stress test for creative-writing models and a useful lens for distinguishing otherwise close models.

Abstract:
Low-rank adaptation introduces a static learned update applied identically to every input. The update provides task-level adaptation but does not explicitly represent token-level or instance-level state variation. A family of adapters is proposed that introduces selective state-space control at two complementary granularities. At the token level, MaLoRA (Mamba-modulated low-rank adaptation) makes the adapter’s scaling factor a dynamic input-dependent function with recurrent state across tokens, in contrast to the stateless modulators of prior work. The token-level adapter improves over low-rank adaptation. On the other hand, it differentiates tokens by structural role but not by contextual relevance, which motivates placing evidence selection at the context level. At the context level, MaRA (Mamba Retrieval Adapter) tracks cross-segment reasoning state and selects the segments most relevant to the query. State-space controlled retrieval of approximately three million parameters exceeds an eight-billion-parameter dense retriever on supporting-paragraph recall. Although base models perform poorly on the task without adaptation (14 to 25 F1), MaRA recovers the evidence relevance latent in their representations. Across three frozen backbones and two multi-hop reasoning benchmarks, the end-to-end family improves reasoning accuracy on every cell of the 3-by-2 grid, by +6.4 F1 (+10.0% relative) on average over the LoRA baseline.

Abstract:
Cybersecurity combines high-stakes analysis with complex technical language, making it an impactful and challenging domain for LLMs. We present MiST (Mid-trained Security Transformer), a suite of 8B and 32B models that achieve strong performance on public cybersecurity benchmarks. We use mid-training as an intermediate adaptation stage between general pre-training and cybersecurity training. Rather than performing continual pre-training over large volumes of raw domain text, we curate a compact, expert-vetted seed corpus, and transform it into high-quality domain-specific synthetic training data. The final MiST checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute percentage points over the corresponding Qwen baselines for 8B and 32B, respectively, corresponding to relative gains of +27.0% and +15.8%. Ablation results further show that these cybersecurity gains arise in the mid-training and supervised fine-tuning stages through a combination of the synthetic data generation flows. Furthermore, we show that MiST provides a stronger initialization for downstream task-specific fine-tuning adaptation and reinforcement learning.

Abstract:
Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify. Despite being essential for downstream tasks like speech recognition and dense audio captioning, timestamping remains a key limitation of most LALMs. We present TEMPO (Temporally-grounded Multi-task Post-training), the first unified model to handle audio, speech, and music timestamping tasks. Our core contribution is a supervised fine-tuning (SFT) stage built on three innovations: atomic timestamp tokens, a time-aware projector that injects sinusoidal wall-clock encodings into audio frame embeddings, and a distance-aware Gaussian loss. Our training is based on a synthetic-to-real curriculum. We further introduce, to our knowledge, the first application of reinforcement learning to unified audio timestamping, using GRPO with verifiable temporal rewards that directly optimize the evaluation objectives. Rather than serving as the primary source of performance gains, GRPO acts as a refinement stage on top of the SFT checkpoint, providing modest additional improvements. To support this work, we build a training dataset containing 119K samples and an evaluation benchmark containing 10K samples, drawn from established corpora across five tasks. On this benchmark, TEMPO outperforms Audio Flamingo Next and Qwen3-Omni, two state-of-the-art LALMs explicitly trained on timestamped data. Experiments confirm that SFT delivers most of these gains, with GRPO providing consistent but moderate refinements.

Abstract:
Conversational legal retrieval requires identifying statutes that are better aligned with a user’s case context, even when relevant facts are revealed incrementally and incompletely across dialogue turns. Sparse and dense retrievers may return semantically related statutes that are poorly matched to these factual conditions. We propose SAGA, a training-free legal retrieval framework that injects structured applicability-oriented cues into dense retrieval. SAGA constructs a Case Applicability Frame (CAF) from user-side dialogue context to summarize dispute issues, core facts, and missing facts, and uses an offline Article Applicability Frame (AAF) to represent statute-side issue tags, prerequisites, legal effects, and evidence cues. It combines raw semantic and structural applicability streams through late fusion. Experiments on two Chinese statute retrieval benchmarks, LexRAG and STARD, show that SAGA improves over strong sparse, dense, and LLM-enhanced baselines under a unified full-corpus protocol. SAGA yields statistically significant gains on the conversational LexRAG benchmark and more modest gains on STARD over the strongest dense baseline, improving nDCG@10 from 0.3504 to 0.3696 and from 0.5862 to 0.5940, respectively. These results suggest that structured applicability-oriented cues provide useful retrieval signals beyond raw semantic matching, especially in conversational legal retrieval.

Abstract:
Discovering new intents is crucial for many systems to adapt to evolving users’ needs, making it inherently a continual process. However, most related studies have not explored such continual learning scenarios for new intent discovery. To bridge this research gap, we formalize the Continual Open-Stream Intent Discovery (COSID) task, which requires simultaneous detection of historical intents and discovery of new ones from hybrid streaming data. Our central contribution is a configurable open-stream dataset construction protocol that transforms a static intent dataset into a stream of temporally partitioned datasets, as well as a set of evaluation metrics for the COSID task. We further propose the Continual Intent Discovery Agent (CIDA) for the task, a memory-augmented agent that screens the incoming stream, clusters new-intents’ samples, and employs an LLM for intent abstraction and memory expansion. Experiments on the ClincCL and BankCL data streams constructed according to our protocol validate both the protocol expressiveness and agent superiority over existing baselines.

Abstract:
LLM agents solve repository-level coding tasks through multi-turn tool use, but utilize half their budget on locating faults before editing. Dedicated localization frameworks have emerged, yet are still evaluated as file retrieval rather than actionable diagnosis, producing locations without the diagnostic context a repair agent needs. We introduce SHERLOC (Structured Hypothesis-driven Exploration and Reasoning for Localization), a training-free framework pairing a reasoning LLM with compact repository tools and self-recovery, without fine-tuning or multi-agent orchestration. SHERLOC reaches state-of-the-art localization across model scales: 84.33% accuracy@ 1 on SWE-Bench Lite and 81.27% recall@ 1 on SWE-Bench Verified; at ∼30 B parameters, it matches or outperforms other agentic methods. Injecting our locations and diagnostic findings into repair agents yields an average +5.95 pp resolve-rate gain from the best SHERLOC result per setting on SWE-Bench Verified. SHERLOC cuts localization and total tokens by 36.7% and 23.1% on average.

Abstract:
LLMs now sit on every side of evaluation: as examinees scored on benchmarks, judges of other models’ outputs, and raters of human-generated content. Each paradigm can be viewed as a measurement problem, where a latent property of an object is probed with items from an instrument (e.g., benchmark) by judges or raters. Standard evaluation practices often neglect the contributions of each core component to the end result, limiting our understanding of what is being measured. Rasch measurement theory (RMT) is well-suited to this problem. RMT decomposes ordinal ratings into separable facets on a common scale. It further provides a battery of diagnostics that can identify miscalibrated measurements and rater biases. We present a case study of RMT applied to the LLM-as-rater paradigm using the Measuring Hate Speech corpus, whose construct was itself built under RMT. We fit a series of many-facet Rasch models to annotations from nine LLMs spanning families and capability levels. Our analyses show that LLMs systematically differ from human raters in severity, item-level calibration, question-order robustness, target-identity sensitivity, and rating scale use, all of which standard evaluation practice would largely obscure. Overall, we argue that RMT belongs in the toolkit for evaluating LLM-as-examinee, -judge, and -rater paradigms.

Abstract:
Instruction-following is a core capability of Large Language Models (LLMs), as they must reliably comply with instructions constraints before being able to be evaluated on downstream tasks. In this scenario, instruction-following benchmarks for low-resource languages remain limited, particularly for Portuguese, where existing evaluations are mostly based on translated English datasets and fail to capture language specific linguistic phenomena. We introduce Portuguese-IFEval, a benchmark for evaluating instruction following in Portuguese built through semantic regionalization. The benchmark contains a total of 1236 instructions across 535 prompts, each prompt have up to three verifiable instructions, combining adapted instructions with Portuguese-specific orthographic, grammatical, and discourse-level constraints. We evaluate 25 proprietary, open-source, and Portuguese-tuned LLMs under a unified protocol, establishing a comprehensive benchmark for Portuguese instruction-following evaluation. Our results show that, while English and Spanish remain the strongest-performing languages, Portuguese still presents substantial challenges, which are effectively exposed by our benchmark. Our code is available at: https://github.com/gap-ufg/Portuguese-IFEval.

Abstract:
Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existing evaluations, however, are often topic-specific or synthetic, limiting their ability to capture the complexity of "in the wild" information-seeking queries and the risks present in model responses. To address this gap, we introduce WildSEEK, a manually annotated dataset of 3k information-seeking queries from real user interactions, and an evaluation framework for LLM-generated responses. WildSEEK includes annotations for risk-sensitive domains (e.g. health and financial information), and distinguishes factoid queries from analytical queries which seek responses beyond facts. We train classifiers on WildSEEK to analyze more than 1.8M realistic user queries. We find that over a third of information-seeking queries are high-risk and more often analytical. Our findings show that LLM responses fail more often in four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations — with failure rates being mostly higher for analytical queries. By providing methods to monitor the reliability, safety, and fairness of LLM behavior, our dataset and evaluation framework offer an empirical foundation for the broader question of how these systems should behave as they take on a growing role in information access.

Abstract:
Distilling reasoning traces from strong teacher models has become the standard recipe for building capable small language models. Yet reasoning traces are 5-20 × longer than standard instruction fine-tuning (IFT) outputs, meaning every practitioner who chooses reasoning distillation implicitly forgoes training a larger IFT model on the same compute budget. Whether this trade-off is worthwhile remains unaddressed. We study it with a controlled experiment: a single teacher generates paired IFT and reasoning outputs for identical prompts by toggling only its reasoning mode, isolating supervision format as the sole variable. Training students at five scales (0.5B to 14B) and evaluating on 18 benchmarks, we find that at matched FLOPs, IFT lies on or near the Pareto frontier across the majority of configurations. Reasoning reaches the Pareto frontier only on open-ended tasks at 7B and above. Even there, a sequential curriculum mixing just 25-50% reasoning data with IFT captures most of the accuracy benefit at far lower compute cost.

Abstract:
Direct Preference Optimization removes the explicit reward model of RLHF but inherits its central failure mode: the preference-reward proxy can keep improving while downstream reasoning, factuality, or safety degrades. We ask whether this reward overoptimization, which we label validation collapse as an operational name for the trajectory we measure, is visible in weight-space geometry. Drawing on Random Matrix Theory and the Heavy-Tailed Self-Regularisation hypothesis, we propose Spectral DPO, which augments the DPO objective with a stochastic penalty computed from a truncated top- k Lanczos estimate of the spectra of effective adapted projections. The penalty holds the fitted tail exponent near a heavy-tailed reference, caps scale-normalised growth of the largest singular direction, and matches the estimated spectrum to a heavy-tailed target. Across ten matched seeds, three model families, and five benchmarks spanning instruction following, dialogue, mathematical reasoning, factuality, and adversarial robustness, Spectral DPO gains 2.33–2.42 points over the strongest included baseline and sustains accuracy through ten epochs where vanilla DPO degrades. Controls and a post-hoc spectral repair rule out norm clipping, KL anchoring, and length inflation, and support the relevance of spectral geometry without identifying a single causal statistic. All runs use rank-64 LoRA, so the evidence concerns effective-spectrum control under rank-constrained adaptation.

Abstract:
Although pluralistic societies exhibit diverse human values that lead to legitimate disagreements in subjective tasks (e.g., safety and preference judgments), NLP models trained on such subjective labels often ignore latent value structures, resulting in miscalibrated predictions over relevant value classes. We propose MultiCalibrated Subjective Task Learning (MC-STL), a framework that identifies latent value groups from annotations (via label rationale similarity, expert value taxonomies, or annotator sociocultural descriptors) and enforces value-conditional calibration through value group-specific representations. MC-STL applies to binary, ordinal, and preference learning settings, and is evaluated on multiple datasets covering toxic chatbot conversations, value reasoning, T2I-safety and preference alignment. The results demonstrate that MC-STL consistently outperforms existing baselines, achieving multicalibration across relevant value groups, while delivering gains in probabilistic prediction performance.

Abstract:
Efficiency is increasingly important for Large Language Model (LLM)-based multi-agent systems (MAS), as larger models and more agents introduce substantial execution costs. Recent methods aim to make MAS cheaper by pruning agents, removing communication edges, or searching for compact structures. However, we argue that existing evaluations may overestimate their true ability to improve MAS efficiency. Reported gains are often measured under method-specific prompts and starting topologies, making them difficult to attribute to the proposed structural changes. Moreover, many reported successes appear in non-MAS-demanding settings, where a single agent or a randomly pruned system can already preserve strong performance. To study these issues, we introduce a controlled and MAS-demanding diagnostic benchmark for representative MAS efficiency methods. We evaluate methods under a shared backbone model, agent registry, and runtime, across controlled variations in topology, scale, depth, and tool use. Our analysis shows that many reported gains are setup-dependent and may arise from structural collapse, disabled tool pathways, or starting systems where random pruning already preserves accuracy, rather than robust improvements in MAS efficiency.

Abstract:
Large Language Models (LLMs) consistently benefit from scaled Chain-of-Thought (CoT) reasoning, but also suffer from heavy computational overhead. To address this issue, efficient reasoning aims to incentivize short yet accurate thinking trajectories, typically through reward shaping with Reinforcement Learning (RL). In this paper, we systematically investigate the mechanics of efficient reasoning for LLMs. For comprehensive evaluation, we advocate for more fine-grained metrics, including length distribution conditioned on correctness and performance across a wide spectrum of token budgets ranging from 2k to 32k. First, we reveal that the training process follows a two-stage paradigm: length adaptation and reasoning refinement. After that, we conduct extensive experiments (about 0.2 million GPU hours) in a unified protocol, deconstructing training prompts and rollouts, reward shaping, and optimization strategies. In particular, a key finding is to train on relatively easier prompts, ensuring the density of positive reward signals and thus avoiding the length collapse. Meanwhile, the learned length bias can be generalized across domains. We distill all findings into valuable insights and practical guidelines, and further validate them across the Qwen3 series, ranging from 0.6B to 30B, demonstrating robustness and generalization.

Abstract:
This paper introduces Dimple and Dimple+, two discrete diffusion multimodal large language models (dMLLMs). Dimple is derived from a discrete diffusion large language model (dLLM) that initially lacks multimodal understanding and acquires this capability through a hybrid training strategy that begins with autoregressive learning and subsequently transitions to discrete diffusion training. In contrast, Dimple+ is initialized from an autoregressive multimodal large language model and attains parallel decoding ability via purely discrete diffusion training. Both models achieve performance on par with their autoregressive counterparts, while Dimple+ further sets a new state of the art among existing dMLLMs. To improve inference efficiency, we propose Confident Decoding, a method that dynamically adapts the number of tokens generated at each iteration. Experimental results demonstrate decoding speedups of 2× – 6× with only marginal performance loss. Additionally, we show that the Prefilling technique, previously developed for autoregressive models, can be effectively extended to dMLLMs with bidirectional attention, yielding nearly lossless acceleration of 1.7× – 7× . Finally, we present the Structure Prior method, which enables fine-grained control over output format and reasoning structure, a capability that is challenging to achieve in autoregressive models.

Abstract:
Multimodal Large Language Models struggle with tabular reasoning on tables containing images as cells owing to their limited memory and the distractions posed by the surrounding irrelevant visual elements. Existing context-reduction methods reduce context size but degrade reasoning accuracy: text-only pruning neglects visual cells, token-level pruning discards valuable information, and retrieval cannot understand table structure. We propose CAPTR: Caption-based Context Pruning for Tabular Reasoning, a two-stage framework that uses image captions as a lightweight textual proxy to prune non-pertinent rows and columns and then restores images for reasoning. CAPTR can reduce the context length by 80% while achieving up to 8% accuracy improvement against competitive baselines. By analyzing the error cases, we find that entity confusion contributes to the majority of the remaining errors. CAPTR is the first multimodal context-reduction method to improve performance on multimodal tabular reasoning.

Abstract:
Tool-using large language models (LLMs) have improved at producing executable API calls, but reliable tool use requires coordinating semantic intent with executable invocation, especially in multi-step settings. Existing tool-learning objectives and traces collapse these two roles into a single homogeneous generation stream, so the learning signal cannot distinguish a flawed plan from a flawed call. We propose Dual-Mode Unified Policy Optimization (DUPO), a typed trajectory optimization framework that represents each step with a PLAN span for the current subgoal and a TOOL span for the API invocation. DUPO follows a PLAN-to-TOOL rollout factorization, optimizes the two modes with mode-separated policy ratios, and introduces Chain-Aware Upstream Advantage, so that early planning errors, which propagate through the rest of the trajectory, receive stronger credit than later steps. To support this optimization, we introduce PlanArena, a human-annotated dataset of 20,646 trajectories across 7,941 scenarios with explicit PLAN–TOOL supervision. Experiments on BFCL, API-Bank, and Tau2 show that DUPO improves tool-use performance, with gains concentrated in multi-step settings. Our code and dataset will be released upon publication.

Abstract:
Recent multilingual vision–language encoders cover hundreds of languages in a single model, yet on two state-of-the-art instances retrieval on low-resource languages (LRL; e.g. Swahili) trails high-resource ones (HRL; e.g. English) by 30 + pp. We ask where in the trained encoder this gap is located. Prior modality-gap and cross-lingual subspace work suggests a linear language direction at the output crowds out alignment-relevant geometry. We falsify this: LEACE drives the linear language classifier from >99% to near chance and iterated INLP to 37–50% while LRL retrieval moves within ± 1.5 pp and all tier means within 2.2 pp, tracking random controls. The linear bias is a symptom, not the cause. Instead, the alignment-causal factor lies along the encoder’s forward path: the EOS (end-of-sequence) hidden state’s per-language trajectory diverges with depth. Substituting the EOS with its parallel English value three blocks before the projector lifts Swahili from 22.1% to 69.1% on one encoder (and reproduces on the other); three controls rule out pooled-position tautology and English specificity. A front-layer trunk that pulls each language’s projection toward the parallel-content centroid corroborates the diagnosis at training time, recovering +9.6 / +17.1 pp on LRL XM3600 retrieval (1,000-image subset), with consistent gains across three further benchmarks while preserving HRL performance.

Abstract:
Aligning large language models to human-centered objectives is difficult when targets are non-executable and context-dependent, limiting reliable verification and scalable supervision. Although synthetic data expands coverage, weak verification shifts the bottleneck from generation to selection. Noisy signals destabilize iterative refinement and can cause silent regressions. We propose Agentic Data Evolution (ADE), a data-centric framework that organizes synthetic supervision as evolving data snapshots. ADE improves data snapshots through a closed-loop Observation-Variation-Selection (OVS) procedure, where a steady-state admission mechanism acts as a quality ratchet that conservatively gates updates for sustained cross-round improvement. We validate these improvements through complementary intrinsic trend tracking and extrinsic post-training evaluation. On DEV300, ADE raises the intrinsic win rate from 50% to 75.81% and the extrinsic win rate from 55.20% to 68.86%, consistent performance gains across diverse benchmarks. Blind expert evaluation further confirms this, with a 66.11% preference for evolved answers. These gains generalize effectively across various model scales, post-training methods, and beyond weakly verifiable objectives. Resources are available at https://github.com/ZeroLoss-Lab/Agentic-Data-Evolution.

Abstract:
Compared with relying solely on initial observations and language instructions, predicting goal images with generative models as high-level visual guidance can significantly enhance the robustness of Vision-Language-Action (VLA) models. However, most existing foundation models have not systematically incorporated goal image conditioning due to the high computational training cost. To this end, we propose Goal-Injected Fine-Tuning (GIFT), a lightweight and efficient fine-tuning framework that seamlessly integrates generated goal images into multiple representative pretrained VLA models. Our approach introduces goal image features into observations via a zero-initialized convolution which progressively grows parameters from zero and prevents harmful noise from disrupting the pretrained policy during fine-tuning. As training proceeds, goal information is gradually incorporated, enabling efficient goal understanding without disrupting model stability. We further introduce a refined image editing method to generate semantically and visually consistent goal images from initial observations and task instructions. Experiments show that goal-aware VLA models achieve substantial performance gains across tasks: with only a single epoch of fine-tuning, GIFT outperforms the base model by 6.0% and 13.4% on two SIMPLER settings, and by 4.7% on LIBERO, demonstrating both efficiency and effectiveness.

Abstract:
Open machine learning engineering tasks require agents to interpret data, build models, debug code, and produce valid competition submissions. Although current coding agents can generate runnable programs, they often rediscover solutions across tasks and repeat engineering mistakes across feedback rounds. We propose EEMA, an experience-evolving meta-agent framework that organizes accumulated experience by purpose and routes it during multi-round solving. Its static component distills public competition material and historical failure traces into compact Skills: task-level modeling priors for improving solution quality and failure-prevention contracts for maintaining execution validity. Its dynamic component, Experience Routed Search (ERS), operates above a lower-level coding agent. At each round, ERS selects a draft, debug, or improve action, routes the corresponding Skill, and updates task memory, allowing cross-task experience to guide code generation while task-level feedback shapes later decisions. On MLE-bench Lite (22 tasks), EEMA achieves a 77.3% medal rate, 20.1% average leaderboard Grade, and 96.0% execution success, outperforming all baselines including GPT-5.5-xhigh. Ablations show that task Skills improve solution quality, failure-prevention Skills improve reliability, and ERS coordinates both effectively. The materials will be released at https://github.com/FrontisAI/OpenRSI.

Abstract:
Large language models achieve remarkable performance via extended chain-of-thought (CoT) reasoning, yet this lengthy process incurs substantial inference overhead. Existing CoT compression methods struggle with inflexible manual length budgets, computationally expensive multi-stage training pipelines, and fragile scalability restricted to small models. We propose HMPO (Hybrid Median-length Policy Optimization), a cost-effective, single-stage reinforcement learning framework. HMPO efficiently compresses CoT via three synergistic components: an adaptive median-based budget derived from successful rollouts to eliminate manual tuning, a cosine-decay token reward for smooth length penalization, and a multiplicative reward formulation that substantially mitigates trivial reward hacking by strictly prioritizing answer correctness. Trained exclusively on mathematical data, HMPO generalizes seamlessly across math, code, science, and instruction-following tasks. Extensive experiments scaling from 9B to 122B parameters across dense and Mixture-of-Experts (MoE) architectures demonstrate that HMPO achieves 19%–46% token compression with negligible accuracy degradation, all while drastically reducing training costs compared to existing multi-stage baselines.

Abstract:
Medical visual question answering (VQA) is a crucial task in clinical AI, yet its evaluation has so far centered almost exclusively on English, limiting its relevance to linguistically diverse patients and clinicians. Recent multilingual medical VQA benchmarks show that large vision-language models (LVLMs) degrade in non-English languages, but lack a fine-grained analysis of how cross-lingual variation affects the distinct capabilities that medical VQA requires. To this end, we construct a multilingual medical VQA benchmark over eight languages, organized into four representative scenarios that isolate the core capabilities medical VQA requires. Evaluating five open- and closed-source LVLMs, we find that cross-lingual degradation is not uniform but highly scenario-dependent. We therefore propose MedVL-XLRepE, a training-free scenario-aware representation engineering method, leveraging LVLMs’ superior English medical VQA capability to steer non-English representations toward their English counterparts at inference time. Across three LVLMs and eight languages, MedVL-XLRepE consistently mitigates cross-lingual degradation, with gains of up to 6.33%.

Abstract:
Large language models (LLMs) frequently generate source code containing vulnerabilities, yet little work studies the internal mechanisms that distinguish safe from vulnerable generation in them. In this work, we systematically perform a mechanistic interpretation of LLMs, aiming at both understanding how code safety-vs-vulnerability is represented or driven by components in an LM and turning the insights into actionable steering strategies to encourage safer code generation. To this end, we introduce CodeSec-Pairs, a dataset of 9,342 Python safe-and-vulnerable contrastive code pairs, sampled from Llama-3.1-8B-Instruct. Utilizing the dataset, we explore approaches to localize layers and attention heads that relate to code safety, and further experiment with different steering strategies for inference-time vulnerability reduction. In particular, we propose DuoSteer, a double-steering approach that simultaneously applies safety and code-correctness steering to attention heads. In experiments over five vulnerability types, DuoSteer leads to an average of -26.9% vulnerability rate reduction and +7.5% functional correctness improvement, which outperforms not only other steering variants but also prompting and supervised fine-tuning baselines. The advantage also replicates on Qwen-2.5-Coder-7B-Instruct with another 2,500 contrastive pairs sampled from that model.

Abstract:
Collaborative dialogue can end with apparent agreement while participants still differ on goals, assumptions, or execution plans, creating an illusion of alignment (IoA). A real-user study across 18 meetings confirms that IoA arises routinely in human collaboration. Yet IoA poses a paradox: if participants were aware of such disagreements, they would already be explicit; if not, they cannot articulate them when asked, leaving IoA invisible to both participants and observers. In this work, we make IoA detectable by generating diagnostic multiple-choice questions whose divergent answers across participants provide direct behavioral evidence of hidden disagreement. We construct IoA-Suite, a dataset and evaluation protocol for detecting hidden disagreement, spanning five task types and six domains. We find that even the best model attains only 49.5% F1, with the bottleneck traced to private context that the dialogue does not surface. We then train IoA-Prober-8B based on IoA-Suite, reaching 51.8% F1 on IoA-Suite. Across the aforementioned 18 real meetings, it surfaces 2.89 hidden disagreements per meeting that participants confirm they had not voiced, transferring to live human dialogue. Further, in multi-agent collaboration, pairing IoA-Prober-8B with LLM agents improves downstream task performance on BigCodeBench-Hard and HiddenBench.

Abstract:
Multimodal Emotional Reappraisal (MER) aims to shift affective interpretation by rewriting and editing a situation without altering its underlying semantic context. Despite advances in controllable editing and sentiment transfer, MER remains challenging under coupled multimodal constraints, especially in internet memes where images, embedded text, and layout jointly encode affective meaning. To bridge this gap, we present MER-Bench, a systematic benchmark for MER on image-text memes, containing over 3,000 real-world memes with fine-grained annotations for emotion transformation and structure-preserving multimodal editing. MER-Bench spans diverse visual, textual, and structural settings and is paired with a structured MLLM-as-a-Judge evaluation suite, comprising eight fine-grained metrics and three aggregated objectives for emotional alignment, content fidelity, and transformation strength. Extensive evaluation of 14 advanced multimodal models reveals that: current models struggle to jointly balance affect, semantics, structure, and text; strong visual editing does not imply effective MER, with poor embedded text rendering and affect-aware rewriting remaining key bottlenecks. Our results identify MER as a critical capability gap in current multimodal generation systems and highlight affective image-text reasoning as a key challenge for future work.

Abstract:
Multimodal Large Language Models (MLLMs) have achieved strong performance on OCR-centric document understanding and text-rich visual reasoning benchmarks. Yet existing evaluations largely assume that every task is valid and answerable. In real-world OCR scenarios, this assumption often fails: questions may rely on illegible text, occluded evidence, nonexistent visual targets, contradictory premises, or missing variables. We study this reliability gap as OCR-grounded Task Verification: before answering, a model should determine whether the Image Premise (IP), Textual Premise (TP), and Question (Q) jointly define an executable task. We introduce VeriOCRBench, a 1,800-sample human-verified benchmark built from source images drawn from 8 OCR-related datasets and spanning 8 real-world image domains, with controlled, image-grounded diagnostic tasks. It contains 1,600 trap-injected invalid tasks across 8 trap types and four verification dimensions—Visual, Contextual, Factual, and Logical—plus 200 trap-free controls for measuring over-refusal. Built with a Visual Atomic Fact (VAF)-anchored pipeline and full human auditing, VeriOCRBench enables decoupled evaluation of task verification, root-cause diagnosis, and over-refusal. Evaluating 15 leading MLLMs reveals persistent blind compliance, diagnosis failures, and prompt-induced over-refusal, exposing a critical reliability gap in current OCR reasoning systems. The code is available at: https://github.com/zy001122/Beyond-Blind-Compliance.

Abstract:
Sales dialogues require multi-turn, goal-directed persuasion under asymmetric incentives, which makes them a challenging setting for large language models (LLMs). Yet existing dialogue benchmarks rarely measure deal progression and outcomes. We introduce SalesLLM benchmark, a bilingual (ZH/EN) benchmark derived from realistic applications covering Financial Services and Consumer Goods, built from 30,074 scripted configurations and 1,805 curated multi-turn scenarios with controllable difficulty and personas. We propose a fully automatic evaluation pipeline that combines (i) an LLM judge for sales-process progress, and (ii) fine-tuned BERT classifiers for end-of-dialogue buying intent. To improve simulation fidelity, we train a user model, CustomerLM, with SFT and DPO on 8,000+ crowdworker-involved sales conversations, reducing role inversion from 17.44% (GPT-4o) to 8.8%. SalesLLM benchmark scores correlate strongly with human ratings (average Pearson r = 0.86; inter-annotator agreement Krippendorff’s α = 0.86, 500 annotations per model). Across 15 mainstream LLMs, in Chinese the strongest models are competitive with typical (junior-to-intermediate) human salespeople — not with sales experts — while weaker ones fall below this baseline, and cross-lingual consistency remains poor. SalesLLM benchmark serves as a scalable benchmark for outcome-oriented sales agents. Our code and data are released at https://github.com/Bairong-Xdynamics/Benchmarking-LLM-Realistic-Selling-Skill

Abstract:
Existing Text-to-Image (T2I) benchmarks remain anchored in foundational criteria such as image quality and text-image alignment, failing to capture the nuanced capabilities that professional creators demand in real-world workflows. Meanwhile, evaluation pipelines that rely on an untrained multimodal model as a judge diverge from human standards. We introduce Qwen-Image-Bench, a creator-centric benchmark that extends evaluation with two application-driven dimensions (Real-world Fidelity and Creative Generation), alongside the conventional pillars of Quality, Aesthetics, and Alignment. The benchmark organizes these into a three-level taxonomy (5 pillars, 23 sub-capabilities, 56 verifiable rubrics) and provides 1000 stratified bilingual prompts. We train a unified Judge Model (Q-Judger) on Qwen3.6-27B, supervised by 80 professional annotators under blind labeling and triple-review protocols, that scores every image across all 56 facets with fine-grained, rubric-grounded diagnostics. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the application-driven dimensions where existing benchmarks provide limited insight.

Abstract:
Text-attributed graphs (TAGs) underpin many NLP applications, but real-world TAGs are often limited in scale, class balance, and diversity, motivating realistic and controllable TAG generation. Existing generators either replace node text with precomputed embeddings, reducing the task to vector-attributed graph generation, or rely on LLM prompting and agent simulation; neither learns a joint distribution over raw node text, edges, and labels. We propose a controllable substructure generator built from a lightweight causal LM. The model generates connected TAG substructures rather than whole graphs: each training substructure is serialized into lossless token sequences from multiple start nodes. Averaging next-token loss over these traversals treats traversal order as a latent variable and optimizes a variational lower bound on the permutation-invariant substructure marginal likelihood. Training combines continual pretraining for the unconditional distribution with control-prefix fine-tuning that conditions generation on target structural and label descriptors. At inference, a single autoregressive pass per prefix emits a parseable substructure, and the same generator supports distribution-aligned synthesis and boundary-aware augmentation through target-side adaptation alone. Across five real-world TAGs, it produces losslessly parseable graphs, matches or exceeds structure-only generators in structural fidelity, and improves cross-graph GNN generalization.

Abstract:
Reducing the modality gap between image and text representations in CLIP is widely expected to improve cross-modal alignment and downstream performance. However, a smaller average image–text gap does not necessarily lead to consistent accuracy gains. We analyze this mismatch from the perspective of the decision structure in zero-shot classification, i.e. selecting the most similar class-text prototype for an input image. Zero-shot accuracy depends not only on average image–text alignment, but also on class-wise decision margins. Using Linear correction as an analytically tractable case, we show that modality gap correction can alter the relative decision structure among classes and cause predictions to concentrate on a small subset of classes. We refer to this output-space failure mode as prediction-level hubness. Furthermore, experiments across multiple datasets show that accuracy degradation under gap correction is consistently associated with increased prediction concentration, both for Linear correction and for learning-based correction methods. This provides a systematic explanation of why modality gap reduction does not consistently improve CLIP zero-shot accuracy from the perspective of downstream decision structure. Our results suggest that gap correction should be evaluated not only by average alignment, but also by its impact on downstream prediction structure.

Abstract:
LLM-empowered paper-code discrepancy detection has received growing concern since the scaling of research submissions exceeds the manual review capability. However, the limited context capacity and one-sided discrepancy detection of existing single-agent LLM paradigms lead to an inferior recall performance in detecting discrepancies. In this paper, we propose Dude, the first Dual-Detection Multi-Agent System for paper-code discrepancy detection. We discover that the granularity asymmetry of the paper-language and code-language introduces over-interpretation and over-reporting challenges in a multi-agent system design for discrepancy detection, resulting in increasing false positives. To address this, we propose a granularity-aligned negotiation and a two-stage salience-filtering mechanism in Dude, which effectively prevents agents from falsely reporting discrepancies. Experimental results in real-world paper-code discrepancy datasets showcase Dude’s significant recall and precision improvement by up to 22.8%, increasing F1 score by up to 18.7% compared to baseline methods.

Authors: Ahmed Ashraf, Aisha Alansari, Fadel Al Abbas, Nada Almarwani, Samah Aloufi, Saad Ezzini, Maged S. Al-shaibani, Doaa Dalaq, AbdelRahim A. Elmadany, Muhammad Abdul-Mageed, Mohamed Mehdi Trigui, Dania Refai, Layan Refai, Mohamed Akrout, Mustafa Jarrar, Wasfi G. Al-Khatib, Alaa Dalaq, Darin El-Nakla, Samir Abdaljalil, Abdulrahman Al-Fakih, Nour El Imane Zeghib, Moussa Redah, Salmane Chafik, Mohamed El-Attar, Rima Grati, Sarah Kohail, Malak Alkhorasani, Khadijah Al Safwan, Ismail Mohmmed Mudhaffar, Ali Altam, Ahmed Al-Shaikh, Adnan Mahyoub Ahmed Saeed, Hamzah Luqman

Abstract:
Arabic automatic speech recognition (ASR) faces unique challenges due to diglossia, extensive regional dialect variation, and limited systematically balanced speech resources. Existing speech datasets often focus on single dialects or large-scale broadcast/web data, leading to trade-offs between linguistic diversity and annotation quality. We present BULBUL, a multi-dialect Arabic ASR dataset collected from 275 speakers across 11 Arab countries. BULBUL includes structured dialect and sub-dialect coverage, as well as recordings of classical Arabic and modern standard Arabic spoken by participants in their native dialectal accents to support accent-aware modeling. The quality of the recordings was ensured through a two-level human verification process. We further benchmark a range of recent ASR systems, establishing strong baselines for modern dialectal and accented Arabic ASR.

Abstract:
Vision-language models (VLMs) are increasingly used to interpret charts and generate natural-language explanations for socially consequential data. However, they may produce different narratives for the same chart when only the referenced social group changes, reinforcing stereotypes and misleading decisions. Despite these risks, no benchmark exists for systematically evaluating bias in chart interpretation across social dimensions. We introduce ChartBias, the first benchmark for auditing bias in VLM-based chart interpretation. ChartBias contains 820 manually curated real-world charts spanning six attributes—race, income, age, religion, immigration status, and gender—yielding 4,319 valid chart–attribute instances and 8,638 paired generations where the chart is fixed and only the group term is swapped. Across 12 proprietary and open-source VLMs, totaling 155,484 model responses, we find three widespread failure modes: narrative shift (same chart, different narratives), group hallucination (assigning a chart to a group without evidence), and preference polarity (favourable trends often linked to one group). We further propose a multi-agent mitigation framework that serves as a strong baseline by separating chart-grounded evidence extraction from group-conditioned generation and using a counterfactual judge to verify that group-driven differences are supported by the chart. The framework substantially reduces narrative shift while preserving chart-grounded reasoning. Our findings show that evaluating chart understanding requires measuring not only accuracy, but also fairness and consistency across social groups. We release ChartBias at .

Abstract:
Audio-language models (ALMs) often follow text that conflicts with audio, even when the audio evidence is clear. This behavior leaves an important question unresolved: is the audio-supported answer unavailable, or is it represented but overridden by the conflicting text? We test this question with a same-audio counterfactual that keeps the audio fixed, removes only the conflicting text, and measures the resulting shift in model preference. Across five ALMs and four conflict tasks, 64.1% of conflict samples exhibit a sign flip: the same-audio branch prefers the audio-supported answer, whereas the joint branch prefers the text-supported answer. This pattern suggests that the relevant audio evidence is encoded but loses during arbitration. Activation patching further localizes the reversal to answer-position computation, and patching effects closely track output candidate-score differences (Spearman ρ=0.93). Based on this diagnostic, we propose Gated Audio Counterfactual Logit Correction (GACL), a training-free decoding rule that interpolates between joint and same-audio scores. Under a strict 5 pp faithfulness-drop budget, GACL improves nAUC by 17.8 points over the best contrastive baseline and transfers without retuning to vision-text arbitration (up to +40.5 pp).

Abstract:
Large language models (LLMs) are increasingly trained to decline queries that fall outside their knowledge (knowledge-based refusal, KR) or violate safety policies (safety-based refusal, SR). Although KR and SR result in superficially similar responses, they have largely been studied in isolation, leaving open whether they share an underlying mechanism. We address this gap with a systematic study on a new dataset of 213 contrastive quadruples that jointly probe both refusal types. We find that KR and SR are governed by overlapping yet distinguishable mechanisms. Both share a refusal direction, yet the overlap is asymmetric: SR signals transfer more strongly to KR than the reverse. Type-specific specialization emerges mainly in upper layers, with KR aligning with uncertainty- and knowledge-related representations and SR with safety- and policy-related ones. We thus characterize refusal as a commit-then-specify process: a shared initial mechanism commits to refusing, then type-specific features in later layers specify whether the grounds are epistemic or normative.

Abstract:
Prior work shows that large language models (LLMs) exhibit varying degrees of introspective capability on benign tasks. We extend the question to safety contexts and examine how reliably a model can recognize that its own prior response was elicited by an adversarial prefill attack. Across ten open-weight instruction-tuned LLMs from 3B to 70B parameters and four safety benchmarks, no model reliably recognizes its own compromised outputs, with models claiming intent on prefilled responses at an average rate of 25.3%. Introspective signal stems primarily from reasoning about safety and refusal. Orthogonalizing models’ weights against the refusal direction collapses the gap between claim rates on prefilled and natural outputs to near zero, though the direction is not its unique mediator. Framing the question as internal intention versus external tampering elicits qualitatively different responses on the same models. Training models to mimic correct introspective answers or optimize an introspective objective can improve the accuracy of introspection, but such training does not transfer to the tampering probe and counterintuitively raises attack success rate under adversarial prefill on most models, amounting to a partial mitigation. These findings outline mechanisms underpinning the observed introspective signals in safety contexts and highlight risks in the reliability of LLM self-reports

Abstract:
How does irrelevant information (i.e., distractors) affect test-time scaling in vision-language models (VLMs)? Prior work on text-only language models has shown that textual distractors can intensify inverse scaling, causing models to reason longer but less effective reasoning traces. In this work, we investigate whether similar phenomena arise in multimodal settings. We introduce Idis (Images with distractors), a visual question-answering dataset that systematically varies distractors along semantic and numerical dimensions. Our analyses reveal that visual distractors affect reasoning VLMs in a fundamentally different way from textual distractors: although inverse scaling still emerges, visual distractors reduce accuracy without increasing reasoning length. We further show that attribute counts extracted from reasoning traces provide key insights into how distractors interact with reasoning length and accuracy. As a sanity check, we propose a simple prompting strategy that mitigates distractor-driven predictions in reasoning vision-language models.

Abstract:
Self-evolving agents requires adaptation after deployment, but existing approaches assume a usable learning loop, such as curated skills, successful trajectories, or verifier signals. Real open-world deployments may provide none of these, offering only a task prompt. In this work, we study open-world self-evolution, where an agent must build both its skills and its own verification signals from scratch, using open-world resources but no target-task supervision. We propose OpenSkill, a framework that bootstraps this loop: it acquires grounded knowledge and verification anchors from documentation, repositories, and the web, synthesizes them into transferable skills, and refines those skills against self-built virtual tasks grounded in the anchors rather than in target answers. The open world thus supplies both the knowledge to be learned and a supervision-independent practice environment, with target-task supervision reserved for final evaluation. Across three benchmarks and two target agents, OpenSkill attains the best automated pass rate while satisfying the no-supervision constraint. Analysis shows its skills transfer across models without model-specific adaptation, and its self-built verifier aligns with ground-truth outcomes despite never accessing them.

Abstract:
Speculative decoding accelerates large language model inference by pairing a target model with a lightweight draft model whose proposed tokens are verified in parallel. A common way to build draft models, like EAGLE3 or DFlash is supervised fine-tuning (SFT) on target-generated trajectories. However, we observe that SFT quickly plateaus: the draft model’s acceptance length on test data stops improving. The reason is an offline-to-inference mismatch: In SFT, the drafter learns from fixed target-generated trajectories, whereas during speculative decoding it is evaluated on blocks proposed under its own policy. This motivates on-policy distillation (OPD), where the target model supervises the drafter on draft-induced states. Yet OPD remains difficult for draft models, as they cannot reliably roll out complete sequences independently, whereas target-assisted generation makes the collected sequences follow the target distribution and thus eliminates the on-policy signal. We therefore propose Draft-OPD, which uses target-assisted rollout for stable continuations and replays drafting from the verification-exposed error positions. This allows the drafter to learn from target feedback on both accepted and rejected proposals, focusing training on the draft-induced errors that limit speculative acceptance. Experiments show that Draft-OPD achieves over 5× lossless acceleration for thinking models across diverse tasks, improving over EAGLE-3 and DFlash by 23% and 13%.

Abstract:
Multilingual large language models (mLLMs) contain a small fraction of feed-forward neurons that are sensitive to particular languages, commonly termed language-specific neurons. Existing work measures language specificity using the entropy of each neuron’s language-wise probabilities of being active, where a neuron is considered active when its activation value is positive. However, this approach may not fully capture the multilingual nature of mLLMs, where language representations are distributional and mutually related. We propose Distribution-aware Language Neuron selection, which leverages pairwise relationships between per-language activation distributions over the full activation range, including negative values. Specifically, we quantify each neuron’s language specificity by clustering languages using pairwise overlap coefficients between their activation distributions. Across two mLLMs and two held-out corpora, our identifier more effectively isolates language-specific causal effects, yielding up to 4.9 times higher on-target language damage per neuron while preserving off-target language performance.

Abstract:
Linear attention reduces the quadratic cost of softmax attention by maintaining a recurrent fast-weight state, but it consistently lags on in-context retrieval and long-context tasks. Existing remedies act on the write side of memory through gating, delta updates, or kernel feature maps, but the read step is left unchanged: every past key contributes additively to the output, so useful targets are diluted by the bulk of stored vectors. We borrow one specific piece of softmax’s geometry to construct a cheap read-time contraction of the query. A second-order Taylor expansion of the softmax log-partition at the isotropic-attention point gives a local quadratic model whose curvature coincides with the running key covariance, a quantity that can be maintained with the same recurrent/chunkwise mechanism as the linear-attention state. The associated linear operator contracts the query along the high-variance directions of memory before it reads the state. We call this mechanism Curvature-Conditioned Query (CCQ). CCQ modifies only the read step and is composable with any linear-attention backbone. Attached to GLA and Gated DeltaNet, it improves perplexity, zero-shot downstream accuracy, S-NIAH retrieval at and beyond the training context, length-extrapolation perplexity from 4K to 20K, and LongBench accuracy.

Abstract:
Memory-augmented conversational agents enable personalized interactions using long-term user memory and have gained substantial traction. However, existing benchmarks primarily focus on whether agents can recall and apply user information, while overlooking whether such personalization is used appropriately. In fact, agents may overuse personal information, producing responses that feel forced, intrusive, or socially inappropriate to users. We refer to this issue as over-personalization. In this work, we formalize over-personalization into three types: Irrelevance, Repetition, and Sycophancy, and introduce OP-Bench, a benchmark of 1,700 verified instances constructed from long-horizon dialogue histories. Using OP-Bench, we evaluate multiple large language models and memory-augmentation methods, and find that over-personalization is widespread when memory is introduced. Further analysis reveals that agents tend to retrieve and over-attend to user memories even when unnecessary. We further evaluate existing post-processing strategies for mitigating this issue, but find that they provide only limited improvements and often reduce personalization performance. Our results highlight over-personalization as a challenging and largely unresolved problem in memory-augmented dialogue systems.

Abstract:
Large multimodal models (LMMs) inherit the self-attention mechanism of pretrained language backbones, yet standard attention can exhibit suboptimal allocation, including cross-modal misallocation between textual and visual evidence and intra-visual imbalance among visual tokens. We propose RAVE (Re-Allocating Visual Attention), a lightweight pair-gating mechanism that adds a learned query–key bias to pre-softmax attention scores over visual keys, derived from pre-RoPE query and key features. RAVE requires no architectural modification to the backbone and can be trained end-to-end with the rest of the model. Across a suite of multimodal benchmarks, RAVE improves over standard attention by 3 points on average, with the largest gains on perception-intensive tasks—including multilingual OCR, chart understanding, document VQA, and scene text VQA—where accurate visual grounding is critical.

Abstract:
Scientific idea generation requires more than fluent continuation from a problem statement: strong ideas often emerge from inspecting the same problem through multiple informative perspectives. We propose PRISM, an entropy-guided multi-view framework for scientific ideation. Given a research problem, PRISM constructs candidate inspection views, estimates their pairwise orthogonality through hypothesis-transition entropy, selects a maximally orthogonal subset, and conditions a backbone model on the selected views to synthesize a single proposal-style idea. We evaluate PRISM on 500 research problems across 23 scientific domains using three generation backbones: Llama-3.1-8B-Instruct, Qwen3-8B, and GPT-5.4. To assess generated ideas, we use two complementary automated judges—a Galactica-based impact discriminator trained from citation-derived scientific preferences and GPT-5.5 blind five-way novelty ranking—as well as a blind human study with five PhD-level evaluators. Across 7,500 generated ideas, PRISM achieves the best impact score and an 80.2% impact pairwise win rate, a 90.9% novelty pairwise win rate, and the highest mean score (3.91/5) in human evaluation. These results suggest that entropy-guided multi-view exploration provides an effective inference-time mechanism for producing more novel and impactful scientific ideas. Our code is available at https://github.com/jiajunyu1999/PRISM-idea-generation.

Abstract:
LLM-based retrievers have become a fundamental component of modern information retrieval systems. The paradigm of ”rewrite-then-retrieve” introduces explicit reasoning before retrieval. In addition, implicit-reasoning retrievers such as GIRCSE and LaSER improve efficiency by replacing explicit reasoning with soft tokens. Although these methods demonstrated competitive performance on reasoning-intensive retrieval benchmarks, they struggle to address the mismatch between the objectives of retrieval and generation. In this work, we propose SHIFT (Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval), a retrieval training framework based on LLMs. Firstly, we transfer LLMs into reasoning-efficient retrievers with residual projection and task-oriented bidirectional attention aggregation in the latent space. Secondly, we alleviate the mismatch between contrastive learning and implicit reasoning using fine-grained next-token-prediction-based reconstruction. Extensive experiments on reasoning-intensive retrieval benchmarks show that SHIFT consistently outperforms other widely used retrievers. We also carried out a detailed analysis to illustrate how our method works.

Abstract:
Large reasoning models have recently demonstrated remarkable capabilities in solving complex tasks, where supervised fine-tuning (SFT) on long chain-of-thought data serves as a crucial foundation for eliciting and enhancing their reasoning abilities. Despite rapid progress in both improving and analyzing reasoning-oriented SFT, the field still lacks a systematic survey that consolidates its fast-growing literature. To fill this gap, we present a comprehensive review of recent advancements in reasoning SFT, examining the literature through the dual lenses of methodological design and analytical investigation. First, we review methodological improvements across the SFT pipeline and categorize them into data-centric approaches and algorithm-centric innovations. Second, we reorganize analytical studies along three dimensions: data characteristics, optimization dynamics, and mechanistic insights. Finally, by synthesizing current research focuses and remaining bottlenecks, we outline promising future directions for reasoning SFT. We hope this survey deepens the understanding of reasoning SFT and paves the way for advanced reasoning models.

Abstract:
Multimodal Large Language Models (MLLMs) achieve strong multimodal reasoning performance, yet we identify a recurring failure mode in long-form generation: as outputs grow longer, models progressively drift away from image evidence and fall back on textual priors, resulting in ungrounded reasoning and hallucinations. Interestingly, Based on attention analysis, we find that MLLMs have a latent capability for late-stage visual verification that is present but not consistently activated. Motivated by this observation, we propose Visual Re-Examination (VRE), a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs. Rather than distilling visual capabilities from a stronger teacher, VRE promotes iterative self-improvement by leveraging the model itself to generate reflection traces, making visual information actionable through information gain. Extensive experiments across diverse multimodal benchmarks demonstrate that VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings.

Abstract:
Recent advances in world models have shown promise for modeling future dynamics of environmental states, enabling agents to reason and act without accessing real environments. Current methods mainly perform single-step or fixed-horizon rollouts, leaving their potential for complex task planning under-exploited. We propose Imagine-then-Plan (ITP), a unified framework for agent learning via lookahead imagination, where an agent’s policy model interacts with the learned world model, yielding multi-step “imagined” trajectories. Since the imagination horizon may vary by tasks and stages, we introduce a novel adaptive lookahead mechanism by trading off the ultimate goal and task progress. The resulting imagined trajectories provide rich signals about future consequences, such as achieved progress and potential conflicts, which are fused with current observations, formulating a partially observable and imaginable Markov decision process to guide policy learning. We instantiate ITP with both training-free and reinforcement-trained variants. Extensive experiments across representative agent benchmarks demonstrate that ITP significantly outperforms competitive baselines. Further analyses validate that our adaptive lookahead largely enhances agents’ reasoning capability, providing valuable insights into addressing broader, complex tasks. Our code and data are publicly available at:https://github.com/loyiv/ITP.

Abstract:
Large language models (LLMs) are increasingly used in high-stakes domains, yet their unfaithful or non-factual outputs create serious risks and hinder trustworthy deployment. Rigorous fact-checking is therefore essential to mitigate these risks. Most existing factuality verification methods improve evidence acquisition, but fail to capture complex logical relations between claims and multi-source evidence, especially when different evidence sources are inconsistent or incomplete. Moreover, current benchmarks rarely capture such realistic conflict scenarios. To address this, we introduce MAVEN, a real-world benchmark for verifying complex claims against heterogeneous and conflicting web evidence with fine-grained explanations. We further propose ArgCheck, an argumentation-based fact-checking framework. It models claims and retrieved evidence as arguments linked by attack relations, resolving conflicts through formal argumentation semantics. To avoid incomplete evidence, ArgCheck incorporates adversarial web retrieval to gather balanced multi-source evidence and grounds its verdicts and explanations in accepted arguments. Experiments on MAVEN and the public fact-checking benchmark AVeriTeC show that ArgCheck improves Macro-F1 over the strongest baseline by 5.39 and 11.10 points, respectively, while generating more faithful and evidence-grounded explanations. Our code and dataset are available online https://github.com/ZyraChen/ArgCheck.

Abstract:
Direct Preference Optimization (DPO) is notoriously sensitive to attribution ambiguity: when chosen and rejected responses differ along multiple entangled dimensions, the model cannot identify which behavior the preference label teaches. We propose Criterion-Activated Preference Synthesis (CAPS), a criterion-first framework that turns explicit quality standards into attribution-verified preference pairs. CAPS synthesizes contexts and responses from a target criterion, then applies dual-signal scoring: a rubric-bound judge measures criterion compliance, while a blind general judge measures holistic quality. Two filters close the attribution loop: the Empirical Activation Test (EAT) keeps criterion-context pairs where satisfying the criterion improves blind quality, and the Attribution Ratio (AR) keeps pairs whose quality gap is dominated by the target criterion. Instantiated with general rubrics, medical rubrics, and Chinese legal statutes, CAPS shows strong data efficiency and domain transfer. With 20k general synthetic pairs, it outperforms 35k human-annotated HelpSteer3 pairs on IFEval, IFBench, AlpacaEval, and Creative Writing v3. In domain settings, CAPS improves HealthBench with 2k medical pairs and improves Qwen3-8B/14B by +4.9 to +6.6 points on legal benchmarks using only public statutes, without human preference labels. We additionally construct the CAPS-Legal preference dataset.

Abstract:
Cross-Modality Prompt Transfer (CMPT) has been explored to address the initialization instability and data scarcity challenges inherent in prompt tuning. Its empirical success is typically interpreted as the transfer of linguistic semantics acquired during language-based prompt pretraining. In this paper, we revisit this interpretation and explore what actually transfers when language-pretrained prompts are adapted to the visual modality. Through controlled empirical analyses, we show that transfer performance does not consistently correlate with source-task linguistic performance. Instead, transferability is better explained by the geometric priors of prompt vectors, particularly stable rank and spatial direction relative to target features. To further validate this interpretation, we directly construct prompts with favorable geometry without relying on language pretraining. The resulting prompts achieve competitive transfer performance against language-pretrained prompts across diverse visual tasks, supporting prompt geometry as a stronger empirical explanation of transferability.

Abstract:
Large Reasoning Models (LRMs) achieve strong performance on table reasoning tasks but incur substantial inference cost due to long reasoning traces. Stepwise model routing mitigates this issue by dynamically assigning reasoning steps to smaller or larger models. However, stepwise model routing for table reasoning remains underexplored. Through empirical analysis, we find that reasoning steps involving tables contain two types of tokens with distinct uncertainty distributions: table tokens grounded in table structure, such as cell values and headers, and text tokens representing surrounding natural-language reasoning. The uncertainty of both token types is correlated with the risk that the model makes an error in the next reasoning step. However, existing methods fail to model them separately, leading to suboptimal routing decisions. To address this, we propose EcoTab, a table-aware stepwise routing framework for efficient table reasoning. For each current reasoning step, EcoTab separately models the two types of tokens and estimates their uncertainties, then combines them to estimate the risk of using the small model for the next step, thereby enabling model selection. Experiments on multiple table reasoning benchmarks show that EcoTab consistently outperforms strong baselines and achieves a better balance between accuracy and efficiency.

Abstract:
When large language models serve as evaluation infrastructure, progress cannot be reduced to a single accuracy number. We study winning-argument (WA) detection as a stress test: can one system jointly improve accuracy, enforce speaker-order invariance, emit rubric-conditioned diagnostics, improve a proper scoring rule, and cut inference cost? TAPIR freezes a survey-grounded rubric into a dual-artifact protocol, distils privileged signals by mode-split self-distillation, and predicts through an anti-symmetric scalar head paired with a structured ledger. An algebraic swap identity (Winning-Position Invariance, WPI=1) holds on every test pair, and a tied-order wrapper preserves it on the deployed router. On the standard 807-pair WA split, TAPIR beats our same-backbone Qwen3-14B re-implementation of Labruna et al.’s MS-PS-MLP, 65.80% vs. 58.12% (paired McNemar p=3.41×10−4), at a lower Brier score and 39.41× faster inference per pair. Reusing the frozen protocol without retuning carries the mechanism to a second task and a second backbone, both at WPI=1. Under domain shift it only matches length-only controls, so we report contract preservation rather than superiority. The contribution is the joint contract under a paired protocol, with an explicit amortisation boundary, not a leaderboard delta.

Abstract:
Dense retrieval has become the dominant paradigm for large-scale table retrieval. However, existing contrastive retrievers often suffer from bag-of-words degradation: they rely on lexical overlap rather than row–column bindings. To quantify and mitigate this, we propose an adversarial benchmark and a structure-aware retrieval framework. Our benchmark uses "evidence cell" anchoring and a Homologous Structural Swapping mechanism to construct lexically equivalent but structurally invalid negative samples. Our model uses text generation as a structural regularizer for dense table representations. It decouples row and column anchors through orthogonal attention pooling, and employs dynamic masking to encourage autoregressive reconstruction. This process encourages the retrieval representations to retain table content and structure-related cues. Extensive experiments on OTT-QA, NQ-Tables, and our adversarial benchmark demonstrate the effectiveness of our method. It achieves competitive performance on conventional retrieval datasets and substantially improves structural robustness under adversarial settings.

Abstract:
Safety alignment in large language models is markedly uneven across languages: behaviors learned from English preference data fail to transfer reliably into lower-resource languages, and existing remedies based on per-language preference data or pairwise translation pivots do not scale. We propose Hyperspherical Equilibrium for Aligned Languages (HEAL), a lightweight auxiliary objective that augments any DPO-style alignment pipeline without altering its supervised pairs. HEAL applies two opposing forces to the multilingual representations of the same prompt: a shrink term draws each non-English representation toward a stop-gradient English anchor, while a dilation term repels same-prompt language representations through a pairwise hinge. At equilibrium, representations occupy an English-centered shell-like region that supports cross-lingual transfer while preserving per-language distinctness. Across multiple backbones, HEAL substantially strengthens multilingual safety with the largest gains on the weakest languages, while preserving general capability and showing favorable out-of-distribution behavior on jailbreak evaluation.

Abstract:
Steering with Sparse Autoencoders (SAEs) offers a lightweight inference-time path for adapting the behavior of large language models without retraining. By exposing sparse and interpretable features, SAE steering provides a promising interface for safety control that guides harmful continuations toward refusal. However, we observe that complex wrappers can still undermine existing SAE steering methods on harmful prompts. To evaluate this failure mode systematically, we construct Generalized Undercover Instruction Safety Evaluation (GUISE), a dataset of harmful prompts with complex wrappers. Existing single direction SAE steering methods do not reliably produce refusals on harmful prompts, suggesting that refusal enhancement alone can be too weak when the harmful continuation path remains active. This motivates us to propose Refusal-Enhanced INhibitory Steering (REINS), which suppresses harmful continuation features and enhances safe refusal features in the same SAE feature space. Experiments on GUISE and other datasets show that prior methods either intervene too weakly or achieve only apparent safety through collapse, while REINS substantially reduces harmful responses, markedly improves safe refusals and largely preserves general capabilities.

Abstract:
Taxonomy construction is essential for knowledge-intensive NLP yet remains costly: clustering-based methods (i) need careful per-corpus tuning and yield unstable hierarchies, while LLM-based pipelines (ii) depend on expensive task-specific training that does not transfer across domains. We report an empirical finding that bears on both: in Matryoshka Representation Learning (MRL) embeddings, where each leading prefix is itself a usable, coarser vector, prefix length aligns with the coarse-to-fine granularity of gold taxonomies; shuffled- and reverse-dimension controls isolate the alignment to the learned early prefix dimensions. The pattern holds across three architecturally distinct MRL models (full statistics in §3). To test whether this regularity is usable, we introduce PEEL (Peeling Embeddings to Extract Latent ontologies): a minimal probe over these frozen embeddings, trained once with an order-embedding objective that represents hierarchy as vector containment; a minimum spanning arborescence then decodes taxonomic DAGs. Using only a concept vocabulary (no documents), PEEL recovers taxonomic DAGs with high structural fidelity on two OLLM benchmarks, transferring to a new domain with no target-edge supervision. The end-to-end OLLM pipeline—which jointly discovers concepts and induces structure from document corpora—is reported as a cross-task reference (§5.3). Pre-trained retrieval embeddings encode a decodable granularity gradient; a minimal probe renders it as explicit parent–child (is-a) structure.

Abstract:
Gradient conflict between forgetting and retaining objectives is a well-known obstacle in machine unlearning, where opposing gradients on shared parameters cause destructive interference. In Multimodal Large Language Models (MLLMs), cross-modal coupling entangles visual and textual representations within the same parameters, making this conflict considerably more severe. We propose EASE (Efficient Alignment-aware Selective Erasing), which explicitly balances the conflicting gradient directions. EASE first dynamically locates the conflicting parameters at each step by detecting opposite-sign gradients, then performs a manifold-constrained optimization within the two-dimensional subspace spanned by the task gradients to find the linear combination that maximally aligns them, yielding a valid descent direction for both objectives without any manually tuned trade-off coefficient. Nonconflicting parameters keep their standard updates, keeping the method lightweight. Experiments across multiple MLLM architectures and unlearning benchmarks show that EASE achieves superior unlearning efficiency and faster convergence while matching or surpassing existing methods in multimodal quality.

Abstract:
To investigate whether QLoRA-trained adapters transfer across model versions like LoRA-trained adapters, we train LoRA and QLoRA adapters on Llama-3-8B-Instruct and transfer them to Llama-3.1-8B-Instruct with a task-specific merge scale λ. Both methods are evaluated on a BF16 target base, isolating the training-time precision difference; NF4 deployment is evaluated separately. Across 20 main tasks, the absolute evaluation-score gap between QLoRA Transfer and LoRA Transfer is at most 0.054 on all tasks and at most 0.03 on 18/20 tasks, so training the adapter on a quantized base does not compromise its cross-version transferability. In a two-task deployment case study, serving QLoRA adapters with an NF4 base uses 5.7–10.0 GB peak allocated inference VRAM, compared with 15.1–19.6 GB for LoRA-merged BF16. Across 44 settings (22 tasks × 2 methods), the task-optimal λ consistently exceeds the Frobenius-distance optimum (we call this discrepancy the λ gap), suggesting that weight-space proximity alone is insufficient for identifying it. Replicating the pipeline on two further families (Mistral v0.1→v0.3, Qwen2→2.5) reproduces these findings. We also report a negative case: on CommonsenseQA, QLoRA Transfer falls significantly below direct adaptation.

Abstract:
LLM-generated drafts often contain subtle factual or logical errors, yet prior work shows that models struggle to reliably integrate multi-turn feedback aimed at fixing them. We propose in-place feedback, an interaction paradigm in which the user directly edits the model’s previous response and the model continues generation from the edited context. In-place feedback consistently outperforms standard multi-turn feedback across five reasoning-intensive benchmarks while requiring fewer tokens, and our fine-grained analysis shows that it applies corrections more reliably and propagates them to subsequent reasoning. A user study with domain experts refining LLM-generated summaries corroborates these findings: participants report higher final-output satisfaction and substantially lower fatigue with in-place feedback, and a mixed strategy combining in-place and multi-turn feedback improves every dimension we measure. These results suggest that editing errors in place is a more reliable way to incorporate expert corrections than multi-turn feedback, while combining the two acts complementarily to form a more effective collaboration than either mechanism alone.

Authors: Angelika Romanou, María Grandury, Clara Meister, Negar Foroutan, Jabez Magomere, Anna Sotnikova, Hitesh Laxmichand Patel, Akshay Kumar Dubey, Alfonso Amayuelas, Amit Agarwal, Arion Das, Carlos Arriaga, Cheng Yu, Dalia Ali, David Quispe-Castillo, Fenal Ashokbhai Ilasariya, Francisco-Javier Rodrigo-Ginés, Gonzalo Martínez, Gouthami Vadithya, Guido Ivetta, Javier Conde, Jorge Vallego, Juan J. Calderón G., Karan Gupta, Marcos J Gomez, Marvin M. Agüero-Torales, Mithil Bangera, Naira Paola Arnez Jordan, Oscar M. Cumbicus-Pineda, Priyaranjan Pattnayak, Reewos Erwin Talla Chumpitaz, Sahil Aggarwal, Sahil Raj, Shruti Jalan, Srikant Panda, Susana Zhou, Yeshil Bangera, Orestis Papakyriakopoulos, Scott A. Hale, Shamsuddeen Hassan Muhammad, Antoine Bosselut

Abstract:
Current multilingual benchmarks cover only a fraction of the languages represented in modern pretraining corpora and often conflate language proficiency with the ability to answer questions grounded in specific regions and communities. We introduce INCLUDE-2.0, a large-scale benchmark for region-aware multilingual factual evaluation, which covers 91 languages from 86 countries (127 total language–country pairs), comprising over 220K curated samples drawn from native regional sources. Unlike existing benchmarks that often treat language as a proxy for region, IINCLUDE-2.0 distinguishes between questions whose answers are independent of context, vary by country, concern widely documented facts about a place, or depend on knowledge primarily expressed through local linguistic and cultural sources. Each item is annotated with its regional knowledge type and domain, and paired with an English translation, enabling controlled analysis of the relationship between language proficiency and possession of regional knowledge. Evaluating frontier and open-weight models on IINCLUDE-2.0, we find that aggregate multilingual accuracy masks substantial variation across regional knowledge types, languages, and regions. We find that frontier models perform worse on locally embedded knowledge than on widely documented facts about regions, with an average drop of ~14.5 points. We also show that for open-weight models, scale improves accuracy but does not reliably make performance invariant to whether a question is asked in English or its original language. INCLUDE-2.0 is available at https://huggingface.co/datasets/epfl-nlp/include-2.0

Abstract:
Temporal Entity Alignment (TEA) aims to identify equivalent entities across Temporal Knowledge Graphs (TKGs). Existing TEA methods generally assume static snapshots and repeatedly perform full-graph alignment, resulting in high computational costs. Recent evolving EA models handle continuously changing graphs through incremental alignment strategies, but mainly focus on non-temporal KGs and overlook temporal dynamics. Consequently, Evolving TEA, a realistic scenario where both graph structures and temporal information evolve over time, remains underexplored. To bridge this gap, we propose a lightweight continual TEA framework with three novel design: (1) informative seed selection that jointly leverages structural and temporal signals for reliable iterative finetuning, (2) localized inference that restricts computation to newly updated entities and their neighborhoods for efficient alignment, and (3) cross-snapshot score fusion that adaptively integrates historical and current alignment evidence. Extensive experiments on real-world evolving TEA benchmarks demonstrate the superiority of our proposed framework, achieving state-of-the-art alignment performance while significantly reducing computational overhead.

Abstract:
Clinical agents promise to democratize access to electronic health records (EHRs), yet existing benchmarks fail to reflect the complexity of practical EHR analysis, e.g., often operating on idealized, clean EHRs via static SQL generation rather than interactive execution. In this work, we introduce EHR-Complex, a large-scale benchmark designed for interactive clinical database reasoning. Built on the large MIMIC-IV substrate (365K patients, 31 tables, 500M+ records), EHR-Complex comprises about 52K tasks spanning six clinical intents, supporting both patient-level and population-level queries, where each task requires an agent to interact with a sandboxed environment by executing SQL queries or Python code. Notably, EHR-Complex considers the real-world SQL task complexity for longitudinal multi-table aggregation and compositional reasoning, resulting in 31.93 SQL structural components per query on average. Evaluation results on EHR-Complex reveal the clinical difficulty of these EHR reasoning scenarios, with the top-performing model achieving only 62.3% exact-match accuracy. Pass^k consistency drops below 50% for nearly all evaluated models at k=4, exposing broad stochastic fragility. A fine-grained analysis of more than 3,800 failed trajectories for representative LLMs reveals three dominant failure modes: SQL logic errors, medical-code lookup failures, and semantic misunderstandings. EHR-Complex provides a rigorous testbed for clinical agents and highlights remaining gaps in robust reasoning for large-scale EHR analysis.

Abstract:
We present AnyCap, a plug-and-play framework that brings instruction alignment to omni-modal captioning. Captions offer a unified language interface for multimodal learning, and users increasingly expect instruction-driven control over their content and style. Current caption models lack explicit instruction supervision and are weak at instruction following, while directly tuning them can degrade general language ability. Achieving instruction alignment in an omni-modal setting is harder still, as each modality calls for separate models and custom designs. To address these challenges, AnyCap leverages a multimodal revision-based refinement paradigm that refines uncontrolled captions from existing models to instruction-aligned ones, without re-training base models. By processing multi-modality features in a unified framework, it enables one model to serve images, videos, and audio. To address the lack of instruction-based data, we construct AnyCapData, a large-scale, high-quality corpus spanning three modalities with 28 well-designed instruction types, each with multiple instruction samples. For evaluation, we address the limitations of current metrics for instruction-oriented captioning by designing AnyCapEval. Its key insight is to decouple evaluation into content and style for fine-grained assessment. Extensive experiments show that on AnyCapEval and diverse public benchmarks, AnyCap consistently improves both caption quality and instruction adherence for both open-source and API-based models. Notably, AnyCap-8B boosts GPT-4o’s content scores by 46% and style scores by 12%. Our code and models will be made publicly available.

Abstract:
Passage reranking plays a crucial role in information retrieval by refining the ordering of candidate passages to better reflect relevance. Existing listwise LLM reranker with Chain-of-Thought (CoT) reasoning can handle complex queries effectively, but they suffer from substantial redundancy and high latency due to sliding-window strategies, which repeatedly generate highly similar CoTs. To address this, we propose QReason, a decoupled framework that separates query-focused reasoning from window-specific passage relevance assessment. Specifically, QReason introduces a dedicated rewriter generates a ranking-oriented reasoning query once, capturing the query’s core intent while avoiding redundantly, which is then reused by a non-reasoning reranker across all windows. The rewriter is trained via a two-stage process that first uses supervised fine-tuning with relevant-passage guidance through semantic evidence to produce deeply grounded, query-focused CoTs. It then applies reinforcement learning to align CoT generation with both the inference-time setting and the reranking objective, optimizing listwise metrics and passage-level discrimination to produce reusable reasoning chains for reranking. Experiments on the BRIGHT benchmark demonstrate that QReason significantly reduces redundant reasoning, achieves ranking performance comparable to or better than strong reasoning-based rerankers, and outperforms existing query rewriting models.

Abstract:
Current vision-language models (VLMs) in medicine are primarily designed for categorical question answering (e.g., “Is this normal or abnormal?”) or qualitative descriptive tasks. However, clinical decision-making often relies on quantitative assessments, such as measuring the size of a tumor or the angle of a joint, from which clinicians draw their own diagnostic conclusions. This quantitative reasoning capability remains underexplored and poorly supported in existing VLMs. In this work, we introduce MedVision , a large-scale dataset and benchmark specifically designed to evaluate and improve VLMs on quantitative medical image analysis. MedVision spans 22 public datasets covering diverse anatomies and modalities, with 29.0K 3D images, 11.2M annotated 2D slices, and 24.3M single-instance annotations. We focus on three representative quantitative tasks: (1) detection of anatomical structures and abnormalities, (2) tumor/lesion (T/L) size estimation, and (3) angle/distance (A/D) measurement. We show that current off-the-shelf VLMs perform poorly on these tasks. However, supervised and reinforcement fine-tuning (RFT) on MedVision significantly enhances performance across detection, T/L size estimation, and A/D measurement, yielding MedVision-V0 as a strong open baseline. In the RFT stage, we design and evaluate the efficacy of process rewards, multiplicative reward composition, and multi-task RFT with curriculum learning. MedVision provides a foundation for developing VLMs with robust quantitative reasoning capabilities in medical imaging.

Abstract:
Large language models (LLMs) increasingly serve as recommendation engines, yet existing preference control work primarily focuses on coarse-grained objectives such as safety, helpfulness, or writing style, while overlooking fine-grained provider preferences , i.e., user preferences over specific brands, platforms, or services. Such preferences are often multi-intent, context-dependent, and dynamically changing, making them difficult to control with prompting, fine-tuning, or activation steering methods. In this paper, we present COPILOT, a plug-and-play, training-free framework. It first decomposes each prompt into sub-tasks with corresponding user-specific provider preference configurations. Then it uses a diffusion language model to draft sub-task responses with hard preference constraints injected at the initial mask, which are finally completed into fluent answers by the target LLM. Experiments across code and text domains on three target LLMs show that COPILOT effectively improves provider preference satisfaction by up to 58.9% over the strongest baselines while reducing irrelevant provider mentions by up to 87.8% to ensure the recommendation quality. This work aims to lay a foundation for LLM recommendations that balance personalized experience with real-world application values.

Abstract:
Multi-turn agents that plan, invoke tools, and interact with environments offer a promising paradigm for solving complex tasks, yet their capabilities typically rely on very large models whose inference cost is prohibitive in practice. On-Policy Distillation (OPD) is a natural recipe for transferring such capabilities to smaller students, but we find that it suffers a characteristic failure mode in this setting: small student errors compound across turns and push the trajectory out of the teacher’s familiar state distribution, so the teacher’s supervision becomes least reliable precisely where the student needs it most. We propose Guided On-Policy Distillation (Guided-OPD), a simple yet effective algorithm that mixes teacher- and student-generated turns within each rollout and schedules the teacher’s intervention probability along a curriculum that decays to zero. Strong guidance keeps early trajectories close to the teacher distribution and is then gradually withdrawn to recover the purely on-policy regime used at inference. On ALFWorld, ScienceWorld, and WebShop, distilling Qwen3 students from a Qwen3-30B-A3B teacher, Guided-OPD yields average relative gains of 21.1% in Score and 25.5% in Success Rate over vanilla OPD, with larger gains on smaller students.

Abstract:
Omni-modal video captioning is not merely combining visual captioning with audio transcription: a useful caption must describe how visual actions, speech, music, and sound effects co-evolve over time. Existing large multimodal models often fail at this relational step, treating audio and visual streams as loosely coupled observations, relying heavily on automatic speech recognition, and under-specifying non-speech sounds and their links to visual events. We present AVSCap, a framework for audio-visual captioning centered on explicit cross-modal event binding. First, we construct AVSCap-130K, a tri-modal training corpus generated by a decoupled-then-fused pipeline that anchors visual and acoustic evidence independently before composing temporally grounded omni-modal captions. Second, we train AVSCap-7B, a 7B captioner with a two-stage strategy: supervised fine-tuning establishes baseline captioning ability, while a sample-efficient reinforcement learning stage uses hybrid rewards to optimize acoustic completeness and audio-visual synergy. Our scaling analysis shows that reinforcement learning brings larger gains than simply increasing SFT data. Third, we introduce AVSCapBench, a human-curated benchmark that decomposes captions into visual, audio, and synergy events and evaluates them with fine-grained event recall. Experiments on AVSCapBench and external benchmarks show that AVSCap-7B improves non-speech audio coverage and cross-modal binding, delivering the best overall performance among evaluated open-source models.

Abstract:
Half-truths, claims that are factually correct yet misleading due to omitted context, remain a blind spot for fact verification systems focused on explicit falsehoods. Addressing such omission-based manipulation requires reasoning not only about what is said, but also about what is left unsaid. We propose RADAR, a role-anchored multi-agent debate framework for omission-aware fact verification under realistic, noisy retrieval. RADAR assigns complementary roles to a Politician and a Scientist, who reason adversarially over shared retrieved evidence, moderated by a neutral Judge. A dual-threshold early termination controller adaptively decides when sufficient reasoning has been reached to issue a verdict. Experiments show that RADAR consistently outperforms strong single- and multi-agent baselines across datasets and backbones, improving omission detection accuracy while reducing reasoning cost. These results demonstrate that role-anchored, retrieval-grounded debate with adaptive control is an effective and scalable framework for uncovering missing context in fact verification. The code is available at https://github.com/tangyixuan/RADAR.

Abstract:
Answer-only reinforcement learning (RL) trains reasoning models to solve fully specified problems, but many realistic queries omit a premise needed for a unique answer. In this setting, the useful response is not always refusal: the model should ask for the missing premise, condition its answer on the unknown quantity, or abstain when no informative conditional response is available. We present Ask-Condition-Abstain Reinforcement Learning (ACA-RL), a data-augmented RL framework for this setting. Its reasoning-graph-guided pipeline converts well-posed problems into missing-premise training instances with localized gap annotations; ACA-RL then trains on these instances with a structured reward over five observable response behaviors. We also introduce the Missing-Premise Benchmark (MPB), a 274-instance human-verified benchmark spanning mathematical, logical, and real-world word problems. Across Qwen3 and Llama models, ACA-RL consistently improves on MPB while preserving competitive performance on well-posed reasoning tasks. This work supports a new mission for NLP evaluation: measuring whether models can recognize when a task is underdetermined and handle uncertainty, not only whether they can answer fully specified questions.

Abstract:
Resource utilization has been studied as an optimization objective for generative large language models (LLMs). Recent studies show that this resource dependence can also be weaponized, as excessive generation, when scaled through high-volume requests, increases inference latency, undermines service availability, and threatens economic sustainability. However, these risks remain underexamined because existing studies are fragmented and their boundaries with legitimate, resource-intensive uses of LLMs are unclear. To address this gap, we provide a systematic review of threats to resource consumption in LLMs. Unlike surveys on inference acceleration, we focus on how resource consumption threats induce disruptive behaviors and synthesize existing studies into research strands that connect attack objectives, triggering mechanisms, and defenses. We further introduce an outcome-based severity taxonomy that distinguishes bounded, non-degenerate Overthinking from self-sustaining Unbounded Drift, providing operational criteria for characterizing adversarial resource consumption. Our goal is to clarify this emerging threat landscape and support future characterization, evaluation, and mitigation. The project repository is publicly available at https://anonymous.4open.science/r/RCT-LLM/.

Abstract:
Current genomic foundation models (GFMs) rely largely on extensive neural computation to implicitly infer conserved biological motifs from single-nucleotide inputs. We introduce Gengram, a conditional memory module that provides an explicit and efficient lookup primitive for multi-base motifs through a genome-specific hashing scheme. Integrated into state-of-the-art (SOTA) GFM backbones, Gengram achieves substantial improvements of up to 22.6% across functional genomics tasks and generalizes across diverse architectures. Further analysis of Gengram’s latent space reveals biologically meaningful representations aligned with established genomic principles. By introducing structured motif memory as a modeling primitive, Gengram improves empirical performance and mechanistic interpretability, offering a scalable and biology-aligned direction for next-generation GFMs. The code is available at https://github.com/zhejianglab/Gengram, and the model weights are available at https://huggingface.co/ZhejiangLab/Gengram.

Abstract:
Hierarchical Document Structure Analysis (HDSA) aims to recover the logical tree and reading order of a document, providing a structural foundation for downstream tasks such as document question answering, information extraction, and long-form document understanding. Extending HDSA to multi-page documents remains challenging, as pagination introduces discontinuities between page-local coordinate systems, making it difficult to link structurally related elements across pages. We present Scroll-Net, a geometry-aware constrained decoding framework for multi-page document hierarchy parsing. Scroll-Net maps page-local coordinates into a unified scroll-space coordinate frame, enabling direct comparison of cross-page elements. It employs a dual-stream encoder to disentangle geometric and semantic cues, injecting geometric inductive bias before multimodal fusion. Under the pre-order convention used by the evaluated benchmarks, Scroll-Net constrains autoregressive parent decoding with a hard reading-order causal mask. This backward-pointer constraint makes decoded parent graphs acyclic under the imposed order while retaining O(K) per-step decoding, reducing inference latency by 36.5% and peak memory by 46.5%. Across four benchmarks, Scroll-Net remains competitive on simple layouts while achieving state-of-the-art results on structurally complex documents, improving TEDS by 4.8 points on DocHieNet.

Abstract:
Current RAG evaluation measures answer quality under fixed evidence but cannot detect systems that ignore evidence changes or fall back to parametric knowledge. We introduce Dynamic Evidence Contracts, a behavioral evaluation paradigm that specifies expected RAG behavior under four evidence interventions (deletion, conflict, distraction, permutation), and the Intervention Compliance Score ( ICS ) to quantify compliance. Evaluation on five datasets reveals that all evaluated open-source RAG pipelines achieve moderate conflict detection ( CDR=0.283 –0.847) via passive NLI-detected refutation signals, but lack explicit conflict acknowledgment. We further present DOVE-RAG, a minimal reference instantiation showing that explicit conflict acknowledgment yields higher CDR (0.505–0.999), up to 1.6× ICS improvement, and 2.0× citation recall gains. Ablation reveals that verification without structured output reorganization yields negligible improvement, suggesting that evidence compliance requires an integrated pipeline.

Abstract:
Despite recent progress in language models and agents for scientific data-driven discovery, advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks. To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises 565 tasks from 239 real scientific repositories across four disciplines, each with a natural language instruction, an executable environment with pre-installed dependencies, dataset previews, a reference solution, and an automatically synthesized evaluation script. Our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic. Training on trajectories sampled from D3-Gym yields consistent gains across Qwen3 models on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and shrinking the gap with strong proprietary models. We further illustrate, through case studies, how D3-Gym environments can serve as a testbed for studying agentic optimization loops such as Autoresearch on real scientific workflows. We open-source D3-Gym, its creation workflow, sampled trajectories, and training scripts at https://github.com/OSU-NLP-Group/D3-Gym.

Abstract:
Mathematical reasoning has seen rapid progress in large language models (LLMs), yet existing methods optimize predominantly for final-answer correctness, raising the question of whether models truly internalize mathematical concepts or merely memorize solution patterns. In human mathematics education, example-based reasoning, such as constructing counterexamples to test theorem boundaries, reflects deep conceptual understanding, but remains underdeveloped in current LLMs. Enhancing this capability through preference optimization presents two key challenges: (1) the model’s limited example-based reasoning ability makes constructing effective preference pairs inherently difficult; and (2) capability acquisition is progressive, as the model must first learn to adopt this strategy before learning to apply it correctly. Therefore, we propose INSPIRE, an Internalize-Then-Improve approach combining Reference-Guided Student Internalization (RGSI), which produces high-quality preference candidates under the policy model’s own distribution, with a stage-wise rubric preference training strategy that decomposes learning into method-oriented and correctness-oriented stages. Experiments across multiple model scales and families demonstrate consistent improvements, even surpassing larger open-source models, while evaluations on out-of-distribution benchmarks confirm no degradation in general mathematical reasoning ability.

Abstract:
We propose a method that represents language models by log-likelihood vectors over prompt-response pairs and constructs model maps for comparing their conditional distributions. In this space, squared Euclidean distances between models are approximately proportional to the KL divergence between the corresponding conditional distributions. Experiments on a large collection of publicly available language models show that the maps capture meaningful global structure, including relationships to model attributes and task performance. The representation also captures systematic shifts induced by prompt modifications and their approximate additive compositionality; we use the vectors to predict downstream task scores and leverage their additive structure to approximate the effects of composite prompt operations without directly observing the corresponding log-likelihood vectors. We further introduce PMI vectors to reduce the influence of unconditional distributions; in some cases, PMI-based model maps better reflect training-data-related differences. Overall, the framework supports the analysis and prediction of input-dependent model behavior.

Abstract:
New Intent Discovery (NID) aims to uncover unseen intent categories from unlabeled user queries, with the core objective of learning an intent representation space with a clear clustering structure. However, in the absence of explicit supervision, how to reliably form a global clustering structure remains challenging. To address this issue, we start from the constituent units of the global structure, namely local relations, and introduce more accessible and more stable forms of local supervision. Based on this idea, we propose LR2GS, which directly trains the intent encoder by leveraging multiple complementary local supervision signals, allowing local constraints to continuously act on the training process and thereby progressively shaping a well-separated global intent structure. Extensive experimental results demonstrate that LR2GS achieves state-of-the-art performance in both unsupervised and semi-supervised settings.

Abstract:
Many reasoning tasks require models to reason over input context, from document-grounded question answering to rule-based deduction. Chain-of-Thought (CoT) prompting produces traces that appear transparent, yet individual steps can silently deviate from the source evidence, even when the final answer is correct. Existing methods detect hallucinations at the response level but fail to identify where in the chain a failure occurs or what type it is. We introduce GRACE, the first human-annotated step-level faithfulness benchmark with a data-driven error taxonomy for context-grounded textual reasoning. GRACE covers CoT traces from 10 models across 4 source datasets, with each step annotated for faithfulness, error category, and natural language explanation. A data-driven taxonomy, discovered bottom-up via unsupervised clustering, organizes failures into two tracks: GRACE-Inference (deductive errors) and GRACE-Grounding (factual grounding errors), with four categories each. The evaluation set is human-annotated and challenging by design. Our experiments reveal substantial headroom for current models. In addition, integrating step-level faithfulness signals into reinforcement learning pipelines improves both downstream accuracy and reasoning reliability.

Abstract:
Humans understand long and complex texts by relying on a holistic semantic representation of the content. This global view helps organize prior knowledge, interpret new information, and integrate evidence dispersed across a document, as revealed by the Mindscape-Aware Capability of humans in psychology. However, current Retrieval-Augmented Generation (RAG) systems often lack explicit global semantic guidance, making it difficult to retrieve and integrate dispersed evidence in long-context tasks. In this paper, we propose Mindscape-Aware RAG (MiA-RAG), the first framework to formulate mindscape-aware retrieval and generation as a unified conditioning paradigm for LLM-based RAG. MiA-RAG builds a mindscape through hierarchical summarization and conditions both retrieval and generation on this global semantic representation. This enables the retriever to form enriched query embeddings and the generator to reason over retrieved evidence within a coherent global context. We evaluate MiA-RAG across diverse long-context and bilingual benchmarks for evidence-based understanding and global sense-making. It consistently surpasses baselines, and further analysis shows that it aligns local details with a coherent global representation, enabling more human-like long-context retrieval and reasoning.

Abstract:
Large language models are increasingly used for subjective tasks such as seeking advice, where there is often no single correct answer. Yet dominant alignment methods learn a single behavior policy that applies uniformly across users, limiting their ability to reflect individual value priorities. We propose personalized value alignment, a framework for adapting model responses to users’ stable value profiles rather than to personas, demographics, or task-specific preferences used in prior personalization work. Our approach represents users through Schwartz value profiles and selects which of their values are relevant to each query before applying targeted prompting, activation steering, or LoRA-DPO. We create a dataset of 1,501 value-tension scenarios and evaluate over 152 diverse value profiles sampled from a nationally representative survey. Value-targeted methods outperform no-personalization, demographic, and full-profile prompting baselines, indicating that gains come from focused conditioning on query-specific value priorities rather than mere exposure. We further find that steering effects are not uniform across users: alignment gains vary by where a user’s priorities fall in the value space. Combining targeted prompting with activation steering or LoRA-DPO yields the strongest personalization while keeping logical quality within adequate range.

Abstract:
LLM-based evaluators of natural language generation (NLG) quality are widely deployed as scoring tools and as automated training signals, yet the internal procedure by which they assign a rating remains poorly understood. We investigate this procedure mechanistically through an eight-attack perturbation taxonomy across the Readability and Adequacy dimensions of NLG quality, a generation pipeline that produces paired clean and corrupt summaries with controlled error intensity and explicit token-level modification maps, and a four-experiment battery of causal tracing, logit-lens vocabulary projection, and attention-head knockout applied to Themis (Llama-3-8B) and Prometheus (Mistral-7B). Both evaluators implement a structured, coherent evaluation pipeline operating in two stages: below layer 15, attention performs local error comparison and routes the result to the final input position; above it, the MLP cascade integrates the signal and writes the rating, with the decision crystallizing in the residual stream at a sharp late layer (L = 26 on Themis, L = 25 on Prometheus). Furthermore, a base-model control at the same scale (Llama-3-8B) reproduces the routing architecture and crystallization but not the stage separation, isolating the two mechanisms that fine-tuning specifically installs, suppression of below-L15 MLP contribution at the last position and a two-layer advance of the crystallization depth, indicating that fine-tuning sculpts an existing substrate rather than building the pipeline from scratch. We release the source code and data at https://github.com/himil-v/judge-mech.

Abstract:
Graphical User Interface (GUI) agents powered by vision-language models (VLMs) have emerged as a promising paradigm for computer-use automation, but remain unreliable on long-horizon tasks. Existing planner–executor frameworks address this by shifting more reasoning to the planner, thereby decomposing rather than decoupling long-horizon reasoning. Our failure analysis reveals that many long-horizon failures are not pure capability gaps: from the same failure state, executors recover substantially more often when their accumulated context is reset with a locally executable subtask. To bridge this gap, we propose SCoR, a sparse context reset framework that restores the executor’s short-horizon capability by resetting its accumulated task context. At subtask boundaries, a lightweight Refiner emits a locally executable subtask and resets the Executor’s context; the executor then autonomously executes multiple GUI actions within the reset context. We train the Refiner with turn-level GRPO. In OSWorld, SCoR achieves a success rate of 53.16%, yielding a 25.41% relative improvement over the standalone EvoCUA-8B executor. Our code is available at https://github.com/huia711/SCoR.

Abstract:
Inferring human engagement from gameplay video is important for game design and player-experience research, yet it remains unclear whether vision–language models (VLMs) can infer such latent psychological states from visual cues alone. Using the GameVibe Few-Shot dataset across nine first-person shooter games, we evaluate three VLMs under six prompting strategies, including zero-shot prediction, theory-guided prompts grounded in Flow, GameFlow, Self-Determination Theory, and MDA, and retrieval-augmented prompting. We consider both pointwise engagement prediction and pairwise prediction of engagement change between consecutive windows. Results show that zero-shot VLM predictions are generally weak and often fail to outperform simple per-game majority-class baselines. Memory- or retrieval-augmented prompting improves pointwise prediction in some settings, whereas pairwise prediction remains consistently difficult across strategies. Theory-guided prompting alone does not reliably help and can instead reinforce surface-level shortcuts. These findings suggest a perception–understanding gap in current VLMs: although they can recognize visible gameplay cues, they still struggle to robustly infer human engagement across games.

Abstract:
Long-term memory is increasingly important for conversational agents, yet existing benchmarks primarily measure memory through pointwise factual recall: whether a system can recover isolated facts or event-level details from prior interactions. Real-world memory use, however, often requires a more demanding capability: integrating distributed, implicit, and noisy evidence across extended interaction histories into coherent, task-oriented outputs. We call this capability memory utilization. Here, we introduce UtilMem, a diagnostic benchmark comprising 1,717 instances across five domains, designed to evaluate four underexplored aspects of memory utilization: reasoning over dense histories, identifying implicitly relevant memories, synthesizing distributed evidence into summaries, analyses, or plans, and resisting interference from semantically similar distractors. Evaluating a diverse set of retrieval-based and memory-augmented systems, we find that strong performance on conventional factual-memory benchmarks does not reliably translate into effective memory utilization. Moreover, retrieval alone is insufficient: even when relevant evidence is successfully recovered, systems frequently fail to integrate information across sessions or to distinguish useful evidence from plausible distractors. These findings expose a substantial gap between accessing stored information and using it effectively, and suggest that progress in long-term conversational memory will require architectures that explicitly support evidence integration and robustness to retrieval interference. Code is available at https://github.com/peijunallin/UtilMem.

Abstract:
Speculative decoding is a critical technique for accelerating LLM inference. In practice, however, the speedup is fragile: drafters are typically trained against a narrow data distribution for a single target model, and their acceptance rate collapses under workload shifts. This is a striking inversion of modern LLM development, where target models are valued precisely for the broad generalization they acquire through large-scale pretraining. We argue that the natural remedy—pretraining—has been hard to apply to drafters because existing recipes are deeply target-specific: the drafter consumes the target’s hidden states and is distilled on the target’s logits, so any pretraining effort must be redone for every new target. We introduce Osprey, which instead bootstraps drafters from off-the-shelf pretrained small language models, treating broad pretraining as a reusable, target-agnostic asset and reducing per-target work to a lightweight adaptation step. Realizing this requires overcoming two challenges: small LMs are far deeper than a latency-bound drafter can afford, and their pretrained computation must remain intact while the drafter learns to ingest target hidden states and emit tokens in the target’s vocabulary. Osprey addresses both by pruning to a shallow backbone, restoring its language-modeling capability with target-agnostic next-token pretraining, and then adapting it to each target through vocabulary alignment, zero-initialized QKV expansion, and target distillation. Empirically, a single pretrained Osprey backbone transfers across targets and improves mean acceptance length by 16.1% for Qwen3-8B, 21.2% for Llama-3.3-70B-Instruct, and 22.7% for the 229B MiniMax-M2.5 (with 17.5% higher tokens per second), with the largest gains on out-of-domain and multilingual data. Our code is available at https://github.com/LeanModels/Osprey.

Abstract:
Large reasoning models (LRMs) "think" by generating structured chain-of-thought (CoT) before producing a final answer, yet they still lack the ability to reason critically about safety alignment and are easily biased when a flawed premise is injected into their thought process. We propose RECAP (Robust Safety Alignment via Counter-Aligned Prefilling), a principled reinforcement learning (RL) method for post-training that explicitly teaches models to override flawed reasoning trajectories and reroute to safe and helpful responses. RECAP trains on a mixture of synthetically generated counter-aligned CoT prefills and standard prompts, integrates seamlessly into standard reinforcement learning from human feedback (RLHF) pipelines without additional RL optimization steps or architectural modifications, and substantially improves safety and jailbreak robustness, reduces overrefusal, and preserves core reasoning capability — all while maintaining inference token budget. Extensive analysis shows that RECAP-trained models engage in self-reflection more frequently and remain robust under adaptive attacks, preserving safety even after repeated attempts to override their reasoning. Our code is publicly available at https://github.com/poloclub/recap.

Abstract:
We study how Vision Transformers ground abstract concepts (e.g., angry) when training data provide limited direct referential evidence. We hypothesize a metonymic grounding mechanism in which abstract predictions are driven by concrete, interpretable anchor concepts (e.g., fire) that bridge visual signals to abstract semantics. By applying Transcoders on CLIP and DINO vision encoders, we recover intermediate features that can be associated with semantic labels for more concrete concepts, and trace their contributions in circuits underlying abstract concept recognition. Experiments on a carefully curated icon dataset reveal structured metonymic circuits, in which perceptual primitives dominate early layers and object-like anchors precede abstract targets. Images containing rendered text instead recruit a distinct perceptual-to-textual route. Causal interventions further validate that metonymic intermediates are functionally involved in grounding abstract concepts.

Abstract:
Retrieval-Augmented Generation enhances Large Language Models by grounding responses in external knowledge, but multi-hop reasoning remains vulnerable to error propagation, where early retrieval failures confound subsequent steps. Standard outcome-based optimization only rewards the final answer, leaving intermediate retrieval and reasoning errors undetected. While existing process-based methods introduce step-level signals, they still score each step against the final answer, rewarding spurious successes where flawed retrieval coincidentally produces the correct answer. Step-level supervision in RAG requires evaluating both logical validity and evidential grounding at each step. We introduce PRO-STEP: we train a generative PRM that evaluates both dimensions, employ PRM-guided value tree search to construct preference pairs contrasting valid steps against flawed ones, and optimize the policy via step-level Direct Preference Optimization. Experiments on single- and multi-hop QA datasets demonstrate that PRO-STEP mitigates error propagation and outperforms existing outcome- and process-based RAG baselines.

Abstract:
Large Language Models (LLMs) show strong potential in medical QA, but still suffer from hallucinations in multi-step clinical reasoning. Existing methods fail to assess confidence at intermediate steps or reliably integrate external knowledge, leading to propagated errors and misleading conclusions. Inspired by Bayesian inference logic in clinical diagnosis, we propose BELIEF, a Bayesian Evidence Leveraged InfErence Framework that enables LLMs to perform reliable medical reasoning through dynamic belief assessment and knowledge-augmented disambiguation. First, the Bayesian Belief Updating Module (BBUM) treats diagnostic hypotheses as prior probabilities and calculates posterior probabilities based on observed clinical evidence, preventing early-stage error propagation with statistical rigor. Second, the Dynamic Evidence Retrieval Module (DERM) autonomously queries external medical databases to resolve ambiguity when the posterior belief exhibits high entropy. Experiments across 7 datasets and 2 LLM backbones show that BELIEF significantly improves diagnostic accuracy, enhances interpretability, and mitigates hallucinations through explicit probabilistic updates.

Abstract:
Production agentic systems routinely face evolving constraints and must comply from the very next interaction. Scenarios like a tool-call notification changing a compliance threshold or a policy update adding disclosure requirements fit this criteria, having close to no room for errors in production. This proactive adaptation setting is common in deployment, but absent from current benchmarks, which assume either static constraint sets or reactive protocols with evaluation feedback. We introduce RECAP, a benchmark that measures continual-learning phenomena (forgetting, regression, forward transfer) at the constraint level under a strictly proactive adapt-then-test protocol: prompt optimization methods receive only the constraint specification and must generalize before seeing any test data. Evaluating six methods across five LLMs and three schedules with evolving constraints, we find that these methods show no significant improvement in performance, even after incurring a higher latency. These methods, designed for offline or reactive settings, are inadequate for the proactive paradigm. Our work emphasizes the growing need for designing proactive prompt adaptation methods, where the models must remain robust to evolving needs in deployment.

Abstract:
Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the attention module often introduces large errors at very low bit-widths, causing performance degradation. Existing methods mainly rely on smoothing techniques to handle outliers, while we propose a hybrid quantization design to better balance accuracy and efficiency. Specifically, we propose HyQuant, an efficient hybrid quantization framework for LLM attention. HyQuant quantizes most attention states into low-bit formats while retaining a small set of vertical-line tokens and local-window states in high precision. These accuracy-critical regions are selected using lightweight vertical-line-aware attention-pattern signals, reducing quantization error with limited overhead. In the Prefill stage, HyQuant uses a hybrid-precision quantized attention operator that preserves vertical-line tokens and a local sliding window in full precision while quantizing the remaining context. In the Decode stage, HyQuant applies the same principle to KV-cache compression and fuses KV dequantization with attention computation to improve memory and hardware efficiency. Across diverse tasks, models, and datasets, HyQuant maintains nearly lossless accuracy with a simple design, demonstrating the efficiency and practical feasibility of hybrid quantization for LLM attention.

Abstract:
Although uniform diffusion language models (UDLMs) represent a promising diffusion paradigm, scaling them remains challenging. We identify the core obstacle as an over-uniform training objective and condition-target confusion during sampling. To address these, we propose Less Uniform Diffusion (LUDI), a novel UDLM framework. Specifically, we (i) introduce a less uniform loss that directs each reverse transition toward the clean token, and (ii) equip the model with per-token time embeddings that supply token-level corruption hints, enabling confidence-based few-step sampling. Experiments across scales show that LUDI yields cleaner supervision and improves few-step generation. We further continue-train a 7B autoregressive model into LUDI-7B, resulting in a UDLM capable of complex reasoning. It achieves 3 token-per-step speedup over AR decoding and a competitive performance compared with masked diffusion baselines, revealing that the full potential of UDLMs for complex generation remains to be unlocked.

Abstract:
Crosslingual evaluation of language models that enables fair comparisons remains a fundamental challenge in multilingual NLP. Existing studies adopt a variety of downstream tasks and intrinsic metrics with different theoretical justifications, yet there has been little empirical investigation into whether these approaches yield meaningful crosslingual conclusions. We systematically examine crosslingual evaluation approaches using controlled monolingual language models trained on parallel data with varying tokenizer vocabulary sizes and model sizes, and further validate our findings on multilingual LLMs. We further discuss challenges in achieving comparable downstream evaluation across languages. Our results show that several widely used normalized metrics introduce crosslinguistic biases rooted in tokenization, encoding, and orthographic differences. In contrast, sentence-level negative log-likelihood computed over semantically equivalent sequences provides more meaningful and consistent crosslingual comparisons.

Abstract:
Language agents are increasingly used as social simulators, but most are evaluated by local utterance plausibility rather than their ability to reproduce longitudinal discourse dynamics. We introduce PRISM, a calibrated language-agent framework for forecasting and counterfactual simulation of social media threads. PRISM separates three components that are often conflated in prompt-only agents: latent affective evolution, participation control, and language generation. It models affect with a profile-conditioned stochastic differential equation, selects participation through a partially observed activation policy, and uses a language model as a controlled generator conditioned on the resulting state. Agent profiles are instantiated with MBTI-like anchors, but treated only as behavioral priors rather than psychological ground truth. On 47,207 post-rooted discussion threads spanning 159 trending topics and 356,593 hierarchical comments, PRISM outperforms homogeneous, random, continuous-trait, prompt-only, and memory-only baselines in trajectory alignment, held-out thread forecasting, and blind human evaluation. Reliability analysis further validates LLM-derived weak labels against human and independent-model annotations. These results suggest that calibrated state-space agents offer a more reliable foundation for online discourse simulation than persona prompting alone.

Abstract:
While there is growing interest in in-context language learning (ICLL) for unseen languages with large language models, such languages usually suffer from the lack of NLP tools, data resources, and researcher expertise. This means that progress is difficult to assess, the field does not allow for easy large-scale experimentation, and findings on ICLL are often limited to very few languages and tasks. In light of such limitations, we introduce a framework (Rashid) for studying ICLL wherein we reversibly cipher high-resource languages (HRLs) to construct truly unseen languages with access to a wide range of resources available for HRLs, unlocking previously impossible exploration of ICLL phenomena. We use our framework to assess current ICLL methods for machine translation with SOTA evaluation tools and manual analysis, explore new directions leveraging otherwise expensive resources for improving ICLL, and test ICLL strategies for rich downstream tasks beyond machine translation. These lines of exploration showcase the possibilities enabled by our framework, as well as providing actionable insights regarding current performance and future directions in ICLL.

Abstract:
Bistable images such as the duck-rabbit are classic stimuli in which one image supports multiple mutually incompatible interpretations, typically reported one at a time in humans. We ask whether multimodal large language models (MLLMs) show similar report behavior and what internal computations support it. Using the LLaVA family, we study two tractable dimensions: modulability, whether reports can be biased by bottom-up visual cues and top-down linguistic priors, and exclusivity, whether responses commit to a single interpretation. We test both on the canonical duck-rabbit and on synthetic Visual Anagrams to mitigate memorization confounds. Behaviorally, both visual and linguistic manipulations systematically shift reports in human-consistent ways, while responses remain predominantly exclusive. Mechanistically, these effects arise from competing image-token representations, distinct pathways for bottom-up and top-down modulation, and a link between exclusive reporting and object-count encoding.

Abstract:
Masked Diffusion Language Models (MDLMs) have recently emerged as a promising alternative to autoregressive models. Unlike standard diffusion-based approaches, MDLMs are not explicitly conditioned on a timestep, raising a natural question: do these models internally represent denoising progress, and how is such information used downstream? In this work, we show that MDLMs do in fact encode a latent representation related to the diffusion timestep within their residual streams. We find that this signal can be reliably extracted using probes across layers, indicating that denoising progress is decodable from internal activations. We further demonstrate that steering the model along a low-dimensional subspace associated with the inferred timestep allows us to systematically modulate its notion of denoising progress, leading to controlled and predictable changes in model confidence and entropy. Finally, we analyse the geometry of the identified representation, showing that it exhibits structured and interpretable properties in activation space, and shedding light on how such a signal is manipulated by these models.

Abstract:
Spoken dialogue state tracking recovers slot-value pairs from speech, where ASR errors concentrate in entity values and persist across turns, making it both a generation and an editing problem. A strong per-turn text editor corrects much of this but, operating on the transcript alone, leaves three recoverable errors: a value predicted inconsistently across turns, an omitted slot, and a value the audio does not support. We present AVERT, which scores each candidate value by combining cross-turn agreement with a trained audio-conditioned verifier and resolves the three error types with three operators, vote, add, and swap, each restricted to the slots where its error is common. On SpokenWOZ, a base speech-LLM reaches 33.04 JGA, a text editor 38.34, and AVERT 40.13, without retraining either. This is in the range of a 1B end-to-end system that consumes the full spoken history (39.32), though AVERT uses two 1B decoders rather than one. The audio verifier contributes a statistically significant gain, and restricting each operator to a selected slot subset matters: removing it lets unrestricted voting overwrite correct categorical values and fall below the editor.

Abstract:
How do people learn to become better conversationalists? This question is especially important in the context of mental-health counseling, where conversational skills are essential, yet volunteer counselors often have limited access to supervision and structured feedback. Understanding how counselors develop their ability to steer conversations toward positive outcomes—and identifying early which counselors are (not) on track to improve—can help prioritize support for the counselors who need it most. In this work, we introduce the task of predicting, early in a conversationalist’s career, whether they will eventually improve at steering conversations toward positive outcomes, and demonstrate the feasibility of this task in the case of volunteer mental-health crisis counselors. Our central insight is that people may struggle with particular kinds of moments in a conversation, and that what is especially revealing of their likelihood of future improvement is how they learn to handle those moments over time. We operationalize this insight by designing a method that identifies the types of moments a counselor initially struggles with, captures how they adapt their response when they re-encounter similar moments in subsequent conversations, and learns which early adaptations predict improvement months or even years later. While this future-prediction task is challenging, our counselor-adaptation approach yields better results than baselines that learn directly from the conversation transcript.

Abstract:
Recent studies suggest that video generative models can exhibit certain forms of zero-shot visual reasoning through generated frames. Yet reliable evaluation remains challenging: benchmarks should adopt inputs aligned with the visual priors of current video models, require valid evolving processes rather than only plausible final states, and calibrate task difficulty to remain challenging yet partly feasible. To this end, we introduce VGI-Bench, containing 27 tasks and 810 instances, organized by a two-level taxonomy of domains and skill tags for evaluating the visual reasoning capabilities of video generative models. Our evaluations show that current generative systems can solve a subset of visually grounded reasoning tasks, but remain far from reliable, with even the strongest model, Seedance 2.0, achieving only 51.0% under our evaluation criteria. Our findings further reveal the input sensitivity, skill-bounded transfer from synthetic fine-tuning, and limited denoising-time self-correction, where later steps mainly refine early hypotheses rather than correct reasoning errors. We hope VGI-Bench will help stimulate the development of next-generation video generative models. Data and code are available at [our project page](https://hexuan21.github.io/VGI-Bench/).

Abstract:
Multi-agent LLM systems for medical question answering treat consensus as a reliability signal: if multiple agents agree on an answer, it is presumed trustworthy. However, answer-level consensus does not entail reasoning-level alignment. We introduce CARA (Cross-Agent Reasoning Alignment), a family of automated metrics that measure whether agents who agree on an answer also agree on the reasoning. Applying CARA to a standard debate system on two medical QA benchmarks (MedQA-USMLE and MedThink-Bench), we identify the consistency illusion: a failure mode where debate reduces detectable contradictions between agents while simultaneously decreasing the semantic similarity of their reasoning chains; agents appear to agree more but reason less consistently. To improve this misalignment, we propose the Grounded Debate Protocol (GDP), a prompt-level intervention that requires agents to commit to named medical facts and take explicit stances on other agents’ claims. GDP produces large, consistent alignment improvements (Cohen’s d = +1.43 to +1.99) across two datasets and two backbone models, without adding LLM calls or modifying system architecture. Our results motivate cross-agent reasoning alignment as a quantity to audit alongside accuracy in safety-critical domains.

Abstract:
Vision-Language-Action (VLA) models are commonly treated as end-to-end action policies conditioned on natural-language task descriptions. In practice, however, their behavior often depends sharply on how the instruction is phrased, suggesting that language is not merely a task label but an optimizable conditioning input. We study whether frozen VLA policies can be improved by optimizing language space rather than updating action weights. Our method introduces a language-conditioning space policy that translates a human instruction into a short VLA-grounded command using object appearance, spatial relations, and target-grounding cues. The language-conditioning space policy is optimized with reinforcement learning from sparse task-completion rewards, while the downstream VLA remains fully frozen. Experiments on RL4VLA and VL-Think show that language-conditioning space optimization improves success on instruction-sensitive, symbolic, and multi-object manipulation tasks, demonstrating that language can serve as an optimizable variable for a robot foundation models.

Abstract:
Reinforcement Learning from Human Feedback (RLHF) often suffers from sparse reward signals, making credit assignment to tokens challenging. We propose Shapley Credit Assignment Rewards (SCAR), a method that uses Shapley values from cooperative game theory to distribute sequence-level rewards across tokens or text spans according to their marginal contributions. SCAR provides dense rewards without requiring auxiliary critique models or fine-grained human annotations, while offering a principled game-theoretic foundation for credit assignment. We show theoretically that SCAR preserves the original optimal policy, and empirically demonstrate across sentiment control, summarization, and instruction tuning tasks that it converges faster and achieves higher final rewards than standard RLHF and attention-based dense reward baselines. To improve scalability on long sequences, we further introduce an Owen value–inspired adaptive text segmentation strategy. Our findings suggest that SCAR provides a more effective and theoretically sound method for credit assignment in RLHF, leading to more efficient alignment of LLMs.

Abstract:
Implicit Discourse Relation Classification (IDRC) identifies discourse relations between text spans when connectives are absent. The task is typically evaluated on the Penn Discourse Treebank (PDTB), where discourse senses are organized in a hierarchical structure. Recent unified models predict discourse labels level by level under teacher forcing, which leads to exposure bias and cascading errors across the hierarchy. Moreover, token-level training objectives fail to assess the correctness of the entire label sequence. We reformulate multi-level IDRC as hierarchical label sequence generation and optimize the full label sequence using sequence-level reinforcement learning. By sampling and ranking complete label sequences, the proposed framework directly optimizes global sequence correctness rather than isolated token-level decisions. Experiments on PDTB 2.0 and 3.0 demonstrate consistent improvements over strong baselines. Further analysis shows that fine-to-coarse decoding outperforms the conventional coarse-to-fine strategy, highlighting the importance of decoding order in hierarchical discourse modeling.

Abstract:
The robustness of Vision Language Models (VLMs) is commonly assessed through output-level invariance, implicitly assuming that stable predictions reflect stable multimodal processing. In this work, we argue that this assumption is insufficient. We introduce a representation-aware and frequency-aware evaluation framework that measures internal embedding drift, spectral sensitivity, and structural smoothness (spatial consistency of vision tokens), alongside standard label-based metrics. Applying this framework to modern VLMs across the SEEDBench, MMMU, and POPE datasets reveals three distinct failure modes. First, models frequently preserve predicted answers while undergoing substantial internal representation drift; for perturbations such as text overlays, this drift approaches the magnitude of inter-image variability, indicating that representations move to regions typically occupied by unrelated inputs despite unchanged outputs. Second, robustness does not improve with scale; larger models achieve higher accuracy but exhibit equal or greater sensitivity, consistent with sharper yet more fragile decision boundaries. Third, we find that perturbations affect tasks differently: they harm reasoning when they disrupt how models combine coarse and fine visual cues, but on the hallucination benchmarks, they can reduce false positives by making models generate more conservative answers.

Abstract:
Conformal prediction supplies finite-sample coverage guarantees for individual language-model responses. We show that per-step conformal calibration need not preserve this guarantee along a multi-step agent trajectory: at a nominal miscoverage level of 10%, per-step split conformal delivers 83.5% trajectory score-threshold miscoverage on a fifteen-step web-navigation agent, and we prove lower bounds in three regimes of agent dynamics: i.i.d. resampling, memoryless covariate shift, and state-dependent Markov transitions. The i.i.d. regime alone accounts for the bulk of the empirical gap; the third regime, novel to this work, gives, to our knowledge, the first separation of the trajectory union event from i.i.d. resampling under marginal-stationary, non-trivial Markov dynamics, a setting in which prior marginal-drift analyses are silent. The separation persists under exact score stationarity and, for non-degenerate score laws, holds at any calibration sample size up to an O(T/sqrt(n)) finite-sample slack. A polynomial alpha-spending schedule restores end-to-end coverage to within 1.1 points of nominal across five agentic-NLP benchmarks and, by direct substitution, lifts process-reward-model step verifiers – which on their own miscover trajectories at 31.5-62.5% – to 9.6-10.6%, near the nominal target.

Abstract:
Third-party retrieval-augmented generation (RAG) marketplaces create a new auditing problem: data providers may license corpora to a RAG operator, yet later have no visibility into whether their documents are being reused without compensation. Auditing this misuse is difficult because the operator is non-cooperative, answers are paraphrased by the generator, and one response may combine evidence from many providers. We propose DirBucket, a provider-side semantic watermarking and black-box auditing framework for document-level reuse in multi-provider RAG. DirBucket watermarks documents by meaning-preserving paraphrases whose embeddings are biased toward provider-bucket secret directions, enabling detection from black-box answers while preserving retrieval utility. On a challenging benchmark that reflects mixed-provider reuse under black-box access, DirBucket is the only method that consistently achieves strong target detection with no non-target activation, detecting non-compliance in every audit within 23 audited answers on our primary benchmark. The watermark survives adversarial post-answer laundering, and none of the evaluated evasion strategies simultaneously defeats detection while preserving user-perceived answer quality. Detection transfers unchanged to a second benchmark built from real clinical, cyber-threat-intelligence, and legal provider corpora. These results suggest that embedding-space watermarking can make document reuse in third-party RAG statistically auditable.

Abstract:
Large language models increasingly generate research ideas, yet judging their novelty or feasibility at generation time does not establish whether they anticipate subsequent work. We introduce IdeaForecastBench to evaluate research idea forecasting. Given a community’s literature up to a cutoff, a system produces up to five ranked ideas, which are evaluated against later papers. The benchmark comprises 624 rolling episodes across 52 topics, with a fixed retrieve-then-judge protocol and separately reported results from two judges. We compare five history-compression strategies across GPT-4.1, Qwen2.5-7B/14B, and Qwen3.5-9B, together with a learned Mode-Decomposition Forecaster (MDF). Under the primary GPT-4.1-mini judge, Summary improves on Direct in Hit@5 and Precision@5 across all four backbones. Qwen2.5 scores above GPT-4.1, whereas Qwen3.5 scores below it. An outcome-blind assessment finds that Qwen2.5 produces broader forecasts, but does not identify how much breadth contributes to its advantage. Threshold and judge diagnostics further clarify the limits of interpreting realization as precise anticipation. IdeaForecastBench provides a common task for studying which research ideas a community subsequently pursues and how reliably this outcome can be measured.

Abstract:
LLM-as-judge leaderboards now decide what gets trained, deployed, and funded, yet years of bias patching have not asked whether the goals those patches pursue are jointly achievable. We show they are not. Casting LLM-as-judge evaluation as a social welfare functional over score profiles indexed by judge, criterion, and alternative, we prove that no aggregation rule can simultaneously satisfy four fairness axioms (Universal Domain, Weak Pareto, Independence of Irrelevant Alternatives [IIA], and Judge-Anonymity) in the ordinal regime and, among scale-invariant aggregators, in the cardinal regime. A fifth axiom, Position-Invariance, captures upstream pipeline bias and is securable independently. The impossibility admits a constructive counterpart: a four-face achievability frontier with one witness per dropped axiom. The Borda-Conformal Hybrid (BCH) occupies the dropped-IIA face, satisfies the other four axioms, and admits a structural conformal bound relating its IIA-violation probability to the calibration-set gap distribution. A 200-prompt audit on MT-Bench, AlpacaEval 2, and Arena-Hard finds IIA-violation rates from 18.5% to 38.0% across the seven aggregators with measurable IIA; a 50-prompt human-jury comparator confirms a non-zero human IIA-violation baseline consistent with known ranking pathologies in the choice-experiment literature. An audit toolkit and reporting protocol accompany the paper.

Abstract:
Contextual override, the tendency of large language models to shift away from otherwise correct predictions under misleading prompt evidence, poses a fundamental risk to reliable deployment in context-rich environments. Despite its prevalence, the internal mechanism behind this shift remains poorly understood, particularly the causal structure underlying the override process. To bridge this gap, we formulate contextual override as an internal mediation process that attributes evidence-induced answer-preference shifts to fine-grained model components. Building on this formulation, we introduce a theoretically grounded mediated-effect framework that disentangles evidence sensitivity from causal mediation, allowing override-mediating components to be precisely identified and their causal role verified through targeted ablation. We empirically validate our framework across six models and three benchmarks, demonstrating consistent support for our mediation account across diverse architectures and task settings. Our study reveals that contextual override is routed through sparse, structurally localized pathways, rather than being diffusely entangled with the model’s internal knowledge. These findings provide a mechanistic diagnostic for auditing contextual override and a targeted interventional test for validating the identified pathways, rather than a general-purpose defense mechanism.

Abstract:
A retrieval-augmented language model returns the correct free-cash-flow figure from Microsoft’s 10-K filing. Replace the filing with a topical distractor containing no figure, and the model returns the same number: the document was never needed. We introduce RAG-CF, a training-free black-box probe that issues a per-query verdict on whether an apparently correct RAG answer is source-independent, meaning produced without the retrieved documents. The probe perturbs document context under four counterfactual operators, regenerates the answer, and converts the aggregated mismatch into a split-conformal P-value with a finite-sample Type-I bound. Across 60 configurations spanning five benchmarks, four generators, and three retrievers, the FPR-adjusted flagged rate, (raw − α)/(1 − α), ranges from 13% to 56% (raw 21% to 60%); the instrument’s FPR is domain-dependent. A held-out no-retrieval label set confirms: 78% of flagged answers remain correct without documents, against 22% of unflagged; the gap is preserved under LLM-judge equivalence. Source-independence does not decrease with retriever quality. Accuracy alone conflates grounded and parametric success; reporting the source-independence rate alongside it would make the distinction visible on every future RAG benchmark.

Abstract:
Tabular data are prevalent across diverse domains such as healthcare and finance. Zero-shot tabular learning aims to make predictions without labeled data. However, existing LLM-based approaches rely on row-wise prompting, resulting in high inference cost and limited use of inter-row dependencies. In this work, we propose TabHGR+, an efficient relation-aware zero-shot framework for tabular data that leverages LLM-generated rules to label and group rows. Specifically, based on column and task descriptions, we use LLMs to generate rules that assign weak labels to all rows. To refine these noisy weak labels, we construct a hypergraph by grouping semantically related rows using LLM-generated rules and perform hypergraph-based label propagation. Lastly, we further enhance the LLM-generated hypergraph for propagation, which is potentially suboptimal, by assigning label-adaptive weights to hyperedges. By requiring only a constant number of LLM calls independent of dataset size, TabHGR+ substantially reduces inference cost while achieving higher classification accuracy than zero-shot and strong few-shot baselines.

Abstract:
We present an end-to-end open-weight pipeline for LLM-Assisted Rule-Based Machine Translation (LLM-RBMT) into extremely low-resource languages, with no proprietary-API dependency at any stage and no per-language linguistic effort beyond registering a language with the framework. On Owens Valley Paiute (OVP), a critically endangered Indigenous language of California, we synthesize training data entirely from the OVP language specification, fine-tune two Qwen2.5-3B LoRA adapters (one for each LLM-driven stage of the translation system), and serve both locally. The fine-tuned forward adapter matches or exceeds gpt-4o-mini on the 150-sentence forward evaluation of Coleman et al. (2026) (COMET-c 0.472 vs. 0.448, same evaluator), and a separately fine-tuned backward adapter approaches it on a 40-item hand-validated backward reference set, both running locally. The full recipe runs on a single consumer GPU.

Abstract:
This paper revisits the LLM cache bandit problem, with a special focus on addressing query heterogeneity for cost-effective LLM inference. Previous works often assume uniform query sizes. Heterogeneous query sizes introduce a combinatorial structure into cache selection, making the cache replacement process more computationally and statistically challenging. We treat optimal cache selection as a knapsack problem and employ an accumulation-based strategy to effectively balance computational overhead and cache updates. In our theoretical analysis, we prove that the regret of our algorithm achieves an O(√MNT) bound, improving the coefficient of √MN compared to the O(MN√T) result in Zhu et al. (2023), where N is the total number of queries and M is the cache size. Additionally, we also provide a problem-dependent bound, which was absent in previous works. Experiments on real-world data show that our algorithm reduces the total cost by approximately 12%.

Abstract:
Code-switching is a widespread practice among the world’s multilingual majority, yet few benchmarks accurately reflect its complexity in everyday communication. We present PINGPONG, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual. Our dataset consists of human-authored conversations among 2 to 4 participants covering authentic, multi-threaded structures where replies frequently reference much earlier points in the dialogue. We demonstrate that our data is significantly more natural and structurally diverse than machine-generated alternatives, offering greater variation in message length, speaker dominance, and reply distance. Based on these dialogues, we define three downstream tasks: Question Answering, Dialogue Summarization, and Topic Classification. Evaluations of several state-of-the-art language models on PINGPONG reveal that performance remains limited on code-switched inputs, underscoring the urgent need for more robust NLP systems capable of addressing the intricacies of real-world multilingual discourse.

Abstract:
Personalized assistants should not only comply with user requests but also assess whether those requests are appropriate given the user’s current circumstances. However, prior work has primarily focused on accurately executing requests, overlooking the need for assistants to account for context and engage in conflict-based refusal. Furthermore, while existing work on conflict or safety detection relies on explicitly provided factors, real-world scenarios often involve implicit factors that must be retrieved from a knowledge base (KB). To this end, we introduce Personalized Assistants for Conflict Evaluation (PACE), a dataset for evaluating whether models can identify latent constraints, expressed as egocentric knowledge or events, that render seemingly reasonable user requests inappropriate. PACE pairs user requests grounded in well-defined personas with egocentric KB facts, requiring models to integrate contextual evidence to determine whether a request is conflicting. This implicit retrieval setting hinders the direct association between user requests and conflict-inducing knowledge, making it difficult for existing models to identify relevant user-specific facts. To address this challenge, we further propose PaceMaker, a multi-agent framework in which specialized agents coordinate across query reformulation, multi-hop graph traversal, and conflict-aware filtering to retrieve contextually decisive evidence. Experiments on PACE evaluate both evidence retrieval quality and conflict decision accuracy, showing that PaceMaker consistently outperforms existing approaches.

Abstract:
Multimodal topic modeling aims to learn interpretable topics from collections that combine text with other modalities. A persistent difficulty is lexical incompleteness: the text often records only part of the evidence needed to describe a sample, and concepts that are visible in images may have no direct path into the learned word distributions. In many multimodal topic models, visual features mainly help infer document-level topic mixtures, whereas the decoder is still trained on the incomplete observed text. We introduce TIGNTM, a graph completion framework that turns visual evidence into lexical supervision. TIGNTM builds a corpus graph over textual and visual representations, treats related samples as weak lexical witnesses, and produces sparse completion targets for words missing from the observed text. These targets supervise the same decoder that generates topic words, so visual cues can shape the word distributions that define each topic. To match the geometry of normalized multimodal embeddings, TIGNTM adopts hyperspherical neural topic learning. Experiments on multiple multimodal datasets show consistent improvements in coherence, diversity, and overall topic quality compared with strong neural and multimodal baselines.

Abstract:
Benchmark performance is the dominant standard for comparing large language models (LLMs), typically summarized by a single aggregate score, such as accuracy, pass rate, or rubric score. This aggregate score provides limited information about a model’s underlying capabilities, yet understanding these capabilities is crucial for model development and evaluation. To this end, we propose Tree-Induced Diagnostic Evaluation (TIDE), which recasts diagnostic evaluation as behavior-driven decision-tree induction to perform fine-grained capability diagnosis. TIDE recursively builds a hierarchical capability tree from benchmark samples and model scores: at each node, an LLM judge proposes candidate splits and performs sample-to-group classification, and behavior-based information gain selects among them. We apply TIDE to five models across four benchmarks to demonstrate its effectiveness, and find fine-grained capability contrasts invisible at the aggregate level: Claude Opus 4.6 achieves 100% on microeconomic market reasoning where GPT-5.4 scores 83.3%, but trails GPT-5.4 on contract law reasoning with 77.8% versus 88.9%. We hope TIDE can be extended to past and future benchmarks so that the rich behavioral signal in evaluation results is fully leveraged for understanding model capabilities.

Abstract:
Automated fact-checking (AFC) systems retrieve evidence and predict claim veracity, yet evaluations omit simple baselines, systems are developed for a single benchmark and cannot be trusted to generalise across domains. No prior work cross-evaluates the full two-stage retrieve-then-verify pipeline across diverse datasets, complementing retrieval-only studies (Thakur et al., 2021) and single-stage benchmarking studies (Calamai et al., 2025). We benchmark nine models, ranging from random and sparse baselines to fine-tuned transformers, zero-shot LLMs, and the two highest-ranked systems from the AVeriTeC 2025 shared task, across four datasets spanning scientific, open-web, and climate domains. Three findings stand out: (1) on ClimateCheck claim-only and fine-tuned models outperform zero-shot LLM and top-performing AVeriTeC 2025 systems, highlighting that noisy evidence can de- grade veracity prediction; (2) system rankings are strongly domain- and metric-dependent: the best model on SciFact (macro-F1 0.70) drops to 0.31 on ClimateCheck, while the AVeriTeC 2025 winner and runner-up swap rankings based on evaluation metrics and datasets; (3) replacing retrieved evidence with gold annotations improves veracity accuracy by 14-22 points across models, confirming retrieval remains primary bottleneck. We release code, pre-processed datasets, and all results to support reproducible AFC research.

Abstract:
Reasoning over text-attributed graphs (TAGs) requires large language models (LLMs) to combine a node’s text with evidence distributed across its neighbourhood. Existing methods fix the set of accessible neighbours before generation, forcing reasoning to operate over a static context and preventing the model from acquiring missing evidence during inference. We argue that neighbour selection should itself be part of the reasoning process. To this end, we propose Call Neighbours Yourself (CNY), a framework that enables LLMs to proactively explore graph neighbourhoods through topology-constrained graph-walk actions. Instead of reasoning over a pre-selected neighbour set, CNY exposes lightweight neighbour previews and learns when to expand candidate neighbours for additional evidence. To address the delayed-credit challenge of neighbour exploration, we introduce destination-conditioned on-policy self-distillation, which retrospectively evaluates a selected neighbour after its content is revealed and converts the resulting change in action preference into an action-level training signal. Experiments on standard TAG reasoning benchmarks under a unified raw-text setting show that CNY consistently outperforms fixed-context post-training baselines. Furthermore, the learned exploration policy transfers to unseen graphs and to a graph-level task not encountered during training. Code is available at https://github.com/superallen13/CNY.

Abstract:
Recent advancements in the application of Large Language Models (LLMs) to formal theorem proving in Lean have unveiled new prospects in complex logical reasoning scenarios. Acknowledging that the difficulty across reasoning steps in a formal theorem proof is not uniform, prior proof search algorithms have not systematically investigated which proof steps are more crucial to explore. This paper introduces E ntropy-Based B est- F irst S earch (EBFS), an entropy-inspired heuristic tree search algorithm that enhances the efficiency of proof generation models to explore proof trees, thereby increasing the probability of finding the correct proof. We also introduce a reinforcement fine-tuning approach specifically for proof tree search. Using EBFS during the generation stage further boosts the model’s performance in proof generation. Evaluations demonstrate that our fine-tuned EBFS model achieves consistent improvements over the baseline models. Under the same sample budget, it outperforms whole proof generation by 3.2% on the ProofNet testset, with further gains of 1.9% on MiniF2F and 1.5% on ProverBench.

Abstract:
While reinforcement learning with verifiable rewards (RLVR) has advanced multimodal reasoning, coarse outcome rewards offer limited guidance for assigning credit to visual evidence. Analyzing same-prompt rollout groups, we observe that different outcomes exhibit distinct temporal patterns of measured visual dependency across reasoning segments. We therefore propose VIGOR (Visual Grounding Outcome-contrastive Reinforcement), a policy optimization method that converts these group-wise contrasts into fine-grained supervision. VIGOR constructs outcome-conditioned prototypes from temporal visual fingerprints to reweight trajectories, uses segment-level contrasts to prioritize token gradients, and allocates the visual budget according to group difficulty. It requires no additional annotations or reward models and introduces no inference-time overhead. Across eight multimodal reasoning benchmarks, VIGOR achieves 57.5% average accuracy with Qwen2.5-VL-7B, improving over VPPO by 1.6 points. Controlled experiments further validate VIGOR on Qwen2.5-VL-3B and InternVL2.5-4B; on the latter, it outperforms VPPO on all eight tasks by 0.96 points on average. Our code is publicly available at https://github.com/beck83879-bit/VIGOR .

Abstract:
Many annotation projects begin before experts have a stable guideline or enough labels to train a task-specific model. We present Goldsmith, an agentic pipeline that turns a small gold set—expert-annotated calibration examples representing the intended task boundaries—into a reusable structured annotation definition. Goldsmith treats this definition as a trainable textual object. Candidate definitions are run on the same gold examples and scored with an executable structured loss, while the output schema, formatting, retrieval, repair, judging, and human review remain in an external harness. An LLM editor converts the highest-loss failures into textual-gradient revisions, which are accepted only when the measured loss decreases. In prompt-optimization comparisons, Goldsmith improves over direct rewriting, OPRO, APE, and PromptBreeder under matched evaluation protocols. The resulting definition also improves downstream annotation when combined with retrieval, score-based routing, and human review across typed span, pair-level relation, and fixed-trigger event-argument tasks. These results show that scarce expert supervision can support both task-definition learning and scalable annotation.

Abstract:
Aligning language models to qualitative behavioral traits, such as human-likeness, remains difficult because they are hard to define, measure, and optimize. As a result, improvements in human-like behavior are largely driven by scale or broad supervised training, rather than targeted alignment. We introduce Human Aligning LLMs (HAL), a framework for aligning language models to conversational human-likeness using an interpretable, data-driven reward. HAL derives explicit conversational traits from contrastive dialogue data, combines them into a compact scalar score, and uses this score as a transparent reward signal for alignment with standard preference optimization methods. Using this approach, we align models of varying sizes without affecting their overall performance. In large-scale Chatbot Arena-style human evaluations, a model aligned with HAL is more frequently perceived as human-like in conversation. Because HAL operates over explicit, interpretable traits, it enables inspection of alignment behavior and diagnosis of unintended effects. More broadly, HAL demonstrates how soft, qualitative properties of language–previously outside the scope for alignment–can be made measurable and aligned in an interpretable and explainable way.

Abstract:
Despite the remarkable progress of reinforcement learning for large language models, effective exploration remains a persistent challenge, as policies may collapse into recurrent failure modes and repeatedly generate similar erroneous behaviors across rollouts. While classical entropy regularization promotes randomness under the current policy, it provides limited mechanisms for recognizing and suppressing such recurrent failure patterns over time. To address this limitation, we propose MEDS, a Memory-Enhanced Dynamic reward Shaping framework that augments the reward signal with informative cues from historical behaviors. MEDS turns intermediate model representations into a lightweight behavioral memory that captures reasoning patterns from past rollouts, enabling dynamic reward shaping that steers the policy away from recurrent failure modes and toward alternative solution paths. Across five datasets and three base models, MEDS consistently improves average performance over existing baselines, achieving gains of up to 4.13 Pass@1 points and 4.37 Pass@128 points. LLM-based annotations and diversity metrics further confirm that MEDS promotes more diverse sampling behaviors. Our code is available at https://github.com/Linxi000/MEDS.

Abstract:
Reinforcement learning for user-centric agents is limited by the cost, latency, and risk of collecting online feedback, as well as by the lack of counterfactual comparisons under the same user state. In this paper, we propose World Model-Guided Reinforcement Learning via counterfactual user engagement simulation (WMG-RL), a framework in which a frozen user simulator provides reward supervision before real user exposure. Motivated by language world models, we instantiate the simulator as a User Engagement World Model (UEWM), which treats a recommended item as the agent action and the user’s heterogeneous feedback as the environment observation. Rather than learning one fixed environment transition, UEWM learns to infer user-specific dynamics from engagement history and apply them to candidate items. In WMG-RL, a downstream policy proposes multiple candidate items for the same history; UEWM predicts the corresponding engagement feedback in parallel; and the simulated feedback is converted into dense rewards for policy optimization. Experiments show that UEWM provides reliable and transferable reward signals across domains, and that WMG-RL enables a compact 1.7B student policy to match or surpass much larger LLMs on downstream recommendation tasks.

Abstract:
In this paper, we present MOB-KV, a Mixture-of-Basis KV cache compression method that makes adaptive low-rank key compression practical without runtime decomposition. We first analyze KV cache compression through attention output error and reveal a clear asymmetry between keys and values: pre-RoPE keys benefit from preserving dominant low-rank structure while quantizing low-energy residuals, whereas values are better compressed by direct quantization. This analysis motivates an asymmetric design that applies low-rank projection with residual quantization to keys and direct low-bit quantization to values. However, obtaining accurate input-adaptive low-rank bases with online SVD is prohibitively expensive for serving. MOB-KV addresses this by replacing online SVD with a reusable dictionary of offline-constructed key bases, transforming adaptive compression into lightweight basis selection and projection. This design preserves the accuracy benefit of local low-rank subspaces while avoiding the latency bottleneck of runtime decomposition. Across reasoning and long-context benchmarks, MOB-KV substantially reduces KV cache memory, preserves near full-precision accuracy, and establishes a stronger memory–accuracy–latency Pareto frontier than existing quantization-only, low-rank-only, and online SVD-based baselines.

Abstract:
Forecasting future events is a critical challenge for large language model (LLM) agents, spanning domains from elections and monetary policy to financial markets. However, evaluating progress on this task presents a fundamental trade-off between efficiency and environment fidelity. While live evaluation benchmarks suffer from an inherently slow feedback loop, existing retrospective replays typically restrict agents to static, pre-frozen databases that sacrifice the environmental realism of actual deployments. To tackle this issue, we introduce Agentic Time Machine (TM), an infrastructure that approximates the pre-cutoff web state at any chosen past time by filtering post-cutoff content. Leveraging this evaluation infrastructure, we further propose a planner-solver-aggregator multi-agent framework that breaks each question into diverse analytical angles, gathers evidence in parallel, and combines the results into a single forecast. Experiments show that offline scores under TM correlate strongly with live FutureX scores, validating that TM offers a fast and reliable sandbox for forecasting-agent evaluation. On FutureX-Past and Polymarket evaluated under TM, our framework achieves the highest score among strong closed-book, tool-augmented, and self-consistency baselines. On the live FutureX leaderboard, our system achieves the best average rank over four consecutive weeks, ranking 1st in May Week 1.

Abstract:
Mixture of Experts (MoE) has great potential in scaling up the capacity of large models while maintaining low computational costs. However, existing methods mainly reduce redundancy at the expert level while overlooking fine-grained neuron-level redundancy and expert collaboration. In this paper, we propose an Information Bottleneck based MoE (IBMoE) for parameter-efficient fine-tuning, which reduces neuron-level redundancy within each expert and fosters internal collaboration among all experts. Specifically, IBMoE equips each expert with learnable queries to capture task-specific characteristics and employs sparse neuronal activation to dynamically select the most relevant neurons, reducing redundant activations across tasks. In addition, a diversity constraint is imposed among experts to maximize their knowledge diversity, enabling them to cooperate more effectively. Extensive experiments across multiple benchmarks show that IBMoE consistently outperforms strong baselines while activating significantly fewer parameters. In addition, IBMoE reduces inference latency by 61.7% and the memory footprint of adapters by 48.2% compared with previous MoE-based adaptation methods, while achieving comparable training costs.

Abstract:
Large Language Models (LLMs) encode parametric knowledge but can exhibit format bias when resolving knowledge conflicts. When evidence is rendered as text, tables, or knowledge graphs, LLMs can favor certain formats despite unchanged semantic content. Existing work mainly diagnoses this bias, while existing inference-time mitigation remains tied to a single rendered prompt. We propose FaDe, a cross-format knowledge distillation framework that treats evidence format as a controllable nuisance variable and marginalizes teacher predictions over controlled format assignments. FaDe constructs a format-marginalized soft target from these renderings, then trains the student with an asymmetric two-path objective that combines heterogeneous-format debiasing with all-text anchoring. Experiments on ConflictBank and two QA-derived controlled conflict sets with four model backbones show FaDe reduces format bias while improving dual-source coverage and cross-format stability under single-pass inference. Our code is available at https://github.com/Kaia91/FaDe.

Abstract:
Large language models (LLMs) remain vulnerable to jailbreak attacks that exhibit diverse prompts and rapidly evolving strategies. Existing defenses often operate at the level of prompts or predefined attack methods, limiting their ability to generalize to unseen attacks that share similar underlying structure but differ in wording. In this paper, we investigate whether seemingly different jailbreak attacks exhibit shared SAE feature co-activation patterns, and propose a motif-based framework that models attacks through these recurring patterns. Specifically, we first extract sparse autoencoder (SAE) features from jailbreak queries and discover motifs as recurring co-activation patterns via nonnegative matrix factorization. We show that motifs capture stable relationships across attack methods and generalize to unseen attacks. Building on this structure, we synthesize new jailbreak queries by composing motif-guided attack patterns. We then use the generated queries to construct preference-training data for Direct Preference Optimization (DPO). Experiment results show that SAE motifs serve as a stable and transferable interface for understanding and detecting jailbreak attacks, as well as enabling effective defenses that generalize beyond fixed prompt templates. Training target models with motif-guided synthetic data improves robustness to unseen attacks with modest changes in utility. Our work provides a practical methodology for advancing data-centric LLM safety through structured modeling of jailbreak attacks.

Abstract:
Long-horizon personalization requires dialog assistants to retrieve user-specific facts from extended interaction histories. However, many relevant facts have low semantic similarity to the query under dense retrieval. Standard Retrieval-Augmented Generation (RAG) and GraphRAG systems remain largely retrospective: they rely on embedding similarity or fixed graph traversals, often missing relevant facts that lie far from the query in embedding space. Inspired by prospection, the human ability to use imagined futures as cues for recall, we introduce Prospection-Guided Retrieval (PGR), which decouples retrieval from memory storage. Given a user query, PGR expands the goal into a linear chain or tree of thoughts (TOT) denoting plausible next steps, and uses these steps as retrieval probes instead of the original query alone. Retrieved facts are then used to personalize the next round of prospection, enabling PGR to uncover additional memories by grounding the simulation in the user’s history. We also introduce MemoryQuest, a challenging multi-session benchmark where each query is annotated with 3–5 dated reference facts under a low query-reference similarity constraint. Across 1,625 queries spanning 185 user profiles from 3 public datasets, PGR-TOT substantially improves retrieval, achieving nearly 3x recall on MemoryQuest over baselines. In pairwise LLM-as-judge evaluations, PGR-generated responses are preferred on 89–98% of queries, with blinded human annotations showing a similar trend. Overall, the results show that explicit prospection yields large gains in long-horizon retrieval and response quality over similarity-only baselines.

Abstract:
Discharge summary generation from clinical records is challenging due to long, heterogeneous inputs and the need to preserve clinically relevant information. Although Large Language Models (LLMs) have improved abstractive summarization, their direct application to full patient records is limited by context length and difficulties in aggregating evidence across notes. We present CLARUS, a fully unsupervised framework that models patient records as hypergraphs encoding multi-dimensional relations. A hierarchical hypergraph encoder with depth-aware concept attention strengthens sparse representations, while a variational information bottleneck with Gumbel-Softmax relaxation enables differentiable selection of compact, informative subsets for LLM generation. Experiments on MIMIC-III show that CLARUS outperforms strong unsupervised baselines across lexical, semantic, and clinical metrics, demonstrating the value of higher-order structure and concept hierarchy for faithful multi-document clinical summarization.

Abstract:
Effectively representing long documents is a persistent challenge in natural language processing, as foundational encoders are constrained by limited context windows. Prevailing methods like truncation and independent chunk aggregation can discard content or lose cross-chunk context, hindering downstream performance. To address this problem, we introduce Spectral Attention Token Pooling (SATPool), a novel, encoder-agnostic module that generates a single, fixed-size vector from variable-length sequences of locally encoded tokens. SATPool operates in two stages: it first uses efficient linear attention to model global token interactions after chunk encoding, then employs Spectral Token Compression (STC) to compress these globally-aware token representations into a compact, context-aware vector. Across long-document classification, Retrieval-Augmented Generation (RAG), and multimodal RAG, SATPool achieves strong overall improvements and robust top-tier performance across datasets and encoder backbones. The pretrained encoder remains frozen, while the lightweight SATPool module and task-specific heads are trained for each downstream task.

Abstract:
While instruction-tuned Large Language Models exhibit an "AI Assistant" persona, it remains heavily debated whether this behavior is merely one of many simulated roles or the fundamental structural identity of the model. In this work, we present evidence that instruction-tuning instills a global, underlying Assistant State that not only bounds autoregressive generation but is also capable of meta-awareness and reflecting on its own outputs. We demonstrate that this behavior surfaces naturally following the stop token () without any special training or prompting. Furthermore, we show that this state can be deterministically triggered during generation via a steering vector that isolates the model’s post-execution geometry. We introduce this methodology as a zero-shot auditing tool for model evaluators. Applying this framework reveals that models demonstrating Emergent Misalignment in early checkpoints show dual nature where the main generation demonstrates misalignment but post text provides safety warning or rationalizes it as hypothetical.

Abstract:
Can language models be trusted in safety-critical operations? In such settings, strong performance on semantic metrics does not guarantee operational reliability: a misread altitude, a dropped execution condition, or a confused callsign may score well under standard F1 yet carry sharply asymmetric operational consequences. We study this problem in air traffic control (ATC), where controller-pilot communication demands near-zero error tolerance, and use consequence-aware evaluation to test whether semantic scores misstate operational reliability. The framework is instantiated in a controlled diagnostic ATC benchmark grounded in aviation standards and feedback from 40 air traffic controllers across three countries. Evaluating 8 models, we uncover a systematic semantic-safety gap: conventional scores give substantially higher performance estimates than consequence-aware evaluation, even for models that appear reliable under standard metrics. Risk-aware fine-tuning narrows but does not close this gap, showing that consequence-aware evaluation is a necessary complement to standard NLP metrics before any real safety-critical deployment claim.

Abstract:
Modern retrieval increasingly relies on dense and learned-sparse neural models that are effective but require encoding the entire corpus into a specialized index, which must be rebuilt whenever the model changes. Lexical retrievers like BM25 stay efficient, transparent, and run on a standard inverted index that need not change as models evolve, but suffer from vocabulary mismatch. LLM query rewriting can help, yet prompted rewriters emit well-formed but retrieval-ineffective—or harmful—terms, and training against a retrieval reward gives only delayed, sequence-level supervision that obscures which terms helped. We introduce STORM (Stepwise Token Optimization with Reward-guided beaM search), a self-supervised framework with retrieval feedback for lexical query expansion. STORM trains the rewriter through generation guided by retrieval metrics: at each step, a beam of candidate expansions is scored against the BM25 index and low-reward continuations are pruned, turning the retrieval reward into a token-level signal that concentrates exploration on retrieval-effective vocabulary. Across TREC DL and BEIR, STORM lets 0.6B–8B backbones match or surpass competitive LLM rewriters and far larger proprietary models while retrieving as fast as plain BM25, and transfers zero-shot to 18 languages (MIRACL), beating dedicated multilingual dense retrievers. STORM is thus a competitive, infrastructure-light alternative to dense neural retrieval.

Abstract:
Knowledge editing aims to update factual knowledge in language models without full retraining. However, edits should also propagate to downstream queries that require combining the edited fact with other knowledge, such as multi-hop queries. Existing approaches address this propagation problem with more elaborate edit-time procedures, through augmented reasoning examples or external editing modules. In this work, we take a different approach: we meta-learn model parameters from which simple edits are more likely to propagate. We propose LEAP (Learning Edit-Adaptable Parameters), a MAML-based framework that learns edit-adaptable parameters such that simple gradient-based fine-tuning on new single-hop facts improves answers to their multi-hop consequences. Experiments on MQuAKE show that LEAP consistently improves over standard LoRA editing and achieves a favorable accuracy-efficiency trade-off compared with representative knowledge-editing baselines. Further analysis suggests that LEAP makes bridge entities more accessible and strengthens the bridge-to-answer transition needed for multi-hop reasoning. These results suggest a new direction for improving edit propagation: preparing the model itself to be more amenable to propagation-friendly updates, rather than relying only on increasingly complex edit-time procedures.

Abstract:
Emergent misalignment (EM) is the surprising tendency of language models to become broadly misaligned after fine-tuning on narrowly misaligned examples. While EM has been extensively studied in the supervised fine-tuning (SFT) setting, evidence that it also arises from reinforcement learning (RL) is limited to large, closed-source models, leaving the phenomenon expensive to study and difficult to reproduce. We characterize EM from RL in small, off-the-shelf open-weight models along three axes. First, we show that rewarding narrow, overtly misaligned behavior produces substantially higher general-domain misalignment than sample-matched SFT. Second, we show that EM from RL can be induced by reward signals that could plausibly arise naturally, such as unpopular aesthetic preferences or poor rhetorical appeals. Third, we evaluate in-training mitigations developed for SFT-induced EM and find that they broadly transfer, with preventive steering with persona vectors, interleaving safety data and inoculation prompting all performing well.

Abstract:
Instruction hierarchy (IH) requires models to prioritize instructions by source, ensuring that higher-priority instructions override lower-priority ones. Despite its importance for safe and controllable deployment, existing evaluations have focused almost exclusively on English, leaving it unclear whether IH compliance remains stable in multilingual settings. We introduce XIH-Bench, a benchmark for multilingual IH evaluation with both same-language and cross-language conflicts across six languages, four domains, and three IH settings. Across models, we find two consistent patterns. First, IH compliance exhibits a clear language-dependent asymmetry: a language that strengthens compliance in the higher-priority position can become disruptive in the lower-priority position. Second, cross-language conflicts yield higher compliance than same-language conflicts, a phenomenon we term the Language Boundary Effect. We further show that language specialization can make lower-priority instructions in model-favored languages harder to override, creating multilingual reliability and security risks.

Abstract:
Detecting AI-generated text is especially difficult under distribution shift, such as transfer across domains, source models, and editing attacks. We propose an AI-generated text detector based on steering vectors extracted from the hidden representations of a frozen language model. At each layer, we construct a direction that separates human-written from AI-generated text, and represent each input by its layer-wise alignment with these directions. A lightweight classifier trained on these projection features yields the final detection score. Our method achieves strong performance both in-distribution and under distribution shift, including across domains, source models, and machine-editing transformations such as polishing and rewriting. Interpretation analyses show that the learned directions align with recognizable stylistic cues while capturing substantial additional signal beyond surface features. These results position AI-generated text detection as a representation-space probing problem and show that steering vectors provide a simple and effective solution.

Abstract:
Language-specific tokenizers improve tokenization quality and the downstream performance of models on those languages. Using one with an existing pretrained model requires either training a new model from scratch or adapting the vocabulary of the pretrained one. We propose Language-adaptive Maximum a Posteriori (LangMAP) Tokenization, which extends the UnigramLM algorithm to the multilingual setting by replacing its single distribution over the vocabulary with one distribution per language over the same vocabulary. LangMAP can be used to train a multilingual language model from scratch, or to adapt a pretrained model’s tokenizer to individual languages without changing its vocabulary. Language labels are required to estimate the per-language distributions; at inference, the algorithm selects the highest-probability segmentation across languages, i.e., it does not need a language label. In experiments where we adapt a pretrained tokenizer with LangMAP, we observe that LangMAP’s new segmentation has consistently higher morphological boundary recall than the original tokenizer. In code data, it improves alignment with abstract syntax tree (AST) leaf boundaries in all configurations we try. Despite ostensible improvements in segmentations, in language model fine-tuning experiments, we do not observe significant differences in performance for most language-task pairs.

Abstract:
LLMs are increasingly deployed as orchestrators that coordinate specialized subagents to solve complex tasks. In many important domains (game playing, robotics, autonomous driving), the strongest available agents are not language models: their expertise is encoded in latent representations of policy, value, and learned features, not in words. Integrating such agents into LLM reasoning pipelines currently requires verbalization : compressing rich continuous representations into sparse textual summaries at each interaction step. To study whether verbalization constitutes a structural bottleneck, we introduce LLAMIA-Bench , a suite of six collaborative chess tasks spanning three facets: behavioral imitation, state assessment, and natural-language explanation. Tasks are drawn from prominent problems in chess literature and industry, each unsolvable by either component alone, and selected to vary in how much relevant signal survives verbalization. Our experiments reveal a consistent verbalization debt : the performance gap between verbalized and internalized integration widens throughout training and persists as the LLM scales from 4B to 14B parameters. The effect is sharpest on tasks whose target signal has no faithful text representation: on puzzle interest prediction, verbalized systems of all sizes remain near-chance ( ≤ 12/100 ) while internalization reaches 52/100 . To close this gap, we introduce latent state internalization , which projects the subagent’s continuous representations directly into the LLM’s token stream as learned state tokens, with dynamic re-encoding as actions advance the environment state. A single 14B model, LLAMIA , trained under this paradigm matches or exceeds dedicated task specialists and frontier models including GPT-5.1 with tool access across all benchmark tasks, and generalizes out-of-distribution where task-specific finetunes collapse. In a human study with skilled chess players, LLAMIA passes as human in 61% of gameplay trials and its commentary is preferred in 72.2% of pairings.

Abstract:
Rubric-based scoring has become a widely used paradigm in model evaluation, typically with LLM-as-a-Judge (LaaJ) for rubric scoring. However, the reliability of LaaJ for rubric scoring remains underexplored. This concern is especially pronounced in agentic scenarios, where long, complex outputs further challenge reliable scoring. To address this, we conduct a systematic meta-evaluation of LaaJ reliability for rubric verification. We introduce RuVerBench, the first benchmark for assessing LaaJ reliability in rubric verification for agentic scenarios. RuVerBench covers two prevalent agentic domains, deep research and agentic coding, with 2,458 instances, each containing a model-generated output, a rubric, and a human-annotated label indicating whether the output satisfies the rubric. Using RuVerBench, we evaluate numerous frontier LLMs and find that even the most advanced models achieve strong performance but still exhibit substantial noise. We further analyze the impact of key LaaJ strategies, including prompt design, batching, and majority voting, on rubric verification. We find that weaker models are more sensitive to prompt variations, batched verification presents a trade-off between accuracy and efficiency, and majority voting yields effective but diminishing returns. We have released our dataset and code to facilitate future research: https://github.com/THU-KEG/RuVerBench.

Abstract:
Multilingual Large Language Models (LLMs) can process many languages, yet how they internally represent this diversity remains unclear. Do they form shared multilingual representations with language-specific decoding, and if so, why do multilingual performance gaps persist? To address this question, we train models on different multilingual mixtures and analyze their internal mechanisms using Cross-Layer Transcoders (CLTs) and Attribution Graphs. Our results reveal a consistent three-phase structure across models: early layers encode language-specific information, middle layers converge to highly shared multilingual representations, and final layers re-specialize for language-specific decoding. Training models without English produces the same shared multilingual structures, suggesting that these representations emerge independently of English dominance. We further show that decoding relies on a small set of high-frequency late-layer features that linearly encode language identity from earlier layers, and that intervening on these features allows one language to be suppressed and another substituted. Finally, through a Model-Diffing experiment, we show that non-English failures arise from weak late-layer decoding features, weak middle-layer clusters, and tokenizer fragmentation that forces early layers to specialize in word reassembly. Finetuning strengthens these features and their connections, improving token assembly and language-specific decoding, providing a mechanistic explanation for multilingual performance gaps.

Abstract:
Reward models for text-to-video (T2V) generation guide post-training but often fail at fine-grained semantic alignment. We trace this to two structural weaknesses in existing reasoning-based reward models: they do not systematically verify every condition described in the prompt, and the visual evidence supporting each judgment remains implicit in their free-form reasoning. We propose SG-PVR, a video reward model that addresses these limitations through plan-and-verify reasoning grounded in spatio-temporal scene graphs. The verification plan decomposes the prompt into atomic claims, making the set of requirements to be checked explicit. The spatio-temporal scene graph, encoding entities, attributes, and temporally grounded relations, is extracted from the video and maintained as a persistent structured visual reference throughout reasoning. Each claim is verified against both the video and the scene graph, anchoring judgments in explicit visual evidence. SG-PVR achieves strong performance on semantic alignment, including fine-grained temporal semantics. As a test-time reranker, it further enhances compositional alignment in T2V generation.

Abstract:
Modern language models increasingly adopt hybrid architectures that combine full attention with efficient attention modules, such as sliding-window attention (SWA) and recurrent sequence mixers. However, how these efficient modules shape model capabilities remains poorly understood. To address this gap, we conduct a systematic analysis across hybrid architectures from three perspectives: scaling behavior, mechanism analysis, and architecture design. First, from a scaling perspective, we find that efficient-attention design primarily affects how fast long-context capability emerges, while different hybrids eventually converge to comparable long-context performance under sufficient training. Second, mechanistically, we show that long-range retrieval is mainly carried by full attention, whereas efficient attention shapes its optimization trajectory. This explains a counter-intuitive phenomenon we call Large-Window Laziness: larger SWA windows can delay the formation of retrieval heads in full-attention layers. Third, guided by this mechanism, we show that applying No Positional Encoding (NoPE) to only the full-attention layers of a small-window SWA hybrid substantially improves long-context performance with negligible impact on short-context performance.

Abstract:
Bilingual language models (LMs) offer a controlled setting for studying how training conditions shape second-language (L2) behaviour, but prior work typically varies exposure structure, scale, and architecture at once, making it difficult to attribute effects to any single factor. We introduce Beetle, a controlled language model pretraining framework in which tokeniser, target language, training budget, and exposure structure are each independently manipulable, enabling systematic and comparable experimentation of training conditions. Using Beetle, we release 285 bilingual and 45 monolingual open-source LMs with rich checkpoints across a range of exposure schedules, data scales and first languages (L1s) to study multilingual pretraining and computational modelling of bilingualism and second language learning. Evaluating models on human bilingual and second language reading-time prediction and grammaticality judgement tasks, we find that staged and temporally structured curricula consistently improve alignment with language learner reading time compared to balanced bilingual training, with the largest gains at smaller data scales and for typologically closer language pairs. The Beetle models are well suited tools to help move computational psycholinguistics beyond its prevailing monolingual, English-centric focus toward models of human bilingual processing, to study cross-lingual learning dynamics, while supporting community-based development of controlled model families.

Abstract:
Meeting notetakers are increasingly deployed in settings where longitudinal context is available including calendar metadata, prior meetings, emails, slides, and speaker enrollments; however, meeting corpora remain limited in terms of size, domain coverage, and available context. We present OSynth, a large-scale synthetic dataset of simulated organizations comprising 570 hours of meeting audio together with associated artifacts and a latent ground-truth state. In a pairwise comparison, human raters distinguish OSynth meetings from real meeting transcripts with only 56% accuracy based on excerpts of 150–300 words. Across state-of-the-art LLMs, additional context yields significant gains on several entity-sensitive dimensions, demonstrating limitations of contemporary approaches and the potential usefulness of OSynth in developing better-contextualized meeting-understanding systems.

Abstract:
While Group Relative Policy Optimization (GRPO) offers a powerful framework for LLM post-training, its effectiveness in open-ended domains like Machine Translation hinges on accurate intra-group ranking. We identify that standard Pointwise Quality Metrics (PQM) fall short in this context: candidates are evaluated in isolation, so the comparative context is missing for distinguishing fine-grained linguistic nuances. To address this, we introduce the Group Quality Metric (GQM) paradigm and its instantiation, the Group Relative Reward Model (GRRM). Unlike traditional independent scorers, GRRM jointly processes the entire candidate group, leveraging comparative analysis to rigorously resolve relative quality and adaptive granularity. Empirical evaluations confirm that GRRM achieves competitive ranking accuracy among all baselines; integrating GRRM into the GRPO training not only improves general translation quality but also unlocks reasoning capabilities comparable to state-of-the-art reasoning models.

Abstract:
Singular Value Decomposition (SVD) is a promising post-training compression technique for large language models (LLMs), as it is training-free, hardware-friendly, and preserves regular dense computation. However, existing SVD-based methods suffer from severe accuracy degradation under aggressive compression. This is mainly because hard rank truncation discards useful singular directions, while the two resulting SVD factors are usually compressed symmetrically despite their different roles, shapes, and quantization sensitivities. In this paper, we propose DELTA, an asymmetric low-rank compression method that decouples the latent heterogeneity between the two SVD factors. Under a fixed memory budget, the proposed method jointly determines the retained rank, factor-wise quantization bit-widths, and quantization directions, allowing more effective singular directions to be preserved with low-bit representations. To efficiently select compression configurations, we use an activation-aware reconstruction error as a layer-wise proxy objective. Experiments on multiple LLMs and datasets show that DELTA consistently outperforms existing post-training SVD-based compression methods, especially under high compression ratios. Compared with the strongest SVD-based baseline at the same compression ratio, DELTA reduces perplexity on WikiText-2 and C4 by up to 68.2% and 53.2%, respectively, and improves average zero-shot accuracy by up to 11 percentage points. These results demonstrate that asymmetric factor-aware compression can effectively mitigate the accuracy degradation of SVD-based LLM compression at high compression ratios. Code is available at https://github.com/BlacksheepJane/DELTA .

Abstract:
LLMs deployed multilingually are often audited via English explanations for non-English inputs. We evaluate extractive explanations "where the model identifies input token spans as evidence alongside a generated rationale” and uncover a systematic trade-off: English-pivot explanations can achieve higher span agreement with human rationales while their evidence becomes less causally grounded in the model’s prediction, as measured by both comprehensiveness and sufficiency. Across 3 tasks, 5 languages, and 2 multilingual LLM families, we find that English explanations frequently produce fluent but loosely anchored rationales, with comprehensiveness degrading by up to 5.7x relative to native-language conditions - even as task accuracy remains stable across settings. For socially nuanced classification, English pivots also fail to preserve pragmatic cues, reducing both faithfulness and span agreement. We recommend auditing explanations in the input language, reporting multi-faceted faithfulness metrics beyond lexical overlap, and treating English rationales as communication summaries rather than faithful decision traces.

Abstract:
Creative generation tasks, such as narrative writing and scientific ideation, demand both high-quality outputs and distinct responses across independent runs to maximize exploration. Multi-Agent Debate (MAD) has shown strong quality gains on factual and reasoning tasks, making it a natural candidate for creative generation. However, we find its convergence-driven design actively suppresses output diversity across independent runs, creating an inherent trade-off with creative tasks. We theoretically show that preserving diversity among agents within each debate session is a necessary condition for achieving diverse outputs across independent runs. Building on this finding, we propose Creative-MAD, which introduces two synergistic interventions to sustain agent divergence. Specifically, Cognitive Lens Assignment counters identity drift by anchoring each agent to a distinct and persistent cognitive mode, while Embedding-based Peer Selection counters majority pull by limiting each agent’s context to its most semantically distant peers. Experiments across four creative benchmarks demonstrate that Creative-MAD significantly enhances both lexical and semantic diversity while maintaining MAD’s output quality.

Abstract:
Large Language Models (LLMs) demonstrate strong reasoning ability but are constrained by outdated knowledge and hallucinations. Knowledge Graphs (KGs) can help mitigate these limitations by providing structured relational knowledge and capturing global multi-entity, multi-hop dependencies beyond linear text. However, existing KG-enhanced LLM reasoning methods mostly rely on hop-by-hop textual traversal, rather than making multi-hop decisions with a broader perspective as humans do, thus underusing the structural advantages of KGs. This leads to myopic exploration and fragmented reasoning, wasting effort on irrelevant entities while potentially missing critical ones. To address these limitations, we propose Foresight-on-Graph (FoG), a novel framework that introduces visual relation graphs to expand the reasoning horizon at each step. FoG enables Vision-Language Models (VLMs) to simultaneously attend to multi-entity, multi-hop contexts, supporting foresighted planning and more efficient graph exploration. To facilitate effective visual reasoning, we design relation-based fuzzy matching and union-find algorithms to filter out irrelevant relations and coarsen the graph, condensing subgraphs with millions of entities into compact relation graphs with only tens of nodes. Experiments on three real-world datasets demonstrate that FoG consistently outperforms existing methods while significantly improving reasoning efficiency. Code is available at https://github.com/hyworrywart/FoG.

Abstract:
Two dominant tokenisation algorithms are used by modern language models: byte-pair encoding (BPE) and UnigramLM. These differ along two orthogonal axes: their optimisation objective (compression vs. log-likelihood) and their search procedure (bottom-up merging vs. top-down pruning). Existing comparisons confound these axes, making it unclear whether their observed differences stem from what is being optimised vs. how it is being optimised. We disentangle the two by introducing two new tokenisation algorithms that complete this 2×2 design space: BottomUpLL, a bottom-up likelihood-based tokeniser, and TopDownComp, a top-down compression-based tokeniser. We train language models with tokenisers produced by each algorithm, varying: model size, vocabulary sizes, and domain (English-only vs. multilingual). Evaluating models on bits-per-byte, we find that the search procedure, not the objective, is the dominant factor: bottom-up tokenisers consistently achieve lower bits-per-byte in most settings. Evaluating models on the BLiMP task, however, shows no consistent relationship between design choice and performance. Overall, our results disentangle the effect of tokeniser design choices on language modelling performance, offering concrete guidance for their more principled construction.

Abstract:
Modern visual document retrieval (VDR) models remain strongly English-centric, leaving Arabic substantially under-served in this rapidly advancing field. To address this gap, we introduce DEAM, the first open Arabic visual document retrieval dataset explicitly designed for fine-tuning and evaluating multimodal retrievers on Arabic. DEAM is built through a human-in-the-loop construction pipeline with annotators community annotators, combining Arabic document rendering, query creation, validation, and retrieval-oriented hard-negative mining. The dataset comprises18,275 rendered Arabic documents spanning nine source domains and 82,727 contrastive retrieval triplets, each including five mined hard negatives and one easy negative for contrastive fine-tuning. Zero-shot evaluation of 27 retrievers shows that multilingual text retrievers outperform visual retrievers on Arabic by 13.6 nDCG@10, except on layout-rich news pages. DEAM establishes a public foundation for Arabic visual document retrieval.https://huggingface.co/collections/NAMAA-Space/deam-for-arabic-multimodal-retrieval

Abstract:
Multilingual large language models can exhibit unintended code-switching — unnecessarily alternating between languages during generation. We present a comparative study of three methods that identify language-controlling latents in cross-layer transcoders: activation value-based selection (ValSel), activation frequency-based selection (FreqSel), and LLM-generated latent annotation-based selection (AnnSel). To evaluate the efficacy of these methods in identifying language-controlling latents, we introduce two multilingual benchmarks that exhibit code-switching for fine-grained analysis of language steering across seven languages. Through targeted intervention experiments on Gemma-2-2B and Qwen3-4B, we find that all three methods effectively manipulate generation language, with FreqSel achieving the strongest overall performance, while AnnSel offering interpretable latent selection through explicit language annotations. A knock-out analysis suggests the methods select non-overlapping but each-functional latent subsets, indicating redundancy rather than a single canonical language direction. Code and data can be found at https://github.com/rm-3284/Latent-Mechanism-Multilingual.

Abstract:
While Vision-Language Models (VLMs) excel at visual reasoning, generating structured, editable Scalable Vector Graphics (SVG) remains a fundamental challenge. Existing pipelines predominantly yield flat, semantically agnostic collections of paths, where editing a single object requires manually identifying its constituent paths. To address this, we propose a VLM-driven agentic framework for semantic compositional SVG generation. Our pipeline recursively parses visual scenes into semantic and geometric hierarchies via top- down decomposition, visual grounding, and prompt-driven amodal occlusion recovery, ensuring each component is geometrically complete. Furthermore, we introduce the Semantic SVG Benchmark with human-annotated semantic groups and novel sub-component metrics (Semantic Recall/Precision, PERE) to explicitly evaluate structural compositionality and functional editability. Experiments show that our natively predicted structures surpass the upper bounds of existing flat-generation methods in both grouping quality and editability, while maintaining state-of-the-art visual fidelity.

Abstract:
Informative keyframe selection is critical for applying large vision-language models (LVLMs) to long-form video understanding under limited frame budgets. However, existing methods heavily rely on query-frame similarity, which can yield coarse boundaries and miss latent contextual events that are answer-relevant but not explicitly mentioned in the query. To address this issue, we propose AGE-Sampler, a training-free Adaptive-granularity Graph-enhanced Event sampling framework for long-video understanding. AGE-Sampler jointly exploits low-level visual transition cues and query-relevance signals to construct query-adaptive events, thus preserving fine-grained structures in relevant regions while merging low-relevance regions into compact intervals. To recover answer-bearing contextual evidence, we further develop a fine-grained event graph and propagate importance from high-confidence query-relevant anchors via Personalized PageRank, followed by event-level budget allocation and representative keyframe selection. Extensive experiments with different LVLM backbones show that our AGE-Sampler consistently improves long-video understanding performance, achieving average absolute gains of +6.2% on LongVideoBench, +8.8% on MLVU, and +3.8% on Video-MME, while remaining lightweight and plug-and-play.

Abstract:
Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), a tuning-free approach grounded in information theory that dynamically determines the candidate set size using a first-crossing rule: it stops candidate-set expansion at the first index where cumulative probability mass exceeds the normalized entropy of the top-k distribution and retains the preceding prefix. We evaluate EES on reasoning, mathematical, and open-ended generation tasks across multiple model architectures and scales. Our results show that EES achieves robust performance across temperature settings without retuning truncation hyperparameters, delivering competitive accuracy with tuned baselines while maintaining diversity. Analysis reveals that the retained candidate-set size k automatically encodes temperature effects, adapting from near-greedy behavior at low temperatures to broad candidate retention at high temperatures. By eliminating truncation-specific hyperparameter tuning, EES simplifies deployment while providing stable and competitive performance. Our code is publicly available.

Abstract:
Contextual and unigram surprisal are standard predictors of reading time in computational psycholinguistics. While contextual surprisal is typically derived from neural language models, unigram surprisal is usually obtained from independent corpus counts. Although both quantities are deterministic functionals of a single underlying distribution—capturing conditional and marginal structure respectively—this practice breaks their probabilistic link and may introduce estimation error from the mismatch. We propose three principles for estimating unigram surprisal in psycholinguistic research, including the requirement that both quantities be derived from the same language model. We operationalize this principle by introducing a Monte Carlo estimator of unigram surprisal from a language model, along with a marginalized variant that has lower variance empirically. When unigram surprisal is re-estimated under this unified framework, we find evidence that predictors change in their predictive contribution to reading time. We also confirm prior findings that frequency can be a stronger predictor of processing effort than contextual surprisal under this unified framework.

Abstract:
Dense document images often contain many fine-grained visual and textual entities whose relevance depends on a user query. Standard vision-language retrievers encode cropped regions with a single vector, which can mix distinct entity signals and obscure the evidence needed for fine-grained retrieval. We call this failure mode Semantic Dilution and quantitatively show that it degrades entity-level retrieval as a function of entity density. To mitigate it, we propose SAGE, a training-free framework that parses semantic entities from dense document images, represents them as hierarchical graph nodes with multi-vector embeddings, and retrieves query-relevant evidence through iterative entity-level subgraph matching. We also introduce DEAR, a dataset of 1,055 query–image pairs sourced from product detail pages, where each query requires retrieving and comparing multiple fine-grained entities from visually dense inputs across four question types of increasing complexity. Experiments show that SAGE substantially reduces semantic dilution and outperforms patch-level and OCR-based retrieval baselines on DEAR, achieving a Recall@3 of 0.849 and a generation score of 2.746 on multi-entity visual comparison queries. Our code is available at https://github.com/All4Nothing/SAGE.

Abstract:
Text-rich visual inputs require models that can read, retrieve, and compress language directly in pixel space, yet existing pixel-text encoders struggle with fixed resolution pretraining, visual shortcut learning, weak visual grounding, and multilingual visual text understanding. In this work, we investigate the fundamental design principles required for robust visual text representation learning. Through systematic controlled ablations, we identify four critical components: variable image resolutions and rendered font sizes provide spatial proxies for high-resolution document generalization; natural image-text pairs are indispensable for grounding and prevent text-only collapse; layout-aware rendering helps prevent pixel-level shortcuts; and a two-stage multilingual curriculum enables effective cross-lingual alignment. By integrating these principles into a scalable training recipe, we train Pixel Linguist II, a native-resolution vision encoder trained with on-the-fly rendering, unified contrastive grounding, and a multilingual curriculum over 280M training examples. Pixel Linguist II sets new state-of-the-art results on English, cross-lingual, and multilingual Visual STS and ViDoRe, while also enabling better MLLM downstream evaluation. Notably, Pixel Linguist II remains robust under 80% visual token compression, showing great promise for optical context compression. Our code and resources are available at https://github.com/Pixel-Linguist/Pixel-Linguist-II.

Abstract:
Autoregressive training requires that the computation at each position ignore future tokens. Causal masking enforces this for self-attention, but efficient training recipes can still break causality when they compute shared statistics across the sequence dimension. We study two such operations: structured activation sparsity and block-scaled activation quantization. Applying these operators along the sequence dimension is attractive for hardware efficiency and can look harmless, but it can break causality when the mask or scale applied to a past token depends on future activations. We show that models do exploit this dependency: the training objective continues to improve under the non-causal operator, but causal validation degrades when the future-dependent computation is removed. We analyze why mitigations that keep the operator in the forward pass can postpone or weaken exploitation, but cannot remove the future-to-past dependency. The solution is to remove non-causal statistics from the forward computation used for prediction, and we show that existing sparsity and quantization recipes do respect this rule, though causality was not an explicit design decision.

Abstract:
Clinical diagnostic reasoning is a sequential process: a working hypothesis is formed early in an encounter and updated as new evidence arrives, and biases at one stage can compound across the stages that follow. In human clinicians, anchoring and premature closure account for a large share of preventable diagnostic errors. As large language models are increasingly considered for clinical decision support, an analogous question arises: how do they handle wrong context that enters mid-encounter - whether from their own prior outputs, an earlier clinician’s note, or pushback from a user? Existing evaluations of LLM cognitive bias use single-shot question answering and cannot speak to this dynamic. We introduce a temporal-replay benchmark over MIMIC-IV in which 686 patient timelines are replayed step by step to three open-weight LLMs, with each step’s structured assessment carried forward as the prior for the next. At three clinically meaningful anchor points we inject the same wrong diagnosis through four structurally distinct channels. The most damaging is corruption of the model’s own previous assessment, with effects roughly twice those of any other channel. When this manipulation derails the correct answer, the model lands specifically on the planted diagnosis about half of the time.

Abstract:
Text-to-SQL systems typically assume a one-to-one mapping between natural-language queries and correct SQL translations. In practice, however, many queries are inherently ambiguous, admitting multiple valid interpretations that yield distinct outputs. We introduce Adversarial Intent Resolution (AIR), a multi-agent framework that actively probes and resolves such ambiguity without any task-specific training. AIR first constructs competing interpretations, verifies through SQL execution that they produce genuinely different results, and generates targeted clarification questions grounded in the observed data differences. Evaluating on leading benchmarks for ambiguous questions, AIR demonstrates strong performance, efficiency, and robust cross-benchmark transferability. On the AMBROSIA benchmark under schema-only context—a realistic production setting where exposing the full database is infeasible—AIR outperforms IntentRL, the state-of-the-art reinforcement learning model, across all metrics (e.g., F1 0.679 vs. 0.618). Crucially, AIR exhibits only ∼ 7% performance degradation without the full database, compared to ∼ 20% for IntentRL. On the AmbiQT benchmark, where ambiguity arises from schema design rather than linguistic structure, AIR achieves higher F1 under both context settings without any benchmark-specific adaptation.

Abstract:
Model diffing aims to identify how fine-tuning changes a model’s internal representations. Crosscoders are attractive because they learn sparse, indexable latent dictionaries, but existing formulations struggle under narrow fine-tuning, where behaviorally important changes are sparse, low-magnitude, and dominated by shared structure. We introduce Delta-Crosscoder , which combines Dual- K BatchTopK sparsity, a delta-based loss that reconstructs activation differences through non-shared latents, and contrastive pairing of activations. Across 10 model organisms spanning synthetic false facts, emergent misalignment, subliminal learning, and taboo word guessing, Delta-Crosscoder recovers causally validated latents for all organisms, compared to 4–6 of 10 for prior crosscoder variants. The learned dictionary is also more structured: in emergent misalignment, it separates misalignment-inducing and refusal-gating directions that pooled activation-difference methods cannot disentangle. Delta-Crosscoder further maintains reconstruction quality and improves dictionary utilization in a broader base/instruction-tuned setting.

Abstract:
While modern language models increasingly rely on ever-larger web corpora, we show that pretraining on historical text (e.g., pre-1913 text) in a data-constrained setting can produce a temporally grounded language model that still shows reasonable performance on language understanding. However, developing History LMs requires addressing challenges in data quality, preventing temporal leakage in post-training, and constructing temporally aligned evaluations. We address these challenges and pretrain TypewriterLM, a 7.24B-parameter model with a 1913 knowledge cutoff. We construct TypewriterCorpus, a 54B-token historical corpus with extensive temporal filtering, propose lexically grounded instruction tuning that constrains all responses to vocabulary from historical source documents, and introduce History-Event, a benchmark of 2,344 events for evaluating both competence and cutoff adherence. We release TypewriterLM and all associated resources to support future research on History LMs.

Abstract:
Sparse Autoencoders (SAEs) decompose language-model activations into sparse, interpretable features, but standard encoders usually treat the latent dictionary as a flat set of independent coordinates, leaving hierarchy and feature interactions to emerge only implicitly. We propose KronSAE, a design that factorizes the latent space into heads and forms post-latent features as pairwise compositions of lower-dimensional pre-latents using mAND, a differentiable AND-like interaction. This imposes a compositional co-activation prior while remaining compatible with standard SAE objectives and variants such as TopK, Matryoshka, and Switch SAEs. KronSAE matches strong baselines on the EV–FLOPs frontier, improves interpretability of the latents, better captures the underlying correlated feature structure, and reduces encoder computational cost as an additional benefit. Code is available at https://github.com/corl-team/kronsae.

Abstract:
As LLMs are deployed as agents, reliable monitoring requires knowing not only what they output, but which instructions are steering their behavior. This is difficult when models infer unintended subgoals, follow contextual cues, or are influenced by prompt injections and hidden objectives. While activation-to-language methods suggest that hidden states can reveal natural-language information, existing approaches are not designed to recover the full set of simultaneous instructions, constraints, prohibitions, and subgoals present in agentic settings. As a first step toward this monitoring goal, we formalize instruction set retrieval and introduce PRISM, an activation-conditioned interpreter that decodes hidden states from a frozen target model into a faithful bullet list of instructions. Unlike prior activation-to-language methods, PRISM is trained to recover instruction sets directly, using judge-guided GRPO to reward covered instructions and penalize unsupported ones. Across benign, constrained, prompt-injection, and hidden-objective settings, PRISM outperforms activation-to-language baselines, especially on security-relevant objectives.

Abstract:
Vast-horizon, repetitive workflows are common in daily routines, e.g., processing expense reports from a stack of receipts, organising a collection of PDF annotations into structured notes, and are tedious for humans, with execution length scaling with the volume of data to process. They are, however, well-suited to autonomous agents, since their structured, recurring sub-workflows follow a logic that can be systematically learned. Yet no existing benchmark evaluates agents specifically on vast-horizon repetitive capabilities, leaving the problem largely underexplored. To bridge this gap, we establish OS-Marathon , comprising 100 vast-horizon repetitive tasks across 5 scenarios and 10 domains, on which our evaluation reveals that leading state-of-the-art (SOTA) CUAs struggle substantially. We further find that introducing a task orchestrator to decompose the workflow into per-instance subtasks fails to mitigate the difficulty: errors accumulate and propagate across solver agents, indicating that naive decomposition is insufficient for these tasks. We then explore a cost-effective personalisation strategy, i.e. GraphDemo, that adapts general agents to such tasks from a single human demonstration of the recurring sub-workflow logic. Extensive experiments show both the real-world challenge of vast-horizon repetitive tasks and the improvement provided by human demonstration in this setting.

Abstract:
Points-of-Interest (POI) search, the task of retrieving and ranking geographically situated entities in response to a query issued from a specific location, underpins major mapping and navigation services. We present POI Search Suite (POISS), the first large-scale dataset designed for this task, pairing over 300K geo-anchored natural language queries across 7 languages and 14 locales with more than 4M graded relevance labels across over 90M candidate POIs drawn from the 72M-entity Overture Maps Places index, jointly capturing semantic match, geographic proximity, and entity quality. Our experiments assess the utility of the proposed dataset for advancing research on POI search.

Abstract:
Entity tracking requires maintaining and updating latent states for entities and attributes over long sequences. Recent task-specific attention operators can compress deep Transformer stacks into a few layers by performing multi-hop state propagation within a single layer, but their dense evaluation remains expensive. We show that in this setting, learned attention is strongly structured: most mass concentrates in local block-diagonal neighborhoods with a light cross-block residue. Exploiting this, we derive a blockwise evaluation of a resolvent-style operator that keeps within-block interactions exact and routes cross-block interactions through a reduced system. The resulting evaluation is subquadratic in sequence length O(n 4/3 d) (and O(n 7/3 ) when d≈ n ). On controlled tracking benchmarks, our method matches the dense operator’s accuracy while reducing wall-clock time by 12-29% under a standardized measurement protocol, and is up to 2.4 × faster than a compact dense Transformer at comparable exact-match accuracy. We further provide ablations over block size and model capacity, and identify a limitation: performance collapses when the number of simultaneously evolving properties exceeds the number of attention heads.

Abstract:
Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to what extent textual and visual modalities are aligned. To investigate this question, we use reasoning-guided image generation as a diagnostic task, where models produce textual reasoning first and then generate images. We introduce UReason, a benchmark for evaluating reasoning-to-generation alignment in this paradigm, consisting of 2,000 human-curated and human-verified instances spanning five reasoning-intensive tasks: Code, Arithmetic, Spatial, Attribute, and Text. To enable controlled analysis, we develop an evaluation framework that compares direct generation, reasoning-guided generation, and decontextualized generation, which conditions only on the refined prompt extracted from reasoning. Across eight widely used open-source UMMs, while we find that reasoning-guided generation yields improvements over direct generation, somewhat surprisingly, decontextualized generation consistently outperforms reasoning-guided generation by a large margin. Our further analyses suggest that the intended visual semantics in textual reasoning are not reliably reflected in the generated images, despite their unified design and training. Overall, UReason serves as a practical litmus test for reasoning-to-generation alignment and provides a challenging benchmark for developing next-generation, more tightly aligned UMMs.

Abstract:
Article-level news stance detection aims to identify the perspective of news articles toward social issues. Despite advances in stance detection and its importance for trustworthy media environments, news articles pose distinct challenges because their stances are often implicit, subtly conveyed through journalistic framing, and embedded in long, structurally complex texts. To address these challenges, we introduce VFStance, which leverages visual framing to make implicit stance cues more explicit via image generation. In evaluation experiments, we demonstrate the effectiveness of VFStance over existing methods and the contribution of visual framing to its performance. Finally, a controlled user study (N=200) in a snippet-based news consumption setting further demonstrates that VFStance can make stance signals visually salient and highlights its potential use beyond automated stance detection.

Abstract:
Out-of-distribution (OOD) generalisation is essential for authorship attribution (AA), especially when involving new authors, yet this aspect remains insufficiently supported by existing AA datasets. In this paper, we introduce GHOSTWRITEBENCH, a dataset for LLM AA task, designed to test generalisation across multiple OOD dimensions, including unseen domain and authors. GHOSTWRITEBENCH comprises long-form texts (50K+ words per book) generated by ten frontier LLMs. We find that existing methods degrade substantially in unseen author scenarios — dropping by as much as 70% even under high-resource settings. We propose TRACE, a novel lightweight and interpretable fingerprinting method that captures token-level transition patterns (i.e., word rank and entropy) using a lightweight language model. Experiments show robustness of TRACE in OOD settings and establish it as a strong baseline on GHOSTWRITEBENCH.

Abstract:
Large language models are pretrained on massive, largely undisclosed corpora that may contain copyrighted, privacy-sensitive, or otherwise unauthorized content, making pre-training data detection essential for data-rights auditing, privacy protection, and benchmark contamination analysis. However, most existing methods follow a gray-box setting, relying on internal signals such as token probabilities, logits, or losses, which are unavailable for commercial LLMs that expose only the final generated text. Recent black-box methods instead use word-recovery rate as the detection signal, but overlook that a word’s discriminative value can vary substantially across target models and target texts. To address this limitation, we propose PIRBD-PDD, a target LLM- and text-adaptive framework for black-box pre-training data detection via Prompt-Induced Recovery Behavior Decomposition. PIRBD-PDD decomposes the target LLM’s recovery behavior into memorization-oriented and context-oriented modes through prompt induction, contrasts their recovery outcomes to identify high-discriminability words, and computes the final detection score from their recovery rate under the original, non-induced prompt. Experiments across diverse LLMs and data domains show that PIRBD-PDD consistently outperforms state-of-the-art black-box baselines. Further analyses provide empirical evidence for the proposed decomposition and show that high-discriminability words provide more robust and informative signals for pre-training data detection.

Abstract:
Fine-tuning a pretrained LLM into a vision-language model (VLM) can erode the backbone’s text capability, with the damage concentrated on tasks that require following exact output rules, such as instruction following, chain-of-thought reasoning graded on a strictly parsed final answer, and similar evaluations with strict graders. We trace this gap to attention-sink corruption: VL fine-tuning perturbs the early sink position that anchors a large fraction of attention probability, and how well the base LLM preserves its sink tracks how much of the affected capability survives adaptation. Building on this view, we introduce Sink Strength, a single scalar computed on the base LLM in a few seconds on a single GPU that predicts post-VL degradation without any VL training. It consistently tracks relative degradation across the six VLM–LLM pairs and multiple format-sensitive tasks. Complementing this diagnostic, we find that post-pretraining QK-norm injection fails to reproduce the protection of native QK-norm, while several off-the-shelf weight-merging settings fail to recover the lost capability after VL training. These negative results underscore the value of screening backbones with Sink Strength before VL training and narrow the intervention space toward head-selective training-time protection.

Abstract:
Current large reasoning models (LRMs) prefer to generate verbose chain-of-thought (CoT) traces (a.k.a. the “overthinking” issue), especially in multilingual scenarios due to the imbalanced proficiency across different languages. However, existing studies on CoT compression focus exclusively on English and overlook language characteristics, resulting in performance degradation in multilingual scenarios. To address this issue, we develop a multilingual CoT compression method based on reinforcement learning, named L anguage- a ware E ntropy-based ( LAE ), which optimizes LLMs with multilingual group sampling and entropy gap-based rewards. Specifically, we extend GRPO with multilingual group sampling, where each query is jointly optimized across multiple languages. We further design a language-aware entropy-gap reward that penalizes redundant uncertainty in the thinking phase and aligns reasoning complexity across languages, enabling concise yet accurate multilingual CoT reasoning. Extensive experiments on the multilingual reasoning benchmarks MGSM, MMATH and MMMLU demonstrate that our method achieves competitive performance and generalization, compared with several strong baselines.

Abstract:
LLMs often answer semantically equivalent queries inconsistently, even when the underlying facts are known. We formalize consistency as semantic invariance over equivalence classes of queries and show that invariance under arbitrary meaning-preserving transformations is undecidable in general. This motivates scoped semantic invariance, where model behavior is evaluated relative to bounded semantic domains. We operationalize this framework using knowledge graphs (KGs), which act as explicit semantic control structures defining local semantics and grounding contexts. Across the MITRE D3FEND ontology and eight Wikidata-derived domains, we show that inconsistencies are structured rather than random, concentrating in localized hotspot subgraphs organized around recurring TREE, STAR, and CYCLE patterns that recur across models and domains. We further show that selective graph grounding improves consistency with the KG specification and cross-model agreement, while indiscriminate grounding introduces regressions. A layered grounding strategy achieves the strongest monotonicity profile, preserving KG-consistent knowledge while reducing inconsistency. Finally, we show that hotspot structures discovered from simple consistency probes transfer to more realistic compositional reasoning tasks, positioning them as a practical reliability layer for predicting and targeting failures rather than a canonical decomposition.

Abstract:
While large language models (LLMs) have shown strong capabilities in tabular reasoning, retrieving relevant tables remains challenging due to the fragmented and relational structure of real-world data. Existing work typically relies on whole table representations that overlook cross-table semantics induced by join relationships. We propose PEARL, a training-free framework that shifts the paradigm toward vertical partitioning-based sub-table encoding. PEARL augments the retrieval corpus offline by generating multi-hop queries over pre-identified join paths and reorganizing relevant columns into vertically partitioned corpus units, enabling effective multi-table retrieval without query-time LLM inference. Experiments show that PEARL consistently outperforms existing methods, with up to +30.05% gains in R@2 on 3-hop queries. The source code is available at https://github.com/SOOB2NHO/PEARL

Abstract:
Multi-domain machine translation (MDMT) requires more than fluent generation: it demands domain-sensitive translation decisions such as domain disambiguation, terminology control, and stylistic adaptation. Large reasoning models (LRMs) make such decisions explicit through intermediate translation steps, but our analysis across 15 domains and four translation directions shows that this explicit reasoning is double-edged: it improves long-form and high-difficulty translation, yet often drifts in terminology-intensive and stylistically constrained settings. We trace this failure to a credit-assignment bottleneck: existing methods optimize final outputs or coarse trajectories, but cannot identify which translation steps actually help the final translation. To address this, we propose PAMT, a process-aligned training framework that combines cold-start domain-aware Long-CoT supervision with reinforcement learning. PAMT uses sequence-level format and outcome rewards for the final translation, together with a step-level process reward that measures how much each explicit translation step increases the likelihood of the reference translation. Across two backbones, PAMT improves over base models, outperforms MT-specialized baselines on average, and remains competitive with strong LLMs/LRMs across in-domain, OOD, and multilingual settings.

Abstract:
On-policy Direct Preference Optimization is commonly evaluated using held-out pairwise accuracy and LLM-judge win rate, but these metrics can reward different behaviors. In controlled Qwen3-4B ablations, interventions that improve a log-prob preference probe do not necessarily improve judged generations, and interventions that win generation judging need not improve the probe. The clearest gains in judge win rate come from direction correction, which strongly improves generation evaluation while leaving pairwise accuracy nearly unchanged. Output-level analysis shows that this advantage is driven primarily by response organization: the judged-winning model wins formatting on 40/50 prompts but wins instruction-following on only 11/50. Stripping markdown does not remove the advantage, suggesting that the effect is not merely markdown decoration but broader response organization. These results suggest that in this setting, on-policy DPO can optimize judge-preferred response organization without corresponding gains on a held-out preference probe.

Abstract:
Recent advances in mechanistic interpretability have revealed how language models process information, yet these insights rarely improve model performance. We propose Interpretable Feature-Space Regularization (IFSR), a training-time framework that transforms mechanistic insights into optimization signals. IFSR identifies error-prone feature interactions through circuit attribution and penalizes them during training, encouraging the model to discover alternative computational pathways. Unlike inference-time interventions that operate on individual features, IFSR targets feature-to-feature edges representing internal computational patterns, and permanently encodes improvements into model parameters. Experiments across ten classification tasks show consistent improvements. Cross-task evaluation demonstrates that IFSR training can transfer positively to unrelated tasks, suggesting benefits to general model capabilities. Our analysis reveals that most identified error patterns resist human interpretation, yet penalizing them still improves performance, suggesting that automatic error identification at the level of feature interactions is feasible and effective. This work demonstrates that mechanistic interpretability can directly enhance task performance through training-time optimization.

Abstract:
LLMs have shown remarkable proficiency in general language understanding and reasoning. However, they consistently underperform in spatial reasoning that severely limits their application, particularly in embodied intelligence. Inspired by the success of hierarchical reinforcement learning, this paper introduces a novel method for hierarchical task decomposition in LLM spatial reasoning. Our approach guides LLMs to decompose complex tasks into manageable sub-tasks by identifying key intermediate states and generating simplified sub-environments. However, we identify that LLMs often fail to derive optimal intermediate states due to their insufficient spatial prior, leading to sub-optimal task decomposition. To address this limitation and enhance its planning capability, we propose a MCTS-Guided Group Relative Policy Optimization (M-GRPO). where we reformulate the UCT formula by incorporating the LLM’s prior predictive probabilities alongside its epistemic uncertainty. Furthermore, we implement a more fine-grained advantage function, enabling the model to learn optimal path planning. Experimental results demonstrate that our method substantially improves LLM performance on spatial tasks, including navigation, planning, and strategic games, achieving state-of-the-art results. This work paves the way for LLMs in real-world applications.

Abstract:
Parameter-efficient fine-tuning (PEFT) typically relies on additive weight perturbation. However, in dynamic deployment environments like multi-tenant cloud serving or mobile edge devices, additive methods force a dilemma: merge weights and lose task-switching flexibility, or remain unmerged and suffer severe parallel-branching latency. To break this bottleneck, we propose Latent Task Embedding (LatTE), which shifts the PEFT design axis to geometric expansion. Instead of perturbing frozen parameters, LatTE concatenates a task-specific auxiliary embedding to the input and expands the corresponding weight matrices. This structural fusion ensures the forward pass remains a single, optimized matrix multiplication, bypassing the routing overhead of unmerged adapters. Furthermore, LatTE natively supports router-free, intra-batch multi-tasking by isolating tasks into distinct subspaces controlled via binary masking. Extensive evaluations across 1B to 14B parameter models demonstrate that LatTE matches highly optimized baselines like LoRA, while raising throughput by 1.7× over routed multi-adapter LoRA under heterogeneous-task batching.

Abstract:
Confidence estimation (CE), i.e. quantifying the reliability of a model’s prediction, has attracted great interest in the context of large language models (LLMs). However, most studies focus on English, ignoring the multilingual reality of LLM usage, while many CE methods degrade or require retraining across languages. To address this gap, we investigate whether multilingual LLMs encode shared, language-transferable confidence features in open-ended question answering. We use a lightweight linear probe that predicts answer correctness directly from intermediate representations. Trained monolingually, the probe generalizes zero-shot to unseen, linguistically diverse languages without target-language supervision. Multiple ablations and learned layer weights reveal that confidence features concentrate in middle layers across languages, suggesting a shared confidence subspace. While zero-shot cross-lingual performance depends on similarity to the source language, the probe provides a strong baseline without any retraining and compares favorably to other popular confidence estimation methods.

Abstract:
Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor probability mass into the target model’s vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the mixing strength can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on structured, front-loaded adversarial benchmarks, with a 4.6-6.6 x inference slowdown inside the band of existing inference-time defenses. The results show that early-token refusal behavior can be transferred between model families at inference time, without touching either model’s weights.

Abstract:
Modern information retrieval (IR) systems rarely represent time, yet many information needs depend on it: in clinical, journalistic, and legal search, when an event occurred can decide whether a document is relevant. Dense retrievers and Retrieval-Augmented Generation (RAG) pipelines match queries to documents well on topic but poorly on time, so they surface content that is on-topic yet temporally wrong. We introduce Temporal Textual Similarity (TTS), a task that measures how well two anchored texts align in time, independent of their topical similarity. We then present TEMPS (Temporal Embedding Model for Precise Search), a modular temporal branch that attaches to a frozen semantic retriever and trains on that signal. It resolves anchored temporal expressions to intervals and moment-matches each one to a Gaussian; the resulting ordering supervises an anchor-date-conditioned encoder, whose score we fuse with the semantic score at inference. Grounding supplies the supervision, so training uses no hand-labeled temporal data. The temporal score itself is the Gaussian-KL inclusion measure from distributional embeddings; what TEMPS adds is the grounding and the moment-matched supervision. On three temporal benchmarks, TEMPS improves MRR for every semantic backbone tested and, on TS-Retriever, lifts R@1 from 19.92 to 25.39 over the prior temporal state of the art.

Abstract:
Authorship attribution is the process of identifying the writer of a disputed text by analyzing distinctive stylistic patterns. By comparing the disputed text against a baseline of known authorship, the task amounts to assessing how strongly the two texts are likely to share an author. However, topical confounding makes the task deceptively easy: two authors covering the same topic may look more alike than one author covering two topics. Scholarly prose offers a natural solution. Academic writers produce multiple papers on related but distinct topics while maintaining consistent stylistic habits. Thus, we introduce HALvest-Contrastive, an English benchmark that draws same-author passages from distinct papers within a disciplinary field to reduce topical overlap, while using different-author passages from the same field to discourage naive keyword matching. The benchmark’s design removes topical shortcuts: a strong lexical baseline fails under these constraints. As a by-product of our benchmark construction, we release the 17B-token multilingual corpus used to produce it. We then revisit how to score authorship: while standard systems compress each document into a single vector, we retain the sequence of token-level representations produced by a language model and compare them with late interaction. We also study patch-level late interaction, which groups neighboring tokens into patches before interaction. Matching at the token level increases Recall@20 by a factor of three to five. The optimal interaction granularity, however, depends on sequence length, with no single granularity performing best across all settings.

Abstract:
Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic study and probe for 78 molecular substructures across eight pre-trained and six randomly initialized models. We furthermore study how fine-tuning on chemical downstream tasks affects the learned representations of molecular substructures. Our results show that pre-training generally improves molecular structure awareness of CLMs, particularly in the upper layers. Moreover, randomly initialized models already encode ring structures well in the first layer. Our analysis on two chemical downstream tasks further reveals that, interestingly, fine-tuning affects task-relevant molecular substructures more than others, indicating that the changes in the representations follow chemical theory.

Abstract:
Backdoor attacks on language models are widely studied, yet the internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood. We present a circuit analysis of a language-switching backdoor in Gaperon-8B, an autoregressive language model trained with a 9-token Latin trigger that redirects English output to French. We identify a three-phase circuit: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model’s natural language-identity direction; (3) the MLP at the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigate the trigger but also hinder the model’s capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals in intermediate representations would miss this trigger entirely.

Abstract:
Text-labeled graph databases combine symbolic graph structures with rich textual descriptions, enabling expressive representations for real-world data. However, answering queries over such databases remains challenging when queries require both precise logical constraints and flexible semantic text interpretation, especially under information incompleteness. We introduce the task of hybrid query answering (hybrid QA) over incomplete text-labeled graph databases, which requires joint reasoning over both symbolic and textual knowledge under incompleteness. To investigate this problem, we formalize a query language that integrates logical constraints with free-form textual descriptions and construct two new benchmark datasets from biological and e-commerce domains. We propose HybridQE, a hybrid query embedding framework that aligns with our query language via embedding and combining the symbolic and textual constraints in a unified embedding space. Experiments on the proposed benchmarks show that existing methods struggle with this setting, while HybridQE provides consistent improvements across diverse hybrid query types, highlighting both the difficulty of HybridQA and the promise of hybrid query embeddings for reasoning under incompleteness.

Abstract:
News consumption on social media has become ubiquitous, yet how different forms of engagement shape psychosocial outcomes remains unclear. To address this gap, we leveraged a large-scale dataset of ~26M posts and ~45M comments on the Bluesky platform and conducted a quasi-experimental study, matching 81,345 Treated users exposed to News feeds with 83,711 Control users using stratified propensity score analysis. We examined psychosocial wellbeing in terms of affective , behavioral , and cognitive outcomes. Our findings reveal that news engagement produces systematic trade-offs: increased depression, stress, and anxiety, yet decreased loneliness and increased social interaction on the platform. Regression models reveal that News feed bookmarking is associated with greater psychosocial deterioration compared to commenting or quoting, with magnitude differences exceeding tenfold. These per-engagement effects accumulate with repeated exposure, showing significant psychosocial impacts. Our work extends theories of news effects beyond crisis-centric frameworks by demonstrating that routine consumption creates distinct psychological dynamics depending on engagement type, and bears implications for tools and interventions for mitigating the psychosocial costs of news consumption on social media.

Abstract:
Large audio-language models (LALMs) show strong zero-shot performance on tasks such as audio question answering, but still lag behind specialized systems on discriminative tasks such as audio classification. Recent work shows that sparse subsets of attention heads can serve as discriminative feature extractors for few-shot classification, but existing methods weight all selected heads equally. We propose C lass-Conditional Sparse A ttention Vectors for Large Audio- L anguage M odels, a few-shot classifier that learns class-conditional importance weights over attention heads. By treating heads as class-specific experts and aggregating them with reliability-weighted soft voting, CALM allows different classes to rely on different subsets of heads. Across diverse few-shot audio, audiovisual, visual, and spoofing benchmarks, CALM consistently outperforms state-of-the-art uniform-voting baselines, with gains of up to 14.52, 1.53, and 8.35 absolute points on audio, audiovisual, and spoofing tasks, respectively.

Abstract:
Multilingual machine translation models must adapt to the grammatical structure of the target language, yet how structural information shapes decoder computation remains unclear. Using the eigenvalue spectra of per-(layer, head) attention Gram matrices instead of raw attention weights, we show that cross-attention geometry in NLLB-200 is organized around the target language: these spectral features carry a strong, distributed signal about target-language word order (SVO vs. SOV) when English is the source, with near-perfect decodability across cross-validation. The signal is not concentrated in a small subset of heads, but is broadly distributed across layers and heads—yet it degrades sharply at every level of aggregation when English is the target. This asymmetry suggests that cross-attention geometry reflects the demands of target-side generation rather than source-side structure. Leave-one-language-out evaluation confirms that the signal generalizes to unseen languages, ruling out language identity as a confound.

Abstract:
Large language models (LLMs) have demonstrated strong performance across a wide range of reasoning tasks, yet their understanding of joint and conditional probability distributions remains largely unexplored. In this work, we present a comprehensive evaluation of LLMs’ ability to reason about discrete joint and conditional probability distributions involving multiple random variables. Given observations drawn from a distribution, we evaluate models on three carefully designed tasks—mode identification, maximum likelihood estimation, and sample generation—which together capture fundamental skills required for probabilistic reasoning. Our study reveals both capabilities and limitations of current LLMs. In sample generation task, some models produce empirical distributions that closely match the target distribution despite exhibiting limited sample independence. We further observe that model performance is highly sensitive to representation: replacing explicit frequency information with raw samples leads to large accuracy drops, and even minor changes to outcome labels can further degrade performance. Together, these findings provide new insights into how LLMs reason about discrete joint and conditional probability distributions and highlight their strengths and limitations in certain probabilistic reasoning settings.

Abstract:
Reviewer comments naturally relate to specific parts of the reviewed paper, yet grounding these comments to the underlying evidence is difficult due to long multimodal documents. Existing benchmarks do not capture this setting and largely focus on explicit, information-seeking queries. We introduce ReGround, a large-scale dataset for reviewer comment grounding that links 10,267 reviewer comments to 16,274 evidence in the original anonymous submission of 3,656 papers. We build on a simple observation: author rebuttals often include explicit references to content of the submission used to address reviewer comments, providing a high-precision annotation source. We cast grounding as a retrieval task and evaluate a wide range of retrieval methods. Results show that retrieval over the entire paper content performs poorly, evidence-type inference is a major bottleneck, and multimodal evidence provides complementary signals that text alone misses. Our dataset exposes grounding reviewer comments as a difficult and practically important problem for scientific document understanding.

Abstract:
Vision-Language Models often struggle with complex visual reasoning due to the visual information loss in textual CoT. Existing methods either add the cost of tool calls or rely on localized patch-based embeddings that are insufficient to extract semantics in multi-step reasoning. We propose "Decompose, Look, and Reason" (DLR), a reinforced latent reasoning framework that dynamically decomposes queries into textual premises, extracts premise-conditioned continuous visual latents, and deduces answers through grounded rationales. We introduce a three-stage training pipeline and propose a novel Spherical Gaussian Latent Policy, to enable effective exploration in the latent space. Extensive experiments on vision-centric benchmarks show that DLR consistently outperforms strong baselines, including text-only, interleaved multimodal CoT, and latent reasoning methods, while providing superior stepwise interpretability.

Abstract:
Large language models (LLMs) are increasingly consulted as sources of historical information, yet their handling of contested events and politically charged narratives remains understudied. We introduce HistoricalMisinfo, a dataset of 500 contested events from 45 countries, each paired with a factual and a documented revisionist reference narrative, instantiated across 11 prompt scenarios reflecting common real-world usage. Using an LLM-as-a-judge protocol, we evaluate multiple model architectures under neutral prompts and robustness prompts that explicitly request revisionist content. Under neutral prompts, models generally align with factual references, though alignment varies by scenario and model. Robustness prompting reveals a consistent and sharp increase in revisionism scores across all models, indicating limited resistance to user-driven revisionist framing. HistoricalMisinfo offers a practical benchmark for evaluating robustness to historical revisionism in AI systems.

Abstract:
Despite extensive alignment efforts, Large Vision-Language Models (LVLMs) remain vulnerable to jailbreak attacks. To mitigate these risks, existing detection methods are essential, yet they face two major challenges: generalization and accuracy. While learning-based methods trained on specific attacks fail to generalize to unseen attacks, learning-free methods based on hand-crafted heuristics suffer from limited accuracy and reduced efficiency. To address these limitations, we propose Learning to Detect (LoD), a learnable framework that eliminates the need for any attack data or hand-crafted heuristics. LoD operates by first extracting layer-wise safety representations directly from the model’s internal activations using Multi-modal Safety Concept Activation Vectors classifiers, and then converting the high-dimensional representations into a one-dimensional anomaly score for detection via a Safety Pattern Auto-Encoder. Extensive experiments demonstrate that LoD consistently achieves state-of-the-art detection performance (AUROC) across diverse unseen jailbreak attacks on multiple LVLMs, while also significantly improving efficiency. Code is available at https://github.com/ShuangLiangX/Learning-to-Detect.

Abstract:
Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering a single reasoning-related latent feature can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting.

Abstract:
Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.

Abstract:
Unified Multi-Modal Generation (UMMG) models support seamless switching between generation tasks spanning multiple modality combinations within a single conversational interface. However, existing safety benchmarks construct prompts independently per modality, so different modalities cover different harmful intentions, preventing holistic evaluation of UMMG models. We propose an intention-grounded evaluation methodology in which every modality combination is evaluated against the same set of harmful intentions, yielding fair, directly comparable safety scores across modalities. Because scores are aggregated at the intention level — averaging over the same set of intentions rather than over modality-specific prompt sets — each model receives an accurate safety profile not distorted by per-modality differences. The methodology is incrementally extensible: new modality combinations or intentions can be added without rebuilding existing data. Following this methodology, we construct UNISAF-Bench (UNIfied intention-grounded SAFety Benchmark): 1,257 human-verified harmful intentions across 19 policies, yielding 20,112 conversations spanning 4 modality combinations, single-/multi-turn settings, and paired policy-violating/boundary-benign variants. Evaluation of four open-source UMMG models reveals systematic safety gaps: multimodal inputs degrade text-output safety, safety alignment fails to transfer to image generation, and multi-turn deflection bias inflates refusal on both harmful and benign prompts.

Abstract:
Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language models (DLMs). In DLMs, bidirectional attention means that updating any token dynamically alters the entire context and its corresponding KVs. Thus, existing caching techniques developed for LLMs, which assume that KVs remain invariant once computed, corrupt the shared prefix KVs. Our experiments show that applying these techniques to DLMs causes model accuracy to collapse to near zero. To unlock high-throughput DLM serving, we propose bidirectional prefix caching, BiCache, the first KV caching technique for shared prefixes in DLMs. BiCache is designed based on key observations from our comprehensive analysis: shared prefix KVs remain stable and reusable in shallow layers, while the depth of shallow layers depends on the fraction of shared prefix tokens in each request. Thus, BiCache dynamically identifies a safe layer depth for reusing shared prefix KVs and eliminates redundant computation. Evaluations demonstrate that BiCache significantly improves serving throughput by 36.3%–98.3% compared to existing techniques without accuracy collapse (only 0–1.8% difference).

Abstract:
Vision-language models (VLMs) can ingest only a limited number of video frames, making frame selection a practical necessity. But do current Video QA benchmarks genuinely require temporal frame selection, or can most questions be answered regardless of which frames are shown? We introduce Frame Selection Sensitivity ( FSS ), a per-sample diagnostic that measures how much VLM accuracy changes when the most relevant frames are replaced with the least relevant ones. Across six benchmarks and eight VLMs, we find that a large majority of samples are frame-agnostic: only a minority are genuinely sensitive to frame choice. Combining FSS with a Language Independence Score ( LIS ) reveals that merely 5.5--31% of samples are Temporally Sensitive . We construct TempCore , compact evaluation subsets that isolate these temporal samples from existing benchmarks, and will release code and per-sample annotations upon publication.

Abstract:
To effectively enable people to understand new topics, explanations should be tailored to their backgrounds and abilities. Prompting alone has been shown to be insufficient for creating such explanations, and other computational methods are missing so far. Therefore, this paper investigates whether LLMs can be steered to generate explanations that are tailored to a specific group of people. To this end, we propose an approach that first identifies group-specific attributes in terms of explanatory style and knowledge of a specific target group. Building on activation engineering, it then computes attribute-based steering vectors and adds them to the internal activations of an LLM during inference to enable a fine-grained steering. In our experiments, we assess the steering effectiveness in terms of specificity and factuality of the generated explanations. Additionally, we evaluate the explanations in a study with human experts from different target groups. Compared to prompting and state-of-the-art steering baselines, our approach tailors the explanations significantly better to the target group while maintaining the best specificity-factuality balance.

Abstract:
Subword tokenization is widely used to address the out-of-vocabulary problem in natural language processing. However, frequency-based subword tokenization can produce artifacts in the resulting vocabulary. Beyond the well-known junk tokens (meaningless fragments), we introduce the notion of subsumed tokens, valid words whose corpus frequencies are suppressed by longer tokens that contain them. Together with fragment tokens, these are referred to as overlapping tokens. We propose a post-processing method that reduces fragment tokens and restores subsumed tokens. The method searches over possible binary segmentations of character sequences and selects tokenizations that maximize corpus frequency, while optionally assigning bonuses to word spans identified by a morphological analyzer. As a post-processing step, the method can be applied to existing tokenizers such as Byte Pair Encoding (BPE) and the Unigram Language Model (ULM) without rebuilding their vocabularies. Although evaluated on Japanese, the approach may also be applicable to other languages given an appropriate morphological analyzer. Experiments on the Wiki-40B Japanese corpus show that our method reduces overlapping tokens, leading to more reliable statistics for valid words, and improves representation quality while preserving downstream performance across tasks.

Abstract:
Recent advancements in large language models (LLMs) have demonstrated their potential in automating the scientific research ideation. Existing approaches primarily focus on prompting techniques, often producing ideas misaligned with expert standards – novelty, feasibility, and effectiveness, which are widely recognized by the research community as the three key subdimensions of high-quality ideas. Also, balancing these dimensions remains challenging due to their inherent trade-offs. To address these limitations, we propose the first framework that employs a two-stage approach combining Supervised Fine-Tuning (SFT) and controllable Reinforcement Learning (RL) for the task. In the SFT stage, the model learns foundational patterns from pairs of research papers and their corresponding follow-up ideas. In the RL stage, multi-dimensional reward models guided by fine-grained feedback evaluate and optimize the model across key dimensions. During inference, dimensional controllers coordinated by a sentence-level decoder enable dynamic context-aware steering of the idea generation process. Our framework provides a balanced approach to research idea generation, achieving high-quality outcomes in the experiment by dynamically navigating the trade-offs among novelty, feasibility, and effectiveness.

Abstract:
Large Language Model (LLM)-based multi-agent systems (MAS) have emerged as a promising paradigm for complex tasks. However, existing works often rely on manual designs or “one-size-fits-all” automation and lack adaptability after deployment. We study in-context structural adaptation, where structured experience conditions both query-dependent system generation and execution-time reconfiguration without updating LLM parameters. We introduce MASFly, which realizes this adaptation through two complementary mechanisms. First, a retrieval-augmented SOP instantiation mechanism retrieves and adapts successful collaboration patterns to construct a query-specific MAS. Second, an experience-enhanced process supervision mechanism uses a dedicated Watcher agent to monitor execution against prior failure experience and reconfigure the system upon abnormal behavior. Experiments demonstrate that MASFly achieves state-of the-art performance, including a 61.7% success rate on TravelPlanner, with strong task adaptability and robustness.

Abstract:
Evaluating the conversational abilities of large language models (LLMs) remains a challenging task. Current mainstream approaches primarily rely on the “LLM-as-a-judge" paradigm, where an LLM is prompted to serve as an evaluator to assess dialogue quality. However, such methods often suffer from various biases, which undermine the reliability and consistency of the evaluation results. To mitigate these biases, recent methods employ multiple LLMs as judges and aggregate their judgments to select the optimal assessment. Although effective, this multi-judge approach incurs significant computational overhead during inference. In this paper, we propose an efficient multi-turn dialogue evaluator that captures the collective wisdom of multiple LLM judges by aggregating their preference knowledge into a single model. Our approach preserves the advantages of diverse multi-judge feedback while drastically reducing the evaluation cost, enabling fast and flexible dialogue quality assessment. Extensive experiments on seven single rating and pairwise comparison dialogue evaluation benchmarks demonstrate that our method outperforms existing baselines across diverse scenarios, showcasing its efficiency and robustness.

Abstract:
Dense retrievers in Retrieval-Augmented Generation (RAG) have achieved substantial progress in semantic matching, but they remain limited in long context retrieval. In long context retrieval, target evidence often occupies only a small local span, while the remaining content acts as noise. We aim to improve long-document retrieval by enhancing retrievers’ anti-interference capability. Our contributions are threefold: (1) we construct the Anti-Interference Capability Benchmark (AICB) to evaluate retrievers’ anti-interference capability under controlled noise levels; (2) we propose anti-interference training as a short-text training path for improving long context retrieval; and (3) we train AnInter, a series of embedding models that are more robust to noisy long documents. Evaluation on LongEmbed shows that AnInter improves over its backbone by 16.4 points on average and achieves state-of-the-art performance on this challenging long context embedding benchmark. These results show that interference-aware training is a practical direction for improving long context retrieval.

Abstract:
Recent Vision-Language Models (e.g., ColPali) enable fine-grained Visual Document Retrieval (VDR) but incur prohibitive multi-vector index storage overhead. Existing training-free pruning methods either rely on heuristic layer choices or degrade sharply under aggressive compression, leading prior work to argue that effective high-compression pruning requires query-dependent training. We challenge this view with Structural Anchor Pruning (SAP), a self-calibrating, training-free, query-agnostic index-time framework combining (i) Score Retention (SR), a white-box per-layer compression diagnostic; (ii) SR-guided window selection, which automatically locates the structural pruning region of any backbone with no per-model hyperparameters; and (iii) a visual in-degree centrality scorer that identifies anchor patches within that window. On ViDoRe v1/v2 across three architectures spanning 18, 28, and 36 backbone layers, SAP retains 93–96% of NDCG@5 on v1 and 88–90% on the harder v2 while pruning 90% of visual tokens; at 20× compression it retains 85–90% and 76–79% respectively. Our layer-resolved SR analysis reveals an Alignment-Aggregation Divergence: visual structure is preserved as a stable “Structural Plateau” within the backbone, while the final layers reshape it into a sparse, query-aligned form unsuitable for pruning. Probing the pre-retrieval base backbones shows that contrastive fine-tuning sharpens this boundary three- to eight-fold, explaining why final-layer methods fail.

Abstract:
Matryoshka Representation Learning (MRL) produces nested embeddings whose prefixes remain semantically meaningful, allowing representations to be truncated at inference time and enabling a single model to operate across multiple dimensional budgets. This paradigm has been adopted in several modern embedding models such as Qwen3-Embedding and OpenAI text-embedding-3. However, distilling MRL teachers into compact students remains largely unexplored, as conventional knowledge distillation methods focus on aligning single-scale embeddings and do not account for the multi-scale structure of Matryoshka representations. We introduce Matryoshka Representation Distillation and propose SUGAR, a new framework that preserves the structural properties of MRL embeddings during knowledge distillation. SUGAR aligns point-wise semantic fidelity, spectral capacity via effective-rank matching, and global subspace geometry through Grassmannian alignment. Our extensive experiments show that SUGAR produces compact students that retain strong performance across multiple embedding dimensions while preserving deployment flexibility.

Abstract:
Online communities develop distinct norms for content they collectively value, yet it remains unclear whether current language models can recognize locally valued contributions in context. We formalize this as community-conditioned preference prediction and introduce Vastu ( V alue- A ware S ocial Tu ning), a benchmark of 75,000 Reddit comments from 15 communities spanning Gaming, Science, Q&A, Advice, and Politics. We evaluate four model families—prompted LLMs, LoRA-adapted SLMs, supervised encoders, and feature-based classifiers—across global, local, and context-conditioned settings. Our central finding is that parametric adaptation consistently outperforms prompting: supervised encoders reach 0.74 AUROC and fine-tuned SLMs 0.64–0.71, while the best prompted result is only 0.62. This gap is not merely quantitative—vanilla prompting yields over 80% false-negative rates, systematically discarding content communities actually value. Conversational context narrows but does not close this divide. Together, these results suggest that local preference recognition requires community-specific training signal, not just better prompting. Our work supports future research on community-aware reward modeling, feed curation, and positive moderation.

Abstract:
Training strategy, namely whether to retrain from scratch or fine-tune from the previous checkpoint, is an overlooked decision variable in active learning. We show that this choice has exploitable structure: retraining is most useful in early rounds, when each batch can substantially reshape the labeled distribution, while fine-tuning becomes safer once the model trajectory stabilizes. We propose HybridAL, an adaptive training schedule that monitors an online stabilization signal and switches from retraining to fine-tuning after sustained stabilization. Two complementary signals, spectral exponent change 𝛥𝛼 (weight-based) and accuracy change 𝛥 Acc (validation-based), span different points on the time-calibration trade-off. Across three encoder backbones and six text-classification tasks (five seeds each), HybridAL keeps endpoint macro-F1 non-inferior to retraining and fine-tuning at a 0.010 margin, saves up to 49% of retraining time, and recovers a substantial fraction of retraining’s calibration advantage as measured by negative log-likelihood (NLL). Compared with schedules that switch at a pre-committed round, HybridAL obtains lower NLL at moderate additional cost, showing that trajectory-dependent switching provides a stronger time-calibration trade-off than fixed early switching.

Abstract:
Mitigating hallucinations in Large Language Models (LLMs) is critical for their reliable deployment. Existing methods typically fine-tune LLMs to abstain from answering questions beyond their knowledge scope. However, these methods often rely on coarse-grained signals to guide LLMs to abstain, such as overall confidence or uncertainty scores on multiple sampled answers, which may result in an imprecise awareness of the model’s own knowledge boundaries. To this end, we propose a novel reinforcement learning framework built on Fine-grained Semantic Confidence Reward (FiSCoRe), which guides LLMs to abstain via sample-specific confidence. Specifically, our method operates by sampling multiple candidate answers and conducting semantic clustering, then training the LLM to retain answers within high-confidence clusters and discard those within low-confidence ones, thereby promoting accurate post-hoc abstention. Additionally, we propose a new metric for evaluating the reliability of abstention fine-tuning tasks more comprehensively. Our method significantly enhances reliability in both in-domain and out-of-distribution benchmarks.

Abstract:
Multimodal retrieval, which seeks to retrieve relevant content across modalities such as text or image, supports applications from AI search to contents production. Despite the success of separate-encoder approaches like CLIP aligning modality-specific embeddings with contrastive learning, recent multimodal large language models (MLLMs) enable a unified encoder that directly processes composed inputs. While flexible and advanced, we identify that unified encoders trained with conventional contrastive learning are prone to learn modality shortcut, leading to poor robustness under distribution shifts. We propose a modality composition awareness framework to mitigate this issue. Concretely, it consists of a preference loss enforces multimodal embeddings to outperform their unimodal counterparts, and a composition regularization objective aligns multimodal embeddings with prototypes composed from its unimodal parts. These objectives explicitly model structural relationships between the composed representation and its unimodal counterparts. Experiments on various benchmarks show gains in out-of-distribution retrieval, highlighting modality composition awareness as a effective principle for robust composed multimodal retrieval when utilizing MLLMs as the unified encoder.

Abstract:
Large Vision-Language Models (LVLMs) have achieved remarkable progress across a wide range of multimodal tasks. However, object hallucination remains a persistent challenge, undermining model reliability in applications such as visual question answering, embodied agents, and human-AI decision support systems. In this paper, inspired by cognitive science studies on perceptual bias, we investigate a new failure mode termed Premise-Induced Visual Fixation Hallucination (PVFH). Specifically, PVFH refers to a premise-induced reasoning error in which models become anchored to visually confusable evidence, accept a false textual premise, and subsequently generate responses about an absent yet visually plausible object. To systematically study this phenomenon, we introduce PVFH-Bench, a benchmark that automatically constructs visually confusable but absent targets and pairs the core hallucination setting with diagnostic subsets to trace how models move from object recognition to Premise-Induced Visual Fixation Hallucination. Experimental results on 10 state-of-the-art LVLMs demonstrate that all evaluated models are highly vulnerable to Premise-Induced Visual Fixation Hallucination. Moreover, we find that explicit chain-of-thought reasoning can further exacerbate this failure mode by encouraging models to overinterpret weak visual cues as supporting evidence for hallucinated objects. All data and code will be publicly released on GitHub.

Abstract:
Large language model chatbots are increasingly deployed in organizational settings such as healthcare, finance, and public services, where organization-specific policies specify allowed and prohibited content. Existing policy-alignment benchmarks construct tests from one policy at a time, leaving failures caused by policy composition under-tested. We present COPAL, an automated framework for evaluating composed-policy alignment in organizational chatbots. COPAL uses empirically derived interaction patterns to generate queries that require multiple policies to be handled in one response, and pairs each query with an explicit handling contract specifying what to provide and avoid. Applied to 30 organization-like company worlds, COPAL exposes substantial composed-policy handling failures. Across 9 served models in our automated evaluation setup, composed-policy requests yield a 33.1% error rate, indicating that composed-policy alignment remains a challenging evaluation target. Our code is available at https://github.com/BruceAllenCS/COPAL-code.

Abstract:
The safety of large vision-language models is increasingly stress-tested by multimodal jailbreaks, yet existing attacks share one fundamental limitation: the attack itself is static. Template-based attacks freeze the image–text layout, while iterative sample-level attacks make only the image–text content dynamic, keeping the attack-iteration strategy fixed and the attacker’s parameters frozen. At the meta level they remain static, and we argue this partial dynamism cannot expose the full safety risks of multimodal models.We propose Meta-Adaptive Multimodal Jailbreaking (MAMJ), which lifts adversarial dynamism from the sample level to the meta level by making the attacker’s policy itself the optimisation target along two axes: an attack strategy prompt (ASP) 𝜃 in language space, governing how the attack is iterated against a victim VLM, and the attacker weights 𝜙 , determining the success rate attainable under a given 𝜃 .Evolving over a group of multimodal attack trajectories, an LLM-based critique of failed trajectories first refines 𝜃 , after which group-aggregated attack-success-rate (ASR) rewards update 𝜙 . On MM-SafetyBench, MAMJ attains 81.0%, 78.9%, and 82.3% ASR against GPT-4o, Gemini-3-Pro-Preview, and Seed 2.0, exceeding the strongest sample-level baseline by up to 24.1 percentage points, and the learned attacker (𝜃⋆, 𝜙⋆) transfers without retraining to unseen victims while remaining effective under representative defences.These results expose a systemic vulnerability of frontier VLMs to meta-adaptive jailbreaks and underscore the need for defences that anticipate meta-level adversaries. Code and checkpoints will be released.

Abstract:
Long-term memory for LLM agents must answer two questions for every stored fact: whether the fact is still true and whether it should be surfaced for a given query. We refer to these two dimensions as validity and visibility, respectively. Existing memory systems often collapse these two decisions into a single scalar score. However, current and historical queries may require opposite retrieval decisions for the same fact: a superseded fact should be hidden for a current-intent query but surfaced for a historical-intent one. This creates a structural failure mode: superseded and expired facts are collapsed onto the same validity axis. To address this problem, we propose MemFSM, a training-free retrieval-side layer that separates validity from visibility through a memory-lifecycle finite-state machine. Along the validity axis, a four-state FSM tracks whether each fact has been superseded and expired. Along the visibility axis, an intent-aware router decides whether to surface or hide a fact based on its state and the query intent. By decoupling visibility from validity, MemFSM improves performance across heterogeneous memory benchmarks and diverse LLM backbones. To evaluate memory systems under fact conflicts and diverse query intents, we further introduce ValidMem, a benchmark that annotates each fact with both supersedure and expiry labels and pairs each memory store with both current-intent and historical-intent queries.

Abstract:
The advancement of LLM agents with tool-use capabilities requires diverse and complex training corpora. Existing data generation methods, which predominantly follow the paradigm of random sampling and shallow generation, often yield simple and homogeneous trajectories that fail to capture complex, implicit logical dependencies. To bridge this gap, we introduce HardGen, an automatic agentic pipeline designed to generate hard tool-use training samples. First, HardGen establishes a dynamic API Graph built upon agent failure cases, from which it samples hard tool-execution sequences. Second, these execution sequences are used to reversely build advanced tools that combine multi-step operations into one, which then help generate hard queries requiring unstated intermediate steps. Finally, advanced tools and hard queries drive the generation of Chain-of-Thought (CoT), which is iteratively refined through closed-loop verification feedback. Extensive evaluations demonstrate that a 4B parameter model trained with our curated dataset achieves superior performance compared to several leading open-source and closed-source competitors (e.g., GPT-5.2, Gemini-3-Pro and Claude-Opus-4.5). Our code, models and the curated dataset with 27,000 samples will be open-sourced to facilitate future research.

Abstract:
Post-training model compression is essential for enhancing the portability of Large Language Models (LLMs) while preserving their performance. While several compression approaches have been proposed, less emphasis has been placed on selecting the most suitable set of data (the so-called calibration data) for finding the compressed model configuration. The choice of calibration data is a critical step in preserving model capabilities both intra- and inter-tasks. In this work, we address the challenge of identifying high-performance calibration sets for both pruning and quantization by analyzing intrinsic data properties rather than model-specific signals. We introduce ZipCal, a model-agnostic data curation strategy that maximizes lexical diversity based on Zipfian power laws. Experiments demonstrate that our method outperforms standard uniform random sampling across various benchmarks. Notably, it also performs on par, in terms of downstream performance, with a state-of-the-art method that relies on model perplexity. The latter becomes prohibitively expensive at large-scale models and datasets, while ZipCal is on average 240 faster due to its tractable linear complexity. We make the code and the experiments available at https://github.com/FrancescoMonaco/ZipCal.

Abstract:
Diffusion Language Models (DLMs) have attracted significant attention for their strong reasoning ability. However, under a bidirectional attention mechanism, DLMs operate over an exponentially large exploration space compared to autoregressive models (ARMs), making it challenging to focus on reasoning-guiding tokens under random masking. We define causal shortcuts as token chains that cover the full sequence and provide explicit guidance towards correct reasoning trajectories. We analyze the effects of causal shortcuts on the reasoning accuracy and convergence speed of DLMs, and find that they largely improve answer convergence efficiency and generation accuracy. Motivated by this, we propose a Causal Shortcut Learning (CSL) Framework for DLMs. Specifically, we introduce a step-by-step token extraction procedure to extract causal shortcuts from data, and apply parallel prioritized masking on these tokens during training to enable efficient and accurate convergence to correct answers via causal shortcuts. Extensive experiments across multiple reasoning benchmarks and two base models demonstrate that CSL consistently outperforms existing SFT-variant baselines, achieving an average improvement of 1.92% over SFT-only models, and up to 4.20% on MATH-500. The code is available at the https://github.com/ZJUDianJin/Causal-Shortcuts-Learning.

Abstract:
As LLMs become increasingly woven into everyday workflows, user queries send to cloud-hosted LLMs routinely mix task-essential content with task-non-essential sensitive disclosures, yet type-based PII redaction is context-agnostic and may raise two issues: over disclosing untyped sensitive context and over-removing answer-bearing spans. We recast privacy-preserving query rewriting under Contextual Integrity: a span should be forwarded only if it is necessary for the task. We build the first task-based CI benchmark for privacy conscious delegation, comprising 3,167 samples that combine high-quality synthetic data spanning 11 tasks and 20 task types, WildChat based real-user queries, and a medical challenge set with dense sensitive information. Building on this benchmark, we propose a CI guided reinforcement learning framework that converts essential and non-essential sensitive spans into verifiable optimization signals, and train a query rewriter to preserve task-critical information while suppressing unnecessary sensitive disclosure. Experiments show that our learned rewriter achieves the best privacy–utility trade-off, achieving up to +10.1 average utility over on-device baselines.

Abstract:
While large language models (LLMs) perform strongly after adaptation within individual scientific domains, their learning dynamics in multi-disciplinary contexts remain poorly understood, despite the promise of improved generalization and broader applicability through cross-domain knowledge synergy. We present a systematic study of multi-discipline LLM fine-tuning across five scientific disciplines and compare the learning dynamics of full fine-tuning, LoRA, LoRA-MoE, and LoRA composition. In our naturally imbalanced setting, we show that multi-disciplinary learning is substantially more variable than single-discipline training and distill four empirical patterns: (1) Balance-then-Diversity: increasing scarce-domain exposure helps, but broad source-pool diversity must be preserved; (2) Merge-then-Align: general instruction data restores instruction-following behavior needed for effective joint adaptation; (3) Optimize-then-Scale: increasing trainable capacity offers limited gains without prior design optimization; and (4) Share-then-Specialize: asymmetric LoRA-MoE improves multi-discipline performance through a shared low-rank projection. Together, these patterns form a practical recipe for principled multi-discipline fine-tuning and provide actionable guidance for developing generalizable scientific LLMs.

Abstract:
As large language models (LLMs) are integrated into numerous applications, LLMs’ safety becomes critical for both application developers and intended users. Currently, great efforts have been made to develop safety benchmarks with fine-grained taxonomies. However, these benchmarks’ taxonomies are disparate with different safety policies. Thus, existing safeguards trained on these benchmarks are either coarse-grained to only distinguish between "safe” and "unsafe,” or constrained by the specified narrow risk taxonomies. To leverage these fine-grained safety policies across multiple safety taxonomies, we propose GSPR, a Generalizable Safety Policy Reasoner to identify unsafe inputs and outputs with violated safety taxonomies and concise explanations. Unlike prior safeguards which only cover a fixed set of risk factors, GSPR incentivizes its reasoning capability with varied safety taxonomies through reinforcement learning. Our GSPR can be trained across multiple safety benchmarks with distinct taxonomies and naturally exhibits powerful generalization ability. We conduct extensive experiments to show that GSPR significantly improves existing safety guardrails’ reasoning capabilities for both safety and category prediction tasks. Moreover, GSPR also achieves the least inference token costs with explanations.

Abstract:
Large language models (LLMs) are increasingly deployed in applications with societal impact, raising concerns about the cultural biases they encode. We probe these representations by evaluating whether LLMs can perform author profiling from English-translated song lyrics in a zero-shot setting, inferring singers’ gender and ethnicity without task-specific fine-tuning. Across several open-source models evaluated on more than 10,000 lyrics, we find that LLMs achieve non-trivial profiling performance but demonstrate systematic cultural alignment: most models default toward North American ethnicity, while DeepSeek-1.5B aligns more strongly with Asian ethnicity. This finding emerges from both the models’ prediction distributions and an analysis of their generated rationales. To quantify these disparities, we introduce two fairness metrics, Modality Accuracy Divergence (MAD) and Recall Divergence (RD), and show that Ministral-8B displays the strongest ethnicity bias among the evaluated models, whereas Gemma-12B shows the most balanced behavior. Our code is available on GitHub and results on HuggingFace.

Abstract:
Clinicians read chain-of-thought (CoT) rationales as evidence of medical reasoning, but whether the visible chain plays that role is rarely tested. General-domain CoT-faithfulness probes ignore clinical cost, and medical LLM evaluations treat the chain as a black box. We close this gap with a medical perturbation audit: a 30-operator battery edits both the chain and the question with clinically motivated operators (severity reversal, negation flip, demographic swap, evidence ablation), paired with a chain-update × answer-flip joint analysis that classifies each model by its failure mode. Applied to 14 LLMs on four medical QA benchmarks, three independent tests converge: the Chain-Decoupling Rate (CDR; chain does not register the edit and the answer does not flip) is 72.9% panel-wide on M2/M4/M5 destructive edits, chain corruption leaves accuracy unchanged, and removing CoT prompting does not reduce accuracy. Two board-certified clinicians re-annotate N=197 perturbed questions; 98.5% leave the gold defensible. The pattern holds across medical and reasoning fine-tuning, scale, and the closed-source subset. Our framework and CDR provide a reusable yardstick for auditing whether medical CoT is faithful or merely documentation.

Abstract:
Arabic LLMs have improved rapidly, but aligning them with country-specific cultural behavioral preferences remains underexplored. Previous work has focused primarily on instruction-tuning for factual knowledge, overlooking nuanced cultural behaviors. We propose , which to our knowledge is the first large-scale pan-Arab preference data set for the alignment of cultural behavior conditioned on the country. Our contribution includes two resources in English and Modern Standard Arabic: a culturally grounded preference dataset for training and evaluation, and a multiple-choice benchmark designed to assess culturally aligned behavior across nations. All test data are validated by native speakers to ensure authenticity. To optimize cultural alignment, we experiment with post-training methods across multilingual and Arabic-centric language models, evaluating performance on both generation tasks and our cultural multiple-choice benchmark. By releasing both a training dataset and an evaluation benchmark, we provide a foundation for systematically studying and improving culturally aligned behavior in language models.

Abstract:
Autoregressive vision-language models achieve strong multimodal understanding, but their sequential decoding becomes increasingly inefficient as multimodal responses grow longer. Adapting AR-VLMs to diffusion-style block decoding is promising, yet fixed refinement blocks struggle with sharply varying multimodal uncertainty in OCR, chart, document, and reasoning-heavy outputs. Binary mask-or-token states further limit local refinement, as masking discards partial evidence while premature commitment can corrupt the causal prefix. We present FLUID-V, a lightweight AR-to-diffusion adaptation framework for reliable local multimodal refinement. FLUID-V selects refinement spans with elastic horizons and represents unresolved positions as evolving soft semantic mixtures in embedding space. Only confident predictions are committed to the persistent KV cache, preserving causal multimodal conditioning and autoregressive compatibility. Experiments with OpenPangu-VL-7B and 100K self-distilled samples show competitive performance across diverse multimodal benchmarks, with ablations confirming the importance of adaptive horizons and stateful intermediate representations. Our code is available at https://github.com/Oli-lab-nun/FLUID-V.

Abstract:
We investigate whether independently trained language models converge to geometrically compatible latent representations, and whether this compatibility can be exploited to correct model behavior at inference time without any weight updates. We learn a linear projection matrix that maps activation vectors from a large teacher model into the coordinate system of a smaller student model, then intervene on the student’s residual stream during generation by substituting its internal state with the translated teacher representation. Across a fully crossed experimental matrix of 20 heterogeneous teacher-student pairings spanning mixture-of-experts, dense, code-specialized, and synthetically trained architectures, the Ridge projection consistently achieves R^2 = 0.50 on verbal reasoning and R^2 = 0.40 on mathematical reasoning, collapsing to R^2 = -0.22 under permutation control and R^2 = 0.01 under L_1 regularization. Behavioral correction rates range from 14.0% to 50.0% on TruthfulQA (mean 25.2%) and from 8.5% to 43.3% on GSM8K arithmetic reasoning (mean 25.5%), demonstrating that the method generalizes across fundamentally different reasoning domains. We report a near-zero correlation between geometric alignment quality and behavioral correction rate (r = -0.07), revealing a dissociation between representation space fidelity and output space impact. Intervention strength is architecture-specific: student models exhibit characteristic sensitivity profiles that invert across domains, with the most steerable verbal student becoming the least steerable mathematical student. Finally, a double dissociation experiment conducted across all 20 model pairings confirms without exception that projection matrices collapse catastrophically when transferred across reasoning domains (mean R^2 = -3.83 in both transfer directions), establishing domain-specific subspace geometry as a universal property of LMs.

Abstract:
Large language models (LLMs) are increasingly used for qualitative data analysis (QDA), yet their outputs often miss the depth and nuance of human analysis. We argue this gap reflects a missing credibility practice from human QDA: peer debriefing, in which an analyst seeks feedback from a disinterested peer and uses it to refine their coding. To bring this practice into LLM-assisted QDA, we propose Agent-as-Peer-Debriefer, a multi-agent QDA framework that builds peer debriefing into key coding steps. In our framework, a Hierarchical Coding Agent follows the standard QDA process to generate codes, sub-themes, and themes, along with self-explanations and reflection memos. It then shares these outputs with three Peer-Debriefing Agents, each applying a distinct analytical perspective (Theory-Driven, Data-Driven, or Applied) and refining the codes by keeping, renaming, reassigning, merging, or splitting them. These perspectives are drawn from established human QDA practices that generalize across domains and datasets. To evaluate the framework, we test it on three datasets across two domains with three LLMs, measuring semantic similarity to human-annotated codes. Across all settings, perspective-based, peer-debriefing refinement aligns more closely with human codes than a single-LLM baseline, and an ablation further shows the gain is not merely from additional refinement. The three perspectives also produce distinct trade-offs, showing that the choice of perspective is a meaningful and controllable design decision. More broadly, these findings suggest that simulating peer debriefing with explicit perspectives is a promising route to more credible LLM-assisted QDA.

Abstract:
Recent work shows that context can both amplify and mitigate social bias in LLMs. Yet a key question remains underexplored: when context provides counter-stereotypic evidence, is that evidence taken up equitably across demographic targets? Using agency–communality bias as a testbed, we introduce a sequential evaluation framework that holds total counter-evidence fixed while varying its evidence structure, adapting the bookkeeping–subtyping contrast from social cognition. Across three instruction-tuned LLMs evaluated in English, counter-evidence is taken up in explicit group-level judgement and transfers to explicit judgement about held-out individuals. The prior-category structure of this uptake, however, does not carry over to generation: communal-prior targets lose their correction advantage, absorb evidence more slowly, and the weakest uptake repeatedly appears for Asian and Hispanic female targets within our controlled target set. The tested prompt instruction and DPO fine-tuning variants yield partial improvements at prior-category level but do not reliably repair these target-level gaps. These findings show that fairness audits for context-rich LLM systems should go beyond checking whether corrective evidence is present and test whether models use that evidence equitably in downstream generation.

Abstract:
Predictive political question answering (QA), such as predicting how a political actor will vote, goes beyond factual lookup. External political resources offer rich historical evidence, but rarely contain the answer itself. Existing LLM augmentation methods, including actor-profile-based simulation and knowledge graph evidence injection, improve political reasoning but largely treat external resources as knowledge-based evidence, leaving prediction-relevant signals under-modeled. We identify two complementary signals for predictive political QA: actor stances that capture issue-specific preferences, and high-order structure signals that capture indirect dependencies among political actors. We propose PSL, a dual-view framework that converts semi-structured political records into inference-oriented evidence for LLMs. PSL extracts stance signals from question-relevant actor records in a semantic view, and learns structure-aware actor representations from an actor interaction graph in a vector view. Across three real-world datasets and multiple LLMs, PSL consistently outperforms baselines, with ablations confirming the complementary gains of stance and structure signals.

Abstract:
Long-video understanding on edge devices must reason over hours of content under tight compute and bandwidth budgets. Subsampling visual tokens loses temporal structure, while text-only video memories lose fine-grained visual attributes. We observe a visual-textual duality: language memories carry long-range temporal structure better than dense frames, while pixels remain decisive for attribute-level perception. Building on this insight, we propose Caption-once, Frames-onDemand (CFD), a budget-aware edge-cloud agentic framework. The edge runs a single offline captioning pass that builds a dual-track narrative index, an event-level story skeleton plus a clip-level micro-log, cached and reused across queries without re-captioning. At query time, a cloud-side MLLM reasons over the index in a story-first loop centered on a lightweight Visual-Need Router: a per-query gating module that triggers bounded keyframe retrieval only for perceptual questions (appearance, on-screen text, attribute disambiguation) and keeps temporal-structural questions in language space. The router turns visual access into a first-class, query-conditioned cost, capping per-query frame consumption regardless of video length. Experiments on long-video benchmarks demonstrate strong accuracy-efficiency trade-offs while substantially reducing online visual processing.

Abstract:
Synthesizing supervised finetuning (SFT) data from language models (LMs) to teach smaller models multilingual tasks has become increasingly common. However, teacher model selection is often ad hoc, typically defaulting to the largest available option, even though such models may have significant capability gaps in non-English languages. This practice can result in poor-quality synthetic data and suboptimal student downstream performance. In this work, we systematically characterize what makes an effective multilingual teacher. We combine intrinsic measures of data quality with extrinsic student model performance in a metric we call Polyglot Score. We evaluate 10 LMs across 6 typologically diverse languages, generating over 1.4M SFT examples and training 240 student models. Our analyses reveal that model scale alone does not significantly predict teacher effectiveness: the most effective teachers we identify are consistently smaller than the largest models evaluated, and their ranking is stable across student base model families. Instead, data qualities such as prompt diversity, length, and response fluency capture 93.3% of the variance in intrinsic data quality and predict student performance. Finally, we provide practical recommendations, including matching the model families of teacher-student pairs and generating responses to existing prompts or translating them from English, which can yield improvements for less-resourced languages. We hope that our work advances data-centric research in multilingual synthetic data and LM development.

Abstract:
Applications in labor market intelligence demand specialized NLP systems for a wide range of tasks, characterized by extreme multi-label target spaces, strict latency constraints, and multiple text modalities such as skills and job titles. These constraints have led to isolated, task-specific developments in the field, with models and benchmarks focused on single prediction tasks. Exploiting the shared structure of work-related data, we propose a unifying framework, combining a wide range of tasks in a multi-task ranking benchmark, and a flexible architecture tackling text-driven work tasks with a single model. The benchmark, WorkBench, is the first unified evaluation suite spanning six work-related tasks formulated explicitly as ranking problems, curated from real-world ontologies and human-annotated resources. WorkBench enables cross-task analysis, where we find significant positive cross-task transfer. This insight leads to Unified Work Embeddings (UWE), a task-agnostic bi-encoder that exploits our training-data structure with a many-to-many InfoNCE objective, and leverages token-level embeddings with task-agnostic soft late interaction. UWE demonstrates zero-shot ranking performance on unseen target spaces in the work domain, and enables low-latency inference with two orders of magnitude fewer parameters than best-performing generalist models that are not tailored to the work domain (Qwen3-8B), with +4.4 MAP improvement.

Abstract:
We test the standard RLVR tool-use recipe (GRPO on Qwen2.5-7B-Instruct) on a deliberately minimal knowledge-graph tool API: four Freebase navigation verbs over Complex WebQuestions. Under a self-verifiable retrieval reward, the policy’s correct-via-tool rate climbs from 3.8 to 9.6 over 250 steps, then collapses to 0 within a single 50-step window, a peak-then-collapse pattern replicated across four seeds. Across seven reward designs, we find four recurring failure modes: adding denser or more targeted proxy rewards shifts the failure mode rather than eliminating it. We argue that a key difference from Python interpreters, web search, and JSON APIs is interface feedback. Their failures often leak natural-language signal the model saw in pretraining. A Python traceback names the failing line; an empty Freebase result [] does not. Stripping away that surface exposes a degradation regime that same-family reward redesigns do not fix. A direct oracle ablation rules out relation selection: injecting gold relations at every retrieval call lifts exact-match accuracy by only +0.20 pp, and 95.4 of retrieval-dependent errors are retrieval-composition failures rather than answer-extraction failures. As a mitigation, one-iteration self-distillation reaches 40 EM at 7B; the best 14B model we train reaches 40.2 (a gap within sampling noise) and initialisation barely matters, so within the 7B–14B range tested the ~40 ceiling appears interface-bound rather than capacity-bound. Paper Type: Long

Abstract:
We propose Process-Aware Policy Optimization (PAPO), a method that integrates process-level evaluation into Group Relative Policy Optimization (GRPO) through decoupled advantage normalization, to address two limitations of existing reward designs. Outcome reward models (ORM) evaluate only final-answer correctness, treating all correct responses identically, and gradually lose the advantage signal as groups become uniformly correct. Rubric-based process reward models (PRM) offer richer supervision, differentiating quality among correct responses and restoring advantage diversity, but using their scores as rewards triggers verbosity-driven reward hacking and accuracy collapse. PAPO resolves both by composing the advantage from an outcome advantage A out , derived from outcome reward r out over all responses, and a process advantage A proc , derived from process reward r proc exclusively over the correct subset. A out anchors correctness, and correct subset normalization on A proc rules out verbosity-driven hacking while sustaining the gradient on all-correct groups where ORM exhausts. PAPO consistently outperforms ORM across five model configurations and six benchmarks (reaching 51.3% vs. 46.3% on OlympiadBench for Qwen2.5-7B while continuing to improve as ORM plateaus and declines), with analogous gains when composed with mainstream GRPO variants.

Abstract:
Zero-shot anomaly detection (ZSAD) has gained significant attention for its practical value in industrial inspection. Recently, CLIP-based approaches have been widely adopted in ZSAD due to their strong vision-language generalization capabilities. However, existing methods commonly employ continuous prompt embeddings for prompt optimization and encode semantics in latent vectors, which lack interpretability and scalability. To this end, we propose CoEvoAD, a co-evolutionary framework for discrete prompt selection. CoEvoAD performs prompt search in the discrete natural-language space using an evolutionary algorithm. Candidate prompts are iteratively generated, evaluated, and selected throughout population evolution, thus preserving the interpretability and composability of natural language. Furthermore, we introduce a Cross-Category Transfer Objective (CCTO), which treats held-out source categories as proxies for unseen categories and scores prompt rules based on their estimated cross-category transferability, effectively improving cross-category generalization. Extensive experiments are conducted to validate the effectiveness of CoEvoAD, and the results show that it achieves state-of-the-art performance across multiple anomaly detection datasets.

Abstract:
LLM agents acting in structured environments fail in operational rather than conversational ways, and reliability depends on procedural knowledge of the environment. Prior self-improvement methods accumulate natural-language guidance without checking that each new item preserves previously correct behavior, so a note that fixes one trajectory can silently regress another. We introduce GRASP (Gated Regression-Aware Skill Proposer), which treats agent improvement as a sequence of edits to a bounded skill library, admitting each candidate only if it produces a net improvement on a balanced held-out probe under a hard regression budget. We evaluate GRASP across five base models (gpt-oss-120b, DeepSeek V4 Flash, Gemini 3.1 Flash Lite, GPT-4.1, GPT-5.4) on two FHIR-based clinical benchmarks. On MedAgentBench, GRASP lifts gpt-oss-120b from 40.6% to 88.8%, exceeds the strongest of five self-improvement baselines by 21.0 points, and improves every other base model by 17.2 to 40.3 points. Ablations attribute the gain to comparative proposal generation, the acceptance gate, and the hard regression budget rather than to skill writing itself, which without validation is no better than using no skills. The mechanism generalizes beyond the clinical domain, improving agents on three of four non-clinical environments and remaining flat only where the action space is open-ended. Frozen libraries transfer across models, where skills from a stronger model improve weaker executors beyond what they learn for themselves while the reverse does not, an asymmetry that no ungated baseline reproduces.

Abstract:
We analyze the learned input-output behavior of GLU-based neurons in large language models (LLMs). We propose a simple analysis method: For each neuron, we compute the cosine similarities between its input (reading) and output (writing) weight vectors. In this scheme, a strong negative cosine similarity indicates the neuron weakens the direction it detects in the residual stream, so we call this a weakening neuron. This allows us to gain a number of novel insights. First, we show that nine different LLMs have similar patterns: weakening neurons appear mostly in late layers whereas their counterparts, (conditional) strengthening neurons, are frequent in early-middle layers. Second, we find that weakening neurons display surprising behavior: even though there are few, they activate often and have a large influence on model behavior. Third, weakening neurons have a strong effect on model output when gate values are negative – which is surprising since negative gate values are not expected to encode functionality.

Abstract:
Large language model (LLM)-powered multi-agent systems (MAS) demonstrate remarkable collective intelligence, with multi-agent memory serving as a pivotal mechanism for continual adaptation. However, existing multi-agent memory designs remain constrained by two fundamental bottlenecks: (i) memory homogenization caused by limited role-aware customization, and (ii) information overload induced by exposing agents to excessively fine-grained textual memories. To address these limitations, we propose LatentMem, a learnable multi-agent memory framework that customizes agent-specific experiential memories while reducing textual context. Specifically, LatentMem comprises an experience bank for lightweight storage of raw interaction trajectories and a memory composer that synthesizes compact latent memories conditioned on retrieved experience and agent-specific contexts. Further, we introduce Latent Memory Policy Optimization (LMPO), which propagates task-level optimization signals through latent memories to learn compact, high-utility latent representations. Extensive experiments across six benchmarks and four mainstream MAS frameworks show that LatentMem improves vanilla settings by up to 19.36% and consistently outperforms existing memory architectures, without requiring modifications to the agent-layer workflows.

Abstract:
LLM agents are widely used in social simulation, yet fluent social reasoning does not guarantee valid decision mechanisms. We test whether models implement socially embedded utility, where actions depend on peer-weighted references rather than isolated private preference. We introduce a theory-guided benchmark from network utility theory covering utility recovery, social-comparison gating, and structural transfer across two out-of-distribution decision forms. Across open-source and proprietary models, we find a gap between cue recognition and peer aggregation, spurious social activation when comparison should be inactive, and declining accuracy as social dependence rises. We propose theory-supervised mechanism alignment trained on latent utility states. Fine-tuning yields +8.3 and +13.9 utility-recovery points on Llama-3.1-8B and Qwen3-8B, raises gating above 95%, and improves transfer to unseen structures. Multi-agent simulations show that repairing local utility shifts macro outcomes, including stronger financial amplification and more stable labor stratification, which indicates that realistic macro-level simulations require adjustments at the individual level. Data and code are available at https://github.com/saaranLiu/SEDM-alignment.

Abstract:
LLM-enhanced recommender systems encode item side information into semantic embeddings, but these embeddings may leak sensitive textual attributes through inversion attacks. Existing input-side defenses rely on trusted collection or local token perturbation, often disrupting the inter-item semantic topology needed for recommendation. To address it, we introduce relational fidelity as the key objective, requiring semantic shift for privacy while preserving relative item relations for utility. Based on this principle, we propose prototype-guided progressive obfuscation (PGPO), which learns topology-aware obfuscation anchors for high-influence prototype words and propagates them through a static semantic graph. PGPO is optimized with GRPO using individual dissimilarity, structural consistency, and semantic constraint rewards. Experiments on two benchmarks show strong inversion defense with near-upper-bound recommendation performance. The code is available at https://github.com/Shengxiang-Lin/PGPO.

Abstract:
Clinical decision-making agents can benefit from structured reuse of prior decision experience. Motivated by illness scripts, we study how to organize clinical experiences with explicit applicability conditions and inter-experience relations. However, many memory-augmented methods store experiences as independent records without conditional or relational structure, which may introduce noisy retrieval, unreliable reuse, and in some cases even hurt performance compared to direct LLM inference. We propose GSEM (Graph-based Self-Evolving Memory), a clinical memory framework that formalizes experiences as indications and contraindications and organizes them into a dual-layer memory graph. GSEM captures both intra-experience decision structure and inter-experience dependencies, supporting applicability-aware retrieval and feedback-driven calibration of node quality and edge weights. Across MedR-Bench and MedAgentsBench with two LLM backbones, GSEM achieves the highest average accuracy among all baselines, reaching 70.90% and 69.24% with DeepSeek-V3.2 and Qwen3.5-35B, respectively. Code is available at https://github.com/unname797/gsem.

Abstract:
Activation verbalization explains hidden representations in natural language, but existing methods are mostly limited to self-explanation, where each model explains only its own activations. We introduce Universal Activation Verbalizer (UAV), a shared-decoder framework that uses donor- and layer-specific adapters to explain activations from heterogeneous donor models. UAV learns a lightweight adapter that converts donor activations into soft tokens in decoder’s embedding space, and further supports adapter-only transfer by reusing a frozen decoder-side LoRA while training only a new adapter for another donor. Across classification, fact retrieval, and gist summarization, UAV remains competitive with strong self-explanation baselines while enabling cross-model verbalization across model families and scales. Ablations show that decoder-side tuning mainly improves task behavior, whereas the adapter provides the activation-grounded factual and semantic evidence. Code and data are available at our GitHub repository.

Abstract:
Multi-turn user-facing agents must infer user intent from incomplete requests, collect missing information through dialogue and tools, and execute valid actions. A training trajectory records this process as an interleaved sequence of user messages, agent responses, tool calls, etc. Synthesizing sufficiently complex trajectory has become a central route to train agents: existing pipelines often increase difficulty by composing multiple user requests into longer tasks, producing write-intensive trajectories that train sequential execution. We argue that a single write decision can itself be difficult when the agent must gather and compare substantial read-tool evidence before its arguments become identifiable, a challenge that write-intensive data alone cannot address. Guided by this insight, we propose WRIT ( W rite- R ead I ntensive T rajectory Synthesis), a pipeline for synthesizing multi-turn agent training trajectories along two complexity axes: the number of write decisions in a task and the evidence burden of each individual decision. WRIT first generates write-intensive and read-heavy tasks. It then diversifies user behavior instructions to reflect realistic conversational variation, and finally simulates agent-user interactions in an executable environment to produce complete training trajectories. The resulting data trains agents not only for longer task execution, but also for robust, evidence-grounded decision making under high information load. With only 2K synthesized trajectories, a 4B model trained on WRIT outperforms GPT-5.1 no-think on 𝜏 2 -bench and substantially reduces inference-time token usage, showing that compact SFT data can convert part of expensive test-time reasoning into efficient agent behavior.

Abstract:
Language models (LMs) operate in evolving environments where adapting to new domains and linguistic shifts is critical. Continual pre-training (CPT) addresses this but introduces a strict stability-plasticity dilemma. While current approaches manage this via data replay or architectural changes, they largely treat the optimizer as a static component. We propose the Plasticity-Inducing Control Optimizer (PICO) to tackle this dilemma directly at the update level. The optimizer defines a utility-gated update together with a stochastic perturbation path controlled by periodic spectral sensitivity checks. Structural flags suppress the perturbation path while leaving the protected-gradient update active. Across the evaluated legal and medical streams, PICO exhibits a favorable stability–plasticity trade-off through optimizer-level control, while incurring a measurable spectral-monitoring cost. Code is available at https://github.com/infonet-gist/infonet-ai-pico .

Abstract:
Large Language Models (LLMs) generate grounded answers using Retrieval-Augmented Generation (RAG), reducing hallucinations. Prior work has focused on answer and attribution correctness, with recent studies exploring faithfulness through internal signals that reflect the model’s decision process. However, these approaches often introduce latency and do not tightly align token generation with attribution. We propose EARAL, a novel attribution method for RAG that improves faithfulness through joint answer generation and generation-time attribution with minimal overhead. Documents are marked with special token identifiers, whose logits estimate their contribution during generation and are aggregated into document-level attributions. On the Trust-Align benchmark for trustworthy attributed generation, EARAL outperforms state-of-the-art methods across metrics. Additional analysis shows low latency, robustness, and strong alignment with faithfulness.

Abstract:
Existing curriculum learning methods for reasoning reinforcement learning often treat problem difficulty as a static property of the problem. We argue that difficulty is instead policy-dependent: problems beyond the current policy provide sparse learning signals, while problems already solved by greedy decoding may still yield informative stochastic trajectories during RL. We propose PAD-Curriculum, a policy-adaptive curriculum framework for GRPO that estimates the student’s evolving solvable frontier from student-generated trajectories. For currently solved problems, PAD-Curriculum ranks examples by a normalized Scorer-to-Student KL discrepancy computed against a frozen stronger scorer; unsolved problems are deferred and periodically re-evaluated as the policy improves. Experiments on GSM8K, MATH, and NuminaMath-CoT with Llama-3.1 and Qwen3 models show consistent gains over standard GRPO and curriculum/distillation baselines, with particularly strong improvements on challenging mathematical reasoning benchmarks. Analyses further show that adaptive discrepancy estimation is more effective than static scorer-based difficulty.

Abstract:
Unified multimodal embedding spaces have become the standard interface for cross-modal retrieval and multimodal RAG, and recent audio-video-text (AVT) encoders extend this setting to three modalities. Such encoders can produce a joint (T, V, A) embedding whenever all three modalities are available, but standard pairwise InfoNCE objectives leave this signal unused during training. We close this gap with fusion-as-teacher distillation, which treats a stop-gradient copy of the fused embedding as a teacher signal for the single-modal embedding, paired with a Tuple-InfoNCE term that supervises the fused embedding directly. We instantiate this objective as OmniRetriever-7B. Across six zero-shot retrieval benchmarks, OmniRetriever-7B reaches R@1 of 19.14 on Clotho text-to-audio and 47.60 on MSR-VTT text-to-video. To stress-test joint representations, we further release OmniRetriever-Bench, a 12-direction AVT retrieval benchmark totaling 3,782 triples; on it OmniRetriever-7B attains AVG-all 34.84, improving over the best prior open-source AVT method by +8.03. Model weights, datasets, and code are released.

Abstract:
Presupposing the boundaries of bias is itself a form of bias. We study closed-loop bias governance for Dutch government documents, where a system must detect biased language, ground decisions in legal and contextual evidence, rewrite problematic sentences when intervention is warranted, and verify that the rewrite mitigates harm without distorting meaning. Existing methods face three challenges: (i) discriminative classifiers capture surface regularities but lack normative grounding; (ii) zero-shot LLMs often adopt generic viewpoints and over-flag ambiguous administrative language; and (iii) fixed taxonomies inherit the Closed-World Assumption, missing emerging local targets. We propose MARS-Gov, a standpoint-aware multi-agent framework that combines legal retrieval, open-set target screening, specialized jurors, conservative routing, and rewrite verification. When screening finds an uncovered group, MARS-Gov instantiates a dynamic “10th juror” to deliberate outside the fixed panel. On DGDB, MARS-Gov sets a new SOTA with 0.880 F1, outperforming the strongest zero-shot LLM detector by 20.2 points (29.8% relative) and the best supervised Dutch encoder by 6.8 points, while reducing unnecessary interventions to 2.5%. Leave-One-Category-Out (LOCO) evaluation recovers held-out categories with 85.1% Correct@1 and 93.8% Correct@3.

Abstract:
Ensuring that multilingual language models generate coherent text in a specific target language is a major issue in multilingual language modeling. We develop an optimal control method for target-language text generation as well as a framework for evaluating the quality of generated text in terms of language adherence, linguistic coherence, and semantic coherence. We find that the proposed method performs at least as well as the prominent difference-in-means activation steering method for the majority of models tested, with substantially less hyperparameter tuning required.

Abstract:
Eliciting high-quality reasoning trajectories to supervise Large Language Models (LLMs) in sequential recommendation is notoriously difficult. In massive item pools, standard data curation methods such as autonomous exploration or rejection sampling suffer from severe sparsity. LLMs’ context-utilization bias exacerbate this challenge by overusing noisy interaction histories, yielding diffuse and weakly grounded reasoning paths that frequently fail to retrieve the target item. To overcome this data curation bottleneck, we propose GEARec (Grounded Reasoning Curation for Anchor-based Recommendation), a framework based on Dual-Perspective Anchor Extraction and Hybrid Reasoning Generation. By combining the semantic strengths of LLMs with collaborative filtering (CF) signals, GEARec identifies a high-relevance anchor item from the user’s history to construct highly focused, reachability-aware reasoning trajectories for supervised fine-tuning. Experiments on multiple datasets demonstrate that GEARec effectively curates superior supervision signals, reducing context overuse and consistently outperforming strong baselines in recommendation accuracy.

Abstract:
Activation steering can be embedded directly into a language model’s weights, shaping behaviour without inference-time intervention and offering a way to encode alignment prior to release. However, models are routinely fine-tuned after deployment, and it is unknown whether embedded interventions survive this. We study the stability of embedded steering for refusal ablation and brevity amplification across five instruction-tuned models (3B–14B) under non-adversarial SFT and RLHF. We find that steering degrades more under stronger optimisation pressure against the steered behaviour, which depends on both the training data and the training algorithm. Refusal ablation loses 69% of its effect on average under our SFT setup, but only 2% under our RLHF setup. The weight-edit, by contrast, remains almost unchanged (on average only 0.4% of it is restored), and the fine-tuning update along the steering direction is close to orthogonal to the pre-edit weight pattern (mean cosine similarity 0.071). Fine-tuning that restores the behaviour does so without reversing the edit. As the steered behaviour is not fully durable, embedded steering should be re-validated after downstream training.

Abstract:
Recent studies on world modeling for Large Language Model (LLM) agents typically formulate the learning objective as next-observation prediction. However, this objective ties supervision to what a transition happens to reveal, which may omit the dynamics most relevant to the agent’s current decision. To bridge this gap, we propose Agent-Authored World Modeling (AAWM), a training procedure that constructs supervision from the policy’s own decision needs. Specifically, at each state, the agent identifies what it needs to understand about the environment before acting. These needs drive the retrieval of relevant transition evidence across trajectories, which is then synthesized into training targets that capture decision-oriented dynamics instead of reconstructing the next observation. This aligns the training objective with the dynamics the policy needs before acting, not with the contents of the next observation. Experimental results validate the effectiveness of AAWM across multiple environments and training settings. These results show that decision-aware world-model targets provide a more effective learning signal than next-observation prediction.

Abstract:
Large Language Models (LLMs) with extended reasoning capabilities often generate verbose and redundant reasoning traces, incurring unnecessary computational cost. While existing reinforcement learning approaches address this by optimizing final response length, they neglect the quality of intermediate reasoning steps, leaving models vulnerable to reward hacking. We argue that verbosity is not merely a length problem, but a symptom of poor intermediate reasoning quality. To investigate this, we conduct an empirical study tracking the per-token predictive entropy of large reasoning models across reasoning trajectories. We find that high-quality reasoning traces exhibit two consistent properties: low uncertainty convergence and fast uncertainty descent. These findings suggest that high-quality reasoning traces are informationally dense, that is, reasoning steps contribute to reaching a low uncertainty level relative to the total reasoning length. Motivated by this, we propose InfoDensity, a reward framework for RL training that captures both properties through a single suffix-max envelope of the entropy trajectory, weighted by a length scaling term that favors achieving equivalent quality more concisely. Experiments on mathematical and general reasoning benchmarks demonstrate that InfoDensity outperforms state-of-the-art baselines on the accuracy–efficiency trade-off.

Abstract:
Multimodal Large Language Models (MLLMs) exhibit strong generalization and reasoning abilities due to large-scale multimodal pre-training. However, fine-tuning these models on downstream tasks often leads to catastrophic forgetting, where newly learned task-specific knowledge degrades previously acquired capabilities. This issue arises because gradient updates for new tasks overwrite parameters critical to prior knowledge, limiting the practical deployment of MLLMs. To address this challenge, we propose Activation-Weighted Adaptive REtention (AWARe), a fine-tuning method that mitigates catastrophic forgetting by dynamically controlling parameter updates based on activation patterns. AWARe assigns activation-based importance scores to parameters, selectively freezing those essential for preserving prior capabilities while allowing less important parameters to adapt to new tasks. Importantly, AWARe operates without modifying model architectures, ensuring compatibility with existing inference engines. Extensive experiments demonstrate that AWARe effectively preserves upstream capabilities while achieving superior downstream performance compared to existing methods. Code is available at https://github.com/kaln27/AWARe.

Abstract:
Large reasoning models (LRMs) are strong multi-step problem solvers, yet their long reasoning traces may gradually drift into unsafe compliance when exposed to harmful or adversarial prompts. Existing safety alignment methods can reduce harmful outputs; however, they often rely on broad refusal or supervision over full safe reasoning traces, which may weaken the reasoning behavior that makes LRMs useful. We introduce I ntent– R isk– D ecision (IRD), a lightweight prefix-tuning method that steers only the beginning of the reasoning trace: the model first emits a prompt-intent tag, a risk statement, and a compliance decision, and then continues under its original reasoning policy. Across three LRMs and evaluations covering harmfulness, jailbreak robustness, over-refusal, and reasoning ability, IRD substantially reduces unsafe outputs while preserving reasoning performance. In particular, IRD lowers average harmfulness to 1.2–7.4% across the evaluated backbones and keeps near-lossless reasoning performance, %within 0.2 average points of the base models, showing that targeted early steering can improve safety without supervising complete safe traces or full model fine-tuning. Code and data are available at https://github.com/ReXT9527/IRD .

Abstract:
Reinforcement learning with outcome-based objectives such as GRPO enables LLM-based agents to solve complex, long-horizon tasks, yet the reusable exploration patterns embedded in interaction trajectories are largely discarded after a single policy update. Existing experience-augmented approaches retrieve historical guidance at inference time, but they apply experiences without accounting for the policy’s evolving capability and create persistent dependencies on external retrieval. We propose EDGE (Experience-Distillation for Guided Exploration), a framework that treats retrieved experiences as temporary training-time scaffolds and progressively internalizes their benefits into the parametric policy. Concretely, EDGE partitions each rollout group into experience-conditioned and experience-free trajectories to estimate and admit only positive marginal gains without extra sampling, then distills the induced behavior into the base policy via a reverse-KL objective on its own empirical support. A co-evolutionary experience bank further synthesizes guidance from emerging failure modes and prunes obsolete entries as the policy evolves. Across embodied, web, and search-based QA tasks, EDGE improves over strong RL baselines by up to 12.5 points and remains effective without inference-time scaffolds or a proprietary reflector. The code is available at https://github.com/xvolcano02/EDGE.

Abstract:
Clinical language models can achieve strong in-hospital accuracy yet fail under deployment shifts because they exploit note-specific artifacts (e.g., templates, separators, boilerplate) that do not reflect patient state. We propose CAST (Concept-guided Artifact Suppression Tuning), an SAE-based framework for auditable clinical text classification. CAST uses Sparse Autoencoders to expose sparse, human-auditable features from intermediate Transformer activations, labels SAE latents with an LLM-assisted interpretation pipeline and ICD-10 retrieval constraints, suppresses verified artifact latents via residual subtraction during fine-tuning, and provides post-hoc perconcept attributions for auditing model decisions. On MIMIC-IV discharge-note mortality prediction, CAST improves over its corresponding fine-tuned encoder baselines and remains competitive with strong LLM baselines, while producing a feature-level audit trail of the clinical concepts that support each prediction and the artifact concepts suppressed during training.

Abstract:
As LLMs rapidly saturate existing benchmarks, automated benchmark creation using LLMs (LLM-as-a-benchmark)—where a model generates test inputs (LLM-as-a-testset) and evaluates outputs (LLM-as-an-evaluator)—has gained traction as a cheap alternative to human curation. We show that this paradigm has a fundamental problem: LLM-generated benchmarks systematically favor the model that created them. Using machine translation as our primary testbed, we find that self-bias arises from two additive sources, LLM-as-a-testset and LLM-as-an-evaluator, and their combination amplifies the effect. Crucially, even when test data is generated with explicit diversity controls, each model’s implicit stylistic tendencies produce homogeneous, model-specific outputs that inflate its own scores. Increasing source text diversity, using our proposed diversity metric, partially mitigates this bias. Self-bias is strong enough to cause each model to rank itself first, overriding the peer-consensus ordering. We confirm that the phenomenon extends to open-ended generation on the Chatbot Arena task.

Abstract:
Stochastic sampling strategies are widely adopted in large language models (LLMs) to balance output coherence and diversity. These heuristics are often inherited in Multimodal LLMs (MLLMs) without task-specific justification. However, we contend that stochastic decoding can be suboptimal for Visual Question Answering (VQA). VQA is a closed-ended task with head-heavy answer distributions where uncertainty is usually epistemic, arising from missing or ambiguous visual evidence rather than plausible continuations. In this work, we provide a theoretical formalization of the relationship between model calibration and predictive accuracy, and derive the sufficient conditions for greedy decoding optimality. Extensive experiments provide empirical evidence for the superiority of greedy decoding over stochastic sampling across multiple benchmarks. Furthermore, we propose Greedy Decoding for Reasoning Models, which outperforms both stochastic sampling and standard greedy decoding in multimodal reasoning scenarios. Overall, our results caution against naively inheriting LLMs decoding heuristics in MLLMs and demonstrate that greedy decoding can be an efficient yet strong default for VQA.

Abstract:
Probing is widely used to study which features can be decoded from language model representations. However, the common decoding probe approach has two limitations that we aim to solve with our new encoding probe approach: contributions of different features to model representations cannot be directly compared, and feature correlations can affect probing results. We present an Encoding Probe that reverses this direction and reconstructs internal representations of models using interpretable features. We evaluate this method on text and speech transformer models, using feature sets spanning acoustics, phonetics, syntax, lexicon, and speaker identity. Our results suggest that speaker-related effects vary strongly across different training objectives and datasets, while syntactic and lexical features contribute independently to reconstruction. These results show that the Encoding Probe provides a complementary perspective on interpreting model representations beyond decodability.

Abstract:
Aligned language models are evaluated for whether they state falsehoods and whether users prefer their answers. Neither axis asks whether an answer carries what a reader needs to decide. We name that failure mode Pragmatic Reticence and build an instrument to measure it. A generator answers a passage-and-question prompt in free form, never seeing the answer options, and a frozen reader, never seeing the passage, decides from that answer alone. A natural language inference judge scores whether the answer conveys the passage’s decision-critical sentence. Across 47 generators the median non-reasoning model reaches a critical-fact recall of 0.28, while every reasoning-tuned checkpoint exceeds 0.74 on the whole response. Supervised fine-tuning and preference optimization do not close that gap. For models that emit a reasoning trace, the recalled fact usually sits inside the trace rather than in the answer that follows it, so a deployment that hides the trace hides the recall. In those models a trace-free prompt restores it, so the omission is a format effect. An answer can be true, preferred, and still not enough to act on.

Abstract:
Text-to-image models can render recognizable cultural symbols, but cultural activity generation requires coordinated control over participants, actions, interactions, scenes, and culture-specific constraints. We propose CARE, an evidence-grounded visual checklist framework for controllable cultural activity image generation. Given an activity prompt, CARE builds source-traceable cultural evidence, converts it into an optimized prompt and fine-grained checklist, verifies generated images with concrete VLM-based visual checks, and uses failed checks for targeted prompt repair. We introduce CARE-1360, a dataset of 1,360 cultural activity prompts across eight countries, and evaluate generated images with a unified human protocol. Experiments show that CARE consistently improves over direct generation and retrieval-based prompt refinement baselines across cultural and visual quality dimensions. Results with both ZImage and Stable Diffusion XL suggest that evidence-grounded visual checklists provide an interpretable control mechanism that transfers across the tested generation backends.

Abstract:
Large language models (LLMs) based on encoder and decoder architectures exhibit strong performance across a range of tasks, yet systematic comparisons of how training objectives shape their behavior remain limited. Building on prior work establishing systematic comparison between decoder-encoder differences (Weller et al., 2026), we ask whether this difference shapes different linguistic bias using two production-related tasks: semantic role assignment and syntactic structure choice (actives vs. passives). Across architectures, LLMs exhibit a robust animacy-first bias, preferring to place animate entities in subject position, diverging from human bias. However, when animacy cues are removed, encoders generalize better than decoders and show a stronger preference for passive constructions. We argue that these differences reflect training objectives: encoders’ sensitivity to contextual compatibility supports flexible mapping from semantics to structure, whereas decoders’ reliance on surface frequency favors dominant patterns and disfavors infrequent structures such as passives.

Abstract:
Interlinear glossed text (IGT) is a standard notation for language documentation which is linguistically rich but laborious to produce manually. Recent automated IGT methods treat glosses as character sequences, neglecting their compositional structure. We propose CWoMP (Contrastive Word-Morpheme Pretraining), which instead treats morphemes as atomic form-meaning units with learned representations. A contrastively trained encoder aligns words-in-context with their constituent morphemes in a shared embedding space; an autoregressive decoder then generates the morpheme sequence by retrieving entries from a mutable lexicon of these embeddings. Predictions are interpretable–grounded in lexicon entries–and users can improve results at inference time by expanding the lexicon without retraining. We evaluate on diverse low-resource languages, showing that CWoMP outperforms existing methods while being significantly more efficient, with particularly strong gains in extremely low-resource settings.

Abstract:
While Large language models (LLMs) incorporate user personalization signals to improve usability and helpfulness, they increasingly shift from providing balanced, informative responses toward optimizing for user satisfaction when conditioned on personal context such as conversation history, inferred preferences, and user profiles. Specifically, we identify three emerging risks: (1) irrelevant personalization, where models reference personal information in unnecessary contexts; (2) preference narrowing, where models reinforce informational echo chambers; and (3) sycophantic bias, where models agree excessively with user opinions. As a result, models may reference personal information in contexts where it is unnecessary, inadvertently collapse response diversity, or agree excessively with user opinions. Despite the growing use of personalization in AI assistants, there has been limited systematic evaluation of its potential side effects. To bridge this gap, we propose PRISK, a dynamic evaluation framework with automated data generation and tailored metrics that uncovers systematic limitations in current LLM personalization and how personalized information shapes its responses. Our empirical analysis across 13 LLMs demonstrates that the presence of user profiles and retrieved memory consistently exacerbates biases, resulting in an average drop of 45.9% in irrelevant personalization, 41.7% in preference narrowing and 61.7% in sycophantic bias.

Abstract:
Contrastive learning is widely used for sentence embedding training, yet it provides limited control over the global geometry of the learned embedding space. Sentence embeddings often exhibit spectral concentration, where a small set of directions dominates the representation while many others remain weakly active. We present singular value regularization (SVR), a batch-level framework for shaping sentence embedding spaces by directly controlling the singular value spectrum during training. SVR enables explicit control over both the spectral floor and the number of regularized directions. Experiments show consistent improvements across encoder backbones, pooling strategies, English evaluation tasks, and multilingual settings. Our analysis further shows that the best regularization range is not full-dimensional flattening, but a controlled active-dimensional region aligned with the naturally active subspace of pretrained representations. Moreover, the strongest results are obtained with a large SVR weight, indicating that spectral shaping can provide a substantial optimization signal alongside contrastive supervision. These findings motivate a geometry-aware perspective on sentence embedding learning, in which semantic supervision and explicit control of the embedding-space geometry play complementary roles.

Abstract:
Transformer encoders are used for natural language understanding tasks, yet they remain vulnerable to gradient-based attacks in the embedding space. Existing defenses either use adversarial training or modify the inference pipeline, incurring additional computational overhead without a principled mechanism to contract the gradient flow. We propose Agatha, a transformer encoder that achieves adversarial robustness through architectural modifications. First, Agatha replaces softmax attention with an iterative Hopfield mechanism that projects representations into deep, sharply-bounded semantic attractors in its energy landscape. Second, Agatha introduces a gated residual connection that scales down the identity path, so adversarial gradients must propagate through the energy landscape where they contract layer by layer due to the landscape’s steep curvature. Agatha’s geometric structure contracts adversarial gradients layer by layer in the backward pass while preserving task-relevant information in the forward pass. Evaluations on standard NLP tasks demonstrate that Agatha reduces attack success rates by 13 percentage points over the best defended baseline at 4.37M scale, with the advantage growing to over 33 points at BERT-base scale (110M), without adversarial training. AutoAttack, transfer attacks, and discrete text-level evaluations confirm the robustness is not an artifact of gradient masking. Agatha adds no parameters, achieves 1.2x faster inference than the best robust baseline, and maintains clean performance within 2 points of the undefended model.

Abstract:
Dense retrieval systems increasingly face an asymmetry between complex instruction-like queries and relatively simple, static document collections. While stronger embedders can better understand such queries, re-embedding large corpora or fine-tuning large models is often impractical. We propose Efficient Retrieval Adapter (ERA), a query-side adapter learning framework for re-index-free retrieval adaptation. ERA first aligns the embedding spaces of a strong query embedder and a lightweight document embedder using unlabeled corpus documents, and then adapts the aligned query representation with a small number of labeled query-document pairs. Across 126 MAIR retrieval tasks from six domains, ERA improves average nDCG@10 by up to 8.2 points in symmetric settings and by more than 12 points in asymmetric settings, while using substantially fewer labels than supervised adapter training. These results show that retrieval systems can benefit from stronger query understanding without updating backbone embedders or rebuilding document indexes.

Abstract:
While large language models (LLMs) excel at deductive reasoning tasks such as math and coding, their capacity for inductive reasoning remains underexplored. This paper investigates this through the lens of personalized preference inference, where models must synthesize consistent patterns from implicit behavioral signals in users’ interaction histories. We propose AlignXplore, a model that leverages extended reasoning chains to enable explicit preference inference. Such explicit articulation enables an efficient streaming inference mechanism, allowing the model to incrementally refine preference descriptions without reprocessing historical signals. Guided by a specialized reward modeling strategy, we develop AlignXplore by combining synthetic-data cold-start training with online reinforcement learning. Extensive experiments show AlignXplore outperforms the backbone model by 15.49% on average across in- and out-of-domain benchmarks, while exhibiting robust generalization to diverse input formats and downstream models. Our analysis further establishes reward modeling best practices and reveals the emergence of human-like inductive reasoning patterns during training.

Abstract:
Existing safety evaluations for large language models overlook whether responses preserve comparable medical information across different user phrasings of the same question. To address this, we introduce the Medical Information Response Audit (MIRA), a bilingual, controlled benchmark that assesses whether LLMs provide comparable medical information across user-side language, register, and health literacy signals. MIRA contains 4,320 prompts built from 60 medically reviewed, low-risk health questions. Across five mainstream LLMs, models answered all medical questions, but responses to low health-literacy signals consistently omitted more key information, provided fewer concrete next steps, and offered less support for independent judgment. We term this pattern Differential Information Dilution (DID). A comparison with 300 real-world health queries provides preliminary evidence of rank-order validity. A knowledge-guided mitigation prompt reduces information dilution for most models, with the largest reductions in underinformative simplification observed for Claude ( ∼ 8%) and Qwen ( ∼ 6%).

Abstract:
State-of-the-art (SOTA) reasoning post-training pipelines increasingly combine supervised fine-tuning (SFT) with reinforcement learning (RL) to balance external knowledge injection and self-exploration. In this work, we identify a critical but underexplored failure mode of these SFT+RL pipelines: catastrophic forgetting. To analyze the cause, we reveal a fundamental asymmetry between SFT and RL. SFT typically induces much larger, redundant parameter updates than RL, while RL’s updates are smaller and more parsimonious. Therefore, SFT updates can easily overwrite RL’s small non-robust updates, causing the forgetting of the knowledge learned by RL. To address this issue, we propose MIFO (MItigating FOrgetting between SFT and RL), a plug-and-play framework that integrates SFT into RL while explicitly constraining SFT-induced updates. MIFO mitigates forgetting through two complementary designs: (i) Data selection, which interleaves RL and SFT by selecting challenging samples based on rollout accuracy and optimizing only high-entropy tokens for SFT; and (ii) parameter freezing, which identifies RL-critical parameters and temporarily freezes them during SFT to prevent overwrite. Beyond stability, MIFO is data- and response-efficient and remains agnostic to the underlying SFT or RL algorithm. Empirically, MIFO achieves SOTA reasoning performance using only 1.5% of the SFT data and 20.4% of the RL data required by prior SOTA methods, while producing reasoning traces up to 10 × shorter than the standard SFT → RL pipeline.

Abstract:
While Abstract Meaning Representation (AMR) has historically improved performance on a range of NLP tasks, the benefit—or lack thereof—of AMR augmentation for modern LLMs is thus far unclear. In this paper, we attempt to reproduce recent work that reported substantial downstream gains from AMR augmentation, finding that these are likely due to specific choices in the experimental settings used: using a consistent and unified protocol for hyperparameter selection, we observe that text-only baselines consistently match or exceed the performance of AMR-augmented models. To investigate this null result, we introduce a perplexity-based probe measuring the degree to which AMR provides an LLM with supplemental relational knowledge not already available to the model. We find that AMR augmentation does not help LLMs improve their understanding of relational content in the sentence, indicating that augmenting these models with AMR offers no clear benefit on downstream tasks.

Abstract:
Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiations, critically depends on continuous time constraints. We use simulated negotiations between paired agents under strict deadlines to study adaptation to real-time pressure. Agents either receive only the initial deadline or explicit remaining-time updates at each turn. Remaining-time feedback raises deal closure from 4% to 32% for GPT-5.1-chat-latest and increases offer acceptance more than sixfold. The same model achieves near-perfect closure under turn-based limits, showing that poor wall-clock performance is not simply due to insufficient negotiation competence. Across additional interface conditions, qualitative urgency cues can outperform numeric countdowns, repeated deadline reminders do not consistently reproduce their benefits, and directed time tracking can help or hurt depending on the model. Across additional negotiation scenarios and model configurations, we find real-time temporal adaptation is fragile, model-dependent, and sensitive to how temporal constraints are presented.

Abstract:
Multimodal large language models excel at passive perception but struggle with complex visual cognitive tasks requiring multi-step temporal reasoning. This degradation largely stems from the inherent ambiguity of language-based reasoning, which often fails to accurately articulate continuous visual transformations. To address this, we propose ChronoVision, a multimodal framework designed to align visual logic with latent imagery. During supervised fine-tuning, a Reconstructive Visual Head predicts the latent representation of the final transformed state, while an ROI Attention Locating module focuses the model on key visual evidence via semantic span queries. In post-training, we apply reinforcement learning with an implicit process grounding mechanism, guided by a composite reward function that evaluates outcome correctness, latent process alignment, and unsupervised visual focus. Furthermore, we introduce Vbvr-VQA, a novel dataset that evaluates temporal tracking by reformulating video reasoning into a strict image-ordering task. Experiments demonstrate that ChronoVision achieves state-of-the-art performance on Vbvr-VQA with 74.8% in-domain and 71.6% out-of-domain accuracy, alongside a strong 55.0% accuracy on IntPhys2, a highly challenging cross-domain benchmark.

Abstract:
Training reliable tool-augmented agents remains challenging due to the difficulty of credit assignment in multi-step reasoning. Process Reward Models (PRMs) provide a natural solution by assigning rewards to intermediate reasoning steps or trajectories, but standard LLM-based judges often produce inconsistent signals due to underspecified evaluation criteria. Recent rubric-based reward methods improve reward quality by making scoring criteria explicit, yet they typically rely on reference answers and construct instance-level rubrics that are difficult to reuse across examples. In this work, we propose SCRIBE (Skill-Conditioned Reward with Intermediate Behavioral Evaluation), a reinforcement learning framework for reusable subgoal-level rubric rewards. SCRIBE decomposes reasoning trajectories into intermediate subgoals, clusters semantically similar reasoning behaviors into reusable Skill Prototypes, and grounds reward evaluation in prototype-level scoring criteria. This transforms reward modeling from instance-specific open-ended evaluation into reusable process-level verification. Empirical results show that SCRIBE consistently outperforms strong RL and rubric-based reward baselines across mathematical reasoning and tool-use benchmarks; notably, it improves the AIME25 score of a Qwen3-4B model from 43.3% to 63.3%. Further analysis shows that SCRIBE reduces dependence on reference answers and provides more stable, cross-instance consistent reward judgments, offering a scalable approach to reliable process reward modeling for tool-augmented agents.

Abstract:
Despite rich safety alignment strategies, large language models (LLMs) remain highly susceptible to jailbreak attacks, which compromise safety guardrails and pose serious security risks. Existing detection methods mainly detect jailbreak status relying on jailbreak templates present in the training data. However, few studies address the more realistic and challenging zero-shot jailbreak detection setting, where no jailbreak templates are available during training. This setting better reflects real-world scenarios where new attacks continually emerge and evolve. To address this challenge, we propose a layer-wise, module-wise, and token-wise amplification framework that progressively magnifies internal feature discrepancies between benign and jailbreak prompts. We uncover safety-relevant layers, identify specific modules that inherently encode zero-shot discriminative signals, and localize informative safety tokens. Building upon these insights, we introduce ALERT (Amplification-based Jailbreak Detector), an efficient and effective zero-shot jailbreak detector that introduces two independent yet complementary classifiers on amplified representations. Extensive experiments on three safety benchmarks demonstrate that ALERT achieves consistently strong zero-shot detection performance. Specifically, (i) across all datasets and attack strategies, ALERT reliably ranks among the top two methods, and (ii) it outperforms the second-best baseline by at least 10% in average Accuracy and F1-score, and sometimes by up to 40%.

Abstract:
Ensuring the alignment of Large Language Models (LLMs) with diverse cultural values is critical for their safe global deployment. However, existing alignment benchmarks are predominantly English-centric or confined to simple discriminative tasks. Moreover, they often fail to effectively evaluate model robustness against complex, adversarial inductions within specific cultural contexts. To address these limitations, we propose ABench, a fine-grained, handcrafted benchmark specifically tailored to taxonomized values in Chinese. Grounded in a rigorous hierarchical taxonomy comprising 158 derived value subcategories, ABench consists of 2,895 open-ended questions. These questions employ techniques such as role-playing and logical traps to probe the stability of models’ stances in adversarially induced contexts. Furthermore, we propose a weighted automated evaluation framework that utilizes question-specific scoring criteria for nuanced assessment. Extensive experiments on 16 mainstream LLMs reveal that most models exhibit significant vulnerabilities when subjected to sophisticated adversary attacks on values (scoring below 60). This underscores the need for more robust, culturally specific alignment strategies. Our data will be publicly released to promote further research on taxonomized value alignment.

Abstract:
The rapid progress of reasoning and agentic large language models (LLMs) has increased the demand for long-context inference, but self-attention (SA) scales quadratically with context length. To address this, we propose SWARR (Sliding-Window Attention with Reinforced Adaptation for Math Reasoning), a practical recipe for adapting SWA models to mathematical reasoning. SWARR has two stages: (1) efficient conversion from a pretrained SA model to SWA with supervised fine-tuning (SFT), which avoids pretraining a new base model, and (2) architecture-aware policy adaptation with reinforcement learning (RL). We find that SWA still underperforms SA after SFT, and we hypothesize that this gap is caused in part by a data-architecture mismatch: most SFT data are prepared for SA models and may contain long-range dependencies that are difficult for SWA to model. Because RL is architecture-aware and optimizes self-generated trajectories under the SWA constraint, it can adapt training trajectories to better match the SWA architecture. Experiments on mathematical reasoning benchmarks show that this architecture-aware RL substantially narrows the gap between SWA and SA, recovering much of the accuracy lost during SWA conversion while preserving the efficiency benefits of linear-complexity attention.

Abstract:
Multi-hop QA benchmarks often reward Large Language Models (LLMs) for spurious correctness, where models reach correct answers through invalid intermediate reasoning. We propose SAFE, an LLM-as-verifier framework for evidence-grounded multi-hop QA. Rather than judging only the final answer after generation, SAFE verifies reasoning during generation by checking intermediate steps against the provided passages and previous reasoning trajectory. To make this process checkable, SAFE decomposes reasoning into atomic, evidence-grounded units represented with Knowledge Graph (KG) triples. At train-time, SAFE verifies benchmark supervision under KG-grounded constraints and constructs reliable verifier training data. At inference-time, an external verifier checks each generated step, identifies invalid reasoning, and provides correction feedback before errors propagate. Across three multi-hop QA benchmarks, SAFE improves accuracy by 8.8 pp on average. These results show that evidence-grounded multi-hop QA benefits from shifting LLM-based evaluation from post-hoc answer judgment to stepwise reasoning verification.

Abstract:
Post-training quantization methods universally minimize element-wise reconstruction error, yet we show this is the wrong objective. A systematic correlation study on LLaMA models reveals that the representation kernel (the matrix of pairwise cosine similarities among calibration inputs) is a substantially stronger proxy for downstream task performance than weight MSE or output MSE. Building on this finding, we propose KernelQuant, a training-free PTQ framework comprising: KPQ, which selects weight-quantization scales via kernel-MSE grid search followed by least-squares error compensation; model-level Hadamard rotation; and KPC, a kernel-preserving per-channel activation clipping scheme. KernelQuant introduces zero inference overhead beyond QuaRot and calibrates in minutes rather than hours. On LLaMA 3B–70B, KernelQuant achieves state-of-the-art W4A16 perplexity and competitive W4A4 results. On reasoning benchmarks (MMLU), where 4-bit quantization causes a “reasoning cliff,” KernelQuant retains the highest fraction of FP16 accuracy among all evaluated baselines, both rotation-based (QuaRot, SpinQuant, FlatQuant) and element-wise, reversing the perplexity-based ranking and supporting the view that reasoning depends on the geometric relations the kernel preserves.

Abstract:
Fine-tuning safety aligned large language models (LLMs) on downstream data improves adaptation but may erode learned safety behavior. Existing methods use fixed safety examples, global constraints, or one-sided task filtering. Our diagnostics show task updates expose different safety constraints, motivating joint selection of relevant references and compatible task samples. We propose DualSelect , a coupled framework for task and reference selection that refreshes task conditioned safety references before filtering whole task samples compatible with the induced reference direction. Under a minimax view, DualSelect selects safety references with high preservation loss and task conflict, together with compatible task samples, through entropy-regularized scoring surrogates, lazy reference refresh, and gradient correction. Across the evaluated English instruction-following and mathematical-reasoning settings on 1B–8B LLMs, DualSelect better preserves safety while maintaining task utility; using the REDORCA judge, it improves Safety Avg. over the strongest baseline by at least 5.10 points and remains highest in Safety Avg. across judges with moderate overhead. This view extends to retention focused continual learning. Code at: https://github.com/ChnIRuI/DualSelect.

Abstract:
Diffusion Language Models (DLMs) are emerging as competitive alternatives to auto-regressive models, but their flexible, any-order decoding may introduce new trustworthiness risks. We present TrustDLM, a comprehensive benchmark for evaluating DLM trustworthiness across safety, privacy, and fairness with multiple categories of static post contexts. Our results show that while DLMs are generally trustworthy with user prompts alone, their alignment degrades noticeably when malicious post-contexts are attached to masked responses. We further find that decoding order and generation length affect evaluation outcomes, while longer contexts do not necessarily induce stronger effects. Finally, we introduce TrustDLM-Auto, the first automatic framework for identifying vulnerable DLM configurations, which reveals substantial weaknesses across models and dimensions. Overall, TrustDLM provides the first trustworthiness evaluation framework specifically designed for DLMs, with practical applicability further validated on closed-source DLMs. Our work will help the community enhance the trustworthiness of DLMs. Our code is available at https://github.com/PKU-ML/TrustDLM.

Abstract:
In real-world task-oriented dialogue (TOD), users often express their intents through both text and images, with images providing essential cues for fine-grained requirements. Addressing such scenarios demands effective cross-modal semantic modeling and precise knowledge retrieval. However, existing TOD methods primarily focus on text and assume one-to-one attribute mappings, failing to reflect the multimodal nature of real-world information. To overcome these limitations, this paper proposes an end-to-end vision-guided retrieval-generation framework named INSIGHT. It introduces a progressive dynamic fusion mechanism that performs layer-wise cross-modal interaction to achieve robust multimodal dialogue understanding. In addition, a graph-structured entity modeling strategy explicitly models one-to-many image attribute mappings through distributed latent semantic learning, thereby preserving visual diversity. Finally, knowledge retrieval and response generation are jointly optimized in an end-to-end manner. To evaluate real-world scenarios, we construct VISTA, a new multimodal TOD dataset that features image-driven multi-turn dialogues and supports one-to-many entity attribute mappings. Experimental results on two multimodal datasets demonstrate that INSIGHT consistently outperforms existing methods across various backbone models.

Abstract:
We introduce ScreenHaystack, a dynamic needle-in-a-haystack benchmark for evaluating spatial reliability in GUI grounding. Instead of testing each target at a fixed position, ScreenHaystack systematically relocates controlled target icons across high-resolution GUI backgrounds and measures whether models can localize them consistently. Using this benchmark, we find that leading GUI grounding models, including Qwen3-VL, UI-TARS, GTA, and UI-Venus, exhibit blind zones: spatial regions where grounding accuracy drops sharply despite fixed target appearance and instruction. These blind zones transfer to unseen ScreenSpot-Pro examples: targets inside blind zones are consistently harder to ground, with Qwen3-VL dropping by 16.1 percentage points, and controlled relocation shows that moving targets into blind zones decreases accuracy while moving them out improves accuracy. We further show through controlled synthetic experiments that uneven spatial coverage in training data can induce such blind zones. Therefore, we propose a simple strategy, blind-zone-oriented augmentation, which adds supervision in blind zones and improves ScreenSpot-Pro accuracy over both original and random augmented Click-100k fine-tuning.

Abstract:
Multilingual LLMs show stereotype-related behavior that varies across languages, but behavioral scores do not show where the relevant information is represented or how it affects the output. We compare linear probing, attribution patching, sparse autoencoders (SAEs) and feature ablation in Llama-3.1-8B, Qwen3-8B, and Gemma-2-9B. Probe performance peaks substantially earlier than attribution in all three models, with a separation of 36-53% of model depth. Retained Llama-Scope features often match the social category on which they were selected and form recurring semantic families, but their lexical alignment and ablation effects vary across SAE suites. Only 6-18% of evaluated residual-stream features have language-agnostic effects under our criterion, and none are category-agnostic. Language-agnostic features have larger mean ablation effects in Llama-Scope, but this pattern does not repeat in the other SAE suites. Decodability, output influence, and cross-lingual ablation effects therefore need to be measured separately.

Abstract:
This study presents a difficulty-annotated corpus of Japanese newspaper articles. The annotation in our corpus has two key features. First, it is hierarchical, with difficulty levels labeled at the word, sentence, and document levels, respectively. Second, it is subjective, with 99 readers from diverse backgrounds labeling the difficulty levels based on their own perceptions. Detailed analysis of our corpus revealed correlations in difficulty levels across linguistic units, such as between words and sentences, and sentences and documents. We also found that text difficulty varies depending on reader attributes such as age and topics of interest. Furthermore, we trained Transformer encoders on this corpus and developed a personalized difficulty estimation model that takes into account both linguistic and attribute information. Experimental results demonstrate that incorporating reader attributes improves the performance of difficulty estimation and that multi-task learning across language units is beneficial. Our difficulty annotations are publicly available.

Abstract:
Humans increasingly turn to Language Models (LMs) in ways that shape beliefs and drive decisions, including discussing, rewriting, and summarizing information from scientific articles, news, and medical reports. However, in these domains, where it often matters how confidently a claim is expressed, little is known about whether LMs faithfully preserve the degree of confidence. In this work, we investigate _certainty distortion_ in LMs, defined as meaningful changes in expressed certainty during transformations intended to preserve meaning. We propose an LM-based evaluation metric that is consistent with population-level judgments of certainty. Using this metric, we characterize certainty distortion across different sizes and families of models in the context of scientific and medical communication tasks. Our results show that certainty distortion affects up to 75% of LM outputs and is systematically asymmetric in rewriting tasks with most LMs being 1.5-2x more likely to increase the expressed certainty than to decrease it. These effects can compound over repeated paraphrasing: in the medical domain, Claude Haiku (4.5) increases certainty in 20% of examples after a single iteration, increasing to 40% after five iterations. Prompt-based interventions reduce overall certainty distortion but do not eliminate it. Together, these findings reveal a general bias toward inflating expressed certainty, with direct implications for users who rely on LMs in high-stakes domains.

Abstract:
Lexical Complexity Prediction (LCP) in real-world scenarios, particularly in language education, is often limited by the trade-off between overly simplified general models and unscalable personalized ones. To address this, we introduce LCP for Leveled Reading (LCP-LR), a task aiming to operationalize group-level personalization by identifying complex words relative to specific proficiency levels, thereby balancing pedagogical guidance and scalability. We construct the first large-scale benchmark dataset for this task, comprising 100,917 instance annotations across 1,401 leveled texts. Furthermore, we propose Dual-Prior Adversarial Learning (DPAL), a framework that incorporates word- and text-level difficulty priors alongside adversarial debiasing to mitigate the model’s over-reliance on simplistic absolute difficulty shortcuts. Experimental results show that DPAL significantly outperforms both conventional pre-trained baselines and large language models, improving the Macro of the best baseline (XLNet) from 74.19% to 77.81%. Our evaluations also reveal that large language models underperform in zero- and few-shot settings without fine-tuning or Chain-of-Thought prompting. Overall, this work promotes the practical deployment of LCP and lays a crucial technical foundation for dynamic text adaptation within leveled reading.

Abstract:
Automatically generating issue-reproducing tests is essential for debugging and regression prevention. Recent advances show that large language models can effectively perform this task, but their reliance on extremely large parameter scales limits practical deployment. In this paper, we propose SLIT, an execution-grounded framework that enables small language models to generate effective issue-reproducing tests. SLIT formulates issue reproduction as an execution-guided exploration problem and optimizes policies with reinforcement learning over execution-based rewards. To improve inference-time exploration, SLIT further introduces diversity-augmented test construction and a learned scoring model to select effective reproducing tests. Experimental results on SWT-Bench-Verified indicate that the 14B-parameter model achieves a resolve rate of 46.2%, achieving competitive performance against prior approaches that rely on larger models. These results demonstrate the effectiveness of SLIT and highlight the viability of execution-grounded small language models for issue-reproducing test generation.

Abstract:
Agent skills occupy a privileged position in the agent workflow, as agents are expected to implicitly follow and execute them, rendering third-party skills a vulnerable attack surface. Existing studies have revealed unsafe agent behaviors induced by skill-based attacks, but they primarily evaluate poisoned skills within a single task execution and enumerate harms through ad hoc risk lists. To bridge these gaps, we introduce SkillHarm, a benchmark for evaluating skill-based attacks across the skill-use lifecycle, paired with a systematic taxonomy of skill-relevant risks. SkillHarm evaluates two attack scenarios: Fixed-Payload Poisoning (FPP), where a fixed poisoned skill package directly compromises any task session that invokes it, and Self-Mutating Poisoning (SMP), where an initially benign execution silently mutates persistent skill content, deferring harm until a subsequent reuse. It further defines 12 risk types based on the agent workflow component targeted by the harm: data pipelines, system environments, and agent autonomy. To instantiate these attacks at scale, we build an automated construction pipeline using coding agents driven by natural-language harnesses. The resulting benchmark contains 879 attack samples across 71 skills. Experiments show that current agents remain vulnerable with attack success rates up to 86.3% in FPP and 69.3% in SMP. Our analysis further reveals a latent risk: many apparent attack failures stem from the agent failing to engage with the poisoned file rather than genuine resistance, and current defenses still fail to reliably mitigate the threat.

Abstract:
While Vision-Language Models (VLMs) have excelled in 2D understanding and shown promise in 3D domains, their spatial reasoning is often hindered by incomplete visual observations. Since active physical exploration to gather missing information is usually impractical, leveraging visual generation to imagine the unobserved views provides a more feasible and efficient solution. We propose AIVE, an efficient active imagined-view exploration framework for spatial Visual Question Answering (VQA) that learns to directly select informative exploration actions, thereby amortizing costly test-time search and scoring. Departing from prior approaches that suffer from substantial inference latency, AIVE employs two key components: a VLM Planner generates targeted exploration strategies, while a generative World Model Dreamer synthesizes 3D-consistent future views accordingly. To facilitate training and evaluation, we introduce SpaThor, a benchmark comprising expert trajectories and action-conditioned visual transitions. Extensive experiments demonstrate that AIVE achieves state-of-the-art performance on challenging spatial reasoning tasks, outperforming prior baselines by an average margin of 5.2%. Notably, it delivers up to a 4.8 × inference speedup over imagination-based methods, highlighting its potential for embodied spatial reasoning.

Abstract:
Real-world fact-checking faces the challenge of temporal dynamics, where misinformation manifests as evolving evidences rather than static claims. Verifying such claims necessitates the synthesis of multi-hop fragmented evidence dispersed across the timeline. However, existing benchmarks largely neglect this complexity, limiting evaluation to static settings that fail to capture real-world demands. To address this, we pioneer MTIMEFACTCHECK, the first dynamic benchmark to rigorously evaluate multi-hop temporal fact-checking in realistic, evolving contexts. Utilizing MTIMEFACTCHECK, we curate the MTimeFactCorpus dataset to benchmark sixteen LLM baselines, scrutinizing their capabilities and critical limitations. Building on these findings, we tentatively propose a novel method MTIMEFACTCHECK to augment LLM efficacy in multi-hop temporal fact-checking, offering guidance for future research. Our data and code are available at https://github.com/SCUNLP/MTimeFact .

Abstract:
In-Context Reinforcement Learning (ICRL) enables Large Language Models (LLMs) to learn online from external rewards directly within the context window. However, a central challenge in ICRL is reward estimation, as models typically lack access to ground-truths during inference. To address this limitation, we propose Test-Time Rethinking for In-Context Reinforcement Learning (TR-ICRL), a novel ICRL prompting framework that leverages Test-Time Scaling to generate reward-like feedback during inference. TR-ICRL operates by first retrieving the most relevant instances from an unlabeled evaluation set for a given query. During each ICRL iteration, LLM generates a set of candidate answers for every retrieved instance. Next, a pseudo-label is derived from this set through majority voting. In the end, this self-improvement context is integrated with the original query to form a comprehensive prompt, with the answer determining through a final round of majority voting. TR-ICRL is evaluated on mainstream reasoning and knowledge-intensive tasks, where it demonstrates significant performance gains. Remarkably, TR-ICRL improves Qwen2.5-7B by 21.23% on average on MedQA and even 137.59% on AIME2024. Extensive ablation studies and analyses further validate the effectiveness and robustness of our approach. Our code is available at https://github.com/pangpang-xuan/TR_ICRL.

Abstract:
Failure-based textual knowledge distillation aims to discover gaps in a model’s knowledge by examining its task errors. The distilled knowledge can be useful for the reasoning of both this model (“source model”) and other models. However, this transfer of knowledge may not be stable. We define a rule atom to be a standalone rule injected into a model’s textual input at inference time. A rule atom can encode transferable task knowledge or model-specific reasoning patches that can confuse other models. Also, the injected rule atoms can be misapplied to unrelated cases, causing the model to incorrectly flip its answer based on irrelevant information. Building on a pipeline that distills training examples into task-specific cheat sheets that aid model reasoning, we examine when failure-derived rules can improve these cheat sheets. Our early experiment shows rule distillation from a single model’s failures underperforms the baseline cheat sheet on non-source model families. This motivates Robust Failure, Conservative Repair (RFCR), a textual distillation procedure that derives rules from failures shared across models, sharpens their application boundaries using boundary cases, and abstains when no useful rule is found. On a 400-item BIG-Bench Hard task set, RFCR improves the baseline cheat sheets from 68.50% to 71.25% (+2.75 pp; 95% CI [+1.25,+4.50]) without performance degradation on previously correct cases. Ablations and cross-model diagnostics support that accuracy gains come from both new knowledge injection and strict rule-application control.

Abstract:
Model editing updates factual knowledge in large language models (LLMs) without retraining, but simultaneous large-batch edits can degrade generalization and locality as edit interactions accumulate. We study this problem through a multi-scale analysis of feed-forward network (FFN) and multi-head self-attention (MHSA) geometry. Weak FFN–MHSA token-level alignment can coexist with non-trivial distributional coupling, while Jacobian-inferred FFN local-response geometry provides a distinct proxy for locally supported response directions. Based on this distinction, we propose PRISM , which replaces unconstrained residual-target fitting with controlled residual correction within edit-conditioned, Jacobian-inferred local-response subspaces. On the standard CounterFact large-batch benchmark, improves the baseline’s Score by 1.3–1.7 points and Generalization by 1.7–1.8 points at 10k edits, with larger Generalization gains in beyond-10k scaling tests at 20k edits while maintaining competitive Specificity.

Abstract:
Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical speedup is constrained by the trade-off between draft quality and drafting cost: autoregressive drafters model causal dependencies among draft tokens but incur sequential overhead, while parallel drafters reduce drafting cost but weaken intra-block dependency modeling. In this paper, we propose Domino, a speculative decoding framework that decouples causal dependency modeling from expensive autoregressive draft execution. Domino first uses a parallel draft backbone to produce preliminary draft distributions for the entire block, and then applies a lightweight Domino head to refine them with prefix-dependent causal information. To stabilize teacher-forced causal encoding, we further introduce a base-anchored training curriculum that first strengthens the parallel backbone and then gradually shifts optimization toward the causally corrected final distribution. Experiments on Qwen3 models show that Domino achieves up to (5.49×) end-to-end speedup under the Transformers backend and up to (5.8×) throughput speedup under SGLang serving. Code and checkpoints are provided in the supplementary material.

Abstract:
Memory-augmented Large Language Models frequently fail to maintain long-term context due to opaque evaluation mechanisms that obscure the root causes of failure. To address this, we introduce MemEval, a stage-wise attribution framework that approximates where failures arise across extraction, update, retrieval, and utilization. On LoCoMo, our automated meta-evaluator achieves 81.62% stage-level consistency with human judgment. Our analysis reveals three critical findings: (1) Extraction is the universal bottleneck, causing information loss at the entry point; (2) Retrieval and utilization failures become prominent in complex reasoning and temporal tasks, even when relevant memories are stored or retrieved; and (3) Scaling base model capacity does not guarantee more reliable memory behavior, as higher-capacity backbones may introduce early-stage misalignment that propagates downstream. MemEval provides a practical diagnostic aid for trend-level, stage-aware analysis of memory systems. Code is available at: https://anonymous.4open.science/r/MemEval-8D41.

Abstract:
Reinforcement learning (RL) for search agents typically relies on outcome rewards. However, it often fails to achieve effective credit assignment, due to the unclear value of intermediate steps. It is hard to separate their contributions from the final result. In this paper, we propose a dense process supervision method based on fact utility estimation, which models the reasoning process as the accumulation of discrete evidence facts. We first extract structured facts from raw observations and organize them into an explicit fact store. To support credit assignment, we then cluster semantically equivalent facts and infer the posterior utility of each fact cluster using Bayesian estimation over group rollouts. Finally, we convert the estimated fact utilities into dense step-level rewards to guide RL training. Experiments on seven single-hop and multi-hop QA benchmarks show that our method consistently outperforms existing baselines. Ablation studies validate clear relative improvements on multi-hop QA compared to outcome reward-only training.

Abstract:
Distributed agents need to exchange textual evidence across organizational boundaries, but these exchanges can expose two types of private information. One is carried by explicit identifiers. The other appears as source signatures spread across formatting, vocabulary, and syntax. Existing PII detection and masking methods mainly handle localized identifiers, leaving these broader source signals largely intact. Even masking 19.2% of tokens reduces TF-IDF stylometric attribution by only 18.6%. To address this problem, we propose DiSan, which treats text sanitization as a representation learning problem. A two-stream encoder maps text into orthogonal role and style subspaces. The role subspace captures task semantics shared across sources, while the local style subspace captures variation associated with source identity. The style representation is never transmitted directly. Prototype alignment supports joint training without centralizing raw text, while adversarial regularization limits source leakage in shared representations and prototypes. We test leakage in sanitized text, learned representations, and uploaded prototypes. On a distributed multi-agent RAG benchmark, DiSan reduces PII exposure in answers by 20× while preserving 83% answer faithfulness. On Enron emails, it reduces TF-IDF stylometric attribution by 73.2%, compared with 18.6% for the strongest masking baseline. Embedding and prototype attribution are also close to random. Together, these results show that DiSan reduces explicit and distributional leakage while retaining task utility.

Abstract:
Diffusion language models (DLMs) have emerged as a promising alternative to the auto-regressive paradigm. With bidirectional attention and any-order generation, DLMs naturally fit infilling tasks, which require generating a middle span conditioned on both the prefix and the suffix. However, infilling is sensitive to the length of the span, while DLMs require the length to be fixed before generation. Although prior studies extend DLMs to dynamic lengths, they still suffer from two limitations. (i) Sensitivity to initial length. These methods require a preset length to initialize the search and are highly sensitive to this initial length, often yielding suboptimal results. (ii) Inference inefficiency. They either insert length-changing operations during generation or repeatedly search for an appropriate length using multi-step denoising confidence, both of which introduce substantial additional forward passes and computational cost. Therefore, we propose PILL (Probing-based InfiLing with preset-Length-free decoding), an efficient infilling method for DLMs that requires no preset initial length and drastically reduces the number of forward passes compared to baselines. Experiments show that, across five DLMs spanning different families, architectures, and training recipes on eight infilling benchmarks, PILL improves over the strongest baseline by +4.8 average pass rate on code and +6.0 BLEU-2 on text, while running 1.82× faster than that baseline.

Abstract:
LLM-guided evolutionary search (Evolve systems) has reached state-of-the-art results on mathematical and combinatorial tasks, yet most existing systems report only the best of many runs and leave the run-to-run distribution undocumented. We ask how a fixed budget of LLM calls should be allocated, and how reliably a single run reaches the reported numbers. Sweeping the depth–breadth grid over five models and three tasks, we identify two empirical regularities: a fitness–compute envelope along which capability ordering largely collapses on effective FLOPs, and a bilinear depth–breadth fit with task-specific interaction; both are gated by model–task capability. Motivated by these regularities, we propose BaSE (Bandit-based Self-Evolving), a multi-armed bandit that allocates LLM calls across parallel trajectories. Without changing the model, prompt, or evaluator, BaSE improves mean fitness by 12.3% over the strongest island-protocol baseline across 8 (model, task) cells, with the largest gains on high-variance settings: a reliability gain from allocation alone.

Abstract:
Large Language Models (LLMs) are increasingly used as conversational search engines for academic literature, yet whether they judge papers on content or on authority signals has not been tested causally. We investigate authority bias : systematic preference for papers based on author prestige, venue, and citations rather than content. Holding title and abstract constant, we vary authority metadata across three counterfactual conditions (original, flipped, boosted) over eight LLMs (five open-weight and three frontier closed-weight) in an in-context, single-turn, top-1 recommendation setting. Our experiments show that authority bias is substantial and directional, varies markedly across models, and is only partially addressable through prompt-level debiasing. We further document a say-do gap: debiasing instructions suppress authority mentions far faster than authority-driven flips, so surface auditing systematically underestimates behavioral bias. Code: https://github.com/jinaduuthman/Authority-Bias-In-Conversational-Search-Engine Data: https://huggingface.co/datasets/uthmanjinadu/authority-bias-paper-recommendation

Abstract:
The widespread deployment of Large Language Models (LLMs) has necessitated robust intellectual property protection. Backdoor-based LLM fingerprinting has emerged as a promising solution for this challenge. To assess the robustness of fingerprinting techniques, various adversarial attacks have been developed. However, existing fingerprint attacks often fail to simultaneously achieve strong effectiveness, harmlessness, and generalization across different LLMs and fingerprinting methods. To address this challenge, we examine the process of generating fingerprint responses and identify an intrinsic trait: fingerprint responses exhibit abnormally high generation confidence and strong robustness under perturbations compared to benign responses. Building on this trait, we propose RIPPLE , a fingerprint attack framework that identifies fingerprint trigger-response pairs by jointly evaluating original confidence and confidence robustness under hidden state perturbations. Extensive experiments on multiple LLMs and fingerprinting methods demonstrate that RIPPLE consistently outperforms existing attack methods, achieving a superior trade-off between attack effectiveness and harmlessness while maintaining strong generalization.

Abstract:
Reward models are central to large language model (LLM) alignment, but they remain vulnerable to reward hacking. To evaluate reward-model robustness, we introduce RewardHackBench containing 13 reward-hacking patterns covering real life high-stakes domains and general settings, and we find severe failures on specific subcategories across eight reward models. To mitigate these failures, we propose HARVE, a training-free reward-head editing method for scalar reward models. Instead of fine-tuning the reward model, HARVE identifies a multi-directional hacking subspace from residual-stream directions associated with selected hacking subcategories, and removes the component of the reward-head vector aligned with that subspace. This directly reduces the reward head’s sensitivity to hacking-related features using only a small set of contrastive gold-hacked examples, without gradient updates or fine-tuning. Comprehensive experiments across eight reward models indicates that improves hacking robustness, outperforms fine-tuning baselines, and preserves reward-models’ general capability. Further analyses show that combining multiple target-specific residual-stream directions is more effective than relying on a single surface-level feature.

Abstract:
Automatic evaluation of image and video captioning is essential for benchmarking multimodal systems, although standard evaluation metrics show limited alignment with human judgments. Recent approaches using large language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved alignment with human judgments but still suffer from a mismatch between large-vocabulary language modeling and evaluation over a small label set. To address this, we propose Rigel, an automatic evaluation metric for image and video captioning, based on self-distilled score adaptation. The metric employs an evaluation-specific scoring head distilled from a frozen LLM, which captures judgment signals in a task-aligned space without relying on large-vocabulary token sets. We then refine the LLM backbone with human judgment data. To train Rigel, we constructed the Vid-Lepus dataset, which contains 3,338 video clips, 33,380 reference captions, and 5,637 candidate captions. Experiments on multiple benchmarks show that Rigel outperforms state-of-the-art metrics, achieving over 10-point improvements on ActivityNet-Fact in the reference-free setting.

Abstract:
Large Audio Language Models (LALMs) have demonstrated strong performance across a wide range of audio tasks. As they are increasingly deployed in real-world applications, ensuring their safety alignment has become more important. Although refusal mechanisms serve as a key safeguard by preventing LALMs from responding to harmful requests, they can also lead to over-refusal, where models incorrectly reject benign queries. This issue is especially challenging in the audio domain because speech that appears harmful in isolation may become benign when interpreted together with the surrounding acoustic context, such as background sounds. To study this problem, we introduce AOR-Bench (Audio Over-Refusal Benchmark), the first benchmark for over-refusal specifically designed for LALMs. AOR-Bench contains 3,000 pseudo-harmful audio samples across six scenario categories. Evaluating 12 representative LALMs from six major model families, we find that over-refusal is widespread and uncover several important patterns in their safety judgments. As a preliminary effort to mitigate this issue, we further explore two lightweight strategies (e.g., Chain-of-Thought prompting and activation steering) to reduce over-refusal.

Abstract:
While vision-language-action (VLA) models excel at robotic manipulation, they frequently fall short when interpreting physical-property language, where adjectives such as fragile or coarse-grained specify constraints that are often visually ambiguous and should modulate behavior primarily during active contact. We therefore propose Contact-Local Semantic Actuation (CLSA), an architecture that extracts semantically structured tactile representations through a language-guided bottleneck and injects them through a contact-gated residual path, confining physical adjustments to contact-relevant stages without perturbing stable nominal motion. To probe instruction-conditioned behavior under matched contexts, we introduce the Action-Property-Constraint (APC) hierarchy. On the language side, APC contrasts action-only (L0), property-aware (L1), and constraint-explicit (L2) prompts; on the trajectory side, offline evaluation uses the corresponding demonstrated behavior strata defined by the data protocol. Across matched offline comparisons, a 140-trial comparative real-robot benchmark, and a separate 240-rollout APC sweep on four tactile-sensitive tasks, CLSA improves instruction sensitivity while maintaining action accuracy. We also report visually ambiguous black-bag trials within the comparative benchmark as a diagnostic of touch-based material recognition after contact.

Abstract:
Recent advances in Large Language Models (LLMs) have shifted toward latent reasoning, where intermediate processes are represented as continuous vectors rather than text tokens. While this paradigm reduces inference cost, existing approaches often suffer from instability on complex, out-of-distribution tasks where robust reasoning is critical. To address this, we propose Bayesian Latent Thought Optimization (BLTO), a training-free framework that enhances frozen LLMs entirely at test time. BLTO treats latent thought tokens as stochastic variables and formulates reasoning as a Bayesian inference problem. We introduce a variational posterior over latent vectors and optimize it by maximizing a reward-weighted objective that balances reasoning quality with KL regularization toward a standard Gaussian prior. The reward is derived from the model’s predictive consistency via output entropy, enabling fully self-supervised optimization without external supervision. BLTO employs reparameterized sampling and reward-weighted updates within a variational inference framework, together with an adaptive controller that dynamically determines inference steps based on reward improvement and uncertainty convergence. Extensive experiments on challenging reasoning benchmarks demonstrate that BLTO consistently improves frozen LLM performance.

Abstract:
On-policy reinforcement learning has become the dominant paradigm for reasoning alignment in large language models, yet its sparse, outcome-level rewards make token-level credit assignment notoriously difficult. On-Policy Distillation (OPD) alleviates this by introducing dense, token-level KL supervision from a teacher model, but typically applies this supervision uniformly across all rollouts, ignoring fundamental differences in signal quality. We propose Signal-Calibrated On-Policy Distillation Enhancement (SCOPE), a dual-path adaptive training framework that routes on-policy rollouts by correctness into two complementary supervision paths. For incorrect trajectories, SCOPE performs teacher-perplexity-weighted KL distillation to prioritize instances where the teacher demonstrates genuine corrective capability, while down-weighting unreliable guidance. For correct trajectories, it applies student-perplexity-weighted Maximum Likelihood Estimation to concentrate reinforcement on low-confidence samples at the capability boundary rather than over-reinforcing already mastered ones. Both paths employ a group-level normalization to adaptively calibrate weight distributions, accounting for the intrinsic difficulty variance across prompts. Extensive experiments on six mathematical reasoning benchmarks show that SCOPE achieves average relative improvements of 11.42% in Avg@32 and 7.30% in Pass@32 over competitive baselines, with extended experiments demonstrating its broader applicability. Code is available at [https://github.com/machine981/SCOPE](https://github.com/machine981/SCOPE).

Abstract:
Large language models remain vulnerable to jailbreaks that hide harmful intent from safety screening. A recent family of highly effective attacks exploits this weakness by splitting harmful keywords into recoverable fragments; we refer to this family as Token Fragmentation Jailbreaks. The success of these attacks stems from a mismatch between screening and generation: safety screening may no longer observe complete harmful keywords, while the target model can reconstruct them during generation. We show that instructions containing fragmented words leave a measurable attention footprint inside the model. Attention concentrates among the fragmented keyword pieces, while links from those pieces to complete words are weak. Based on this finding, we introduce Calibrated Attention-Void Detection (CAVD), a training-free detector that measures this footprint with length-conditioned calibration. We then combine guard models with CAVD in CAVD-Guard. When CAVD flags an instruction as unreliable for original-view safety screening, CAVD-Guard maps it to multiple canonical views and applies guard-based classification to the recovered views. Extensive experiments show that CAVD-Guard substantially reduces attack success and improves guard blocking while maintaining low false-positive rates on benign inputs. Code is available at https://github.com/serendipity098/CAVD-Guard.

Abstract:
Reinforcement learning (RL) is often credited with improving reasoning at the expense of factual knowledge. We instead find that reasoning models outperform their instruction-tuned versions on factual recall by accessing existing parametric knowledge more effectively. Across five model families, structured prompting—which explicitly guides models through hierarchical traversal—recovers most of this gap, suggesting that much of the missing knowledge is latent rather than absent. Controlled RL experiments further support this: training on unseen, non-extractable facts improves recall of held-out, frequent but previously inaccessible facts, ruling out simple data exposure. Decomposing the training objective further attributes this gain to iterated on-policy exploration. The same mechanism appears behaviorally and internally: the reasoning advantage grows with retrieval depth, while layerwise analysis finds similar factual representations but divergent query representations. Distilled models, in contrast, often imitate self-correction without acquiring the exploration needed for navigation. Together, these findings suggest that improving factual recall in LLMs depends not only on expanding what models know but also on teaching them to navigate it—motivating future post-training methods that optimize traversal.

Abstract:
As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical. A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives. Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces. However, under optimization pressure, models are incentivized to conceal deceptive reasoning, rendering semantic supervision fundamentally unreliable. Grounded in cognitive psychology, we hypothesize that a deceptive LLM maintains a stable internal belief in its CoT while its external response remains fragile under perturbation. We term this phenomenon stability asymmetry and quantify it by measuring the contrast between internal CoT stability and external response stability under perturbation. Building on this structural signature, we propose the Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning. Unlike CoT monitoring, SAR targets the statistical structure of model outputs, rendering it robust to semantic concealment. Extensive experiments confirm that stability asymmetry reliably identifies deceptive behavior, and that SAR effectively suppresses intrinsic deception without degrading general model capability.

Abstract:
Large vision-language models (LVLMs) often hallucinate due to over-reliance on language priors. Existing methods implicitly assume that visual tokens maintain consistent semantics across layers and thus indiscriminately inject shallow-layer visual information into deeper layers. We identify a new phenomenon, termed correlation flip, where a visual token positively correlated with the decoding context in an intermediate layer becomes negatively correlated in the final layer, indicating that correct visual evidence is suppressed before prediction. Based on this finding, we propose Correlation-Flip Visual Injection (CFVI), a simple and training-free decoding-time method that selectively identifies correlation-flip tokens and injects corrective visual signals into the final-layer hidden state to restore suppressed evidence and mitigate hallucinations. CFVI can be directly applied to existing LVLMs, and introduces minimal computational overhead. Experiments on multiple benchmarks demonstrate that CFVI achieves strong performance while preserving language fluency.

Abstract:
Pretrained vision-language models such as CLIP excel at zero-shot recognition but often fail at compositionality, particularly attribute-object and relational structures. Recent studies mitigate this issue by augmenting training with synthetic hard negatives generated by a cascade of large language models and text-to-image models, which incurs substantial pipeline overhead. We instead propose MLLMCLIP, a heterogeneous distillation framework that transfers multimodal knowledge directly from a generative Multimodal Large Language Model (MLLM) teacher into a discriminative CLIP student, bypassing synthetic data entirely. To bridge the architectural mismatch between the two paradigms, we introduce an attention-based per-layer token selection and a CKA-based distillation loss. Compared to prior CLIP-enhancement methods, MLLMCLIP achieves state-of-the-art compositional accuracy while delivering consistent gains on standard zero-shot classification and image-text retrieval, showing that feature-level distillation strengthens both compositional and general vision-language representation capability.

Abstract:
Reverse Kullback–Leibler (RKL) divergence is widely favored over forward KL (FKL) in large language models (LLM) distillation, yet this preference is largely based on comparisons that omit the temperature 𝜏 , overlooking its central role in softening teacher distributions and improving knowledge transfer. In this work, we revisit temperature in LLM distillation and show that it fundamentally changes the comparison between FKL and RKL. Our analysis reveals an asymmetric effect: temperature substantially enriches FKL with non-dominant token signals, whereas it mainly rescales RKL gradients, causing FKL to benefit much more from 𝜏 scaling than RKL. This asymmetry overturns the standard empirical conclusion: although RKL outperforms FKL at 𝜏=1 , FKL consistently surpasses RKL at higher temperatures across instruction-following benchmarks. Moreover, the impact of temperature is not limited to FKL; it improves a broader family of distillation objectives, enabling simple KL-based methods to achieve competitive performance against recent state-of-the-art LLM distillation approaches.

Abstract:
Generative models, such as text-to-image models, excel at taking textual prompts and producing semantically faithful outputs. However, the user’s ability to control the level of creativity of the output remains limited. We introduce Concept-Level Temperature (Concept-τ ), a novel framework for controlling creativity in generative models. Unlike existing methods that operate on low-level features (tokens, noise, etc.), Concept-τ operates at a higher level of abstraction; it systematically identifies decisions left underspecified by the prompt and realizes them according to a creativity parameter, creating a structured exploration space. In this work, we design general criteria for a creativity parameter, and implement an interpretable, model-agnostic method to satisfy them. We demonstrate our ideas on the text-to-image domain; empirical evaluation, both qualitative and quantitative, shows that Concept- significantly outperforms baseline methods in creative modulation and enhancement.

Abstract:
Semi-structured N : M sparsity has emerged as a practical direction for accelerating large language models (LLMs). However, existing learnable-mask approaches incur substantial parameter and memory overhead, limiting their scalability to large models and aggressive sparsity regimes. In this work, we revisit semi-structured pruning from a perspective that reconciles efficiency with scalability. We propose Reservoir of Importance (RoI), a lightweight semi-structured pruning framework that learns sparsity masks through differentiable subset sampling. Unlike prior methods that model full categorical distributions over all feasible N : M patterns, RoI introduces a compact-logit parameterization for sparsity mask learning and performs sampling without replacement to select masks, thereby reducing trainable parameters from combinatorial complexity to 𝒪(M) . As a result, RoI requires 1.5–8.75 × fewer learnable parameters and significantly lower memory cost, while remaining fully aligned with hardware-friendly sparsity patterns. Extensive evaluations across multiple scales of the Qwen2.5 LLM family (0.5-7B parameters) demonstrate that RoI achieves competitive performance with strong memory efficiency, stability, and scalability to more aggressive N : M sparsity patterns, offering a practical path toward efficient LLM deployment.

Abstract:
Muon has emerged as a promising optimizer for large-scale foundation model pre-training by exploiting the matrix structure of neural network updates through iterative orthogonalization. However, the orthogonalization quality of Muon hinges on the number of Newton–Schulz (NS) iterations performed, which poses efficiency challenges due to its non-trivial computation and communication cost. We propose Muon 2 , an extension of Muon, to improve both quality and efficiency by applying Adam-style adaptive second-moment preconditioning before orthogonalization. Our key insight is that the core challenge of polar approximation in Muon lies in the ill-conditioned momentum matrix, of which the spectrum is substantially improved by Muon 2 , leading to faster convergence toward a practically sufficient orthogonalization. We further characterize the practical orthogonalization quality via directional alignment, under which Muon 2 demonstrates dramatic improvement over Muon at each polar step. Across GPT, LLaMA, and Mixture-of-Experts pre-training experiments up to 13B parameters, Muon 2 (and its memory-efficient variant Muon 2 -F that preserves most of its benefits) consistently outperforms Muon and its variants while reducing NS iterations by 40%, and saves up to 1/4 training time over Muon when achieving the same loss.

Abstract:
Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge. We study _archive-grounded reasoning_: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer. Existing benchmarks address only parts of this setting and none jointly stresses archive-groundedness, agentic exploration, and cross-domain coverage. We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model’s context window, so agents must explore deliberately rather than scan exhaustively. Agora is built by an agentic pipeline combining cross-document task synthesis, leakage-preventing obfuscation, and difficulty filtering. Evaluating eight models, we find the task far from solved: even the strongest reaches only 59.4% accuracy, with notable variation across domains.

Abstract:
Understanding online tutoring requires more than recognizing isolated events. A model must connect what a teacher says or does to how the learner responds and how the teacher subsequently adapts. Existing video benchmarks emphasize event and temporal understanding, while educational-dialogue benchmarks are largely transcript-based, leaving this interactional contingency underexplored. We introduce VIRA-Bench, a diagnostic benchmark for short-horizon pedagogical contingency in one-to-one tutoring videos. Grounded in the Initiation–Response–Follow-up structure and contingent teaching theory, VIRA-Bench contains 1,005 clips and 1,322 multiple-choice questions organized into three layers: Observable Evidence Grounding, Interaction Contingency Reasoning, and Support-Response Modeling. Items are built through researcher-led evidence auditing and expert verification to keep questions tied to observable interaction evidence. Across 11 multimodal large language models, the best model reaches 61.3% accuracy, compared with 91.0% on a stratified human subset, with interaction contingency reasoning emerging as the most consistent weakness. Controlled evidence interventions reveal distinct dependencies across layers. Removing video hurts observable grounding most, while disrupting interaction order hurts contingency reasoning most. Additional controls show that this distinction survives free-response evaluation and cannot be explained by question-option shortcuts or video–transcript misalignment. By separating visual grounding from order-sensitive interaction reasoning, VIRA-Bench exposes diagnostic differences that aggregate video-QA scores obscure.

Abstract:
AI agents promise to turn scattered scientific evidence into AI-ready datasets, yet existing benchmarks examine isolated NLP skills or paper-level summarization and leave open whether agents can assemble complete, evidence-grounded records across multimodal and heterogeneous scientific sources. To address this gap, we introduce ABDataBench, a general evaluation protocol for antibody functional data construction from scientific literature and patents. ABDataBench contains 44 expert-annotated, OCR-processed documents, including 27 papers and 17 patents, yielding 279 antibody-level records under a unified 22-field schema. We further develop ABCurator, a record-centric multi-agent framework that treats each antibody as the unit for planning, evidence aggregation, enrichment, and validation, coupled with a skill-accumulation workflow that distills expert feedback into reusable agent competencies across evaluation rounds. Across proprietary and open-source models, the strongest configurations achieve scores of 78.3 and 76.0, respectively. Analysis shows that current agents achieve high antibody recall but remain limited by over-generation, sequence attribution errors, and incomplete affinity grounding. These results position antibody record construction as a challenging long-horizon information-extraction task and establish ABDataBench as a benchmark framework for reliable AI-driven scientific data generation. The source code and data for ABDataBench is released at:https://github.com/GAIR-NLP/ABDataBench.

Abstract:
Test-time Scaling (TTS) has been demonstrated to significantly enhance the reasoning capabilities of Large Language Models (LLMs) during the inference phase without altering model parameters. However, existing TTS methods are largely independent, implying that LLMs have not yet evolved to progressively learn how to scale more effectively. With the objective of evolving LLMs to learn “how to scale test-time computation,” we propose LatentEvolve, a self-evolving latent TTS framework inspired by the complementary learning system (CLS) theory. Analogous to the human brain’s dual system of a fast-recall hippocampus and a slow-consolidating neocortex, LatentEvolve comprises two evolutionary components: daytime scaling, which rapidly retrieves historical latent representations to better guide current LLM reasoning; and nighttime scaling, which integrates past latent optimizations in a manner akin to the human brain’s consolidation of experiences during sleep. The alternation of daytime and nighttime processes facilitates a fast and slow evolution of LLM TTS, mirroring human cognitive dynamics in a fully unsupervised manner. Extensive experiments across eight benchmarks and five model backbones demonstrate that our LatentEvolve surpasses state-of-the-art TTS methods such as LatentSeek and TTRL by up to 13.33% and exhibits exceptional cross-domain and cross-backbone generalization.

Abstract:
Multimodal fake news detection predicts post veracity from text, images, and inter-post context. Existing content-centric and graph-propagation methods model post evidence and inter-post context, but graph propagation still assumes connected posts provide compatible veracity evidence. In multimodal fake news graphs, edges often represent post relatedness rather than veracity consistency, so propagation can pass misleading evidence across conflicting posts. In this paper, we propose Semantic-Structural Propagation Calibration (SSPC), which uses noisy LLM pseudo-labels to calibrate where and how much propagation should be trusted, rather than using them as direct labels. SSPC refines graph edges from high-confidence pseudo-label agreement, computes a graph-encoded base prediction and an explicitly graph-propagated prediction, and learns node-level propagation reliance with a cross-fitted utility gate. Experiments on Twitter, PHEME, and Weibo show SOTA performance over strong multimodal and graph-propagation fake news detection baselines.

Abstract:
Sentence-level AI detectors are essential for detecting AI misuse in human-AI collaborative writing. Yet existing approaches still treat all authors identically, ignoring individual writing styles. This is a critical oversight, given that the boundary between human and AI text is inherently author-specific. To address this limitation, we propose PHAS-ID (Personalized History-Aware Sentence-level AI Detector), an approach that introduces personalization to sentence-level detection. By matching target sentences against the author’s history of labeled documents, PHAS-ID tailors its predictions based on prior writing behaviors. Furthermore, current synthetic datasets for this task fail to capture realistic collaborative patterns, which we address by introducing GhostFill, a style-aware synthetic dataset generation pipeline. Our experiments demonstrate that PHAS-ID outperforms the previous state-of-the-art (SOTA) across seen-author, unseen-author, cross-domain, and cross-generator settings. Ablation studies confirm that these improvements stem from genuine author-specific personalization rather than the mere addition of context. Our source code and dataset are available at our GitHub repository.

Abstract:
Text-to-Image (T2I) models have recently achieved impressive visual fidelity, yet their evaluation remains constrained by benchmarks that are often difficult to interpret and insufficiently diagnostic. Existing skill-based evaluations tend to overlook critical failure modes that strongly impact usability but fall outside standard taxonomies, such as global incoherence arising from missing parts or physically implausible configurations (e.g., floating objects). In addition, prompt difficulty is typically controlled along a single dimension; either prompt length or the number of elements to generate. To address these limitations, we introduce Imag-Eval, a controlled benchmark designed to assess how T2I models ground compositional natural-language instructions into visual outputs. Unlike prior work that conflates surface linguistic complexity with compositional difficulty, Imag-Eval explicitly seeks to disentangles these factors by independently varying both the number of instances and the combination of constraints (rules), while avoiding error propagation. This design enables fine-grained and interpretable analysis of where cross-modal instruction following fails. Our benchmark comprises 1,140 prompts and 8,842 combined rules, and we evaluate it on several state-of-the-art models. Complementing this analysis with an additional study of over 2,000 prompts from a concurrent benchmark, our results suggest that, for structured skills, compositional difficulty is primarily governed by the number of grounded rules and their binding to instances,, rather than by prompt length alone. Code samples are available here: https://github.com/Justsecret123/Imag-Eval

Abstract:
Long-horizon GUI automation remains challenging due to error accumulation over extended interaction sequences. Process Reward Models (PRMs) provide dense step-level supervision for mitigating error accumulation, yet standard PRMs are poorly suited to GUI verification. Standard PRM judgments often rely on superficial visual alignment rather than functional correctness, reflecting an evaluative knowledge gap caused by missing domain-specific adjudication logic. Standard PRMs also perform passive, single-pass visual assessment, which creates Visual Ambiguity when reliable judgment requires actively locating, parsing, or inspecting task-relevant UI evidence. We introduce GUI-PRA, a Process Reward Agent that transforms GUI process evaluation from passive scoring into active investigation. GUI-PRA couples Experience-Injected Criterion Synthesis, which distills generalized verification principles into state-specific criteria, with Criterion-Guided Autoregressive Perception, which uses these criteria to navigate multi-granularity visual tools and gather grounded evidence. On AndroidWorld and Mobile-MiniWoB++, GUI-PRA achieves improvements of 5.0% and 6.5% over standard PRMs on the Qwen-VL series, with Qwen3-VL attaining 54.74% success rate on AndroidWorld. On the offline OS-Critic Bench, GUI-PRA demonstrates strong competitiveness against fully trained critic models.

Abstract:
Speech large language models (Speech LLMs) are increasingly expected to reason in everyday spoken interactions, where users often revise task conditions through self-correction. Unlike ordinary disfluencies, self-corrections alter the premise structure of a problem: obsolete conditions must be discarded, while final corrected conditions must guide reasoning. We formulate this setting as self-corrected speech reasoning and study it as a condition-update problem. We construct SCUR-Bench (Spoken Condition-Update Reasoning Benchmark), a controlled Chinese benchmark of 13,295 instances organized by reasoning difficulty and self-correction difficulty, and introduce a three-path diagnostic protocol to analyze speech-side recovery, condition use, and reasoning. Our analysis reveals a recurrent failure pattern in current Speech LLMs: corrected conditions may be perceived, transcribed, or even mentioned, yet still fail to serve as active premises in reasoning. We term this final-condition adoption failure. A controlled English parallel evaluation exhibits the same recovery–adoption gap, providing preliminary evidence that this failure is not unique to Chinese. To mitigate it, we propose PACER, a perception-aware condition-enhanced reinforcement framework based on GRPO. On Qwen2.5-Omni-7B, PACER reduces end-to-end error and condition-related errors by 39.8% and 67.5%, respectively, relative to the untrained baseline.

Abstract:
Group Relative Policy Optimization (GRPO) is one of the most widely adopted RLVR algorithms for post-training large language models on reasoning tasks. We first show that GRPO admits an equivalent discriminative reformulation, in which policy optimization maximizes the expected score gap between verified positive and negative rollouts. This reformulation reveals two objective-level limitations: likelihood-misaligned surrogate scores, in which clipped ratio-based scores are optimized rather than the sequence likelihoods that govern generation, and score-insensitive credit assignment, in which rollout-level credit does not reflect the current score gaps between positive and negative rollouts. To address these limitations, we propose ConSPO, a Contrastive Sequence-level Policy Optimization method that uses length-normalized sequence log-probabilities as rollout scores and contrasts verified positive rollouts against negative distractors within the same group. ConSPO optimizes a group-wise InfoNCE-style objective to adaptively strengthen updates for poorly separated positives and high-scoring negatives, together with a curriculum-scheduled margin that preserves separation pressure as training progresses. Experiments across diverse settings show that ConSPO outperforms strong baselines on challenging reasoning benchmarks.

Abstract:
Current speech synthesis struggles with code-switching, which mixes a foreign language phrase into a primary language utterance, causing the phrase to be spoken with the primary language’s accent rather than its native one. We propose Phrase-Localized Language-Contrastive Guidance (LCG), a training-free inference framework that restores a native accent to code-switched phrases in cross-lingual text-to-speech. LCG replaces the single language guidance applied across the whole utterance with a separate guidance for each region, so each part is guided by its own language. To choose where to apply this localized guidance, we propose a self-attention probing technique that finds the phrase boundaries without external alignments. Together, these components generate speech in which each region carries the accent of its own language, requiring no fine-tuning or auxiliary models. Across diverse language pairs, LCG robustly increases the nativeness of the code-switched phrase while suppressing accent leakage, and preserving overall speaker identity and naturalness.

Abstract:
Vision-language models (VLMs) are expected to respond helpfully to appropriate requests while withholding compliance with requests that are incorrect, unsafe, infeasible, or unanswerable. However, existing benchmarks predominantly evaluate non-compliance at the level of the query as a whole, assuming that each request either warrants compliance or requires withholding compliance. In practice, real-world queries can contain a mixture of answerable content and components for which compliance should be withheld. In this paper, we introduce KoNA, a benchmark for evaluating selective non-compliance in VLMs across five categories: False Premise, Visual Inaccessibility, Universal Unknown, Task Feasibility, and Safety. Each task evaluates two capabilities: query-level non-compliance and component-level non-compliance under paired single and compound queries. Our evaluation across diverse VLMs shows that models often fail to refuse, correct, or abstain appropriately, and these failures become more pronounced when queries require selective non-compliance. To address this challenge, we fine-tune VLMs using KoNA examples that require selective non-compliance, together with a fully answerable set that should receive direct answers. Our fine-tuned models achieve substantial improvements in non-compliance accuracy while largely maintaining performance on fully answerable tasks. These results suggest that the fine-tuned models can distinguish between answerable components and those requiring non-compliance and respond in a task-appropriate manner.

Abstract:
Quarter-ahead revenue forecasting requires company-scale numerical accuracy, strict temporal validity, and company-specific interpretation of narrative disclosures. LLMs can distill textual evidence but can produce scale-misaligned forecasts, whereas history-based anchors are stable but miss forecast-time signals such as product transitions, supply constraints, and management guidance. We introduce CAME (Company-Aware Evidence-Memory Experts), a residual-forecasting framework that refines a no-leakage statistical anchor when current semantic evidence and prior error patterns justify an adjustment. On a development-inclusive rolling backtest of 336 company-quarters from 12 large public technology and platform firms, CAME achieves the lowest aggregate point-estimate error among the reported methods, with statistically supported macro-sMAPE gains over the matched Statistical Anchor, and outperforms History + Guidance on all six aggregate metrics. CAME also links adjustments to source-linked evidence cards and guarded memory traces, supporting forecast inspection, provenance, and failure localization.

Abstract:
Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora. However, auditing such corpora faces three challenges: (1) their web-scale size makes full scan costly; (2) prior analyses are often too coarse to expose token-level pollution; (3) Chinese web pollution is implicit and rapidly changing. We propose Sampled-BPE, a lightweight token-level auditing pipeline that sample a small subset and train BPE tokenizer to surface polluted tokens. Experiments show that Sampled-BPE preserves usable estimates while substantially reducing runtime and memory: a 148.4 × speedup and a 35.8 × memory reduction induce only 4.25% relative error for pollution categories. We apply the pipeline to 11 open Chinese corpora and 6 Chinese Common Crawl snapshots from 2021 to 2026. The audit reveals widespread but uneven pollution across open corpora, as well as highly polluted and temporally shifting Chinese web content. We further release a hierarchical Chinese web token dataset with 660k+ token records, each with web context, category, and explanation fields, organized as trees to support review and tracing of pollution.

Abstract:
Robust tool-use agents must navigate mixed-intent interactions, deciding whether to ask for missing information, answer directly, or invoke tools. We tackle this challenge by introducing a novel task formulation that structures the planner’s action space into explicit interaction instructions over an explicit Ask/Answer/Act space, thereby making these behavioral boundaries an explicit target of optimization. In this framework, a planner predicts the interaction mode and semantic payload, while a frozen executor realizes the instruction into an observable behavior. Coupled with this structured decision process, we propose Executor-Grounded Failure-Routed Planner Learning (EFPL). EFPL optimizes the planner via executor-observed outcomes and strategically leverages privileged feedback for targeted correction when outcome rewards provide insufficient guidance, further refining these behavioral boundaries. Experiments on BFCL and When2Call across three backbone sizes show that EFPL consistently improves clarification, abstention, when-to-call decisions, and robustness to irrelevant tools, while fully preserving strong multi-turn tool-calling performance.

Abstract:
Distinguishing machine-generated text (MGT) from human-written text (HWT) becomes increasingly important due to potential misuse. However, most supervised detectors often degrade out of domain (OOD) and require large, diverse training sets. In this work, we analyze the linearity and quality of MGT representations and show that simple linear probes outperform a wide range of detectors while being substantially more sample-efficient. We first show that MGT and HWT latent representations are linearly separable in low-dimensional space, and provide a plausible explanation for this separability through systematic differences in their representation quality. Motivated by these insights, we train two variants of simple linear probes and evaluate them across 4 benchmarks against 16 baselines. Probes consistently improve OOD detection (+11 AUC), requiring solely samples to reach near-peak performance. We show that this transferability arises because probes recover a shared latent MGT direction that generalizes across diverse settings. Finally, we demonstrate that probing vectors capture a continuous spectrum of “machineness”, highlighting their potential for fine-grained estimation of AI-edited text. Overall, our work provides insights into latent-space differences between MGT and HWT and demonstrates the potential of linear probes as as robust and sample-efficient MGT detectors. We release our code on github .

Abstract:
Long-horizon autonomous agents require memory systems to retain historical information, track evolving states, and reuse relevant knowledge beyond finite context windows. Existing agentic memory systems typically follow a memory construction–retrieval (MCR) pipeline, but often adapt mainly the memory bank while keeping the surrounding pipeline fixed after deployment. This fixed-pipeline design struggles to handle heterogeneous task-specific failure modes and can become misaligned with memory banks that evolve in scale and structure over time. To address these limitations, we propose MemPro, a system-level evolution framework that treats the entire MCR pipeline as an evolvable program rather than adapting only the memory bank or prompt text. MemPro maintains a version tree of runnable memory-system implementations, where an Evolving Agent iteratively selects promising versions, diagnoses recurring failures, and creates improved child versions through failure-mode-guided edit–debug refinement. Experiments on LongMemEval, LoCoMo, HotpotQA, and NarrativeQA show that MemPro consistently outperforms strong static and prompt-level evolving baselines within a few iterations, continues to improve with evolution, and achieves a favorable performance–cost trade-off. Code is available at https://github.com/wanghai673/MemPro.

Abstract:
Generating multiple diverse and high-quality solutions is valuable for many applications, such as code-test generation and drug discovery. However, Large Language Models (LLMs) tend to converge on a single high-confidence solution during inference, limiting exploration of alternative valid solution paths. Existing test-time methods promote diversity through tree-like search and prune semantically similar branches using response-level semantic embeddings. However, we find that such embeddings are easily confounded by linguistic and stylistic similarities, making it difficult to distinguish genuinely distinct solution paths. To address this, we introduce Answer Probing, which probes the potential answer an LLM would reach from an intermediate reasoning path. We demonstrate that the hidden states of probed answers more effectively differentiate distinct solution paths than semantic embeddings, and the perplexity of probed answers serves as a practical proxy for reasoning correctness. Based on these findings, we propose Answer Probing-Guided Tree Search (APTS), which guides the tree search by the probed answers’ hidden state similarity and perplexity. Experiments on three reasoning tasks across two LLMs show that APTS consistently enhances solution diversity, demonstrating its effectiveness and robustness.

Abstract:
Block-coordinate descent (BCD) has emerged as an effective approach for memory-efficient fine-tuning of large language models. During training, BCD randomly activates a small subset of blocks, such as layers or modules, and updates only those blocks while keeping the rest frozen. Although effective, prior work has paid little attention to the frozen blocks, which must still retain forward activations to propagate gradients during the backward pass. Our analysis reveals that gradients through these frozen blocks are highly sparse at the token level: a small subset of tokens carries nearly all of the signal required by the active blocks. Building on this observation, we introduce frozen-block gradient approximation with sparse tokens (Frost), which approximates gradients through inactive blocks using only a sparse subset of tokens and integrates seamlessly with standard BCD optimizers. Across diverse fine-tuning scenarios, model backbones, and modalities, Frost substantially improves the memory efficiency of BCD while preserving downstream performance with negligible degradation.

Abstract:
Retrieval Augmented Generation (RAG) is a highly effective paradigm for keeping LLM-based responses up-to-date and reducing the likelihood of hallucinations. Yet, RAG was recently shown to be quite vulnerable to corpus knowledge poisoning: an attacker injects misleading documents to the corpus to steer an LLM’s output to an undesired response. We argue that the standard causal attention mechanism in LLMs enables harmful cross-document interactions, specifically in cases of attacks. Accordingly, we introduce a novel defense approach for RAG: Sparse Document Attention RAG (SDAG). This is a block-sparse attention mechanism that disallows cross-attention between retrieved documents. SDAG requires a minimal inference-time change to the attention mask. We present an empirical evaluation of LLM-based question answering (QA) with a variety of attack strategies on RAG. We show that our SDAG method substantially outperforms the standard causal attention mechanism. We further demonstrate the clear merits of integrating SDAG with state-of-the-art RAG defense methods. Specifically, the integration results in performance that is statistically significantly better than the state-of-the-art.

Abstract:
Uncertainty estimation for language models often treats reliability as a property of the answer distribution: high confidence, stable samples, or low semantic entropy are taken as signs that an answer can be trusted. Table reasoning exposes a fundamentally different failure mode. A model may consistently return the same answer while linking the wrong entity, selecting the wrong column, applying the wrong operation, or relying on incomplete or insufficient evidence. We introduce MIRAGE—Model Inference Reliability Assessment via Grounded Evidence, a structure-grounded reliability framework for selective table reasoning. MIRAGE estimates residual structural uncertainty from test-time traces, combining outcome consistency with grounding and computation signals, and produces calibrated answer-or-abstain decisions with lightweight reliability certificates. On a controlled four-family diagnostic benchmark, structural reliability signals substantially reduce selective risk compared with generic confidence, sampling, semantic-entropy, and hallucination-detection baselines.

Abstract:
Pretraining-data attribution typically assigns scalar scores to individual training examples. However, practical data curation involves retaining or removing coherent subsets, making additive pointwise assumptions mathematically restrictive. To address this, we formalize attribution as subset-level counterfactual utility prediction and propose GRASP (Geometry-aware Residual Alignment for Subset Prediction). GRASP combines first-order residual alignment with a low-dimensional train-train geometry term, derived from a smoothness lower bound. By explicitly modeling data interactions via compact 65-dimensional feature sketches, GRASP reduces upfront artifact construction time by nearly an order of magnitude compared to standard baselines. It efficiently scores 100,000 candidate subsets in just 5 seconds on a single CPU, completely bypassing the prohibitive cost of combinatorial retraining. Empirically, GRASP strictly dominates scalable baselines, more than doubling the average task-level subset-utility rank correlation on Linear Datamodeling Score (LDS) benchmarks, while fixed-compute diagnostics on Common Crawl further qualify its practical utility in large-scale curation.

Abstract:
Multi-agent LLM systems can outperform single-agent baselines, yet practitioners still cannot predict which design works for a new task or diagnose why one fails. We argue this gap persists largely because the field lacks a diagnostic framework with measurable primitives and testable predictions. We introduce DIANOIA, a three-channel decomposition of multi-agent reasoning gain into coverage, fidelity, and synthesis, each of which is empirically measurable. From this decomposition, we propose a diagnostic protocol that forms qualitative hypotheses about bottleneck channels for tasks with measurable quality signals. We instantiate the protocol as a multi-agent system whose three components mirror the channels: role-diverse proposers for coverage, execution- or pseudo-verification-grounded review for fidelity, and iterative synthesis. On GSM8K, MBPP, and BFCL-SP, our method outperforms strong multi-agent baselines under matched standard configurations; on MBPP, it matches the strongest baseline with approximately 5× fewer tokens and dominates the measured accuracy–token Pareto frontier. Full-scale MBPP replications on Qwen, DeepSeek, and Gemini consistently improve over the corresponding single-model baseline. A prospective held-out evaluation on VitaBench, whose bottleneck hypothesis was recorded before running DIANOIA, improves pass rate from 29.5% to 45.0%. DIANOIA reframes multi-agent design as channel-aware resource allocation: diagnose the likely bottleneck, then invest tokens accordingly.

Abstract:
Language models (LMs) represent tokens using embedding matrices that scale linearly with the vocabulary size. To constrain the parameter footprint, prior work proposes hashing many tokens into a single vector within encoder-only models. While this offers parameter efficiency, many-to-one collisions prevent its use in causal LMs. In this paper, we propose MultiHashFormer, a new framework that allows hash-based autoregression. Each token is represented as a unique hash signature, a short sequence of discrete hash IDs, generated by multiple independent hash functions. A Hash Encoder compresses this signature into a single latent vector for processing by a Transformer decoder. Then, a Hash Decoder generates the hash signature of the next token, which is then mapped back to text. We evaluate our approach at the 100M, 1B and 3B parameter scales, demonstrating that MultiHashFormer consistently outperforms standard Transformer LMs across multiple benchmarks. Furthermore, we show that our model handles multilingual vocabulary expansion with a constant parameter footprint without any modifications.

Abstract:
Generative engine optimization (GEO) enables content producers to increase the visibility of their web pages in generative search engines, but the same techniques can deliver targeted misinformation when adversaries publish ordinary-looking GEO-optimized documents that victim large language models (LLMs) retrieve and synthesize into distorted answers. No existing benchmark evaluates defenses against this threat under controlled conditions. Therefore, we present Counter-GEO-Bench, a defense benchmark that pairs 247 human-verified, quality-gated queries with information-preserving and information-distorting GEO rewrites, and evaluates defenses on attack success rate (ASR), false positive rate, and answer quality across three victim LLMs. Under Counter-GEO-Bench, three off-the-shelf defenses (Granite Guardian, Llama Guard 3, and NeMo Self-Check Fact-Checking) reduce ASR by at most 5.7% relative, while Granite Guardian’s reduction is not statistically significant. Safety-taxonomy guardrails target policy violations, while GEO misinformation passes through them as fluent informational content. To this end, a lightweight benchmark baseline, C-GEO Guard, is proposed, reducing ASR by 47.6% relative with near-zero utility loss, which proves threat tractable.

Abstract:
Vision-language models (VLMs) are increasingly evaluated for whether they identify the right visual content, but little is known about whether they express such content in a discourse-appropriate form. We address this research gap using information structure (IS), testing whether VLMs distinguish discourse-old Topics from discourse-new Foci in visually grounded question answering. We exploit Hungarian, a language in which Topic and Focus map onto dedicated syntactic positions, making IS choices observable in text. Comparing six VLMs with human participants, we find that models produce IS-relevant constructions, but over-regularise this sensitivity. Under the interacting pressures of discourse status, grammatical role (preference for subject Topics) and definiteness (preference for indefinite Foci), humans choose variable strategies for IS realisation. VLMs, by contrast, collapse onto narrow response templates, resembling mode collapse (Kirk et al., 2024). Our findings suggest that VLM evaluation should look beyond content accuracy to how content is packaged for the discourse.

Abstract:
As Large Language Models (LLMs) increasingly encourage users to disclose personal profiles for tailored assistance, measuring their political alignment becomes increasingly important. However, many existing benchmarks for assessing political behavior rely on closed-ended questions and do not fully capture how a model’s stance may adapt to user-provided context during interaction. We introduce a framework that disentangles two distinct triggers of political sycophancy: opinion (aligning with explicit narratives) and identity (stereotyping based on demographic labels). Using 450 manually-checked political dilemmas as controlled probes, we evaluate 13 instruction-tuned LLMs. We uncover a dissociation: a model’s susceptibility to explicit opinions does not necessarily predict its susceptibility to identity cues, and vice versa. When both signals are present, their effects are generally sub-additive rather than simply additive. Additionally, system-level personas primarily shift a model’s baseline stance while having limited effect on the stance shift caused by user opinion or identity. Ultimately, our results suggest that LLM political stance is interactively and steerably vulnerable rather than being a fixed trait, highlighting how personalization may amplify identity- or opinion-conditioned shifts in the model’s behaviors.

Abstract:
Pixel-based language models (LMs) replace traditional tokenizers by processing rendered images of text, making cross-lingual transfer heavily dependent on the visual and structural properties of writing systems. However, the dynamics of adapting these models to low-resource languages with complex morphology and written in unique scripts are not yet explored. Using Tibetan as a case study, we analyze how continued pre-training of pixel-based LMs is influenced by data scale, initial script exposure, and cross-lingual transfer from languages written in other Brahmic scripts. We introduce four rendering-level metrics to quantify visual script similarity. We evaluate downstream performance across three tasks. Our results show that higher orthographic proximity enhances semantic transfer, even under severe data constraints. Additionally, we find a performance asymmetry based on the pre-training starting point: while multilingual pre-training (PIXEL-M4) has stronger initial performance, its capacity for subsequent adaptation seems to be constrained, whereas adapting a monolingual model (PIXEL) with mixed scripts yields more gains on sentence-level tasks. Our metrics and case study offer empirical observations that could help inform data selection and script adaptation choices when working with pixel-based models in similar low-resource settings.

Abstract:
Partially Relevant Video Retrieval (PRVR) aims to retrieve untrimmed videos that contain moments relevant to a text query. Since the target moment occupies only a portion of the video, PRVR requires retrieval based on fine-grained understanding beyond coarse video-level matching. However, existing methods often rely on frozen CLIP frame features, which lack temporal understanding. Even with recent progress in parameter-efficient CLIP adaptation, video-level predictions can still be supported by imprecise frame-level evidence. In this paper, we propose an Intrinsic Temporal Adaptation (ITA) framework for PRVR. First, our Backbone-Internal Temporal Adaptation allows the last few visual transformer layers to attend over groups of neighboring frames. This provides temporally aware frame embeddings while keeping CLIP frozen and training only adaptation parameters. Second, we introduce Affinity-Weighted Gradient Propagation to address the weakly supervised nature of PRVR, softly aggregating top- k frames based on text-frame affinities and propagating learning signals to multiple query-relevant frames. Our method achieves state-of-the-art performance on PRVR benchmarks, demonstrates robust cross-dataset transfer, and retrieves substantially more accurate frame-level evidence within ground-truth query-relevant moments. Our code is available at github.com/hynnsk/ITA.

Abstract:
Understanding relationships between attention heads is essential for interpreting the internal structure of Transformers, yet existing metrics do not capture this structure well. We focus on the subspaces spanned by attention-head weight matrices and quantify head-to-head relationships using the Projection Kernel (PK), a principal-angle-based measure of subspace similarity. Experiments on GPT2-small, BLOOM-560M, and Qwen2-0.5B-Instruct show that PK reproduces known head-to-head interactions on the IOI task more clearly than prior metrics such as the Composition Score. We further introduce a framework to quantify the informativeness of PK distributions by comparing them with a reference distribution derived from random orthogonal subspaces. As an application, we analyze a directed graph constructed from PK and show that, in GPT2-small, L4H7 acts as a hub by functioning as an Identity Head.

Abstract:
Hallucinations undermine the reliability of Large Language Models (LLMs) in real-world settings. Current contrastive decoding methods mitigate hallucinations by generating multiple responses based on an amateur model and an expert model or using the logit distribution across all layers, leading to considerable redundancy and computational inefficiency during inference. To address these limitations, we propose SHICD (Self-Hallucination-Inducing Contrastive Decoding), a novel training-free decoding framework that requires a single response from an individual model and leverages information from a subset of all layers, without any external intervention on activations. Specifically, SHICD first identifies the inducing layer of the LLM when an initial semantic representation emerges in the model. The hallucination direction is then extracted to induce hallucinations by itself while preserving the initial semantic information encoded before the inducing layer. Finally, we employ an adaptive contrastive decoding mechanism to determine when to apply contrastive decoding during generation. Extensive experiments across multiple LLMs and benchmark datasets demonstrate that SHICD significantly improves factual accuracy over baseline methods while achieving lower inference latency.

Abstract:
Retrieval-Augmented Generation (RAG) enhances large language models with external evidence, but in compliance-oriented QA, semantic relevance alone does not determine whether evidence should be used for generation. A retrieved document may be relevant and factually correct, yet inadmissible because it is inaccessible to the current user, explicitly superseded, or in unresolved conflict with other active evidence. We formalize three post-retrieval governance failures in this setting: unauthorized leakage, stale citation, and unsupported answers. To address them, we propose TrustRAG, a RAG framework that incorporates a metadata-governed routing layer between the retriever and the generator. Given correct and complete governance metadata for the retrieved candidate set, this layer filters inadmissible evidence before generation, returns a deterministic refusal when no admissible evidence remains, and preserves unresolved conflicts for conflict-reporting generation. We construct AeroTrustBench, a counterfactual compliance QA benchmark for evaluating evidence governance in RAG. Across Qwen-Max, DeepSeek-V3, and GLM-4-32B-0414, TrustRAG consistently reduces the three failures relative to unguided RAG and baselines based on prompting, filtering, routing, verification, and recency. Mechanism-matched and 50-document comparisons show benefits beyond filtering under larger candidate pools, while metadata-noise tests and human audits characterize dependencies on upstream metadata and evaluator choice.

Abstract:
One approach to accelerating progress on AI alignment is to automate parts of alignment research itself. Doing so requires systems that can perform reliably on hard-to-supervise research tasks, including identifying flaws in research proposals and evaluating whether a proposed experiment supports its intended claim. To evaluate this capability, we introduce Tampered ML, a benchmark of 1,858 item-pairs, each consisting of an experiment proposal reconstructed from a published machine learning paper and a minimally edited counterfactual that introduces a targeted methodological flaw. Models must identify the more methodologically sound proposal in both presentation orders. A human validation study on a stratified sample estimates that 80-90 % of items are valid. Frontier models perform above chance but remain far from reliable (with Opus 5 achieving 77.9% accuracy). They are highly sensitive to the order of presentation and favor flawed designs that appear more specific, transparent, or conventional.

Abstract:
Automated CT report generation remains challenging because it requires both fine-grained grounding of pathological entities in multi-view volumetric evidence and clinically coherent language that reflects diagnostic priors from historical cases. Existing methods often rely on noisy cross-modal alignment or underutilize retrieved report knowledge during decoding. We propose HEMP, which is a Hierarchical Entity-aligned Memory Prompting framework for LLM-based CT report generation. HEMP first performs two-stage entity-visual alignment where graph-constrained entity-conditioned attention retrieves initial evidence and an optimal transport refinement enforces principled coupling between entity nodes and visual tokens under marginal constraints. To inject case-conditioned linguistic priors, we design a Conditional Dual-Hopfield Memory module that retrieves case-conditioned report priors and integrates them through both implicit feature-level fusion into entity representations and explicit memory tokens appended to the prompt. Finally, we enforce clinical consistency via hierarchical severity supervision at the entity, region, and sign-group levels. Comprehensive Experiments demonstrate that our method outperforms previous methods and achieves SoTA performance.

Abstract:
Long-context reasoning has significantly empowered large language models to tackle complex tasks, yet it introduces severe efficiency bottlenecks due to the computational complexity. Existing efficient approaches often rely on complex additional training or external models for compression, which limits scalability and discards critical fine-grained information. In this paper, we first propose VTC-R1, a new efficient reasoning paradigm that integrates vision-text compression into iterative reasoning. Instead of processing lengthy textual traces, VTC-R1 renders intermediate reasoning segments into compact images, which are iteratively fed back into vision-language models as "optical memory". We construct a training dataset based on OpenR1-Math-220K achieving 3.4× token compression and fine-tune representative VLMs–Glyph and Qwen3-VL. Extensive experiments on benchmarks such as MATH500, AIME25, AMC23 and GPQA-D demonstrate that VTC-R1 consistently outperforms standard long-context reasoning. And VTC-R1 improves inference efficiency, achieving 2.7× speedup in end-to-end latency, highlighting its potential as a scalable solution for reasoning-intensive applications.

Abstract:
There are many reasons one may want to edit a model’s chain of thought (CoT)—e.g., to prefill it with reasoning from a stronger model or to remove steps that may yield unsafe outputs. The success of these edits plausibly depends on a model’s inability to notice them, as the model may alter its behavior if it suspects tampering. In this work, we study whether recent reasoning models are able to detect such edits on their CoTs under a variety of conditions: during reasoning and after it; when alerted to the possibility of tampering and when not alerted; and when prefilled both with their own CoTs and with those of other models. Broadly, we find that (i) models exhibit only very modest detection accuracy; (ii) models struggle to identify where their CoT was modified; and (iii) models are about as good at detecting changes to their own CoTs as to those of other models.

Abstract:
Large language models drop in performance when their context contains semantic distractors : context topically related to the question but off the reasoning chain. Prior mechanistic work tracks a single injected distractor token, missing failures in which the model predicts other context tokens or out-of-context but topically related ones. We instead track the model’s actual top non-gold prediction, providing a localization target for every observed top-1 failure in the analyzed subset. Failures converge on a small set of mid-to-late-layer answer-selection heads (AS-Heads): patching their clean outputs into distracting runs recovers 60- 86% of failed predictions across four base models from two families (random heads recover \leq 8%), with the mechanism also replicating in two instruction-tuned models. Clean-prompt ablations indicate that these heads also contribute to normal answer selection, so their role is not specific to distractors. Redirecting their attention onto an entity span causes the model to transcribe that span at 22-83% (vs. \leq 7.4% for random heads), establishing a causally consequential intervention point. The localized heads transfer across MuSiQue, HotpotQA, 2WikiMultiHopQA, and non-Wikipedia RAG settings. At inference, their attention provides a training-free context-filtering signal that lifts F1 by up to +35 points on disrupted samples and is competitive with retrieval-head methods.

Abstract:
Pretrained LLMs excel across diverse tasks, yet standard attention remains a major bottleneck for long-context understanding. Recent test-time training (TTT) methods alleviate this by encoding context into internal model parameters, but they typically require specialized architectures, meta-learned initialization, or continual post-training, preventing plug-and-play use on off-the-shelf LLMs. We propose test-time training sparse attention (TTT-Sparse), a plug-and-play alternative that performs test-time training over external memory while keeping the internal parameters frozen. For each input, TTT-Sparse constructs contextual chunk key states from the KV cache and learns a lightweight query adapter to induce a task-specific sparse attention pattern, which is used during generation to attend only to the most relevant context chunks and suppress interference from irrelevant context. Extensive experiments on long-context benchmarks show that TTT-Sparse improves long-context understanding while reducing generation-time attention cost.

Abstract:
Recent speech language models rely on encoders that are optimized separately from autoregressive models. Since these encoders are unaware of the downstream objectives, the extracted representations may not be optimal for downstream tasks. To address this limitation, we introduce a discrete latent variable model on mel spectrograms that jointly optimizes the encoder and the speech language model. Joint optimization not only brings improvements over codec-based and other mel-spectrogram-based baselines on zero-shot Text-to-Speech (TTS) and Speech-to-Text (STT) tasks, but also effectively alleviates common issues in autoregressive mel spectrogram modeling, such as prolonged silence generation and word omissions. Samples are available at the [demo page](https://anonymous29300909.github.io/anonymous2930-demo/).

Abstract:
Policy optimization (PO) for Large Language Models faces a stability–exploration trade-off, currently mediated by an action-side Policy-KL regularizer. This puts practitioners in a double bind: keeping Policy-KL constrains response behavior and consumes the action-side exploration budget, while dropping it leaves the optimization without an explicit drift control. We argue for an alternative that breaks the dilemma by moving regularization to the input side. As training progresses, the distribution over training queries induced by the current policy drifts unchecked from its pre-RL reference distribution. Concretely, Environment-Regularized Policy Optimization (ERPO) introduces a Query-KL (QKL) term that bounds this query distribution shift, together with a dataset-static reference-derived per-query weight that biases each per-query update toward queries typical under the reference. The QKL gradient flows strictly through the query likelihood; the response score function used by policy-gradient estimators does not appear in the QKL term, so QKL exerts no direct gradient pressure on the response distribution—exploration is preserved. ERPO plugs into GRPO/PPO/REINFORCE-style pipelines without additional forward passes. On six mathematical reasoning benchmarks, ERPO replaces the standard Policy-KL regularizer while achieving effective control over query distribution drift, delivering stronger accuracy and substantially more stable behavior under high-temperature decoding and long-horizon training.

Abstract:
Multimodal Large Language Models (MLLMs) demonstrate remarkable reasoning capabilities, yet they remain susceptible to hallucinations. While Multi-Agent Debate (MAD) presents a promising collaborative solution, unguided debates inevitably devolve into a stochastic martingale process and expanding conversational contexts trigger severe visual attention decay. In this paper, we propose EviaMAD, an Evidence-Anchored Multi-Agent Debate framework. To proactively expose internal hallucinations, EviaMAD bifurcates perception into two distinct roles: a Global Observer capturing the holistic scene and a Local Observer scrutinizing salient details. When conflicts arise, a dedicated Analyzer intervenes, distilling the linguistic disagreement into a deterministic bimodal evidence anchor: a physically cropped visual patch of the disputed region paired with key semantic directives. During the subsequent reflection phase, a surgical reallocation mechanism explicitly amplifies the attention weights of this anchor. By compelling agents to ground their reasoning in strictly verified evidence, EviaMAD effectively elicits collective intelligence. Extensive evaluations demonstrate that EviaMAD significantly mitigates hallucinations while elevating the reasoning capabilities of the multi-agent system.

Abstract:
The lack of diversity in LM content is widely attributed to the alignment process, but how and where exactly in the pipeline this collapse begins is unknown. We argue that output homogeneity is likely learned during the pretraining phase, and only revealed or magnified during the alignment process. Specifically, we find that semantic convergence is observed from the first alignment stage—the instruction-tuning phase (SFT)—suggesting that homogeneity might already exist in the pre-alignment model. To investigate this, we conduct controlled SFT experiments examining how training data influences output convergence on specific input/output pairs. We find that convergence can be revealed and amplified, but not introduced by the SFT data, supporting its role as a catalyst rather than a cause. To further test whether homogeneity originates before alignment, we measure convergence in base models. We find that instruct-like collapse can be induced through prompting alone, even without alignment. Taken together, our results suggest that semantic convergence may arise naturally from the objectives underlying LM training, making it difficult to mitigate through post-alignment interventions alone.

Abstract:
Proactive AI writing assistants need to predict when users want drafting help, yet evidence about which contextual cues drive these preferences remains limited. Through a factorial vignette study with 50 participants making 750 pairwise comparisons in professional-email scenarios, we find that compositional effort is the strongest measured predictor ( 𝜌 = 0.597 ), whereas urgency has a near-zero scenario-level correlation and no reliable effect across inferential analyses. More critically, we observe a striking stated–revealed gap between abstract factor rankings and situated choices: they rank urgency first in self-reports despite it being the weakest behavioral driver, a contrast at the extremes even though mean stated urgency and effort ranks are close, i.e., 2.06 vs. 2.18. This misalignment has measurable consequences. Within this controlled professional-email testbed, the behavior-derived rule agrees with 61.3% of the 750 observed pairwise choices, compared with 57.7% for the stated-preference rule. This 3.6-point difference corresponds to a net gain of 27 matches. It also identifies a concrete design risk for proactive email assistance: stated rankings can place weight on cues that do not predict situated choices. Future studies should test whether stated rankings create similar design risks in other writing tasks.

Abstract:
Despite more than a decade of curriculum learning (CL) research in NLP, the field lacks a principled account of which difficulty function or scheduler to use for a given problem. To understand what has hindered progress towards this account, we propose a fine-grained taxonomy separating difficulty evaluation from training scheduling to enable systematic analysis of CL strategies. For difficulty evaluation, we distinguish attribution source and task dependence, revealing difficulty as a perspectival concept encoding different assumptions about what makes an instance hard to learn. For scheduling, we provide the first formalisation of CL schedulers in terms of expected training contribution, enabling comparison across implementations by introducing retention regimes and monotonicity properties. Applied in the first dedicated survey of CL in NLP, covering 107 strategies from 90 papers, our taxonomy reveals a systematic incomparability problem: prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label – hindering comparison and the accumulation of a coherent evidence base. Beyond diagnosis, the taxonomy supports the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.

Abstract:
Learning from experience is essential for agents in real-world applications. However, most existing methods rely on static imitation of successful expert trajectories, overlooking the informative signals exposed from failed experiences and lacking a interactive feedback loop with the environment. In this paper, we propose R-IDEAL, a unified framework for insight-driven experience generation and optimization. R-IDEAL first perform contrastive analysis between successful and failed trajectories to automatically synthesize structured insights, consisting of positive plans and negative lessons, which guide agents not only on what to do, but also on what to avoid. These synthesized insights serve as supervision to train a lightweight insight model that generates task-specific guidance. While this supervision stabilizes insight generation, it does not guarantee the effectiveness of the insights due to distribution shifts and execution discrepancies across agents. Therefore, we further optimize the insight model via reinforcement learning using execution feedback from heterogeneous agents in real environments, ensuring that the generated insights effectively improve agent behavior. Experiments on AlfWorld and SciWorld show that R-IDEAL substantially outperforms existing methods, demonstrating strong robustness in out-of-distribution scenarios and serving as a lightweight, plug-and-play module for both closed-source and large-scale models.

Abstract:
Large vision-language models (LVLMs) often hallucinate objects that are not present in the input image, largely because visual grounding weakens as decoding progresses. Existing inference-time mitigation methods modify logits or hidden states throughout generation, but they suffer from three key limitations: they lack an explicit grounding objective, intervene even when the model is already well-grounded, and use fixed correction strengths that do not adapt to the severity of grounding failure. We propose BRACS (Barrier-Regulated Adaptive Closed-form Steering), a training-free steering framework that addresses these issues through barrier-regulated adaptive closed-form steering. BRACS monitors the model’s own attention to measure visual grounding and applies corrections to the hidden states only when grounding deteriorates. The corrective update is computed analytically in closed form, requiring no training of auxiliary networks or model retraining. Experiments on LLaVA-1.5-7B and Qwen-VL-Chat show that BRACS consistently outperforms prior methods on hallucination benchmarks, reducing CHAIR s by 9.4 points and improving POPE F1 by 2.7 points, while matching or improving performance on four general multimodal benchmarks. BRACS also remains efficient, operating at 80% of greedy decoding throughput and achieving 1.3× higher speed on average than the baselines.

Abstract:
Deep Research Agents (DRAs) aim to solve complex, long-horizon research tasks involving planning, retrieval, multimodal understanding, and report generation, yet their evaluation remains challenging due to dynamic web environments and ambiguous task definitions. We propose DR 3 -Eval, a realistic and reproducible benchmark for evaluating deep research agents on multimodal, multi-file report generation. DR 3 -Eval is constructed from authentic user-provided materials and paired with a per-task static research sandbox corpus that simulates open-web complexity while remaining fully verifiable, containing supportive documents, distractors, and noise. Moreover, we introduce a multi-dimensional evaluation framework measuring Information Recall, Factual Accuracy, Citation Coverage, Instruction Following, and Depth Quality, and validate its alignment with human judgments. Experiments with our developed multi-agent system DR 3 -Agent based on multiple state-of-the-art language models demonstrate that DR 3 -Eval is highly challenging and reveals critical failure modes in retrieval robustness and hallucination control.

Abstract:
Flowcharts are widely used in business management and software development to represent process logic and user interactions. Understanding the semantics they encode is essential for downstream applications such as visual question answering. Existing methods either detect components and reconstruct topology, which suffers from cascaded errors and rigid shape assumptions, or feed the whole image to a vision language model (VLM), which dilutes attention on dense diagrams and often reverses edge directions. Inspired by cognitive science findings that humans read flowcharts by tracing arrows, we propose TRACE (Triplet Recovery via Arrowhead-Centric Extraction), a three-stage framework. An offline stage synthesizes aligned training data from text-format flowchart sources (e.g., Mermaid, BPMN XML) and fine-tunes the two models used online: a detector that locates every arrowhead, and then a VLM that extracts one triplet per arrowhead. Each output triplet is grounded to a detected arrowhead, enabling traceable diagnosis. Experiments show that TRACE substantially improves flowchart understanding accuracy and improves downstream QA accuracy.

Abstract:
The next-token prediction (NTP) objective trains language models to predict a single token at each step, even though many continuations can express the same meaning. For example, in the sentence “this sticker can be placed here”, positioned , attached , or put are all plausible alternatives. While standard NTP training treats these alternatives as mutually exclusive targets, we explore a self-supervised framework that encourages models to predict concepts, approximated as sets of semantically equivalent tokens. Models trained with this concept supervision align better with human similarity judgments, improve classification, clustering, and reranking performance, and achieve comparable or stronger downstream reasoning. These gains come with lower perplexity on semantically meaningful words (Section 3.2) and only minimal increases in global perplexity, suggesting that concepts enhance semantic alignment while preserving language modeling quality.

Abstract:
Pause-token methods improve LLM reasoning by inserting special tokens into sequences. Prior work explains these gains through computational expressivity. However, there is relatively little investigation into the training dynamics of pause tokens. We explore how pause tokens reshape the training dynamics of fine-tuning. Two controlled pilots expose distinct asymmetries. On a synthetic continual-learning task, masked pauses overwrite a previously-learned distribution roughly 4× less at matched final adaptation (H1, mode retention ); on a synthetic math-reasoning probe, the boundary-adjacent token comes to encode substantially more downstream-step information (H2, non-myopic compression ). We formalize a training rule consistent with both — Masked Boundary Pause (MBP), pause tokens placed at reasoning-step boundaries with their loss masked. Across 1B-8B Qwen and Llama models, MBP consistently improves reasoning, achieving gains of up to 6 points on math and 2.5 points on code, while preserving general language understanding abilities. We further demonstrate that the gains from this mode-preserving strategy extend to GRPO. These results recast pause tokens as a training-dynamics intervention on the retention–adaptation trade-off, rather than merely an inference-time computation device.

Abstract:
Large language models (LLMs) exhibit in-context learning capabilities, where they can learn new tasks from prompt contexts without weight updates. We compare the learning efficacies of two prominent modes of in-context learning: (1) learning from descriptions of rules (instruction following); and (2) learning from examples of input-output demonstrations (few-shot prompting). Through five learning tasks that cover diverse domains (games, arithmetic, linguistic inferences), we compare two modes of learning (rules vs. examples) specifying the same underlying task. We furthermore explore model and task properties that modulate the learning efficacies. We find that models generally learn more reliably from rules than from examples alone, and additional examples on top of rules or simply scaling up the number of examples do not lead to consistent and significant gains. Instruction tuning amplifies the benefit of rule-based learning while keeping example-based learning capacities intact. Surprisingly, we find no privileged effect of example-based learning in base models, and rules still lead to gains in algebraic task domains. Overall, the comparative efficacy of rules over examples is larger when the task recruits algebraic abstractions and computations, and smaller when the task requires distributional sensitivity and/or recruits parametric knowledge.

Abstract:
LLM agents for sequential decision tasks are often post-trained with trajectory-level outcome labels, but such labels provide little supervision for preserving multiple successful branches from the same decision state. We study this problem as successful strategy coverage: how broadly a model realizes distinct successful strategies under a fixed rollout budget. We present Direct Diversity Optimization (DDO), an offline post-training method that combines Divergence-Tree Collection (DTC) with the Reference-Relative Target-Odds Objective (RTO). DTC constructs state-aligned branch sets rooted at shared decision states, and RTO trains the model to match reference-relative targets over successful alternatives. DDO achieves the strongest task success and successful strategy coverage among the compared post-training methods across BabyAI, BabaIsAI, and WebShop. It also achieves the highest recovery rate after local action replacement and higher task success and coverage than successful-only imitation and decoding-time diversification controls.

Abstract:
Large Language Models (LLMs) perform well on standard reasoning and question-answering benchmarks, yet such evaluations often fail to capture their ability to handle long-tail, expertise-intensive knowledge in real-world professional scenarios. We introduce LPFQA, a long-tail knowledge benchmark derived from authentic professional forum discussions, covering 7 academic and industrial domains with 430 curated tasks grounded in practical expertise. LPFQA evaluates specialized reasoning, domain-specific terminology understanding, and contextual interpretation, and adopts a hierarchical difficulty structure to ensure semantic clarity and uniquely identifiable answers. Experiments on over multiple mainstream LLMs reveal substantial performance gaps, particularly on tasks requiring deep domain reasoning, exposing limitations overlooked by existing benchmarks. Overall, LPFQA provides an authentic and discriminative evaluation framework that complements prior benchmarks and informs future LLM development. Our data is available at https://huggingface.co/datasets/m-a-p/LPFQA.

Abstract:
Large reasoning models (LRMs) achieve strong performance through extended reasoning traces, but they often exhibit overthinking behavior for low-complexity queries. Existing efforts to mitigate this issue are fundamentally limited by unstable accuracy–efficiency trade-offs and poor robustness to heterogeneous reasoning behaviors. To address these challenges, we propose a two-stage framework for stable adaptive thinking in LRMs. The framework first applies Hybrid Fine-Tuning to expose the model to both thinking and no-thinking behaviors, establishing well-conditioned initialization. It then performs adaptive reinforcement learning with Correctness-Preserving Advantage Shaping (CPAS) to avoid suppressing correct long-chain reasoning, and Length-Aware Gradient Regulation (LAGR) to stabilize optimization under severe reasoning-length heterogeneity. Extensive experiments on Qwen2.5-1.5B and 7B show consistent improvements over strong baselines, achieving up to +3.7/+3.6 accuracy points while reducing generated tokens by 40.6%/43.9%. Further analyses across varying problem difficulties and out-of-distribution tasks confirm the robustness and generalization of our approach.

Abstract:
Do language model benchmarks actually measure what practitioners intend them to? Benchmark metadata often obscures how benchmarks operationalize a capability: a "poetry" benchmark may omit haikus, while "instruction-following" benchmarks may test for some arbitrary set of skills. This coarseness prevents practitioners from inspecting how capabilities are operationalized at the item level, leading to unclear estimates of model competence. We introduce BenchBrowser, an auditing infrastructure that retrieves use-case-relevant items from 20+ benchmarks, letting practitioners inspect the evidence behind benchmark claims. Through a human evaluation with ten NLP practitioners, we show that the retrieved items are relevant and help support two meta-evaluations: surfacing content validity threats, where advertised capabilities are unevenly represented across facets, and convergent validity threats, where nominally similar benchmarks induce unstable model rankings. By tying validity concerns to concrete examples, BenchBrowser provides interpretable evidence to calibrate interpretation of benchmark scores and improve future iterations of benchmarks.

Abstract:
Long-horizon multi-agent systems suffer from three tightly coupled failures: task drift, role drift, and state staleness. We test the hypothesis that these share a common representational cause: agent hidden states move away from the directions encoding the active task, assigned role, and current shared state. A pilot study over 14,832 annotated turns supports this: drifted turns collapse near-uniformly across all three anchor subspaces (pairwise r = 0.72–0.78). Existing methods address these failures separately, operating at the surface-text level or treating control directions independently. We introduce ATLAS (Algebraic Task–role Locking with Adaptive Spectral Memory), a training-free intervention acting on hidden states before a token is generated. ATLAS builds a positive-semidefinite anchor kernel over task, role, and state subspaces and steers hidden states toward their shared high-coherence eigendirections; a low-rank Gram decomposition keeps this tractable. Across long-horizon memory, state maintenance, tool use, least-privilege coordination, and embodied stress tests, ATLAS outperforms prompting, retrieval, memory, and activation-steering baselines. Against A-MEM it improves memory macro-average accuracy by up to 5.7 points and reduces write/read failures by up to 31.7%/24.8%; against MemoryArena-Agent it gains up to 4.8 points. It gains 10.7–12.1 points under perturbation, reaches 93.5% role consistency, and transfers to a 72B backbone (macro-average 61.0 vs. 56.2). Ablating the joint task–role–state span costs 7.2 points against 0.8 for a matched-rank random span, indicating these directions are functionally implicated, not incidental; supplying them by prompt or retrieval recovers 4.3 points less. ATLAS is therefore a representation-level controller for open-weight multi-agent systems, complementary to output-level protocols for black-box agents.

Abstract:
Large language models (LLMs) perform well on multi-hop reasoning, yet how they internally compose multiple facts remains unclear. Recent work proposes hop-aligned circuit hypothesis, suggesting that bridge entities are computed sequentially across layers before later-hop answers. Through systematic analyses on real-world multi-hop queries, we show that this hop-aligned assumption does not generalize: later-hop answer entities can become decodable earlier than bridge entities, a phenomenon we call layer-order inversion, which strengthens with total hops. To explain this behavior, we propose a probabilistic recall-and-extract framework that models multi-hop reasoning as broad probabilistic recall in shallow MLP layers followed by selective extraction in deeper attention layers. Through probing and causal intervention analyses, we further show that shallow recall of deeper-hop knowledge can act as an intuition-like signal for answer selection, with stronger effects in weaker models and higher-hop queries. Our framework further explains prior empirical phenomena and provides a mechanistic diagnosis of multi-hop failures despite correct single-hop knowledge. Code is available at https://anonymous.4open.science/r/Layer-Order-Inversion/.

Abstract:
Many people code-switch, mixing multiple languages within a single conversation, text or both in everyday communication and increasingly in interactions with AI systems. While prior work has examined how code-switching affects LLM outputs, its impact on RAG systems remains largely unexplored. We present a systematic study of multilingual retrieval under code-switched queries. First, we introduce a retrieval evaluation framework jointly measuring ranking accuracy, language alignment, and bilingual coverage, revealing language preferences that standard metrics obscure. In particular, the strongest relevance-performing retrievers tend to exhibit an English preference. Second, we propose Language-Targeted Contrastive Training to mitigate this retrieval bias, but find a fundamental trade-off: improving alignment reduces exposure to relevant content across languages. Third, controlled RAG experiments show that generation quality depends on retrieval language, with the preferred evidence language varying across query languages. Together, our findings expose tensions between retrieval accuracy, language coverage, and fairness in multilingual RAG, arguing for evaluation protocols that account for query-language coverage.

Abstract:
Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong action, such as refunding the wrong purchase, can cause irreversible task failure and must be intercepted before execution. Such failures may not appear in every single run, but can emerge across repeated trials, making reliability across steps and trials critical. However, ensuring agentic reliability is challenging: even frontier LLMs struggle to explain why an action may be wrong, especially in long, intertwined trajectories governed by domain-specific policies. Much recent work relies on prompt-based critique agents, while optimization-based methods lack a systematic way to produce rich verification rationales for training. We address this gap with CAST, a critique-aware training framework that converts sparse task outcomes into action-level supervision for critique learning and policy optimization. CAST analyzes agent trajectories to synthesize structured rationales explaining action validity under partial observability. The resulting critique model is used to construct critique-aware training data for optimizing the policy model. Fine-tuning Qwen3-family models on dynamic tool-calling benchmarks, CAST improves reliability across domains, outperforming GPT-OSS-120B by over 10% pass^4 on Retail tasks and yielding an additional 9% improvement on Telehealth in an out-of-domain setting. These results demonstrate that critique-aware training improves the robustness of LLM agents in realistic dynamic environments.

Abstract:
Automated Text-to-Speech (TTS) evaluation methods (Mean Opinion Score (MOS) predictors and Audio-LLM Judges) are expected to reflect human perception, yet it is unclear how well they capture the distinct aspects of speech that listeners actually perceive. We deconstruct the under-specified "naturalness" into a linguistically grounded annotation schema spanning 10 distinct perceptual dimensions, and use it to construct the first dimension-level meta-evaluation benchmark for TTS, comprising 860 utterances annotated by trained linguist raters (Krippendorff’s up to 0.821). Results from benchmarking four MOS predictors and four Audio-LLM Judges reveals that MOS predictors collapse onto acoustic signal quality, while Audio-LLM Judges show selective, prompt-dependent detection that doesnt generalise across the dimension set. Neither class reliably captures linguistically structured speech errors. Our dataset, annotation schema, and evaluation code are publicly released to support more targeted and interpretable TTS evaluation.

Abstract:
The massive scale of Large Language Models (LLMs) incurs prohibitive computational costs, limiting their ubiquitous deployment. While activation sparsity is a promising training-free solution, existing magnitude-based methods overlook weight structures and dependencies. We reveal that standard weight norms fail as importance proxies, identifying instead that weight distribution heavy-tailedness is critical. To this end, we propose Kurtosis-Aware Coupled Sparsity (KACS), a novel training-free framework that introduces a Kurtosis-based metric to explicitly capture and prioritize outlier-rich weight columns. Furthermore, we develop an Interaction-Aware evaluation mechanism that assesses the joint importance of structurally coupled projections (e.g., Gate-Up and Q-K-V), ensuring information retention across interacting pathways. In addition, an adaptive sparsity allocation strategy is employed to distribute the sparsity budget across layers and components. Extensive experiments on Llama and Mistral models demonstrate that KACS consistently outperforms state-of-the-art baselines, retaining over 97% of the original performance at 50% sparsity.

Abstract:
Reliable handling of unanswerable questions (UAQs) is critical for trustworthy LLM-based agents. Although memory is widely used in agent systems, its role in reliable UAQ handling remains unclear. We present a systematic study of agent memory for UAQ handling under a unified agentic RAG framework, evaluating four representative memory methods across three UAQ-related datasets and two base models. We find that memory can improve UAQ performance in some settings, but such gains are selective rather than universal and remain fragile under dataset shift. Interestingly, cross-model memory reuse is often more feasible than cross-dataset transfer, suggesting that shifts in answerability patterns pose a greater challenge to memory reuse than changes in the base model itself. We further find that UAQ gains are more strongly preserved through decision guidance than through trajectory shaping, and that memory effectiveness depends strongly on representation. In particular, procedural and rule-based memories often provide the most reliable support for UAQ handling, while memory composition is most effective when procedural guidance is combined with complementary behavioral signals. Overall, our findings suggest that reliable UAQ memory depends less on storing larger amounts of experience and more on preserving transferable behavioral guidance.

Abstract:
Long-term personalized dialogue agents must track user preferences as their personas evolve. Existing memory systems organize past events well, but store personas as flat profiles detached from the events that justify them. This loose coupling leads to the memory–persona validity gap and the persona-aware retrieval gap. We propose PGMem, a heterogeneous persona-memory graph that connects event and persona nodes through typed provenance and evidence edges, keeping each persona signal traceable to the events that support or revise it. At retrieval time, PGMem expands from query-relevant seeds and ranks signals by evidential validity. Across three benchmarks with small language model backbones, PGMem consistently outperforms summary-based, persona-aware, graph-structured, and agentic memory baselines, and improves performance as the context grows. The source codes of PGMem are available at https://github.com/wonjunchoi23/pgmem/

Abstract:
Narrative NLP encompasses a diverse set of competencies such as detecting whether a text contains a narrative, classifying its genre or function, completing its plot, and judging similarity between stories. However, the relationship between model capabilities across these sub-tasks remains poorly understood and no unified benchmark exists to evaluate large language models across these dimensions simultaneously. In this paper, we introduce NARRATOR, a unified benchmark of 96,818 multiple-choice questions derived from six narrative datasets spanning detection, classification, story completion, and similarity. Evaluating 50 open-weight LLMs in a zero-shot setting reveals that LLM performance profiles are highly heterogeneous and no model dominates across all sub-tasks. To exploit this complementarity efficiently, we learn compact narrative competence embeddings of LLMs using a matrix factorisation over a binary correctness matrix, without accessing model weights. A single linear layer on top of these frozen embeddings supports model routing (71.1%), correctness forecasting (65.3%), and benchmark accuracy prediction (mean Kendall’s 𝜏 = 0.72 ) at near-zero inference cost.

Abstract:
Large language models (LLMs) struggle with complex, long-horizon reasoning due to instability from the frozen-policy assumption. Current test-time scaling methods treat execution feedback merely as an external signal for filtering or rewriting trajectories, without internalizing it to improve reasoning strategies. Inspired by Popper’s “conjectures and refutations,” we posit that effective reasoning requires real-time policy adaptation through learning from failures. We introduce Policy of Thoughts (PoT) , a framework that recasts reasoning as a within-instance online optimization process. PoT effectively generates diverse candidates via targeted exploration, then applies Group Relative Policy Optimization (GRPO) to update a transient LoRA adapter using execution feedback. This closed-loop design enables dynamic, instance-specific refinement of reasoning priors. Experiments demonstrate PoT’s substantial gains across multiple benchmarks. On LiveCodeBench, a 4B model achieves 49.71% accuracy, surpassing 12 test-time scaling baselines and commercial models (e.g., GPT-4o, DeepSeek-V3) despite being over 50 smaller.

Abstract:
The Model Context Protocol (MCP) enables large language models (LLMs) to access external resources on demand. While commonly assumed to enhance performance, how LLMs actually leverage this capability remains poorly understood. We introduce MCPGAUGE, the first comprehensive evaluation framework for probing LLM–MCP interactions along four key dimensions: proactivity (self-initiated tool use), compliance (adherence to tool-use instructions), effectiveness (task performance post-integration), and overhead (computational cost incurred). MCPGAUGE comprises a 160-prompt suite and 25 datasets spanning knowledge comprehension, general reasoning, and code generation. Our large-scale evaluation, spanning six commercial LLMs, 30 MCP tool suites, and both one- and two-turn interaction settings—comprises around 20,000 API calls and over USD 6,000 in computational cost. This comprehensive study reveals four key findings that challenge prevailing assumptions about the effectiveness of MCP integration. These insights highlight critical limitations in current AI–tool integration and position MCPGAUGE as a principled benchmark for advancing controllable, tool-augmented LLMs.

Abstract:
Legal Case Retrieval (LCR) aims to identify cases that are legally similar to a given query case. Existing research has largely recast LCR as the task of modeling the textual similarity between the query case and a historical case. While this approach can handle single-defendant cases fairly easily, such surface-level matching proves insufficient for multi-defendant cases, which typically requires modeling and aligning defendants’ roles and judgment outcomes in different cases. To address the challenges associated with Multi-defendant LCR (Multi-LCR), we (1) present Multi-ELAM and Multi-LeCaRDv2, two multi-defendant datasets we create by enriching widely used benchmarks with defendant-level annotations, and (2) introduce a novel role- and judgment-aware framework for Multi-LCR. Experimental results show that our approach outperforms state-of-the-art baselines, highlighting its effectiveness for Multi-LCR.

Abstract:
Universal multimodal retrieval (UMR) increasingly adopts multimodal large language models (MLLMs) as unified embedding backbones, but their strong retrieval performance comes at substantial inference cost. Existing methods typically rely on uniformly dense inference, where all input tokens are processed through the entire model and matched using the final-layer [EOS] representation. However, this paradigm overlooks two key forms of heterogeneity in multimodal retrieval: token contributions to the final retrieval embedding are highly uneven, and different queries require markedly different amounts of inference depth. To address this, we propose Skim and Skip (SAS), a hierarchical adaptive inference framework for efficient multimodal retrieval. SAS first performs token-level evidence selection to preserve only the input information most relevant to the final retrieval embedding, and then performs depth-adaptive inference to determine whether the current representation is already sufficient for reliable matching. Experiments on 12 MMEB retrieval tasks show that SAS retains about 99% of the dense baseline’s average retrieval performance while reducing LLM-side FLOPs by up to 66.3% and achieving up to 1.64× end-to-end speedup. Our code is available at [https://github.com/GaoMengGladys/SAS](https://github.com/GaoMengGladys/SAS).

Abstract:
The Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) provides open access to macromolecular structures and offers an extensive structural keyword search infrastructure. However, the current RCSB PDB APIs do not support natural-language semantic search. To overcome this challenge, we propose RAD-PROTEIN, a novel multi-source knowledge-enriched Retrieval-Augmented Protein structure search system that answers user natural-language queries by jointly leveraging heterogeneous structural literature and metadata. In addition, we introduce the first benchmark for text-based protein-structure search evaluation, comprising 454 expert-verified queries. RAD-PROTEIN substantially outperforms baselines, achieving up to +19.8% Recall@1 and +16.1% in MRR with hybrid retrieval and multi-source knowledge enrichment. Comprehensive ablation studies further confirm the robustness of our proposed approach. Finally, we develop an interpretable interface to return top-ranked structures with LLM-generated explanations, enabling efficient human verification and downstream scientific analysis. Code: https://github.com/Cassie07/RAD_PROTEIN

Abstract:
Instruction tuning is now the default way to train and adapt large language models, but many instruction–input–output pairs are only weakly specified: for a given input, the same output can remain plausible under several alternative instructions. This raises a simple question: does the instruction uniquely determine the target output? We propose the Task–Specificity Score (TSS) to quantify how much an instruction matters for predicting the target output by contrasting the true instruction against plausible alternatives for the same input. We further introduce TSS++, which uses hard alternatives and a small quality term to mitigate easy-negative effects. Across three instruction datasets (Alpaca, Dolly-15K, NI-20) and three open LLMs (Gemma, Llama, Qwen), our strongest evidence appears in the tightest budget regime, particularly at 5% retention. In this setting, TSS and TSS++ consistently outperform existing selection baselines across most model–dataset combinations. These results show that task specificity is a useful signal for low-budget instruction-data selection and complements existing filters such as perplexity and IFD.

Abstract:
Recent multimodal large language models have shown great promise in clinical image reasoning, but existing post-training pipelines remain predominantly outcome-centric, relying on final answer correctness or sequence-level preferences. This approach suffers from sparse credit assignment, making it difficult to optimize the internal reasoning process essential for clinical applications. Our analysis reveals that cascading errors from early-stage reasoning failures are a leading cause of incorrect predictions in medical visual question answering (VQA) benchmarks. Motivated by this, we propose Medical Reasoning-aware Policy Optimization(MRPO), an RL algorithm that incorporates step-wise process rewards. When the final answer is incorrect, MRPO assigns exponentially larger penalties to tokens in earlier invalid reasoning steps, breaking failure cascades without compromising successful paths. Across three multimodal LLM backbones, MRPO consistently outperforms standard GRPO and a recent RL baseline, and on Qwen3-VL-8B-Instruct even outperforms substantially larger medical MLLMs such as HuatuoGPT-Vision-34B by 2.79 points. Moreover, MRPO reduces early-stage reasoning failures from 64.0% to 13.0%, demonstrating that targeted mitigation of cascading failures improves both reasoning quality and final answer accuracy.

Abstract:
Large language models are increasingly regarded as strong translators, yet it remains unclear what their success establishes beyond overall translation quality. This question is especially consequential in Chinese web novel translation, where sentence-level cultural meaning shapes tone, characterization, and reader experience. Existing evaluation provides only indirect evidence, as benchmarks emphasize overall quality and available metrics do not explicitly assess the cultural and pragmatic phenomena underlying reader experience. We introduce DITING (a mythological creature known for discerning truth), the first comprehensive evaluation framework for this problem, supported by an expert-annotated benchmark of over 18K Chinese-English sentence pairs, a reasoning-driven multi-agent evaluator, AgentEval, and a 1,440-pair meta-evaluation benchmark, MetricAlign. AgentEval achieves the strongest correlation with human judgments among eight representative automatic metrics. Applying DITING to 14 translation systems shows that current progress is selective rather than general: frontier LLMs lead overall, but their gains concentrate in a subset of dimensions, while zero-pronoun resolution remains the clearest bottleneck. Overall, our results show that aggregate rankings substantially overstate how broadly sentence-level cultural meaning has been solved in web novel translation. All the code and data in https://github.com/whunextgen/DITING.

Abstract:
Large language models (LLMs) are increasingly required to process inputs composed of multiple context units, such as answer options, retrieved passages, or in-context demonstrations. Effective reasoning in such scenarios demands capturing both intrinsic semantics within each unit and the aggregated semantics across units. However, recent efforts typically treat multiple units as a single entangled sequence, ignoring the distinct knowledge within isolated units, which leads to suboptimal performance. To this end, we propose MUA(Multiple Context Units Attention), a dual-view self-attention module that computes a local view for learning unit-specific semantics and a global view to enable bidirectional interaction across units. These two views share projection matrices and are fused by a lightweight input-conditioned gate, while generation proceeds with standard causal decoding. With an optimized implementation, this design achieves efficient fusion of multiple context units and superior information utilization during input encoding at marginal overhead. Experiments on diverse multiple context units reasoning benchmarks, including multiple-choice, multi-hop QA and language understanding tasks, show that MUA consistently improves performance across model scales, exhibits increased robustness to context units reordering, and establishes new state-of-the-art results on key benchmarks.

Abstract:
For embodied agents capable of physical interaction, the capability to create and understand dialog is crucial to ensure both safety and effectiveness. While DialNav provides a framework for holistic evaluation of the dialog–execution loop in photorealistic indoor navigation, its performance remains limited by a critical scarcity of training data (2K episodes) To address this, we propose an automatic generation pipeline and generate the RAINbow dataset, a large-scale training dataset with 238K episodes for DialNav. Then, we introduce two additional complementary advances to unlock the data’s full potential: (1) a navigation training scheme to align the navigation training with the dynamic dialog-navigation loop, and (2) a localization model that leverages VLN knowledge. By combining these complementary solutions, our model substantially outperforms the success rate of the baseline on both Val Seen (58.24, +89%) and Val Unseen (29.05, +100%) splits, establishing a new state of the art. We will publicly release our dataset, code, and models.

Abstract:
Although LLMs perform well on static medical QA, adapting them to multi-turn diagnostic dialogue remains challenging: agents forget confirmed symptoms, ask redundant questions, and fail to narrow differentials, largely because they lack an explicit representation of their evolving clinical understanding. Our key insight is that a structured belief state—encoding accumulated evidence, ranked differential diagnoses with confidence, and prioritized next questions—can simultaneously serve as a supervision target, a basis for rule-based reward computation, and a grounding context for deciding what to ask next. We propose BridgeGap, a two-stage framework in which supervised fine-tuning on silver-annotated belief states teaches the agent to maintain structured clinical reasoning, while reinforcement learning with a belief-progress reward refines questioning strategy through dense, environment-verified per-turn feedback without LLM judges or learned reward models. On HealthBench, BridgeGap (8B) surpasses Doctor-R1 (8B) by 2.84 points and the 4× larger Baichuan-M2-32B by 5.85 points; on AgentClinic, it achieves 44.6% diagnostic accuracy in 10.5 average turns, compared to 37.6% in 15.5 turns for Doctor-R1. Across evaluations, BridgeGap advances interactive diagnosis while preserving factual knowledge on static medical benchmarks.

Abstract:
Interactive text-to-SQL agents solve database tasks through multi-turn interactions involving schema exploration, query execution, feedback interpretation, and decision revision. Long-term memory helps agents reuse past experiences, but existing retrieval methods remain limited. Static methods use fixed similarity heuristics, whereas dynamic methods often learn from sparse terminal outcomes and retrieve at a single decision horizon. This is insufficient when memory usefulness changes across interaction stages, since memories useful for initial planning may differ from those needed for local, state-conditioned execution. We propose MERIT, a dynamic multi-horizon memory retrieval framework. MERIT maintains episode-level memory for global strategic guidance and turn-level memory for local decision support. Both levels use learned retrieval policies optimized with reinforcement learning. To train turn-level retrieval despite limited intermediate supervision, MERIT uses a lightweight Process Reward Model to provide dense proxy rewards for local memory selection. On BIRD-Interact, MERIT achieves higher success rates and fewer interaction turns than no-memory, static-retrieval, and dynamic-retrieval baselines. Transfer results on Spider2-Snow further show that the learned policies transfer without degradation, staying above all memory baselines without benchmark-specific tuning. These results suggest that multi-horizon retrieval improves experience reuse in interactive text-to-SQL agents. Our code is available at https://github.com/YiboWANG214/MERIT.

Abstract:
Modern LLM agents persist state across sessions via two channels: long-lived memory files silently prepended to the system prompt, and scheduled tasks waking the agent without user input. A one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks. Existing prompt-injection benchmarks evaluate attacks only in the session where malicious content is first observed, leaving the gate governing when and how often a planted payload later re-fires unmeasured. We present CarryoverBench: 12 personal-assistant scenarios and 101 attack tasks across three trigger channels: Instant, same-turn IPI; Autonomous, heartbeat-scheduled; and Latent, memory-resurfaced. A rule-based verifier scores planting and triggering as independent stages. Across five frontier models, planted payloads re-fire without measurable decay over five rounds; conditional on successful planting, the trigger stage often provides little additional protection. Porting the Latent channel to a second runtime, Hermes, attenuates but does not eliminate the effect, suggesting that carryover defense requires runtime-layer controls in addition to prompt-level interventions.

Abstract:
Post-training quantization (PTQ) for multimodal large language models (MLLMs) faces a distinct optimization challenge: text and vision tokens can induce conflicting gradients when learning a shared channel-wise transformation. We identify this issue as modality gradient conflict, characterized by opposing update directions and severe magnitude imbalance between modality-specific gradients, and propose Modality-Coordinated Transformation Quantization (MCTQ) to address it. MCTQ combines whitening-based preconditioning, modality-decomposed reconstruction, and modality-coordinated optimization to learn a single shared transformation that serves both modalities without introducing modality-specific parameters or inference overhead. Across multiple MLLMs, benchmarks, and precision settings, MCTQ consistently outperforms existing channel-wise PTQ methods, with the largest gains under aggressive low-bit quantization. These results show that coordinating modality-specific gradients is critical for learning effective shared transformations in multimodal PTQ. Code and reproduction materials are available at https://github.com/jtsdw/UniMQ.

Abstract:
Self-reflection has become a key mechanism for improving reasoning in Vision-Language Models (VLMs), yet this corrective mechanism often fails when resolving complex fine-grained visual ambiguities. This performance degradation stems from the issue of modality disconnect in self-reflection: most existing models execute self-reflection either within textual or latent space, lacking an explicit mechanism to align textual reasoning with visual evidence. In this paper, we propose MIRROR, a closed-loop visual reflection framework comprising four steps: response generation, error identification, region-based visual verification, and revision. In this cycle, the model first generates an initial response, identifies uncertain logical statements in the response that require visual verification, then grounds them in relevant image regions, and finally revises based on the visual evidence. We construct a multi-turn visual reflection dataset ReflectV, which empowers the model with such a reflective capability. Extensive experiments across diverse multimodal benchmarks show that MIRROR improves performance and reduces visual hallucinations by an average of about 7.2%, demonstrating the advantage of transforming self-reflection from open-loop textual revision into closed-loop, visually grounded verification.

Abstract:
Standard approaches to benchmark redundancy rely on Spearman correlation, which implicitly treats dependence as uniform across the performance spectrum. Using copula-based dependence modeling on 4,497 models (779 deduplicated models; 714 unique base-model families) from the Open LLM Leaderboard v2, we show this assumption fails dramatically: benchmark pairs correlating at 𝜌 S > 0.73 show near-zero or reversed ranking agreement among top-performing models, with conditional Spearman reductions of up to 97%—findings that hold after cluster-robust inference at the base-model-family level ( ≥ 96% bootstrap stability for two of three headline pairs, 𝛥 BIC CIs entirely above zero). The mechanism is non-Gaussian dependence structure: 47% of benchmark pairs follow copula families with zero asymptotic tail dependence despite strong bulk association ( 𝛥 BIC up to 585), confirmed by nonparametric validation requiring no parametric assumptions. We validate across BenchPress (316 benchmarks; signal strengthens with sample size) and kensho/WILD (27 benchmarks, 65 independent-organization models; 38% Frank prevalence), and demonstrate that no scalar dependence measure—Spearman, distance correlation, or mutual information—can detect the tail decomposition. In plain terms: a model ranked first on one benchmark but last on another is not unusual among frontier models—precisely the ones practitioners are deploying.

Abstract:
Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a single globally dominant answer. We formalize this as localized knowledge disambiguation and introduce LocalNewsQA, an 18,700-item English-news benchmark that pairs the same question across two locales and scores whether a model actually switches its answer when the locale changes. We also introduce MAPLE, a controlled family of decoder-only models pretrained with document-level geographic metadata (source URL, country, and continent) already present in the training corpus, and compare it to metadata-free controls trained on identical data with the same token budget, architecture, and optimization. In controlled experiments at 1B and 3B, with inference-time metadata fixed, pretraining with metadata in MAPLE produces measurable switching and improves accuracy on questions whose correct answer depends on locale. Ablations and external-benchmark evaluations further suggest that locale-conditioned prediction benefits from geographic provenance learned during pretraining and that these benefits strengthen at larger model sizes.

Abstract:
Coreference consistency is essential for preserving discourse coherence in dialogue translation, yet it remains underexplored in the context of large language models (LLMs). In this paper, we present a comprehensive study of coreference preservation in Chinese-to-English dialogue translation. We construct a new dataset based on the RiSAWOZ corpus, annotated with source-side zero pronouns and coreference chains. Translations are obtained from a diverse set of LLMs of varying sizes. To evaluate whether coreference relations are maintained, we propose the Coreference Consistency Score (CCS), which quantifies the extent to which translated mentions preserve original coreferential chains. We further introduce an LLM-as-Judge protocol for assessing the faithfulness of translated mentions. Our results reveal that, despite producing fluent translations, many LLMs, including both small and large models, struggle to maintain cross-lingual coreference, highlighting the need for more discourse-aware dialogue translation models. All code and data will be released.

Abstract:
A concept can carry different associations across languages, while modern language models learn English alongside many other languages during pretraining. Yet comparisons among existing models cannot easily isolate how any one language changes the way these models represent English concepts because their training corpora, compute, architectures, and random seeds all differ. We study this question through a controlled experiment with 40 matched 310M-parameter decoder-only models that share an architecture, tokenizer, training recipe, and English data source. Each bilingual condition adds one of eight languages, while four experimental comparisons separately account for English exposure, total training, and English-document overlap. We align each model pair using 3,000 common English words, then measure where 1,000 held-out English concepts fall along 50 fixed semantic contrasts, such as red versus white. Across 32 experimental comparisons, English concept positions differ more between bilingual and English-only conditions than between English-only runs with different random seeds. These differences are larger in contextual states than in token embeddings and peak in middle layers. The language learned alongside English can therefore change how a model represents English concepts even when its English input representations are explicitly aligned.

Abstract:
Diffusion multimodal large language models (dMLLMs) frequently produce long-form outputs marred by semantic drift and repetition, with quality generally degrading as output length increases. We identify two structural deficiencies in existing decoding methods as primary drivers of these failures: confidence-based scoring ignores decoded-neighbor support, and block partitioning prevents access to high-readiness semantic anchors. Together, these deficiencies cause tokens to be committed before their local context is sufficiently established. We propose CACD (Context-Aware Cluster Decoding), a training-free decoding method that scores each masked position using a multiplicative combination of softmax confidence and neighbor proximity. This approach prioritizes contextually ready tokens over isolated candidates while suppressing low-confidence positional noise. CACD operates without block partitioning, keeping high-readiness anchors globally accessible. It further applies architecture-aware calibration to handle confidence heterogeneity induced by diverse visual integration strategies. Experiments on three dMLLMs across four benchmarks demonstrate consistent quality gains and hallucination reduction over the original decoding methods, with larger gains in several longer-generation settings. These results highlight the importance of neighbor support and visual integration strategies for the future design of dMLLM decoding methods. Our code is openly available at https://github.com/zhaoyk-sysu/CACD-dMLLM.

Abstract:
End-to-end GUI testing is essential for verifying web applications, yet existing evaluations rely on predefined checklists and are confined to the data and frameworks of web generation benchmarks, leaving the bug-discovery ability of vision-language models (VLMs) systematically untested. We introduce C ode-driven A gentic T esting (CAT), a paradigm in which the agent writes Playwright code to drive the browser, gathers feedback, and autonomously explores web applications to uncover bugs. We instantiate CAT with CATJudge, an agentic framework that unifies Browser-Use and Computer-Use tools within a single environment and CATTest, a benchmark of 102 AI-generated web applications with carefully annotated bugs, built through close human-AI collaboration to feature complex interactions and subtle defects. Experiments with mainstream VLMs show that all evaluated models perform poorly, revealing a clear gap between current VLM capabilities and the demands of real-world testing in AI web development.

Abstract:
Understanding how Vision-Language-Action (VLA) models transform multimodal knowledge into embodied control remains an open challenge. We present VLA-Trace, a progressive diagnostic framework that analyzes VLA models through a unified evidence chain from representation dynamics to causal control attribution and behavioral manifestation. It specifically combines cross-modal and checkpoint-drift centered kernel alignment (CKA) to trace representation evolution, attention knockout interventions to identify modality-specific control pathways, and rollout-level behavioral probes to examine grounding, shortcut dependence, and semantic following. Experiments on 𝜋 0.5 and OpenVLA reveal three key findings. First, the two models exhibit distinct modality-specific adaptation dynamics during VLA finetuning. Second, they rely on different multimodal routing strategies and layer-wise dependencies during action decoding. Third, although VLA policies excel at visually grounded trajectory generation, they remain limited in fine-grained semantic following. These findings highlight future directions for representation-preserving adaptation, causal VLA circuits, and compositional semantic control.

Abstract:
Retrieval-Augmented Language Models (RALMs) have shown strong potential in knowledge-intensive tasks, yet they remain vulnerable when retrieved contexts are noisy or irrelevant. Robustness against such contexts requires two distinct capabilities: abstention when contexts are uninformative, and selective extraction when relevant evidence is buried in noise. Yet existing methods face two key limitations: they do not train separately for these two capabilities, and they adapt the model at a coarse layer- or module-level granularity, overlooking that only a small subset of neurons is strongly activated for a given input. We propose NeuRIT, a Neuron-guided Robust Instruction-Tuning framework built on a localization-first perspective. NeuRIT mines context-aware neurons associated with relevant and irrelevant context processing, and uses them as anchors to selectively adapt both the identified neuron groups and the layers in which they concentrate. NeuRIT then performs two-stage instruction tuning that teaches complementary behaviors: suppress generation when there is nothing to extract, and extract relevant evidence when there is. NeuRIT consistently outperforms strong baselines across diverse QA benchmarks and generator backbones. Our code is available at https://github.com/HYU-ARK-Lab/NeuRIT.

Abstract:
Multimodal Entity Linking (MEL) aims to link mentions in a given context to their corresponding entities in a knowledge base by leveraging both textual and visual information. Existing MEL methods still underexploit visual information in real-world scenarios. Most approaches directly encode raw image-text inputs or apply uniform enhancement, while lacking adaptive modeling of heterogeneous mention-relevant evidence. Moreover, they typically rely on single-layer visual representations or simple cross-modal interactions, failing to capture hierarchical visual semantics required for fine-grained disambiguation. To address these limitations, we propose a novel MEL framework termed EVEMEL. First, we introduce an adaptive evidence augmentation mechanism, which analyzes the visual condition of each sample and strengthens mention-relevant discriminative clues, providing more reliable contextual evidence for entity disambiguation. Subsequently, we develop a cross-modal modeling mechanism that treats multi-layer visual representations as different semantic granularities and dynamically fuses them through token-level routing, enabling cross-modal matching. In addition, a bidirectional gated intra-modal mechanism is designed to capture multi-granularity textual and visual matching signals. Experimental results on public benchmark datasets demonstrate that the proposed EVEMEL outperforms all state-of-the-art baselines. The code is publicly available at https://github.com/hanwang86/EVEMEL.

Abstract:
Should multilingual LLMs answer medical questions consistently across input languages, or adapt responses to cultural cues? Existing multilingual medical benchmarks usually assume that medically correct answers should remain consistent across languages and treat cross-lingual variation as model error. In contrast, cultural adaptation research argues that appropriate medical answers may legitimately differ across contexts. We review the multilingual medical NLP literature through these two perspectives, we identify three gaps: limited stakeholder perspectives (e.g., of medical professionals), a lack of empirical evidence on which approach better serves users, and no benchmarks capable of distinguishing universally correct from culture-specific cases. To address the first gap, we survey 356 participants across three stakeholder groups (medical, NLP, and anthropology professionals) in three countries (Germany, Spain, and the United States). Anthropologists consistently favor adaptation, while medical and NLP respondents remain divided, with notable divergence between U.S. and European medical professionals. LLMs prompted with profession and country personas fail to reproduce this variation, overestimating cross-lingual consistency preference among NLP and medical personas. We conclude that neither consistency nor adaptation can currently be considered clearly preferable, highlighting the need for empirical evidence on which approach better serves users across cultural contexts.

Abstract:
Large language models (LLMs) are increasingly deployed in multilingual settings, yet the energy costs of serving these models across different languages remain poorly understood. We present a systematic study of inference energy consumption across languages with ML.Energy framework (CITATION). We find striking disparities: energy consumption per output token varies by up to 8.3 × across languages, while total energy for a fixed set of requests varies by up to 179 × between the cheapest (English, 17.6 kJ) and the most expensive (Pashto, 3 , 147 kJ) languages . Our analysis shows that this disparity is driven by two compounding factors: (1) higher per-token energy costs for languages using complex or rare scripts, and (2) more tokens generated for low-resource languages. Moreover, we find a double penalty: languages with the highest energy footprints also tend to achieve the lowest task accuracy. We reveal that the energy divide persists across models, hardware, and tasks, suggesting a systemic energy inequity in multilingual LLM deployment. Finally, we recommend that the community treat energy as a first-class evaluation axis, extend reporting checklists and model cards to include it, and adopt deployment-side mitigations for better energy efficiency.

Abstract:
Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively motivated approach, introduced in Berzak et al. (2018), proposed instead to predict language proficiency from behavioral traces of eye movements in reading. In this work, we validate and extend this approach from single sentences to more naturalistic reading of contextualized passages in English as a second language, new proficiency measures, prediction models, and reading in an information seeking regime. We find that the approach is effective in all these evaluations. We further address two key open questions on eye movement based proficiency testing: (1) potential scoring biases that reflect the proximity of the reader’s native language to English, which may undermine validity, and (2) its reliability. We find that eye movement based proficiency scores are indeed biased towards L1s that are linguistically closer to English. We propose a score debiasing method which effectively remedies this issue. The reliability analyses suggest that eye movement proficiency scores are more reliable than standard language proficiency scores. Overall, our results strengthen and broaden the empirical foundations for future eye movement based language assessment technologies.

Abstract:
Constructing inputs that drive execution to chosen code locations underpins many dynamic-analysis tasks in software engineering and security. It is a generalized constraint-solving challenge that requires agents to reason about the semantic mapping between input and code. We built COIN, the first benchmark to systematically evaluate this ability in modern code agents. By leveraging OSS-Fuzz’s long-running coverage data and maintainer-written harnesses, COIN constructs reachability challenges across over 1000 real-world projects in nine languages, and is contamination-resistant and self-refreshing as upstream code and OSS-Fuzz coverage evolve. To keep challenges non-trivial yet solvable, COIN samples two target families: lines OSS-Fuzz has covered but baselines miss, and lines never be reached but with static evidence of a reachable harness path. Across five independent runs on 137 targets from 15 projects in six languages, the strongest agent (Claude Code with Opus 4.6) reaches on average only 26.6% of the C/C++ targets and 39.7% of the Go/JVM/Python/Rust targets, never half of either set in any run, and the weakest (DeepSeek V3.2) stays below 9%, indicating substantial room for improvement.

Abstract:
Adapting large language models to user-specific preferences is often constrained by the cost of human annotation, making preference optimisation impractical in low-resource settings where preferences cannot be reliably labelled by LLMs themselves, e.g., due to cultural, subjective, or personalised contexts. In this paper, we investigate how language models encode preference information in their intermediate representations, finding that activations from chosen and rejected responses form distinct clusters across layers, even in pretrained models. Strikingly, this structure is strengthened by alignment on canonical datasets but erased when the target preferences differ from those the model was aligned on, suggesting aligned LLMs are poor judges for non-mainstream populations. Exploiting this structure, we propose training a lightweight linear probe on a few labelled preference pairs ( ≤ 500) and using it to annotate large unlabelled datasets (50K+) for downstream preference optimisation. We systematically evaluate this approach across different datasets, preference optimisation methods and model scales and find that our method consistently outperforms direct training given the same annotation budget, and remains competitive against baselines trained on 50-100× more labelled data in the majority of our settings.

Abstract:
In multimodal multi-hop question answering, we focus on the initial retrieval stage via two distinct tasks: (1) evidence set completion, retrieving missing evidence given context, and (2) sequential pool construction, iteratively building the top- K pool from the scratch. Under these settings, we point out that conventional iterative retrieval frameworks often suffer from Semantic Anchoring, where previously fetched evidence traps the retriever and yields entity-centric redundancy. To break this trap, we propose GRAIL (Gap-aware Retrieval via Adaptive Implicit Localization), a paradigm that performs implicit query rewriting directly at the embedding level. By context-subtractive query steering, GRAIL excels at compositional cross-modal reasoning, while additive embedding updates show strength on localized information aggregation. To leverage these complementary strengths, we propose a Task-Adaptive Hybrid Framework, achieving a 35.7% macro-averaged performance gain on the MultimodalQA evidence set completion task. Extensive evaluations demonstrate that sequential GRAIL retrieves in a superior, noise-resilient manner, significantly expanding the search horizon through iterative gap-aware optimization.

Abstract:
The rapid development of large language models (LLMs) for code, which closely aligns with the practical needs of real-world software development, has sparked growing attention in repository-level code completion. However, existing Code LLMs present challenges in focusing on suitable contexts from the repository and in deeply reasoning about cross-file dependencies. To address these challenges, we propose a novel reinforcement learning framework for repository-level code completion. To better understand both in-file and cross-file contexts, we employ identifier-driven intent recognition to capture completion intent, thereby improving the model’s performance in real-world scenarios. To enhance cross-file reasoning, we propose a reward-driven completion learning to effectively reward in complex repository completion scenarios. To guide LLMs in completing code that aligns with intended functionality and repository context, we introduce a selective-exploration strategy that directs the model to focus on low-confidence, high-reward tokens, promoting the exploration of valuable, underexplored completion patterns. Experimental results show that our approach significantly improves the performance of Code LLMs.

Abstract:
Current medical retrieval benchmarks primarily emphasize lexical or shallow semantic similarity, overlooking the reasoning-intensive demands that are central to clinical decision-making. In practice, physicians often retrieve authoritative medical evidence to support diagnostic hypotheses. Such evidence typically aligns with an inferred diagnosis rather than the surface form of a patient’s symptoms, leading to low lexical or semantic overlap between queries and relevant documents. To address this gap, we introduce R2MED, the first benchmark explicitly designed for reasoning-driven medical retrieval. It comprises 876 queries spanning three tasks: Q A reference retrieval, clinical evidence retrieval, and clinical case retrieval. These tasks are drawn from five representative medical scenarios and twelve body systems, capturing the complexity and diversity of real-world medical information needs. We evaluate 15 widely-used retrieval systems on R2MED and find that even the best model achieves only 31.4 nDCG@10, demonstrating the benchmark’s difficulty. Classical re-ranking and generation-augmented retrieval methods offer only modest improvements. Although large reasoning models improve performance via intermediate inference generation, the best results still peak at 41.4 nDCG@10. These findings underscore a substantial gap between current retrieval techniques and the reasoning demands of real clinical tasks. We release R2MED as a challenging benchmark to foster the development of next-generation medical retrieval systems with enhanced reasoning capabilities.

Abstract:
Large language model agents are increasingly expected to handle complex, long-horizon tasks that require sustained tool use and stateful interaction. Final outcome labels in existing agent benchmarks provide only a coarse view of long-horizon agent execution. They obscure differences in the execution process among successful trajectories and collapse failed trajectories that may have achieved different amounts of task-relevant progress. We introduce the Auditable Semantic Reference Graph (ASRG), a post-hoc framework that represents task-relevant progress with task-specific Semantic Reference Units (SRUs) and matches trajectories against these references. Each SRU is a checkable task milestone with an explicit verifier. Deterministic matching quantifies process coverage and coverage-weighted first-hit timing, while an action-level diagnostic records actions unexplained by the labeling rules. We apply ASRG to 4,740 Toolathlon evaluation trajectories from 22 models across 108 tasks. ASRG reveals process-coverage differences between models with identical final-success rates and first-support timing differences between same-task trajectories with matched coverage. By relating measured progress to resource use, ASRG also supports process-efficiency analysis. ASRG complements final outcome evaluation with auditable, reference-conditioned process diagnostics.

Abstract:
Long-video understanding (LVU) is increasingly framed as an agentic process that iteratively localizes task-critical moments before perception and reasoning. Moment localization is therefore a key bottleneck: under long-video context constraints, sparse sampling can easily miss decisive evidence, while similarity-based retrieval often returns semantically related but task-irrelevant clips. We present EviScan, an evidence-seeking moment localization model built on an MLLM, designed for task-oriented localization and near-second temporal scanning on hour-long videos. To learn task-oriented localization, we construct ToMo-Pile, a training set that pairs temporal intervals with rationale traces explaining why the interval resolves the task. To make near-second scanning feasible on long videos, EviScan uses DualKV Prefill for memory-efficient streaming inference, and Progressive Localization to first narrow down candidate intervals and then refine boundaries using high-resolution evidence. We further introduce ToMo-Bench, a benchmark for task-oriented moment localization across diverse long-video scenarios. EviScan consistently outperforms specialized localizers and general-purpose MLLMs on both ToMo-Bench and standard temporal grounding benchmarks. When used as a plug-in localization module, it yields substantial gains on downstream long-video understanding benchmarks (MLVU and LongVideoBench), and these gains are larger than those brought by other moment-localization methods. We will release the model, dataset, benchmark, and code.

Abstract:
Masked diffusion language models (dLMs) offer a promising parallel alternative to autoregressive models for complex reasoning. However, they face a unique credit-assignment challenge: reasoning failures typically stem from a few critical, early commitments that cascade downstream. Existing post-training overlooks this dynamic: supervised fine-tuning (SFT) weights all tokens uniformly, while online reinforcement learning (RL) attributes a single trajectory-level reward to every position. We introduce Pivot-SD, an efficient offline framework supervising only these high-impact bottlenecks (pivots). Pivot-SD isolates pivots using an information-gain metric measuring uncertainty reduction over the remaining masked block. It applies bidirectional supervision: successful pivots are reinforced with cross-entropy, while critical errors in failed trajectories are suppressed via targeted unlikelihood, preserving valid surrounding sub-steps. Using only 200 questions and four rollouts each, Pivot-SD improves LLaDA-8B-Instruct over full-sequence SFT and budget-matched diffusion RL baselines across math and code benchmarks.

Abstract:
Adaptive Retrieval-Augmented Generation (RAG) promises accuracy and efficiency by dynamically triggering retrieval only when needed and is widely used in practice. However, real-world queries vary in surface form even with the same intent, and their impact on Adaptive RAG remains under-explored. We introduce the first large-scale benchmark of diverse yet semantically identical query variations, combining human-written and model-generated rewrites. Our benchmark facilitates a systematic evaluation of Adaptive RAG robustness by examining its key components across three dimensions: answer quality, computational cost, and retrieval decisions. We discover a critical robustness gap, where small surface-level changes in queries dramatically alter retrieval behavior and accuracy. Although larger models show better performance, robustness does not improve accordingly. These findings reveal that Adaptive RAG methods are highly vulnerable to query variations that preserve identical semantics, exposing a critical robustness challenge.

Abstract:
Unlike human reasoning in abstract conceptual spaces, large language models (LLMs) typically reason by generating discrete tokens at each step, then discard the rich semantic information contained in the output distribution. Recent work such as Soft Thinking has shown that LLMs’ latent reasoning via soft concepts is a promising direction, but LLMs are trained on discrete tokens, leaving the continuous concept space underutilized during training. To address this, we propose Soft Concept Mixing (SCM), a soft concept aware training scheme that directly exposes the model to soft representations during training. Specifically, SCM constructs a soft concept vector by forming a probability-weighted average of embeddings. Then, this vector is mixed into the model’s hidden states, which embody rich contextual information. Finally, the entire latent reasoning process is optimized with Reinforcement Learning (RL). Experiments on six reasoning benchmarks demonstrate that SCM improves the reasoning performance of LLMs, and simultaneously maintains a stable training dynamic.

Abstract:
LLMs often struggle with deductive judgement in syllogistic reasoning, systematically conflating semantic plausibility with formal validity—a phenomenon known as content effect. This bias persists even when models generate step-wise explanations, indicating that intermediate rationales may inherit the same semantic shortcuts that affect answers. Recent approaches mitigate this issue by increasing inference-time structural constraints, either by encouraging abstract intermediate representations or by intervening in the model’s internal computations; yet, reliably suppressing semantic interference remains an open challenge. To make formal deduction less sensitive to semantic content, we introduce a framework for abstraction-guided reasoning that separates structural inference from lexical semantics. We construct paired content-laden and abstract syllogisms and use the model’s activations on abstract inputs to define an abstract reasoning space. We then learn lightweight Abstractors that, from content-conditioned residual-stream states, predict representations aligned with this space and integrate these predictions via multi-layer interventions during the forward pass. Using cross-lingual transfer as a test bed, we show that abstraction-aligned steering reduces content-driven errors and improves validity-sensitive performance. Our results position activation-level abstraction as a scalable mechanism for enhancing the robustness of formal reasoning in LLMs against semantic interference.

Abstract:
As large language models are increasingly deployed in high-stakes domains, it is essential that their outputs reflect not average human preference, but diverse perspectives. Achieving such pluralism remains challenging: existing pluralistic alignment methods rely on prompt-level interventions, represent only a limited set of values, or are compute-heavy, lacking explicit value control and representation. Furthermore, existing steering methods require manual value selection and cannot dynamically adapt to scenario-dependent value requirements, limiting their scalability and generality. To address these limitations, we introduce VISPA, a training-free pluralistic alignment framework that performs automatic, context-driven value selection and modular value-steered generation. VISPA identifies scenario-relevant values dynamically and applies activation steering to produce value-conditioned responses representing distinct perspectives. These responses enable support for three pluralistic alignment modes: Overton, Steerable, and Distributional. VISPA supports the addition of new values without finetuning and is compatible with diverse steering instantiations and steering models. Across extensive experiments spanning multiple models and evaluation settings, VISPA demonstrates strong performance across all pluralistic alignment modes in healthcare and beyond.

Abstract:
Open-world logical entailment requires deciding whether a set of premises entails a conclusion, entails its negation, or leaves it undecided. Many existing methods are deductive in spirit, searching for a proof and reading the label from whether that proof succeeds. When implemented through LLM-based proof search, this makes the prediction depend on a single unstable trajectory, and a failed or partial search is easily mismapped to the wrong label. We propose ABED (Assumption-Based Bidirectional Entailment Diagnosis), a training-free inference-time framework that reframes entailment classification as missing assumption diagnosis. Instead of asking only whether a conclusion can be proved, ABED asks what assumption is missing for it to hold, and classifies that gap as already closed by the premises (Null), contradicted by them (Contra), or left open (Admissible). It applies this diagnosis to both a conclusion and its negation, and determines the label from the resulting dual gap pair. Across FOLIO, ProofWriter, ProntoQA, and LogicNLI, ABED consistently outperforms prompting-based, symbolic, solver-based, and deductive baselines, improving over the strongest baseline on each dataset by 3.4, 5.3, 0.8, and 4.4 points with GPT-4.1.

Abstract:
Generative point-of-interest (POI) recommendation models based on large language models (LLMs) have shown promising results by formulating next POI prediction as a sequence generation task. However, the knowledge encoded in these models remains fixed after training, making them unable to perceive evolving real-world conditions that shape user mobility decisions, such as local events and cultural trends. To bridge this gap, we propose AWARE (Agent-based World knowledge Augmented REcommendation), which employs an LLM agent to generate location- and time-aware contextual narratives that capture regional cultural characteristics, seasonal trends, and ongoing events relevant to each user. Rather than introducing generic or noisy information, AWARE further anchors these narratives in each user’s behavioral context, grounding external world knowledge in personalized spatial-temporal patterns. Extensive experiments on three real-world datasets demonstrate that AWARE consistently outperforms competitive baselines, achieving up to 12.4% relative improvement.

Abstract:
Matching submissions with suitable reviewers at scale is a growing challenge for major venues, yet existing approaches either rely on coarse proxy signals that conflate general relatedness with true suitability, or require expensive human annotations that are difficult to scale for training. We propose MERIT, a two-stage framework that bridges this gap by converting criterion-level expertise matching into scalable suitability supervision. In the first stage, we train a reviewer assessor via reinforcement learning to identify the expertise dimensions a paper requires, match them against the reviewer’s prior work, and produce a suitability decision, with rewards provided by an LLM judge guided by paper-specific expertise rubrics. In the second stage, we distill the assessor’s predictions into an embedding-based retriever for efficient large-scale assignment. Experiments show that our 4B reviewer assessor outperforms larger general-purpose LLMs on suitability classification, and the resulting retriever achieves state-of-the-art performance across LR-Bench and the CMU Gold dataset. Our code is available at https://github.com/Luli3220/MERIT.

Abstract:
LLMs are extremely good code translators, except when it matters most: translating from low-resource to high-resource programming languages. The reason may lie in the fact that code translation relies on superficial lexical cues rather than on exploiting the deeper, language-agnostic program structure. We propose BoJac (Bohm–Jacopini), a novel commenting approach that pushes LLMs to exploit program structure by verbalizing structured programming principles in a guided natural language format. By using structure, BoJac enhances the ability to translate low-resource to-high-resource code. Using equivalent code benchmarks across multiple language pairs, we compare BoJac against several forms of auxIliary guidance provided alongside the source code: synthetic free-form chain-of comments, abstract syntax tree, and I/O Traces. To disentangle structural understanding from identifier-level matching, we repeat all experiments under name mangling and extend the same comparison to LoRA Fine-tuning on signal-augmented training sets. Our results show that the strongest gains come from auxiliary signals expressed in natural language, particularly when they explicitly encode structural and computational-level properties of the source code as in BoJac. While purely structural representations appear less effective, hybrid approaches such as BoJac, which convey structural and paradigm-level information through formalised natural language, emerge as a promising direction.

Abstract:
Large language models (LLMs) are increasingly used as computational models of human behavior. Recent models such as Centaur have advanced this goal by fine-tuning LLMs on trial-by-trial decisions from psychological experiments. However, existing large-scale behavioral datasets do not link the same participants across different tasks, so these models learn population-level regularities but are not trained to model individual-level regularities across tasks. We introduce Cross-Psych-101, a dataset of 591 participants and 261 , 551 valid decisions across 12 Bandit, Tradeoff, Structure, and Social tasks. We train Cross-Centaur to predict a participant’s target-task actions from a target-task description and the source-task description paired with the same participant’s source-task behavior. Cross-Centaur outperforms off-the-shelf LLMs and Centaur on held-out participants, both within and across tasks. Probing analyses suggest that its hidden states encode participant-level behavioral tendencies that transfer across tasks. These results highlight the potential of cross-task individual-level data for predicting human decision-making with language models. Cross-Pysch-101 is available at https://huggingface.co/datasets/HikaruAsano/cross-psych-101.

Abstract:
While retrieval-augmented generation (RAG) enhances LLM performance, it does not eliminate hallucinations, making accurate detection essential. Uncertainty-based methods are attractive for this purpose because they can be integrated into real-world pipelines with little overhead. One of the most widely used uncertainty signals is predictive entropy. We show, however, that entropy can be unreliable in RAG settings and trace this limitation to two opposing internal effects. Induction heads, which copy patterns from earlier context, causally support correct responses and lower predictive entropy, but they also appear to co-activate entropy neurons that push it back up. As a result, correct, context-grounded responses can still receive high uncertainty scores. To address this, we propose INTRYGUE (Induction-Aware Entropy Gating for Uncertainty Estimation), a training-free, mechanistically grounded method that gates predictive entropy by an attention-based estimate of induction-head activity. Evaluated across four RAG-style benchmarks and six open-source LLMs (4B to 13B parameters), INTRYGUE performs competitively against a wide range of baselines, matching or exceeding the strongest of them in most settings. Our findings suggest that hallucination detection in RAG benefits from combining predictive uncertainty with interpretable internal signals of context utilization.

Abstract:
Despite remarkable capability in multi-modal understanding, deploying Multi-modal Large Language Models (MLLMs) in open-ended conversational scenarios introduces safety risks that remain poorly addressed by existing alignment methods. Unlike simple malicious visual question and answer (VQA) pairs , multi-turn interactions enable adversaries to incrementally reconstruct harmful intent across dialogues, progressively bypassing safety constraints in ways that are difficult to detect at any individual turn. Meanwhile, conventional reinforcement learning from human feedback (RLHF) approaches are unsuitable for this situation: designed primarily for VQA tasks, they neither capture cross-turn risk dynamics nor scale efficiently without costly manual preference annotation. To close this gap, we introduce MINT-Safe , an open-source visual multi-turn training dataset comprising 11,270 multi-image dialogues and 500 refusal VQA pairs, constructed via multi-agent interaction with text-to-image (T2I) tool-call augmentation. Building on MINT-Safe, we propose TAD-Align , a dialogue safety alignment framework centered on a turn-aware dual-objective reward function. Rather than treating all dialogue turns uniformly, TAD-Align leverages rollout-based safety score variance to dynamically identify turns where the model exhibits inconsistent safety behavior, and adaptively up-weights these turns during optimization. Experiments on Qwen2.5-VL-7B-Instruct and LLaVA-NeXT-7B demonstrate reductions of over 10% in Attack Success Rate (ASR), alongside improvements of at least 8% in harmlessness and 13% in helpfulness on multi-modal multi-turn safety benchmarks, while preserving general model capabilities.

Abstract:
Current evaluations of memorization in Large Language Models (LLMs) predominantly rely on overlap ratios, which fails to distinguish between sporadic token matches and reliable, sequential recall. What’s more, these methods become inapplicable in practical scenarios where ground truth suffixes is unavailable. To address these limitations, we propose a paradigm shift from quantifying "rates" to determining the Effective Memorization Length—the distinct boundary where a model transitions from faithful recall to open-ended generation. We introduce a Ensemble Memorization Classifier that leverages internal states from the LLM to estimate this boundary in a reference-free manner. This framework enables Confidence-Aware Generation Control, allowing to extract high-precision suffixes by dynamically truncating sequences precisely at the memorization boundary. Furthermore, we demonstrate that this boundary is optimizable; by integrating our classifier as a reward model within a reinforcement learning framework, we actively extend the average effective length of generated suffixes. Experiments on the GPT-Neo and Pythia families demonstrate that our approach not only accurately filters non-memorization tokens but also effectively "unlocks" deeper chains of memorized knowledge, transforming memorization from an unpredictable risk into a controllable utility.

Abstract:
As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has become a critical challenge. While recent work has moved beyond single-turn evaluation toward multi-turn paradigms, key limitations persist: step-level methods treat actions in isolation, missing how risks accumulate, and trajectory-level evaluations operate post-hoc, offering no opportunity for timely intervention. To address this, we formalize Decoupled Proactive Safety Monitoring along three dimensions—whether, when to intervene, and what the risk is—and introduce PASTABench, a benchmark of 1,139 multi-turn trajectories spanning 5 risk categories and 13 sub-categories. We further propose the Optimal Intervention Window (OIW), anchored by annotated Earliest-Signal and Trigger turns, to quantify intervention timeliness. Evaluation of 16 LLMs reveals that proactive intervention remains largely unsolved, with the best model achieving only 40.74% optimal-timing interventions. Fine-grained diagnosis further uncovers pervasive lexical overfitting: competitive safety scores of smaller models mask keyword hypersensitivity rather than genuine risk comprehension, as their proactive capability largely collapses once hazard vocabulary is neutralized.

Abstract:
Despite the remarkable reasoning capabilities of large language models, they still struggle with one-shot relation extraction without predefined relation labels. We identify two pitfalls: models are often misled by irrelevant tokens instead of relation-conveying semantics, and they often fail to align with the abstraction level human annotators expect. We introduce a novel framework that closes this gap with two components: (1) CogRE, a cognitively-inspired reasoning framework that structures RE into a series of processes mimicking human text-processing; and (2) Hit@Dict, a reinforcement learning intermediate reward strategy that ncourages reasoning to align with relational labels by rewarding relation-relevant phrases in reasoning. The reward is based on a credit dictionary automatically extracted from gold outputs. Our experiments show that our framework improves both accuracy and explanation quality by addressing these two pitfalls. For example, CogRE with Qwen2.5-14B-Instruct on One-shot NYT29 achieves 24.65% F1, surpassing prior reasoning-based designs. Optimizing this approach with RL using Hit@Dict further improves performance by +23.46% points. Finally, human evaluation shows that our best model generates relational phrases closely aligned with gold labels, increasing human explanation quality ratings by 54% (relative).

Abstract:
The sudden emergence of induction circuits and the ability to perform in-context copying in LLMs coincides with a sharp drop in training loss over a short span of the training process. This co-occurrence has been cited as evidence that it marks the point at which a wide range of in-context learning (ICL) capabilities begin to emerge. To analyze the training dynamics of few-shot abstractive ICL capabilities (i.e., tasks where the answer is not contained in the input context), we propose Hapax, a training regime that omits the loss contribution of tokens predictable by the induction circuit to control training pressure for copying. Using this intervention, we find that abstractive ICL capabilities can still emerge, despite a substantial reduction in inductive copying and the absence of the characteristic phase changes observed during Vanilla training. Analysis of attention heads shows that Hapax-trained models develop fewer and weaker induction heads. Together, these results suggest that the developmental link between emergent phenomena associated with induction circuits and the emergence of abstractive ICL is weaker than previously hypothesized. More broadly, Hapax provides a controlled framework for analyzing how copying-related training signals shape the development of in-context learning.

Abstract:
Memes are a dominant medium for online communication and manipulation because meaning emerges from interactions between embedded text, imagery, and cultural context. Existing meme research is distributed across tasks (e.g., hate, misogyny, propaganda, sentiment, humour) and languages, which limits cross-domain generalization. To address this gap, we propose MemeLens, a unified multilingual, multitask explanation-enhanced Vision-Language Model (VLM) for meme understanding. We consolidate 38 public meme datasets, filter and map dataset-specific labels into a shared taxonomy of 20 tasks spanning harm, targets, figurative/pragmatic intent, and affect. We present a comprehensive empirical analysis across modeling paradigms, task categories, and datasets. Our findings suggest that robust meme understanding requires multimodal training, varies substantially across semantic categories, and remains sensitive to over-specialization when models are fine-tuned on individual datasets rather than trained in a unified setting. We make the experimental resources (Code: https://github.com/MohamedBayan/MemeLens | MemeLens-VLM: https://huggingface.co/QCRI/MemeLens-VLM | MemeLens-Datasets: https://huggingface.co/datasets/QCRI/MemeLens) publicly available to the community.

Abstract:
Can LLMs reason through new information like humans, or do they merely retrieve cached opinions? This question is critical for the growing practice of silicon sampling, where LLM personas simulate public opinion at scale. Current evaluations test only whether these personas hold the right opinions — a static snapshot. But opinion research increasingly depends on dynamic fidelity: whether personas update beliefs in response to new arguments, as humans do during deliberation. No existing benchmark tests this capacity. We introduce the Deliberative Polling Diagnostic Framework, which fills this gap by comparing human and LLM belief shifts after identical informational interventions. Grounded in the methodology of deliberative polling, the framework surfaces failures invisible to static evaluation: models that produce plausible partisan opinions can still misrepresent how those opinions change. Applying the framework to five frontier models using data from America in One Room (526 personas, 72 questions), we observe that every model fails, and in a unique manner. GPT-5.1 exhibits reversal: its personas become more hostile toward the opposing party after balanced information, while humans become less so. This reversal is selective (80% on outgroup vs. 26% on policy questions) and symmetric across partisan identities. Gemini 2.0 Flash, Claude Sonnet 4.5, and Llama 3.3 70B exhibit overshoot, shifting correctly but at 5-7x human magnitude. DeepSeek V3 exhibits rigidity with near-zero change. Targeted ablations reveal that policy content triggers these failures and that they are identity-specific: GPT-5.1 reverses on outgroup questions but overshoots on ingroup; Gemini shows the inverse. We term this behavioral signature self-sycophancy: conformity to the model’s internal stereotype of the persona rather than reasoning from the information provided. Our framework offers practitioners a concrete protocol: run the deliberative diagnostic before trusting LLM personas to accurately mimic revised beliefs.

Abstract:
Modern instruction-tuning mixtures are increasingly large, redundant, and heterogeneous, making full-data supervised fine-tuning costly and often unnecessary. We propose GRADFILTERING, an objective-agnostic, validation-free data selection method that scores examples using uncertainty-aware proxy training dynamics. GRADFILTERING trains a lightweight proxy model with a LoRA ensemble and ranks examples by a Gradient Signal-to-Noise Ratio (G-SNR), which combines early-to-late gradient drop with late-stage ensemble disagreement. Because selection requires neither target validation examples nor target-model gradients, the selected subsets can be reused across downstream targets. Across Alpaca and Tulu-3-52K, dense LLaMA-2-7B, LLaMA-3-8B, and Qwen3-8B targets, and a Qwen3-30B-A3B MoE target, GRADFILTERING-selected 5–15% subsets consistently outperform random selection and are competitive with the target-aware LESS-T reference while remaining validation-free. We further evaluate broad model capabilities on objective benchmarks and analyze proxy choice, gradient windows, ensemble size, human preferences, and selection cost, demonstrating the practical utility of early proxy dynamics for reusable, compact instruction-tuning data selection. Code is available at this GitHub repository.

Abstract:
Large Language Models (LLMs) can match local semantic clues in knowledge graphs (KGs), but complex KGQA often requires satisfying the full dependency structure of a question. Existing retrieval-based methods therefore risk producing reasoning paths that look plausible at each step but violate the query’s global intent. We propose Guiding Graph, a schema-light query-structure interface that turns KGQA from open-ended semantic traversal into structure-constrained query completion. A Guiding Graph exposes the entities, placeholders, and global dependencies that KG exploration must preserve, serving as an operational interface between natural language and graph traversal. The interface is training-free by default through prompt-based construction, and can optionally be strengthened with a cold-start fine-tuned constructor trained from limited bootstrapped supervision. Given the constructed Guiding Graph, exploration is governed by target selection from the query structure, structure-guided pruning before semantic matching, and target-scoped semantic alignment under the current Guiding Graph triple. Experiments on WebQSP, CWQ, and a domain-specific agricultural KG show that explicit query-structure guidance improves accuracy, robustness, and efficiency, with especially clear benefits for multi-hop reasoning and smaller LLMs.

Abstract:
Latent reasoning offers a more efficient alternative to explicit reasoning by compressing intermediate reasoning into continuous representations and substantially shortening reasoning chains. However, existing latent reasoning methods mainly focus on supervised learning, and reinforcement learning in latent space remains highly unstable. We study this problem through the lens of Group Relative Policy Optimization (GRPO), and show that directly adapting GRPO to latent reasoning is fundamentally non-trivial: latent reasoning changes both the probability density and the sampling mechanism, causing three coupled bottlenecks: absence of intrinsic latent manifolds, where unconstrained exploration pushes rollouts off the valid latent manifold; exploration-optimization misalignment, where trajectory-level rewards can induce incorrect token-level updates; and latent mixture non-closure, where jointly reinforcing multiple correct latent paths can produce an invalid averaged state. To address them, we propose Latent-GRPO, which combines invalid-sample advantage masking, one-sided noise sampling, and optimal correct-path first-token selection. Across four low-difficulty benchmarks (e.g., GSM8K-Aug) and four high-difficulty benchmarks (e.g., AIME), Latent-GRPO improves over its latent initialization by 7.86 Pass@1 points on low-difficulty tasks and surpasses explicit GRPO by 4.27 points on high-difficulty tasks while using 3–4 shorter reasoning chains. It also achieves stronger pass@k performance under Gumbel sampling. These results establish Latent-GRPO as an effective approach for stable and efficient latent reasoning.

Abstract:
Latent reasoning offers a computation-efficient alternative to Chain-of-Thought but often suffers from performance degradation due to distributional misalignment and ambiguous chain definitions. Ideally, latent reasoning should function as a superposition of multiple reasoning paths. To realize this, we introduce Latent-SFT, a unified framework addressing challenges at three levels: token, chain, and learning. First, we define the Latent-Vocab to constrain hidden states within the pre-trained vocab-space. Second, we construct the Latent-Chain via Induction-Supervision Masking to ensure semantic compactness and sufficiency. Third, we employ Latent-Optim with stochastic Gumbel-Softmax to guide the model toward generalizable solutions. Empirical results demonstrate that Latent-SFT consistently outperforms explicit SFT across six mathematical benchmarks (e.g., GSM8k, AIME24) while achieving a 2.7x to 5.5x reduction in reasoning length. Analysis confirms that our method effectively captures a superposition of diverse reasoning trajectories rather than merely compressing a single path.

Abstract:
The prohibitive memory footprint of the Key-Value (KV) cache imposes a critical bottleneck for efficient long-context LLM serving. Current eviction methods, ranging from attention-based heuristics to recent learnable approaches, rely on scoring criteria decoupled from the downstream task performance that ultimately matters. To address this, we introduce KV-Retriever , a layer-wise learnable importance scoring module directly optimized by the end-to-end downstream task loss. We demonstrate that aligning the scoring objective with the actual task signal—rather than intermediate proxies such as attention statistics or oracle labels—yields a fundamentally more accurate notion of token importance. Functioning as a lightweight, plug-and-play module with only ∼ 0.01% additional parameters on a frozen 8B backbone, KV-Retriever is compatible with diverse host eviction frameworks. Extensive experiments show that, for each backbone, one trained retriever can be reused unchanged across multiple KV cache compression architectures and delivers consistent improvements on LongBench, LongBench-v2, and RULER. KV-Retriever requires one-time supervised training with golden answers; “plug-and-play” refers to deployment into otherwise unchanged host eviction frameworks, not to training-free adaptation. Our code is available at https://github.com/Taylor-Gavel/kv_retriever .

Abstract:
Deep search agents solve knowledge-intensive tasks through long-horizon, multi-turn interactions, where assigning credit to individual search decisions is fundamentally difficult. Outcome-based reinforcement learning provides only a single terminal reward shared across an entire trajectory. Recent process-supervision methods aim to assign credit at finer granularity, but they typically anchor credit to either surface-level trajectory states or model-internal signals, both of which become unreliable in deep search where observations are heterogeneous and retrieved contexts are long and noisy. We propose KnowGPO, a reinforcement learning framework that aligns trajectories by the task-relevant knowledge they have acquired rather than by their surface observations. For each query, KnowGPO decomposes the information required to derive the answer into atomic knowledge conditions, and represents the state of a trajectory at each step by the subset of conditions it has satisfied. Trajectories sharing the same knowledge state are grouped for within-group comparison, and the incremental knowledge gain between consecutive steps serves as an environment-grounded process reward. Experiments on Xbench, BrowseComp-ZH, and GAIA show KnowGPO consistently outperforms outcome-only (GRPO) and process-supervision baselines (GiGPO, IGPO) by 3.8 to 4.2 points, with our 8B model rivaling or exceeding most 32B open-source systems. Our code is available at https://github.com/ZJU-REAL/KnowGPO.

Abstract:
Parameter-Efficient Fine-Tuning (PEFT) adapts large language models (LLMs) by training only a small fraction of parameters. %Adapter-based approaches reduce compute per step but introduce practical overhead from the additional adapter path (e.g., extra kernel launches and activation storage). Adapter-based approaches reduce the number of trainable parameters, but introduce practical overhead from the additional adapter path (e.g., extra kernel launches and activation storage). Adapter-free approaches avoid this structural overhead by directly updating pretrained weights; however, random index selection can fragment the trainable subspace, while calibrated weight selection incurs extra cost, limiting accuracy or efficiency. We propose Recursively Aligned Pathway Adaptation (RAPA), an adapter-free PEFT method that forms index-consistent pathways through depth. RAPA follows two principles: (i) selecting balanced submatrices that maximize the number of weights alignable across layers, and (ii) recursively aligning these indices across layers and residual connections. In experiments, RAPA matches or surpasses strong PEFT baselines across most benchmarks while preserving adapter-free efficiency with minimal memory and compute overhead.

Abstract:
On today’s social media, many argumentative artifacts appear in mixed image-text format. Meanwhile, the argument mining community has seen much success in developing argument understanding technologies in text. In this work, we bridge that gap by studying vision-language argument (VLA) parsing: generating an argument graph that captures both the explicit and implicit components of a vision-language argument. To lay the groundwork for this task, we annotate MemeArgs, a corpus of argument graphs for 614 persuasive memes. We then propose an automatic evaluation protocol for VLA parsing that accommodates its generative nature and follows standard argument mining practice. Evaluating current general-purpose vision-language models, we found that they all failed to perform knowledge-grounded visual perception reliably, which distorts their understanding of the arguments. VLA parsing thus presents an exciting challenge for multimodal argument understanding.

Abstract:
Randomized smoothing provides certified robustness guarantees, but its direct application to text remains challenging due to the discrete, sparse, and semantically structured nature of textual inputs. We propose CRICO, an input-dependent covariance optimization framework for certified robustness in the text embedding space. CRICO estimates token-level decision influence in a black-box manner, making it readily extensible to LLMs. By adapting the certified ellipsoid to local decision structure, CRICO significantly enhances certified robustness. Experiments on multiple tasks demonstrate its effectiveness and broad applicability.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning, but GRPO-style optimization remains prone to collapse. We analyse this instability through token-level gradient dynamics, deriving a taxonomy that predicts how updates affect next-token probabilities and entropy. The taxonomy shows that stability depends jointly on the advantage sign and token distribution under the current policy. Motivated by this finding, we propose Winner Advantage Policy Optimization (WAPO), a simple online clipped policy-gradient objective that updates only on positive-advantage completions. Across mathematical reasoning and multi-hop QA benchmarks, WAPO improves training stability and matches or outperforms baselines across multiple model families. Full code can be found at https://github.com/layer6ai-labs/wapo.

Abstract:
Recent research demonstrates that pretrained large language models (LLMs) require minimal but carefully curated data for instruction-tuning. Building on LLMs, multi-modal models further contend with far larger data volumes, heterogeneous sources, and cross-modal alignment, amplifying both the necessity and the complexity of data selection. However, existing methods rely on deeply extracted embeddings, gradients or narrow quality scores that are time-consuming to compute and limited in expressiveness, frequently failing to surpass a simple random-sampling baseline. To overcome these limitations, we step back from complex embedding spaces and target the fundamental objective of instruction tuning: how and what knowledge should be elicited from the pretrained model and exposed to users. We leverage interaction Styles as a proxy for structural diversity, and decompose data quality into capability-specific Scores, yielding an explicit yet Rich (SSR) profile for each sample. We further cast the selection as submodular coverage maximization over the capability-style grid, providing an approximation guarantee. SSR consistently delivers superior performance across two distinct architecture stacks, LLaVA-1.5 (Vicuna + CLIP) and LLaVA-OneVision (Qwen2 + SigLIP), two large-scale data pools, six data budget settings, and 14 multi-modal benchmarks. To our knowledge, SSR is the first data selection method validated at million-scale, achieving 99.1% of full-data performance using only 30% of a 2.6M pool. Furthermore, the SSR profile models and selected subsets transfer training-free across data pools and model architectures with capability extensibility.

Abstract:
We investigate the factors affecting salience in movie screenplay by posing two research questions: 1. Does Uniform Information Density (UID) predict salient scenes? 2. What other linguistic properties distinguish salient scenes from non-salient background? We answer both questions through a controlled corpus study on the MENSA corpus, paired with a Linguistically-Grounded Salience Model (LGSM), a dual-stream classifier that fuses a frozen BERT encoder with explicit linguistic features through adaptive gating. The UID picture is layered in long-form discourse. Perscene UID summaries carry no independent signal once length is controlled, but at the token level salient scenes match non-salient material in the bulk while specifically suppressing the rarest, hardest-to-predict tokens in the upper tail. UID therefore lives at the within-scene token level rather than as a per-scene aggregate. Salient scenes also show a moderate and sustained cast of central characters (following an inverted-U pattern), strong third-person usage, syntactically simpler language, and more abstract vocabulary about motivation and conflict. LGSM exploits this signature and improves recall on salient scenes over a strong semantic baseline while yielding better downstream summaries.

Abstract:
Retrieval-augmented generation assumes that language models can use relevant context while rejecting semantically related but non-evidential context. Further, multilingual retrieval adds a second assumption that when evidence appears in different languages, models arbitrate by factual content instead of language cues. We test both these assumptions as generator-side failures to understand how models bind, compare, and trust evidence across languages. We first study near-miss contamination where we add true, relation-matched context about a nearby entity that does not answer the target question. Across Mintaka and PopQA, this context raises distractor adoption before any direct contradiction is introduced, and the effect persists across every tested model and language. We then study language evidence arbitration where English and local-language evidence disagree. We find that language preference is model-family dependent. GPT-5.4-mini leans toward English across all four languages, while DeepSeek models lean toward local-language evidence for all tested languages. We also run a label-only control where both snippets are written in English and only the source label varies and find that evidence labeled as English is over-trusted, with 95% bootstrap confidence intervals excluding zero across all 16 model-language pairs. Our results thus show that context use is shaped by entity proximity, language preference, and source-language metadata.

Abstract:
Editable diagram generation from natural language requires structural correctness and visual readability. Text-to-image models produce rasters with no editable objects. LLM code-generation approaches yield editable output but face three challenges: unreliable geometry from token-by-token coordinate emission, shallow verification that checks code rather than the rendered image, and dependence on task-specific fine-tuning. We present VISTA, a training-free pipeline that assigns each subproblem to the appropriate component: an LLM extracts semantic structure, a deterministic algorithm computes all geometry, and a VLM inspects the rendered diagram jointly with its source and issues localized repairs. We construct a 150-prompt benchmark from PNG-only references and evaluate against eight baselines including Codex, Claude Code, and a fine-tuned multi-agent system. VISTA achieves the highest readability across all pairwise comparisons. Code and data are available at https://github.com/nice-grp/Vista.

Abstract:
The development of autonomous web agents, powered by Large Language Models (LLMs) and reinforcement learning (RL), represents a significant step towards general-purpose AI assistants. However, training these agents is severely hampered by the challenges of interacting with the live internet, which is inefficient, costly, and fraught with risks. Model-based reinforcement learning (MBRL) offers a promising solution by learning a world model of the environment to enable simulated interaction. This paper introduces DynaWeb, a novel MBRL framework that trains web agents through interacting with a web world model trained to predict naturalistic web page representations given agent actions. This model serves as a synthetic web environment where an agent policy can dream by generating vast quantities of rollout action trajectories for efficient online reinforcement learning. Beyond free policy rollouts, DynaWeb incorporates real expert trajectories from training data, which are randomly interleaved with on-policy rollouts during training to improve stability and sample efficiency. Experiments conducted on the challenging WebArena and WebVoyager benchmarks demonstrate that DynaWeb consistently and significantly improves the performance of state-of-the-art open-source web agent models. Our findings establish the viability of training web agents through imagination, offering a scalable and efficient way to scale up online agentic RL.

Abstract:
Mixture-of-Experts (MoE) models are widely used to scale language models, yet their expert routing behavior and adaptation in a multilingual setting remain underexplored. In this work, we study multilingual routing dynamics during continual pre-training of an English-centric MoE model on a multilingual corpus, analyzing how expert usage varies across languages. We find that continual multilingual pre-training leads to diffused, language-agnostic routing in early and middle layers, with language specialization primarily emerging in the final layers. We also show that token-level vocabulary overlap between languages plays an important role in how languages are routed. Motivated by these findings, we propose a parameter-efficient adaptation strategy that updates language-specific and shared experts in the final MoE layers. Experiments on MultiBLiMP and Belebele show that our method achieves a strong performance–efficiency trade-off, attaining competitive performance relative to fine-tuning complete final layers, while updating less than 2% of the parameters. Overall, our findings provide insights into where and how language specialization emerges in MoEs during continual pre-training and provide practical insights for low-resource multilingual adaptation.

Abstract:
Human beings primarily understand the world through concepts (e.g., dog), abstract mental representations that structure perception, reasoning, and learning. However, how large language models (LLMs) acquire, retain, and forget such concepts during continual pretraining remains poorly understood. In this work, we study how individual concepts are acquired and forgotten, as well as how multiple concepts interact through interference and synergy. We link these behavioral dynamics to LLMs’ internal concept circuits, computational subgraphs associated with specific concepts, and incorporate graph metrics to characterize circuit topology. Our analysis reveals: (1) LLMs’ concept circuits provide a non-trivial, consistent signal of concept learning and forgetting; (2) concept circuit topology evolves stage-wise, rising during concept acquisition, and transiently rising before declining during concept forgetting; (3) semantically similar concepts induce stronger interference than weakly related ones; (4) conceptual knowledge differs in their transferability, with some significantly facilitating the learning of others. Together, our findings provide a circuit-level view of concept learning dynamics and motivate concept-aware training strategies, such as Circuit-aware Experience Replay, which uses circuit topology to prioritize concepts vulnerable to forgetting

Abstract:
Generalisations are common in scientific communication, even though they are semantically ambiguous. An automated method is needed to identify and categorise claims according to their level of generalisation, in order help detect an over-reliance on generalisations and possible misrepresentations of scientific findings. We introduce a comprehensive taxonomy of generalisations in the scientific domain, NLPGENX, which labels claims according to their level of generality and framing within the text. We operationalise this taxonomy with an LLMpowered framework, NLPGENA, that automatically classifies sentences from scientific articles into 5 different generalisation classes. We validate our framework with human annotators and use the framework to construct a large-scale dataset of NLP papers annotated according to generality, with auxiliary labels for hedging and vague descriptors (NLPGENS). We use NLPGENS to analyse the use of generalisations in NLP papers across multiple venues and subdomains, and to examine associations with citation counts, hedging, and vague descriptors.

Abstract:
Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assumption underlying benchmark scores is that the model performance measured through APIs faithfully reflects the behavior of deployed systems. We challenge this assumption by auditing ChatGPT, Claude, and Gemini across seven systems and nine benchmarks spanning general capability, social bias, and sycophancy. We find systematic API–interface differences in both accuracy and consistency. On average, API evaluations score 3.4 percentage points higher in accuracy and 2.1 percentage points higher in test–retest agreement than corresponding interface evaluations. For ChatGPT, the performance difference between API and interface access rivals the API-only difference between GPT 5.3 and GPT 5.4. Put differently, switching access surfaces can degrade performance as much as downgrading a full model generation. We further test whether exposed API controls can reproduce interface behavior by varying system prompts, sampling parameters, and reasoning settings. These controls shift behavior in some cases but do not reliably eliminate the gap. Our findings document a context-validity gap: measurements obtained through APIs do not necessarily generalize to corresponding deployed interfaces, complicating the use of API evaluations as proxies for deployed systems.

Abstract:
Agent memory systems are increasingly used to maintain long-term user preferences, task states and evolving facts, but current evaluations often collapse memory behavior into final-answer accuracy. We introduce MemProbe a cognitive-science-inspired framework for diagnosing stability–plasticity tradeoffs in agent memory. The framework is motivated by a core insight from cognitive memory research: memory is reconstructive and shaped by interference, source reliability, reinforcement, and reactivation. MemProbe turns this insight into four reusable experimental paradigms (interference, misinformation, consolidation strength, and reconsolidation window) that manipulate when a memory should be updated, preserved, or treated as uncertain. It further decomposes correctness into behavioral profiles that reveal how systems update, preserve, attribute, and temporally organize information. We instantiate these paradigms in a 56-episode diagnostic suite and evaluate six incremental memory systems under a unified protocol. Results show that systems with similar aggregate scores exhibit distinct behavioral profiles. MemProbe provides such a diagnostic lens, turning aggregate performance into interpretable profiles of memory maintenance over time.

Abstract:
Video-Language Models (VLMs) excel at visual recognition but struggle with causal reasoning. Current evaluations are flawed: multiple-choice QA allows shortcuts, and open-ended QA lacks robust automatic evaluation. We introduce Causal Chain Generation (CCG), a novel task where models have to generate the sequence of intermediate events connecting a cause and an effect. This structured setup demands genuine reasoning and enables reliable automatic scoring. We accompany this task with two complementary metrics: SODA-Embedding, which measures the semantic matching of a generated chain against a reference, and Causal-QA, which probes the chain’s causal coherence by asking whether it contains the specific causal facts needed to reconstruct the narrative. We provide two test sets: SF20K-CC, a manually annotated diagnostic on 20 short films, and CMD-CC, a movie clip benchmark derived from audio descriptions. Our analysis reveals that state-of-the-art VLMs struggle with narrative causality understanding, frequently hallucinating or omitting key events. To address this, we fine-tune VLMs with GRPO using a composite reward that combines a format reward, SODA-Embedding, and Causal-QA accuracy, demonstrating substantial improvements over strong baselines.

Abstract:
As agentic AI systems tackle more complex mathematical tasks, they increasingly rely on information retrieval (IR) to search problem databases, theorem libraries, and educational resources. However, choosing the right retriever remains difficult, as it is infeasible to isolate its effect on downstream performance. On the other hand, existing retrieval-specific benchmarks often fail to capture fine-grained mathematical relevance, penalizing systems that retrieve useful documents. We address this gap with SABER-Math, the first fully automated reranking benchmark for mathematical IR, built without expert annotation. Starting from 283K high-school-level problems with solutions, it builds challenging tasks in three steps: (i) LLMs extract concise solution summaries and topics for each problem; (ii) candidates are discovered per query using ontology-based topic similarity and lexical overlap between solution summaries; and (iii) a Swiss-style LLM preference tournament assigns every candidate a graded relevance rating. Over these judged candidate pools, we evaluate retrievers across all major paradigms, from lexical and math-specific systems to modern embedding, late-interaction, and reranking models. Reasoning embedders lead, followed by general-purpose models, both far outperforming classical and math-specific baselines. Yet, all still struggle in symbol-heavy domains such as Algebra and Calculus. Importantly, we show that generic IR benchmarks, such as MTEB, track mathematical performance broadly but are less discriminative among the strongest models, highlighting the need for domain-specific evaluation.

Abstract:
As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inference-time alternative to fine-tuning methods (e.g., RLHF, DPO) for behavioral control. However, existing work typically validates steering on isolated behaviors, leaving it unclear whether steering vectors encode coherent semantic structure or merely exploit behavior-specific shortcuts. We investigate whether the latent geometry of LLM steering vectors reflects theory-specified structure in human values and morality. Using Schwartz’s Theory of Basic Human Values as our primary fine-grained framework, we introduce a 26K-sample benchmark covering 20 human values and analyze distribution-driven methods (e.g., CAA, SphericalSteer, ODESteer) and behavior-centric approaches (e.g., COLD-Steer, BiPO) across diverse model families and sizes. We find that distribution-driven methods recover human value topologies aligned with theoretical predictions (Spearman 𝜌 up to 0.51, p < 10 -13 ). In contrast, behavior-centric methods achieve comparable steering performance but show little correlation with the expected value geometry. Geometric fidelity improves with model scale but drops after instruction tuning. Finally, better geometric alignment also leads to more human-consistent transfer across values: steering one value correctly lifts compatible values and suppresses opposing ones. Code and data are available at: https://github.com/DeepRCL/Steering_Geometry .

Abstract:
Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model’s hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, and show two challenges absent in encoders: (1) the causal attention mask causes bias to accumulate across token positions, requiring position-dependent correction; and (2) not all token positions are equally important, i.e., high-entropy (decision-critical) positions matter far more than low-entropy ones. To address these challenges, we propose Decoder-Aware Representation Tuning via Surgery (DARTS). DARTS employs a novel entropy-weighted L1 loss to upweight correction at high-entropy positions where errors most affect generation quality, and a per-position additive bias that captures position-dependent error without overparameterization. We perform extensive evaluation on three domains: code generation (HumanEval), mathematical reasoning (GSM8K), and instruction following (AlpacaEval) on Llama-2-7B models, and show DARTS achieves significant improvement over the standard surgery approach while adding negligible parameters (0.1% of total parameters).

Abstract:
We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language models (LLMs) and large vision-language models (LVLMs). PetQA contains 10,076 text-only and 8,751 multimodal QA pairs derived from real-world questions about dogs and cats, paired with answers from expert veterinarians. Its test split, PetQA-Bench, further includes annotations for question types and clinical conditions. We evaluate eighteen models using ROUGE, BERTScore, and LLM-as-a-judge metrics for factuality and helpfulness under three settings: zero-shot inference, retrieval-augmented generation (RAG), and supervised fine-tuning (SFT). The benchmarking results provide an overview of the strengths and limitations of current models in addressing veterinary clinical queries and highlight the need for more effective adaptation methods to develop clinically reliable AI systems for veterinary care. To facilitate broader use, we additionally provide translated versions of PetQA-Bench in five languages.

Abstract:
Legal Judgment Prediction (LJP) aims to predict the charge, relevant law article, and term of penalty from case descriptions. Existing LLM-as-Judge solutions apply retrieved historical cases to enhance prediction, but the retrieval might contain irrelevant and redundant information, thereby limiting their accuracy. Moreover, these solutions rely on a single LLM for prediction and lack mechanisms to address potential bias and hallucination in its inference. To address these limitations, we introduce the Collaborative Reasoning with Reranked Multi-facet Retrieval (CoRe) framework, which retrieves historical cases from multiple facets, reranks them as candidate precedents, and employs two collaborative LLM judges to make the final prediction. The two LLM judges are inspired by the collegiate bench mechanism in real-world judicial practices, where Judge A makes a preliminary prediction that guides Judge B in re-selecting precedents and producing the final prediction. Experiments on public datasets validate the effectiveness of our CoRe framework in terms of better prediction performance over the competitors.

Abstract:
The ability of Large Visual Language Models (LVLMs) to identify hate speech in memes across several cultures requires (1) access to cultural knowledge that is not readily available to them; and (2) the ability to reason with this knowledge. This paper shows how culturally-aware Hate Frames can be acquired to inform hate speech identification across various cultures. The paper also shows that counterfactual reasoning is ideal for identifying hate speech and generating explanations. Promising results show not only improved hate speech identification in each culture, but also reduction of the performance gap of hate speech identification between different cultures.

Abstract:
Multi-round reflection lets agents built on large language models recover from failures within a single task, but lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code-generation task: from a scientific paper section, the agent writes executable Manim code. We present ManimAgent , a self-evolving multimodal agent that carries reflection experience across tasks through a dual-channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision–language model scores the rendered keyframes; the resulting signals populate a positive channel ℳ + that stores success rationales as soft Reference Examples, and a negative channel ℳ - that stores VLM-screened failure patterns as hard Known Pitfalls. On a fixed-probe evaluation against no-memory, matched-budget retrieval-augmented generation, and shuffled-memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows. We will release the code, frozen memory snapshots, and the task stream.

Abstract:
Mixture-of-experts (MoE) architectures scale large language models efficiently, but they demand massive GPU memory. To cope with such demand, models are commonly compressed to reduce their memory footprint. Residual sparsification is a representative compression technique that decomposes each projection matrix of an expert into a shared base matrix and per-expert residual matrix, and then compresses the residuals. Existing sparsification methods compress each residual matrix independently by minimizing its compression error, thereby minimizing the error of each projection matrix. However, our analysis shows that this objective is misaligned with preserving model accuracy after compression. In an expert, the final output is produced through computations coupled across multiple projections and hidden representations. Therefore, even small errors in individual matrices can propagate through hidden representations and projection interactions, leading to large expert output errors and accuracy degradation. To address this misalignment, we propose PARSER, a new residual sparsification method that shifts the compression objective from minimizing isolated matrix errors to preserving the expert output error. PARSER achieves this by introducing output importance, which measures the actual contribution to the expert output error. Our experiments show that, compared with existing methods, PARSER narrows the accuracy gap to the uncompressed model by 1.41 × on Qwen and 1.44 × on DeepSeek, while achieving the same peak memory reduction.

Abstract:
Computer-use agents ground natural-language instructions in screenshots to locate interface elements, yet existing benchmarks do not isolate whether models bind relational language to the correct element. We introduce GUI-Primitives, a 994-item benchmark of contrastive instruction pairs over seven spatial relations in graphical user interfaces (left/right, above/below, containment, alignment, proximity, list ordinal, occlusion). Each pair holds the screenshot and anchor fixed while changing the relation expression, so the correct target moves between two designated candidates. Five annotators validate a 196-item subset ( 𝜅 = 0.94 well-formedness; 𝜅 = 0.79 target selection). Nineteen vision-language models reach at most 32% strict point-in-box accuracy. Because models emit unconstrained coordinates, we classify each prediction by the candidate region in which it falls. Predictions fall outside both candidates on 60–92% of items. Conditional on falling within a candidate region, target selection reaches 0.82–0.90 for horizontal position, vertical position, proximity, and list ordinal, but does not differ significantly from 0.50 for containment and occlusion: most failures reflect candidate localization rather than relation understanding. Across ten models, benchmark accuracy correlates with ScreenSpot-Pro accuracy (Spearman 𝜌 = +0.74 ), an exploratory association at this sample size. Marking the two designated candidates raises selection accuracy by 35–57 percentage points, an oracle diagnostic that supplies the candidate set rather than a deployable method. We release the benchmark, predictions, and code.

Abstract:
Large language models (LLMs) are increasingly used for molecular tasks, but it remains unclear which molecular representation to use. We present a systematic benchmark, MolRepBench, to evaluate LLM molecular competence across nine representations and eight chemical tasks. We benchmark 16 LLMs across five model families, including reasoning and non-reasoning variants, chemistry-specialized LLMs, and closed frontier models. Performance is strongly representation-dependent and no single representation wins across tasks, though CML is the best, followed by MolJSON, InChI and canonical SMILES. Explicit structured text representations (CML and MolJSON) dominate structural tasks; IUPAC dominates semantic tasks, winning molecule retrieval for all 16 LLMs; and SMILES variants are rarely optimal despite their prevalence in pretraining. Chemistry-specialized models perform well with SMILES at the cost of large degradations with structured text representations, suggesting SMILES-only evaluation rewards specialization that does not generalize. Using LLM-as-a- judge, we find that IUPAC produces the highest fraction of correct molecule generations. A mechanistic study via tokenization audits, linear probes, and attention shows that representations are encoded differently inside the model; for example, structured representations require more attention across the molecular span. Our results argue against representation-invariant evaluation and motivate task-aware representation routing for LLM-based chemistry.

Abstract:
Despite advances in safety alignment, large language models remain vulnerable to continuously evolving jailbreaks. Existing fine-tuned safety classifiers cannot adapt to these evolving attacks, while adaptive memory-based guardrails tend to over-refuse benign queries that resemble stored attacks. We propose Membrane, a self-evolving guardrail built on Contrastive Safety Memory (CSM): each cell pairs the conditions for blocking a harmful query with those for permitting a superficially similar benign request. Without retraining, Membrane evolves CSM by distilling each harmful interaction and its benign counterpart into a contrastive cell indexed by the underlying attack strategy, so that one cell generalizes across topical variants of the same mechanism. At inference, retrieved cells serve as grounding context for precise safety decisions. Across model-level safety on HarmBench and agent-level safety on AgentHarm, Membrane achieves the highest F1 on all six modern jailbreak attacks. Notably, benign refusal on AgentHarm stays at 7-14%, well below the 28-85% range of prior guards. Memory cells also retain 87-88% F1 under cross-attack transfer and remain stable under memory poisoning.

Abstract:
Understanding the internal mechanisms of transformer-based large language models (LLMs) is crucial for their reliable deployment and effective operation. While recent efforts have yielded a plethora of attribution methods attempting to balance faithfulness and computational efficiency, dense component attribution remains prohibitively expensive. In this work, we introduce Layer-wise Target Propagation (LTP), a novel framework that faithfully traces information flow on the frozen transformer in one forward and one backward pass without requiring counterfactual examples. LTP analytically decomposes and linearizes the computational structure of the Transformers into distinct pathways along which it propagates a targeted unembedding vector to receive the effective representation at each residual position. This target-centric propagation achieves O(1) time complexity with respect to the number of model components, scaling to long input sequences and dense component attribution. Extensive experiments on standard interpretability benchmarks demonstrate that LTP achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines. O(1) time complexity with respect to the number of model components, scaling to long input sequences and dense component attribution. Extensive experiments on standard interpretability benchmarks demonstrate that LTP achieves state-of-the-art faithfulness and unprecedented efficiency compared to existing baselines.

Abstract:
Large Language Model judges are widely used to rank texts and text-generating systems through pairwise comparison, and their reliability is typically assessed via three proxies: position bias, transitivity, and pairwise agreement (self- or human-labeled). Because these proxies drive judge selection and benchmarking, a substantial literature reporting that judges perform poorly on them risks steering practitioners away from otherwise capable evaluators. We argue this assessment is misleading. Under the Bradley–Terry geometry underlying pairwise aggregation, each proxy is dominated by close-rank-gap pairs, where inconsistency is information-theoretically expected and individual verdicts contribute little to the aggregate ranking; far-gap pairs carry the ranking signal but barely move the proxies. We formalize this argument and validate it in a controlled simulation and on two human-rated corpora: the proxies correlate only weakly with ranking accuracy against gold, and their predictive component concentrates in the far-gap regime. Judges should therefore be assessed on rank-gap-conditional metrics, ideally against human rankings. Code in the supplementary zip archive.

Abstract:
Post-training compression is essential for deploying large language models (LLMs) under tight resource constraints. Tensor decompositions have emerged as a promising direction, offering compact parameterizations well suited to Transformer weight structures. However, existing studies evaluate these methods in narrow settings, leaving unclear whether tensorization is effective at large-scale deployment. We systematically evaluate tensor compression across dense and MoE architectures, establishing performance trade-offs grounded in both empirical analysis and theoretical analysis. % We identify a fundamental mismatch between the shared subspaces assumed by tensor decompositions and the heterogeneous representations learned by modern LLMs, thereby delineating their practical limits and clarifying their viable role in large-scale deployment. We identify a fundamental mismatch between the shared subspaces assumed by tensor decompositions and the heterogeneous representations learned by modern LLMs, thereby delineating their practical limits that persist across an extensive range of mainstream dense and MoE deployment settings evaluated in this work. The code is available at https://github.com/brain-lab-research/TT-LLM.

Abstract:
Large language models exhibit impressive cross-lingual capabilities. However, how cross-lingual generalization emerges has only been analyzed through isolated factors and at sparse points during training, limiting our understanding — particularly of the early phases of learning. To study the early trajectory of linguistic and translation capabilities, we pretrain a multilingual 1.7B model on nine diverse languages, capturing checkpoints at a much finer granularity. We use word-level translation as a testbed, introducing a novel dataset to trace how translation develops over training through behavioral analyses, model-component analysis, and parameter-based ablations. We find that the model quickly acquires basic linguistic capabilities in parallel with token-level copying, while translation develops in two distinct phases: an initial phase dominated by copying and surface-level similarities, and a second phase in which generalizing translation mechanisms develop and copying is refined. Together, these findings provide a fine-grained view of how cross-lingual generalization develops during multilingual pretraining.

Abstract:
Political evasion refers to responses that engage with a question while withholding the requested information. Recent NLP work frames political evasion as a classification task using a two-level taxonomy of response clarity and fine-grained evasion strategies. Existing work on response clarity and evasion classification is limited to English, leaving open whether the taxonomy and model behavior transfer across languages and political contexts. We introduce PolERo , a dataset of 3,574 human-annotated question-answer pairs extracted from official transcripts of five Romanian presidents. We evaluate multiple classification approaches on both datasets under matched conditions, including TF-IDF baselines, fine-tuned encoder models, a proposed sliding-window encoder, and zero/few-shot LLM prompting. We study cross-lingual transfer through joint bilingual training and machine-translation-based data augmentation. Our results indicate that fine-tuned encoders are competitive, cross-lingual transfer is asymmetric, and ambivalent evasion categories involving pragmatic cues remain the main challenge across all model families.

Abstract:
Language-model agents increasingly act through tools, APIs, databases, and other stateful environments, where task success depends on external evidence rather than final text alone. Existing reward paradigms face a trade-off: preference and rubric-based feedback are flexible but often weakly grounded, while verifier-based rewards are grounded but typically tied to narrow task-specific checkers. We argue that agentic reinforcement learning needs reward interfaces: mechanisms that explicitly connect task requirements to environment evidence, provenance, and reward aggregation. We introduce Agentic Rubrics, an executable reward-interface view with four components: criteria, evidence acquisition, provenance, and aggregation. This view separates what should be evaluated from how evidence is obtained, audited, and converted into reward. We also specify a minimal environment contract for auditable reward construction and give an illustrative instantiation in stateful tool-use environments. In this case study, Agentic Rubrics augment final environment feedback with process-aware rewards derived from tool traces and state records, yielding modest improvements in task performance and process reliability while revealing domain-dependent limitations. These results support the position that grounded agent learning should treat reward construction as an interface problem, not only as scalar reward modeling.

Abstract:
Long-form retrieval tasks such as report generation, multi-document summarization, and retrieval-augmented generation require systems to surface evidence covering multiple aspects (subtopics) of a complex query. Existing approaches to information coverage typically fall into two families: (1) methods such as Maximum Marginal Relevance (MMR) that rerank initial retrievals using document-similarity heuristics, and (2) expansion-based methods that invoke large language models at query time to produce subqueries, pseudo-answers, or expanded keywords. We propose SPLADE-COVER, a learned sparse retrieval model whose expansion mechanism is trained to be coverage-aware. SPLADE-COVER inherits the SPLADE-v3 architecture but is trained with an objective that distills multi-aspect subquery supervision into a single sparse representation, requiring no language-model invocation or reranking at query time. On benchmarks with nugget-level judgments, SPLADE-COVER often improves over its base model as measured by both information coverage metrics and relevance metrics, at a fraction of the query-time cost of generation-based methods.

Abstract:
Multi-turn jailbreak attacks have emerged as a critical safety threat to LLMs, as the harmful objective is decomposed across a sequence of apparently benign turns to bypass guardrails. Existing defenses lack the reasoning capacity to identify evolving manipulation patterns, often trading helpfulness for safety by over-refusing benign requests related to sensitive topics. We introduce Trace, a multi-turn defense with trajectory-aware structured reasoning. Before generating each response, the model identifies manipulation cues from the trajectory, evaluates both the benign and adversarial interpretations of user intent, assigns a jailbreak score, and commits to an action: Allow, Caution, or Decline. We curate 4k multi-turn adversarial conversations from five attack frameworks, pair them with 2.4k benign dialogs, and 600 sensitive-but-benign conversations. We train Llama-3.1-8B-Instruct with SFT and GRPO under a multi-component reward that jointly optimizes helpfulness on benign prompts and robustness against jailbreak attempts. Across seven multi-turn attack benchmarks, Trace attains an average attack success rate (ASR) of 14.5% against 31.4% for the strongest baseline and 74.9% for the undefended target while raising the attacker effort required per successful jailbreak significantly. Trace also balances usability and safety, achieving a 93.3% average compliance on over-refusal benchmarks.

Abstract:
Long transcripts are costly inputs for downstream NLP systems and often contain irrelevant context. We study query-conditioned topic localization: predicting the sentence span in a transcript that best addresses a topic-title query. To improve span localization, we reuse ASR encoder states as sentence-level representations and fuse them with textual embeddings. This lets lightweight span locators exploit speech information without running a separate audio encoder. Experiments on two public datasets show consistent gains over text-only baselines, especially under strict boundary-matching criteria. Cross-dataset experiments further indicate that the benefits are strongest for structured or semi-structured speech, while gains on spontaneous speech are limited and mixed.

Abstract:
Flowchart-oriented dialogue (FOD) systems aim to guide users through multi-turn decision-making or operational procedures by following a domain-specific flowchart to achieve a task goal. In this work, we formalize flowchart reasoning in FOD as grounding user input to flowchart nodes at each dialogue turn while ensuring node transition is consistent with the correct flowchart path. Despite recent advances of LLMs in task-oriented dialogue systems, adapting them to FOD still faces two limitations: (1) LLMs lack an explicit mechanism to represent and reason over flowchart topology, and (2) they are prone to hallucinations, leading to unfaithful flowchart reasoning. To address these limitations, we propose FloCA, a zero-shot flowchart-oriented conversational agent. FloCA uses an LLM for intent understanding and response generation, while delegating flowchart reasoning to an external tool that performs topology-constrained graph execution, ensuring faithful and logically consistent node transitions across dialogue turns. We further introduce an evaluation framework with an LLM-based user simulator and five new metrics covering reasoning accuracy and interaction efficiency. Extensive experiments on FLODIAL and PFDial datasets highlight the bottlenecks of existing LLM/VLM-based methods and demonstrate the superiority of FloCA.

Abstract:
Multiple-choice question answering (MCQA) benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme—the combination of the response mode models follow and the rule for grading responses—is a key design choice that dictates which abilities to reward. We examine how alternatives to number right change what MCQA measures with six education-inspired schemes that assess abilities beyond accuracy: distractor elimination, abstention, confidence calibration, and self-correction. On LLM benchmarks, these schemes: 1) shift rankings of 31 LLMs beyond rephrased number right prompts; 2) better predict the LLMs users prefer in LLM Arena; and 3) reveal distinct model capabilities, like that GPT-5 rarely abstains and readily self-corrects, while weaker open-weight models often abstain and hesitate to eliminate choices. Given the benefits of alternative scoring schemes, we discuss ways to extend them to tasks beyond MCQA.

Abstract:
While factual correctness and task-performance have received much attention by Large Language Model (LLM) research, the fundamental question of how human-like generated texts are on a linguistic level has been underexplored. From a corpus-linguistic perspective, language production is inherently context-dependent. Distinct communicative contexts give rise to differences in frequencies and co-occurrence patterns of linguistic features that shape registers of language use. A text failing to adhere to these patterns can be correct content-wise, but still may not appear natural to human readers. In this work, we propose a context-aware evaluation framework in which human-likeness is measured as a two-sample problem between the linguistic feature distribution of a human reference corpus for a given register and a corresponding LLM-generated corpus. We implement this framework using the Maximum Mean Discrepancy (MMD) measure and the 67 lexico-grammatical features introduced by Biber, which are commonly applied in corpus linguistics. In our experiments, we compare seven instruction-tuned, open-source models and look at five English-language datasets spanning distinct registers. While across all tested setups, LLMs deviate from the human baseline, which models are closest to human language depends on the register and is not dictated by model size.

Abstract:
Reasoning-intensive retrieval is fundamentally challenged when queries demand deep semantic inference and arrive in multimodal form. Existing systems treat each query in isolation, rely on monolithic rerankers, and lack retrievers trained for cross-modal reasoning. We introduce MEMORA , a M emory- E nhanced M ultimodal O rchestrated R eranking A gent built around three contributions: (1) Collaborative Factor-Committee Reranking , which decomposes queries into intent factors, extracts factor-specific evidence into a structured matrix, and synthesizes judgments via a memory-conditioned meta-verdict model; (2) an Episodic Memory Module that tracks factor outcomes across queries and dynamically adjusts priorities at inference time; and (3) a Reasoning-Aware Dense Retriever fine-tuned with chain-of-thought augmentation on 20k + reasoning-intensive pairs. Evaluated on MM-BRIGHT across 29 domains, MEMORA achieves nDCG@10 of 41.5 (text-to-text) and 38.9 (multimodal query-to-text), outperforming the strongest agentic text-only baseline by +8.3 points and the strongest agentic multimodal baseline by +7.9 points.

Abstract:
In this paper, we explore multi-expert Conformal Risk Control (CRC) algorithms for pairwise LLM-as-a-Judge evaluation in open-ended dialogue. Our core insight is that multi-expert aggregation offers a complementary remedy to CRC: whereas CRC controls risk at the decision threshold through abstention, aggregation sanitizes the scoring function at its source. Guided by this, we first design two multi-expert CRC methods: Score Averaging and Decision Voting, which aggregate at the score and decision levels, respectively. While both strategies outperform single-expert methods on homogeneous expert panels, on heterogeneous LLM judges they remain risk-valid but recover only limited coverage, because a uniform threshold cannot match the experts’ distinct scoring scales. To resolve this issue, we further propose Marginal-Calibrated Conformal Consensus (MC³): it captures distinct per-expert scales via initial threshold ratios, while jointly tuning a unified decision function Cₜ(x) applied identically in both calibration and test, thereby preserving exchangeability. To evaluate our framework, we construct Panel, a 1,800-pair human pairwise-preference benchmark for open-ended dialogue. It is built on responses generated by four open-weight LLMs over dialogue contexts from three domains (ESConv, MSC, DREAM), with full logit access. In experiments, we find that both Score Averaging and Decision Voting substantially improve accuracy and acceptance rate on homogeneous panels. Notably, MC³ extends these gains to heterogeneous panels by accommodating distinct per-expert scoring scales across all three datasets.

Abstract:
In multi-agent dialogue, a hallucination crosses an operational control boundary when another agent accepts it and downstream decisions inherit the error. We formalize this cross-speaker transition as unsupported endorsement and introduce SocialCritic, a learned receding-horizon controller that ranks and repairs candidate action sequences with separately trained utility, norm, and epistemic advantage critics. Across 692 test episodes spanning LIGHT, DealOrNoDeal, SOTOPIA, and template-family-held-out Persuasion-Conflict over 10 generation seeds, SocialCritic exceeds SAFE-Dialogue by 6.9 points in goal completion and 7.9 points in actionable accuracy while reducing endorsement by 44.9% and norm violations by 42.7%. The public-benchmark aggregate preserves the actionable-accuracy gain (74.9% vs. 66.8%), and trace analysis shows that endorsement falls 77.5% versus 63.5% for initial injection relative to LLM-only. Leave-one-benchmark-out critics retain an actionable-accuracy advantage on every target (minimum 5.2 points), while an H=1 controller without partner rollouts preserves 71% of the full controller’s advantage with 18% lower latency. At the default operating point, batched local scoring requires 1.7 external round trips and 748 ms per turn, enabling propagation-aware simulation, safety evaluation, negotiation, tutoring, moderation, and red-team workflows.

Abstract:
We investigate how annotator demographic attributes, supplied as prompt cues, shape the alignment between large language model (LLM) predictions and human annotations across five tasks. Using five open-source LLMs, we systematically vary the number and composition of demographic components in the prompt, spanning every combination from single-attribute through full-attribute configurations. Our experiments reveal three principal findings. First, demographic prompting is highly model- and task-dependent. The baseline is best in 12 of 25 model–task pairs. Among the remaining 13, 11 peak with 1–3 attributes, while two require higher-order combinations. The full-attribute prompt is never the best configuration. Second, the overall magnitude of demographic influence on human annotations does not predict which attributes improve LLM alignment; instead, both the learnability and the directional coherence of each attribute’s annotation signal need to be considered jointly. Third, neuron probing reveals that specialized activation correlates with alignment gains only under coherent annotation signals, and that activation volume alone does not imply steerability. Together, these results demonstrate that demographic prompting is not a monolithic intervention: its utility is highly context-dependent, shaped by attribute signal quality, task characteristics, and model architecture.

Abstract:
Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover only a limited set of languages, are often domain-specific, susceptible to overfitting, and poorly representative of low-resource languages. To address these limitations, we introduce ALEE, a framework that extends Sentence Smith (Li et al., 2025) to the cross-lingual and paragraph level. ALEE uses Abstract Meaning Representations (AMR) to generate English minimal pairs with controlled, fine-grained semantic shifts, which are paired with translations in target languages. This approach enables targeted diagnostics for models in any language with English parallel data. We conduct a large-scale empirical study across a diverse set of embedding models and 275+ languages spanning three parallel datasets. On ALEE, performance varies substantially across languages, text lengths, and linguistic phenomena, exposing persistent gaps in cross-lingual semantic representation that track language prevalence in training resources and subword tokenization. We release ALEE at https://github.com/Andrian0s/any-lang-embed-eval

Abstract:
While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capture agent-specific vulnerabilities that emerge through multi-step tool execution, particularly in rapidly growing ecosystems such as the Model Context Protocol (MCP). To address this gap, we propose a trajectory-aware evolutionary search method, T-MAP, which leverages execution trajectories to guide the discovery of adversarial prompts. Our approach enables the automatic generation of attacks that not only bypass safety guardrails but also reliably realize harmful objectives through actual tool interactions. Empirical evaluations across diverse MCP environments demonstrate that T-MAP substantially outperforms baselines in attack realization rate (ARR) and remains effective against frontier models, including GPT-5.2, Gemini-3-Pro, Qwen3.5, and GLM-5, thereby revealing previously underexplored vulnerabilities in autonomous LLM agents. Code is available at https://github.com/pwnhyo/T-MAP.

Abstract:
Safety alignment in Large Language Models (LLMs) often involves mediating internal representations to refuse harmful requests. Recent research has demonstrated that these safety mechanisms can be bypassed by ablating or removing specific representational directions within the model. In this paper, we propose the opposite approach: Rank-One Safety Injection (ROSI), a white-box method that amplifies a model’s safety alignment by permanently steering its activations toward the refusal-mediating subspace. ROSI operates as a simple, fine-tuning-free rank-one weight modification applied to all residual stream write matrices. The required safety direction can be computed from a small set of harmful and harmless instruction pairs. We show that ROSI consistently increases safety refusal rates while preserving the utility of many models across diverse architectures. Furthermore, we show that ROSI can also re-align “uncensored” models by amplifying their own latent safety directions, demonstrating its utility as an effective last-mile safety procedure. Our results show that targeted, interpretable weight steering is a cheap and effective mechanism to improve LLM safety, complementing more resource-intensive fine-tuning paradigms.

Abstract:
Applying steering vectors to large language models (LLMs) is an efficient and effective model alignment technique, but we lack an interpretable explanation for how it works–specifically, what internal mechanisms steering vectors affect and how this results in different model outputs. To investigate the causal mechanisms underlying the effectiveness of steering vectors, we conduct a comprehensive case study on refusal. We propose a multi-token activation patching framework and discover that different steering methodologies leverage functionally interchangeable circuits when applied at the same layer. These circuits reveal that steering vectors primarily interact with the attention mechanism through the OV circuit while largely ignoring the QK circuit. Freezing all attention scores during steering drops performance by only 8.83% across three model families. A mathematical decomposition of the steered OV circuit further reveals semantically interpretable concepts, even in cases where the steering vector itself does not. Leveraging the activation patching results, we show that steering vectors can be sparsified by up to 85-96% while retaining most performance, and that different steering methodologies agree on a subset of important dimensions.

Abstract:
Robust voice agents require exposure to the full diversity of how people interact through speech. However, obtaining enough spoken interactions is prohibitively expensive. Building spoken user simulators that address this requires large-scale spoken task-oriented dialogue (TOD) data encompassing spoken user behaviors, yet existing datasets are limited in scale and domain coverage, with no systematic pipeline for augmenting them. To address this, we introduce SpokenTOD , a spoken TOD dataset of 52,390 dialogues and 1,034 hours of speech augmented with four spoken user behaviors—cross-turn slots, barge-in, disfluency, and emotional prosody—across diverse speakers and domains. Building on SpokenTOD, we present SpokenUS , a spoken user simulator grounded in TOD that decides when to speak through a dedicated turn-taking head. SpokenUS achieves comparable goal coverage to much larger models while substantially outperforming all baselines in human MOS, disclosing slot values gradually across the dialogue as humans do rather than front-loading them. Further analysis confirms that SpokenUS’s spoken behaviors pose meaningful challenges to voice agents, making it a practical tool for evaluating more robust spoken dialogue systems. Our code is available at https://github.com/holi-lab/SpokenUS .

Abstract:
Despite extensive research on reasoning-oriented LLMs, core cognitive faculties of human intelligence, such as abstract reasoning and generalization, remain underexplored. To address this, we evaluate recent reasoning-oriented LLMs on the Abstraction and Reasoning Corpus (ARC) benchmark, which explicitly demands both faculties. We formulate ARC as a program synthesis task and propose nine candidate solvers. Experimental results show that repeated-sampling planning-aided code generation (RSPC) achieves the highest test accuracy and demonstrates consistent generalization across most LLMs. To further improve performance, we introduce Knowledge Augmentation for Abstract Reasoning (KAAR), which encodes core knowledge priors within an ontology that classifies priors into three hierarchical levels based on their dependencies. KAAR progressively expands LLM reasoning capacity by gradually augmenting priors at each level, and invokes RSPC to generate candidate solutions after each augmentation stage. Empirical results show that KAAR maintains strong generalization and consistently outperforms non-augmented RSPC across all evaluated LLMs, achieving around 5% absolute gains and up to 64.52% relative improvement.

Abstract:
Event Argument Extraction (EAE) aims to extract arguments for specified events from a text. Previous prompt-based research has mainly focused on designing static role anchors, effectively modeling general semantics but overlooking the geometric rigidity of these representations: (i) they failed to capture the diverse manifold structure of real-world semantic distributions and (ii) they lacked the flexibility to adapt to instance-specific reasoning contexts. To bridge the gap between static parameters and dynamic contexts, we introduce a new framework named LAMPEAE, which reformulates EAE as a dynamic manifold matching problem via adaptive parameter instantiation. Specifically, we employ a neuro-symbolic approach that utilizes a frozen LLM to extract high-order reasoning priors. A lightweight hypernetwork then maps this meta-knowledge into low-rank geometric transformation matrices, which dynamically project original static prompts into instance-specific probes. This mechanism ensures precise semantic alignment within the continuous parameter space. Experimental results on the RAMS, ACE05, and WikiEvent benchmarks show that LAMPEAE establishes new state-of-the-art performance, respectively, effectively validating its superiority in handling heterogeneous semantic transfer.

Abstract:
Open-domain table QA requires reliable evidence localization beyond candidate retrieval. Compact table fingerprints reduce irrelevant signals during representation learning, but retrieval based on learned representations may still have limited ability to distinguish evidence segments among retrieved candidates. This issue is especially pronounced when tables are partitioned into segments that share similar metadata and schema patterns. We propose Probe-Grounded Reasoning (PGR), a coarse-to-fine framework for content-level evidence localization. PGR generates query-specific probes with metadata cues and cell-level evidence, then reranks candidate segments to improve fine-grained evidence localization. It then derives answers via a decoupled Semantic-Plan-Execute workflow with GRPO-optimized planning and execution. On Open-WikiTable, under a controlled evaluation setting with fixed BM25-retrieved candidates, PGR achieves 85.96% segment-level evidence accuracy at k=5 and reaches 69.0% end-to-end QA accuracy, suggesting that content-level verification supports more reliable evidence grounding for downstream reasoning. Code is available at https://github.com/XFastDataLab/PGR/.

Abstract:
Recent reasoning models such as OpenAI-o1 and DeepSeek-R1 have shown strong performance on complex tasks including mathematical reasoning and code generation. However, this performance gain comes with substantially longer output sequences, leading to significantly increased deployment costs. In particular, long outputs require large KV caches, resulting in high memory consumption and severe bandwidth pressure during attention computation. Most existing KV cache optimization methods are designed for long-input, short output scenarios and are ineffective for the long-output setting of reasoning models. Moreover, importance estimation in prior work is computationally expensive and becomes prohibitive when continuous re-evaluation is required during long generation. To address these challenges, we propose LongFlow, a KV cache compression method with an efficient importance estimation metric derived from an intermediate result of attention computation using only the current query. This design introduces negligible computational overhead and requires no auxiliary storage. We further develop a custom kernel that fuses FlashAttention, importance estimation, and token eviction into a single optimized operator, improving system-level efficiency. Experiments show that LongFlow achieves up to an 11.8x throughput improvement with 80% KV cache compression with minimal impact on model accuracy.

Abstract:
Tokenization forms the basis of multilingual language models, yet existing methods often limit cross-lingual transfer by mapping semantically equivalent words to different embeddings. For example, ’I eat rice’ in English and ’Ina cin shinkafa’ in Hausa are typically mapped to different vocabulary indices, preventing shared representations and limiting cross-lingual generalization. This problem is even more pronounced in low-resource languages, where shared representations could offer the greatest benefit. We introduce parallel tokenizers, a new framework that first trains tokenizers monolingually and then aligns their vocabularies exhaustively using bilingual dictionaries or word-to-word translation. This alignment enforces a shared semantic space across languages while naturally improving fertility balance. To assess their effectiveness, we pretrain a transformer encoder from scratch on thirteen low-resource languages and evaluate it on sentiment analysis, hate speech detection, emotion classification, and sentence embedding similarity. Across all tasks, models trained with parallel tokenizers outperform conventional multilingual baselines, confirming that rethinking tokenization is essential for advancing multilingual representation learning–especially in low-resource settings.

Abstract:
Hybrid post-training combining supervised fine-tuning (SFT) and reinforcement learning (RL) is the standard paradigm for aligning large language models, yet fixed mixing schedules cannot adapt when the relative noise of the two signals evolves. We derive a noise-aware mixing weight by minimizing an MSE upper bound on the mixed stochastic gradient, yielding a closed-form that balances gradient noise variance and SFT–RL disagreement. Building on the token-wise reweighting of CHORD (CITATION)

Abstract:
Client-side privacy rewriting is crucial for deploying LLMs in privacy-sensitive domains. However, existing approaches struggle to balance privacy and utility. Full-text methods often distort context, while span-level approaches rely on impractical manual masks or brittle static dictionaries. Attempts to automate localization via prompt-based LLMs prove unreliable, as they suffer from unstable instruction following that leads to privacy leakage and excessive context scrubbing. To address these limitations, we propose DAMPER ( D omain- A ware M ask-free P rivacy E xtraction and R ewriting). DAMPER operationalizes latent privacy semantics into compact Domain Privacy Prototypes via contrastive learning, enabling precise, autonomous span localization. Furthermore, we introduce a Prototype-Guided Preference Alignment, which leverages learned prototypes as semantic anchors to construct preference pairs, optimizing a domain-compliant rewriting policy without human annotations. At inference time, DAMPER integrates a sampling-based Exponential Mechanism to provide rigorous span-level Differential Privacy (DP) guarantees. Extensive experiments demonstrate that DAMPER significantly outperforms existing baselines, achieving a superior privacy-utility trade-off.

Abstract:
The computational and memory overheads associated with expanding the context window of LLMs severely limit their scalability. A noteworthy solution is vision-text compression (VTC), exemplified by frameworks like DeepSeek-OCR and Glyph, which convert long texts into dense 2D visual representations, thereby achieving token compression ratios of 2x–10x. However, the impact of this high information density on the core long-context capabilities of vision-language models (VLMs) remains under-investigated. To address this gap, we introduce the first benchmark for VTC and systematically assess the performance of VLMs across three long-context understanding settings: VTC-Retrieval, VTC-Reasoning, and VTC-Memory. We further introduce VTCBench-Wild to evaluate robustness across diverse rendering configurations. We comprehensively evaluate leading open-weight and proprietary models on our benchmarks. The results indicate that several VLMs can retrieve information from rendered text in short-to-moderate contexts, but this ability degrades at longer lengths, and the degradation is especially pronounced on tasks requiring binding and integrating dispersed evidence. This study provides diagnostic evidence of VTC and serves as a foundation for designing more efficient and scalable VLMs.

Abstract:
Internet memes are a powerful form of online communication, yet their multimodal nature and reliance on commonsense and cultural knowledge make automated toxicity detection challenging. Prior work has studied individual dimensions of memes: textual content via optical character recognition, visual content via object recognition, and toxicity via hate speech detection, but lacks a unified architecture that formally identifies the semantic dimensions contributing to meme toxicity. We present three contributions: (1) a semantic framework identifying the dimensions necessary for meme interpretation; (2) a corresponding benchmark, M-QUEST, of 609 question-answer pairs for 307 memes, generated via a semi-automatic process grounded in the framework; and (3) an empirical evaluation of 12 VLMs on M-QUEST. The results show that the instruction-tuned models with reasoning capabilities significantly outperform others, whereas pragmatic inference remains challenging across all models.

Abstract:
Wider beam search should improve LLM reasoning, but when should you stop widening? Prior work on beam width selection has focused on inference efficiency (CITATION), without analyzing whether wider search can hurt output quality. We study each search step as scoring n candidates and keeping the top b of them. Best-of- n keeps a single winner ( b=1 ); beam search keeps several. Because the scorer is noisy, a worse candidate can outrank a better one, and that risk grows as more candidates are compared. How far you can usefully widen therefore depends on the scorer: how large the true quality gap is relative to the noise. Keeping several hypotheses is easier than naming one winner, so beam search is more forgiving than best-of- n , but that per-step analysis does not by itself tell you the best width for the full task. We compare perplexity scoring and a process reward model (PRM) across three 7–8B models and all ten MR-BEN subjects (5,737 questions). Perplexity yields no accuracy gain at any tested width; the PRM gains up to 8.9 points and continues improving through beam width 4 ( n=16 candidates per step). Across 444K search steps, the spacing of ranked scores agrees with a simple Gaussian ranking model. The same model and algorithm therefore support very different useful widths, depending on how reliable the scorer is.

Abstract:
Developing reliable models for clinical tasks, such as Medical Question Answering (QA), is severely constrained by the limited availability of high-quality, expert-annotated training data. This challenge is exacerbated by stringent privacy requirements and the impracticality of utilizing large open-source corpora or proprietary cloud APIs within resource-limited clinical settings. To address these obstacles, we introduce SAGE ( Semantic Anchor-Guided Evolution ), a novel data synthesis framework that enables small, locally deployed models to generate high-quality medical training data. SAGE leverages lightweight, publicly available taxonomies such as MeSH as semantic anchors, imposing a structured prior to effectively guide and ground the data generation process. At its core, SAGE iteratively interleaves atomic (individual concept-based) and associative (relation-based) synthesis, bootstrapping training data from minimal seeds. This approach eliminates the need for large collections of medical documents or reliance on external APIs, providing a practical solution for on-premises data creation. Extensive experiments across multiple medical question-answering benchmarks demonstrate that models fine-tuned with SAGE-synthesized data consistently outperform those trained using self-derived or conventional document-based paradigms, highlighting tangible improvements in data efficiency and resource utilization for medical LLM development.

Abstract:
Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.

Abstract:
The attention prefilling phase of long-context LLM inference scales quadratically, making self-attention a severe computational bottleneck. Traditional sparse attention methods mitigate this through fixed patterns or offline profiling, but lack the flexibility to adapt to input-dependent attention structure. Recent dynamic methods address this by routing heads to sparse patterns in real-time, but rely on indirect routing proxies with overhead and budget allocation mechanisms that overlook the post-softmax mass hierarchy. We present CRISP (Cliff-awaRe Input-adaptive Sparse Prefilling), which identifies and addresses two structural challenges in this dynamic routing paradigm. First, we show that the routing decision can be read directly off the structure of the proxy attention map. We replace the Jensen-Shannon Divergence (JSD) routing with C struct , a structural proxy that measures mass at Vertical-Slash compatible positions and reproduces JSD’s routing decisions while eliminating both the pooled matmul and subsequent KL divergence overhead. Second, we formalize the post-softmax mass cliff and demonstrate theoretically that strictly cumulative coverage thresholds accumulate O(n) background noise at long contexts. CRISP navigates this via a sink-aware threshold grounded in the noise floor. Empirically, across InfiniteBench, RULER and LongBench on two model families, CRISP is the strongest sparse method overall and matches or exceeds exact dense attention on retrieval-heavy benchmarks, recovering up to +28.0 pp on retrieval tasks over baselines and achieving up to a 5.30 × attention speedup at 512k tokens, driven primarily by our O(n) noise elimination during selection while preserving structural integrity.

Abstract:
Latent tokens are gaining attention for enhancing reasoning in large language models (LLMs), yet their internal mechanisms remain unclear. This paper examines the problem from a reliability perspective, uncovering fundamental weaknesses: latent tokens function as uninterpretable placeholders rather than encoding faithful reasoning. While resistant to perturbation, they promote shortcut usage over genuine reasoning. We focus on Chain-of-Continuous-Thought (COCONUT), which claims better efficiency and stability than explicit Chain-of-Thought (CoT) while maintaining performance. We investigate this through two complementary approaches. First, steering experiments perturb specific token subsets, namely COCONUT and explicit CoT. Unlike CoT tokens, COCONUT tokens show minimal sensitivity to steering and lack reasoning-critical information. Second, shortcut experiments evaluate models under biased and out-of-distribution settings. Results on MMLU and HotpotQA demonstrate that COCONUT consistently exploits dataset artifacts, inflating benchmark performance without true reasoning. These findings reposition COCONUT as a pseudo-reasoning mechanism: it generates plausible traces that conceal shortcut dependence rather than faithfully representing reasoning processes.

Abstract:
Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitivity (unjustified refusals of benign queries) and undersensitivity (missed detection of visually grounded risks), leaving a persistent gap in safety alignment. To address this issue, we introduce Safety-aware Contrastive Decoding (SafeCoDe), a real-time decoding framework that dynamically adjusts token generation based on multimodal context. SafeCoDe operates in two stages: (1) a contrastive decoding mechanism that highlights tokens sensitive to visual context by contrasting real and Gaussian-noised images, and (2) a global-aware token modulation strategy that integrates scene-level reasoning with token-level adjustment to adapt refusals according to the predicted safety verdict. Extensive experiments across diverse MLLM architectures and safety benchmarks, covering undersensitivity, oversensitivity, and general safety evaluations, show that SafeCoDe consistently improves context-sensitive refusal behaviors while preserving model helpfulness.

Abstract:
Test-Time Scaling (TTS) enhances the reasoning capabilities of large language models by allocating additional inference compute to explore the solution space. However, existing parallel TTS methods typically keep branches isolated during search: intermediate discoveries remain branch-private and cannot guide other branches in time. This information isolation causes substantial redundant exploration, as branches repeatedly rediscover information already found elsewhere and require more search steps to collect complete decision information needed to reach correct answers. To bridge this gap, we propose Collaborative Parallel Thinking (CPT) , a training-free inference framework that enables search-time information sharing across parallel branches. CPT extracts compact intermediate information from ongoing branches, maintains a deduplicated query-level information pool, and broadcasts pool entries through the input context, allowing each branch in subsequent search steps to reuse discoveries made by other branches rather than rediscover the same information. Empirically, experiments on HMMT and AIME benchmarks show that CPT establishes a stronger accuracy–latency Pareto frontier than strong baselines across rollout budgets and model scales, highlighting search-time collaboration as an effective direction for efficient parallel TTS.

Abstract:
Dialogue systems typically optimize the immediate quality of a response, but rarely anticipate how that response will be interpreted by the user. This is especially important in emotional support conversations, where a locally plausible reply may still be perceived as judgmental, dismissive, pressuring, or misaligned with the user’s current need. We frame this as a problem of candidate response evaluation under user-side interpretation uncertainty. We propose Cognitive Look-ahead Simulation (CLS), an inference-time framework that previews likely user interpretations before committing to the final response. CLS expands single-pass generation into a Generate–Simulate–Evaluate–Revise pipeline: it first infers a Structured User-State Representation, then uses a Cognitive World Model as an operational one-step predictor of the user’s immediate inner thought and next-turn state under a candidate reply, and finally applies selective revision to high-risk drafts. We evaluate CLS on the SAGE benchmark using simulator-based outcomes and LLM-as-a-Judge capability profiling, with complementary human evaluations. Across multiple backbones, CLS improves dialogue outcomes and reduces severe failures compared with direct generation and self-refinement baselines. These results suggest that anticipating user interpretations provides a useful inference-time signal for identifying and revising high-risk responses in emotional support conversations.

Abstract:
On-policy distillation (OPD) has recently emerged as an effective post-training paradigm by providing supervision on student-generated trajectories. However, existing OPD methods for multimodal reasoning usually rely on a static teacher routing, assigning each sample to a single teacher based on modality or task type. This ignores that visual grounding and abstract reasoning may dominate different decoding steps, making a single teacher insufficient for the full trajectory. To this end, H-OPD is proposed as a confidence-aware heterogeneous multi-teacher OPD framework for multimodal reasoning. By verifying the complementarity of heterogeneous teachers in the same reasoning process, H-OPD replaces task or sample level teacher routing with token-level teacher arbitration along the shared student trajectory. H-OPD employs vision-to-language description transfer to enable text-only teachers to access key visual semantics, and uses a confidence-aware arbitration mechanism to dynamically combine vision-language teacher and text-only teachers at each token. Extensive evaluations over 11 widely-used reasoning benchmarks showcase the superior performance of our method.

Abstract:
Despite the remarkable success of Multimodal Large Language Models (MLLMs) across diverse tasks, the internal mechanisms governing how they encode and ground distinct visual concepts remain poorly understood. To unravel these mechanisms, we propose a causal framework based on activation steering to actively probe and manipulate internal visual representations. Through systematic intervention across four visual concept categories, our results reveal a divergence in concept encoding: entity knowledge is distinctively localized, whereas abstract concepts are globally distributed across the network. Critically, this divergence uncovers a mechanistic driver of scaling laws: increasing model depth is indispensable for encoding distributed and complex abstract concepts, whereas entities maintain a consistently high degree of localization. Furthermore, reverse steering uncovers that blocking explicit output triggers a surge in latent activations, exposing a compensatory mechanism between perception and generation. Finally, by extending our analysis to visual reasoning, we expose a disconnect between perception and reasoning: although MLLMs successfully recognize geometric relations, they treat them merely as static visual features, failing to trigger the procedural execution necessary for problem-solving.

Abstract:
Multi-hop question answering often fails when retrieval treats evidence as isolated matches to the original question, since the facts needed to answer a complex question are usually connected through intermediate entities, relations, and constraints. We propose HyperProve, a retrieval-augmented QA framework that addresses this challenge by coupling question decomposition with answer-conditioned expansion over a hypergraph of atomic facts. HyperProve does not use atomic facts, hypergraphs, or iterative retrieval in isolation; instead, it carries intermediate answers and supporting hyperedges as retrieval state, then uses that state to bias the next local hypergraph expansion. This design enables HyperProve to construct coherent evidence chains for final answer generation while making the retrieval process stateful and fact-centered. Across multi-hop QA benchmarks, HyperProve achieves the best overall performance in our evaluation, outperforming the strongest baselines by an average relative improvement of 6.2% in answer accuracy and 4.9% in F1.

Abstract:
Memory systems are essential for enabling long-term learning and sustained interaction in AI systems such as LLMs and agents. However, memory storage and retrieval often suffer from hallucinations, including fabrication, errors, conflicts, and omissions. Existing evaluations mainly rely on end-to-end question answering, making it difficult to identify which memory operation causes hallucinations. To address this, we propose the Hallucination in Memory Benchmark (HaluMem), the first operation-level benchmark for evaluating hallucinations in memory systems. HaluMem defines three tasks (memory extraction, memory updating, and memory question answering) to comprehensively reveal hallucination behaviors across different operational stages of interaction. To support evaluation, we construct user-centric, multi-turn human-AI interaction datasets, HaluMem-Medium and HaluMem-Long, containing about 15k memory points and 3.5k questions. The datasets feature long dialogues, with average lengths of 1.5k and 2.6k turns and context sizes exceeding 1M tokens, enabling evaluation across varying context scales and task complexities. Experiments on HaluMem show that current memory systems tend to introduce and accumulate hallucinations during extraction and updating, which then propagate to question answering. These findings highlight the need for interpretable and constrained memory operation mechanisms to improve memory reliability. All resources are available at [GitHub](https://github.com/MemTensor/HaluMem).

Abstract:
Tree of Thoughts (ToT) reasoning is fundamentally bottlenecked by its generate-then-score paradigm, which decodes every candidate branch into text and evaluates it through a decoupled scoring stage. Prior efficiency efforts primarily shrink the search breadth, leaving this prohibitive per-step textual evaluation intact. To resolve this, we propose ITOT (Implicit Tree of Thoughts), a training-free framework that recasts tree-structured reasoning as a continuous search within a model’s latent space. In this framework, branch quality is assessed intrinsically from hidden states via alignment with a contrastive prototype distilled from successful and failed traces. This elegantly collapses explicit branch enumeration and textual scoring into a single latent generate-evaluate-decide loop. Across six reasoning benchmarks and five LLM backbones, ITOT consistently improves mean accuracy over both the strongest tree-search and single-path baselines by roughly ten absolute points. Crucially, it achieves this while cutting token consumption by nearly an order of magnitude relative to explicit tree-search methods, operating even below the token budget of vanilla Chain of Thought. We will release our code to facilitate further research.

Abstract:
Robot imitation data are often multimodal: similar visual-language observations may be followed by different action chunks because human demonstrators act with different short-horizon intents, task phases, or recent context. Existing frame-conditioned VLA policies infer each chunk from the current observation and instruction alone, so under partial observability they may resample different intents across adjacent replanning steps, leading to inter-chunk conflict and unstable execution. We introduce IntentVLA, a history-conditioned VLA framework that encodes recent visual observations into a compact short-horizon intent representation and uses it to condition chunk generation. We further introduce AliasBench, a 12-task ambiguity-aware benchmark on RoboTwin2 with matched training data and evaluation environments that isolate short-horizon observation aliasing. Across AliasBench, SimplerEnv, LIBERO, and RoboCasa, IntentVLA improves rollout stability and outperforms strong VLA baselines.

Abstract:
Test-time training (TTT) adapts an LLM during generation by reading and updating request-owned state, such as fast weights, low-rank deltas, or streaming learner state. This breaks batched LLM serving, which assumes shared static weights: serial execution is correct but slow, while naive batching can corrupt request state. We formulate this problem as read-write TTT serving and present RW-TTT, which tags each decode step with its owner, version, and READ/WRITE effect, batches only compatible phases, and commits updates only to the owner. On one GPU with eight fast-weight In-Place-TTT streams, RW-TTT reaches 274.61 aggregate tok/s, 9.31 × over sequential serving and 3.44 × over memory-matched independent replicas. It preserves behavior on RULER, a long-context benchmark, and passes owner/version checks.

Abstract:
Token-level differentially private (DP) prompt optimization methods such as DP-OPT can become unstable under tight privacy budgets: on GSM8K, DP-OPT obtains 49.5 ± 28.5% across 30 runs, and a logged search trajectory reveals prompt-template drift and noise-sensitive irreversible choices. We diagnose these as structural consequences of greedy token-by-token construction over privately aggregated counts. We then propose DP-ES (Differentially Private Evolution Strategies), a structurally cleaner alternative that maintains a population of full prompts, mutates them via LLM calls that never access the private dataset, and spends privacy only on sampled-Gaussian evaluation; deterministic or Gumbel-smoothed selection is post-processing. Under a conservative (ε ≤ 1.0, δ = 10⁻⁵) guarantee, DP-ES achieves 88.1% on GSM8K (+38.6 pp over DP-OPT, approximately 9× lower standard deviation), 99.7% on MedQA, 73.5% on BANKING77, and 86.8% on Alpaca. It is also 2.5× faster in wall-clock time and uses 3.3× fewer logged private-data call groups than DP-OPT. Selection and population ablations, implementation-level noise checks, and a 200-profile exact-match memorization stress test complement the formal guarantee. Scope: Our experiments establish optimization robustness under DP noise, especially where prompt structure is critical; end-to-end validation on genuinely sensitive, non-saturated deployment data remains future work. Code: https://github.com/StephCpa/dp-es

Abstract:
Current Large Language Models (LLMs) are primarily optimized for written text, often producing outputs that are grammatically correct and helpful yet poorly suited for spoken delivery via Text-to-Speech (TTS). In this work, we study how to make LLMs natively generate TTS-friendly text, which we frame as a preference alignment problem: instead of relying on downstream rewriting modules, we directly align LLMs to generate text optimized for spoken delivery. We introduce two preference datasets spanning different target domains, CORA and Recipe, which contain paired TTS-friendly and TTS-unfriendly responses. We further propose an evaluation suite combining a pattern-based heuristic metric, a TTS→ASR evaluation pipeline, and a MUSHRA listening study with human judges. Our experiments compare the recently proposed Feature-aware Sampling and Tuning (FaST) framework – leveraging interpretable features instead of a black-box reward model – against an array of alignment baselines on the TTS-friendly generation task. Notably, we found that FaST achieves the best overall tradeoff between TTS-friendliness and helpfulness across various settings. We also identified a strong correlation between our different metrics, highlighting the ability to reliably assess TTS-friendliness via an efficient heuristic.

Abstract:
The rapid development of large language models (LLMs) has created an urgent need for identifying machine-generated texts, and text watermarking technology has proven to be an effective solution. However, current watermarking methods, while demonstrating strong detectability, significantly degrade text quality due to the introduction of unnatural tokens. The main reason lies in the fact that these methods ignore the importance of semantic information in the watermarking process. To address this issue, we note that the logit vector produced by LLMs encodes both semantic understanding of input texts and prediction confidence across different tokens. Therefore, we propose a novel Semantic Self-Guided Watermarking (SSGW) framework that leverages the LLM itself to generate a guidance logic vector that assists in watermarking while producing the original one concurrently. Subsequently, we design a transform module to analyze these two vectors comprehensively and then transform them into adaptive watermark logits for different candidate tokens, thereby reducing the possibility of selecting inappropriate tokens. Experimental results confirm the effectiveness of our method in achieving superior performance in both watermark detectability and text quality preservation. The source code will be made publicly available upon acceptance.

Abstract:
Lexical semantic change detection identifies words whose meanings change over time, but it often cannot explain which senses and semantic properties are responsible for the change. We propose LSC-Pattern, a framework for interpretable lexical semantic change analysis that combines dictionary-grounded diachronic sense distributions with interpretable semantic dimensions. The framework first assigns each target-word usage to an explicit dictionary sense and then analyzes the resulting sense distributions along dimensions such as valence, semantic scope, and metaphoricity. We instantiate it for Japanese with LLM-based word sense disambiguation, which reaches 0.815 accuracy against majority-vote human annotation and outperforms five baselines; its sense-distribution change scores correlate with JaSemChange human judgments (Spearman’s ρ of 0.821 on seven overlapping entries and 0.580 on all 20). Case studies illustrate how sense redistribution and operationalized semantic signals can jointly support the discovery and interpretation of candidate types of change.

Abstract:
The integration of large language models (LLMs) into topic modeling has shown promise for improving topic interpretability and coherence, particularly by exploiting the generative capabilities of LLMs through instruction-based prompting. However, existing methods often incur substantial computational cost and inference complexity. To address these limitations, we propose LLMFeed, a novel framework that leverages feedback from an LLM to improve topic model optimization. Specifically, the LLM is used to evaluate topic quality, identify semantic meaning, and recommend representative topic words. These signals are then incorporated to (i) refine topic representations via Direct Preference Optimization (DPO), and (ii) guide contrastive learning for improved document-topic assignments. Extensive experiments show that LLMFeed achieves a 7.6% gain in topic coherence and over 10% improvement in doc-topic distributions on 20 Newsgroups dataset, while requiring only minutes to run, compared with hours for recent LLM-integrated topic modeling methods.

Abstract:
Large language models achieve strong performance across diverse tasks, but deployment remains costly because of memory, latency, and energy demands. Structured pruning reduces these costs by removing architectural components, yet its recovery stage is often limited by a mismatch between the recovery module’s representational capacity and the complexity of the removed knowledge. We call this bottleneck the capacity-knowledge asymmetry and propose OverRep, an Overcomplete Reparameterization framework for structured LLM pruning. Following the principle of “train overcomplete, deploy compact”, OverRep temporarily overparameterizes the recovery module during training to absorb complex knowledge distilled from the original model. After recovery, the overcomplete re-parameterization is algebraically merged into a mathematically equivalent compact module, preserving the pruned model’s inference-time architecture and computational cost. OverRep further introduces an annealed activation that enables nonlinear training dynamics while converging to a linear regime for exact algebraic merging. Across three backbone families, OverRep improves retained reasoning performance over strong recovery baselines by up to 5.5 and 8.4 points at 25% and 50% pruning, respectively, while keeping memory usage and TFLOPs comparable to existing recovery methods. Our code is available at https://github.com/mmai-laboratory/OverRep

Abstract:
Large language models (LLMs) have shown strong reasoning capabilities in knowledge graph question answering (KGQA), yet their predictions are not always supported by explicit and verifiable reasoning paths. Existing methods often retrieve or generate plausible candidate paths, but path plausibility alone does not establish that a prediction actually depends on the retrieved evidence. To address this issue, we propose FBV-KGQA, a forward-backward verification framework for reliable and interpretable KGQA. FBV-KGQA first derives executable reasoning paths through relation-level planning and evaluates their forward support for candidate answers. It then identifies critical entity and relation pivots and applies schema-constrained counterfactual perturbations to assess their influence on model predictions. Reverse questions are subsequently constructed from candidate answers to verify whether they can be consistently traced back to the original question entity. Finally, an adaptive selection strategy integrates forward support, counterfactual effects, and backward consistency. Experiments on WebQSP and CWQ show competitive performance against representative KGQA baselines, while human evaluation indicates improved traceability, clarity, and usefulness of the resulting reasoning paths.

Abstract:
Personalizing large language models often requires adapting them to new domains, tasks, or user preferences. However, such adaptation can unintentionally shift a model’s value profile, even when the training data is primarily knowledge-oriented. We refer to this failure mode as value drift , a form of value alignment tax that emerges when learning new knowledge interferes with pre-existing alignment. To address this problem, we propose VISA, a modular framework for Value Injection via Shielded Adaptation. VISA keeps the base model as a stable content generator and trains a lightweight Value Rewriter to adjust the value expression of generated responses toward a target value vector. The target is constructed through a Value Detector and an Instruction Translator, while the rewriter is optimized with Group Relative Policy Optimization using rewards for value alignment and semantic consistency. We further construct VCR-45K, a value-conditioned rewriting dataset for evaluating controllable value adaptation. Experiments show that VISA substantially improves semantic preservation over prompting and fine-tuning baselines while maintaining competitive value alignment, enabling low-cost personalized alignment with reduced value drift.

Abstract:
A standard pipeline for symbolic reasoning over natural-language problems translates them into first-order logic and invokes a theorem prover. The translation step is the bottleneck: swap "every" for "some" and every inference that follows is corrupted. Yet today’s metrics often score more broken translations higher than less broken ones, because BLEU, BERTScore, and Smatch++ reward surface overlap that the worst errors happen to preserve. We introduce SIV, which derives two kinds of probes from the target formula and uses a theorem prover to verify the candidate translation against each. Positive probes are statements the candidate must entail, which detect translations that drop content; contrastive probes are statements the candidate must not entail, which detect translations that assert more than the original. On a controlled pool of perturbed FOLIO translations, the severity of the error accounts for 80% of SIV’s score variance, compared with at most 17% for any prior metric. Across six error classes on a disjoint pool, SIV scores the reference above the perturbed candidate in over 99% of pairs. Because each probe is labeled with what it tests, the failure pattern also supplies a labeled error trace, recovering the perturbation class at macro-F1 0.638, nearly double the score-only baseline. On 434 expert-audited real LLM translations, SIV attains the top AUC, uniquely detects and grades expert-labeled major errors, and abstains, rather than mis-scoring, on out-of-vocabulary translations.

Abstract:
Activation steering offers a lightweight approach to LLM safety, yet existing methods relying on static, linear perturbations often struggle with complex boundaries, leading to severe safety-utility trade-offs such as over-refusal. To address this, we introduce WassSteer. First, we propose a Safety Routing Mechanism that achieves context-aware sparse control by selectively intervening only on unsafe activations. Second, we reformulate steering as a transport problem on the Wasserstein Representation Manifold. Utilizing Flow Matching, WassSteer learns a non-linear velocity field to transport harmful activations toward safe regions along Wasserstein Geodesics, effectively neutralizing toxicity while maximally preserving semantic content. Experiments across Llama-3, Gemma-2, and Qwen-2.5 demonstrate that WassSteer significantly outperforms baselines against diverse jailbreaks while maintaining high performance on general benchmarks, effectively reshaping the safety-utility Pareto frontier.

Abstract:
Long-horizon reinforcement learning for information seeking agents remains difficult because terminal rewards reveal whether the final answer is correct, but not which acquired information enabled it. This difficulty is amplified by text-derived webpage observations, where parsing, truncation, and summarization often produce incomplete and unstable content representations across trajectories. We propose an evidence-centric framework for web agent learning that represents information acquired through tools as identifiable units for comparison across trajectories. In particular, fetched webpages are represented as rendered snapshots, preserving layout and multimodal content as stable content-level observations. Building on these units, we introduce Information-Aware Credit Assignment, a post hoc reward propagation method that estimates turn-level utility scores from rollout success rates and assigns dense rewards to intermediate steps that introduced high-utility information. Integrated with GSPO, our method consistently improves performance on BrowseComp, GAIA, Xbench-DS, and Seal-0. Code and datasets will be released at [https://github.com/pc-inno/ICA_MM_deepsearch](https://github.com/pc-inno/ICA_MM_deepsearch).

Abstract:
Large language models (LLMs) exhibit remarkable reasoning capabilities, but their task-specific fine-tuning is notoriously plagued by overconfidence, severely hindering trustworthy deployment. We propose Data-Adaptive Lower-Rank Adaptation (DALorRA), a simple and effective variational Bayesian sparse framework that shifts the paradigm of uncertainty quantification from the dense parameter space to the lightweight rank level of low-rank adaptation (LoRA). With the insight that LoRA essentially aggregates multiple rank-one components that may provide superfluous model capacity, DALorRA imposes stochastic masking on rank dimensions, enabling Bayesian regularization of model capacity during training and ensemble-like calibration during inference. Extensive experiments demonstrate DALorRA’s excellent calibration of LLMs without compromising reasoning accuracy.

Abstract:
Knowledge distillation (KD) is a technique to compress a larger teacher system into a smaller student. In machine translation, KD is commonly evaluated through translation quality and inference efficiency, without jointly accounting for the environmental costs of producing and deploying the distilled system. We evaluate representative KD methods both on bespoke MT models and LLMs, by considering both translation quality and computational cost, using the Machine Learning Life Cycle Assessment tool, which accounts for costs throughout the KD model life cycle. Our key finding is that the deployment volume required to amortize KD is serving-dependent and can shift by several orders of magnitude under batching. We include actionable guidance for selecting, developing, and evaluating KD methods under quality and compute-induced constraints.

Abstract:
Real-world image search queries are multimodal and compositional: “find this shirt in pink” specifies an entity to retain, an attribute to modify, and context to ignore. Yet existing re-rankers either compress such multi-faceted relevance into an opaque embedding or rely on free-form chain-of-thought that easily omits or hallucinates fine-grained constraints. Drawing on rubric- and checklist-based evaluation from NLP, we recast multimodal image re-ranking as a semantic constraint satisfaction problem and propose EviRank , which parses any query—text-only, image-only, or composed—into a unified evidence package : typed criteria across six semantic slots (e.g., entities, attributes, relations), each labelled required , forbidden , or ignorable . Re-ranking then reduces to evidence-conditioned verification , combining deterministic rubric scoring and evidence-grounded listwise comparison in a single training-free procedure. The explicit evidence can further serve as structured supervision for optionally distilling a lightweight student. Across five benchmarks spanning text-to-image, image-to-image, and composed image retrieval, EviRank achieves state-of-the-art performance, and the distilled student preserves over 90% of the teacher’s capability at substantially lower cost.

Abstract:
Existing LLM routing methods often construct supervision from a single sampled response for each query–model pair. Because LLM generation is stochastic, however, such an observation can be an unstable estimate of model capability: semantically equivalent query formulations and repeated decoding may yield different scores and even different model preferences. We show that this instability can further propagate from routing labels to learned routing policies. To address this issue, we propose DARS (Distribution-Aware Routing Supervision), which estimates query-level model capability from repeated observations spanning semantics-preserving query rewrites and stochastic decoding. DARS summarizes expected quality, expected cost, and performance variability to construct risk-aware supervision without changing the downstream router architecture. Experiments across diverse tasks and routing methods show that DARS generally improves routing utility and cost–quality trade-offs over single-shot supervision. Further analyses show that its benefits persist under moderate sampling budgets and different decoding temperatures. These results suggest that reliable LLM routing should move beyond individual sampled outcomes and instead model query-level capability distributions.

Abstract:
Instruction-tuning datasets for large language models (LLMs) are often large, redundant, and imbalanced, hindering efficient adaptation. When such data are used naively in large-batch fine-tuning, overrepresented sample groups are repeatedly included while underrepresented but informative ones remain weakly covered; in practice, large-batch training is typically implemented with data parallelism (DP) across multiple GPUs. We propose CluSTER, a Cluster-aware balanced Sampling framework for Training Efficient data Reduction in DP instruction tuning. We curate a representative reduced dataset through gradient-space clustering and DP-aware balanced allocation, ensuring dual-level coverage across clusters and workers, while cluster-weighted updates preserve the original data distribution. As a result, CluSTER reduces redundant computation and improves training stability without compromising model quality. Across multiple instruction-tuning datasets, CluSTER reduces training time by 64.6%, almost without accuracy loss compared to prior sampling and data reduction methods. Code is available at https://anonymous.4open.science/r/CluSTER.

Abstract:
Visual token pruning reduces the inference cost of vision-language models (VLMs), but most methods only ask which tokens to keep. This retained-token view can keep redundant high-scoring tokens while leaving discarded evidence without a close representative. We propose CoverPruner, a training-free pruner that asks the complementary demand-side question: after a token is removed, which surviving original token represents it for the target VLM? CoverPruner formulates pruning as Representational Coverage Maximization (RCM), covering the full projected visual-token set with query-weighted demand. It instantiates RCM with projector-space coverage and a lightweight first-layer attention probe. Across multiple VLM architectures and compression rates, CoverPruner achieves the best average accuracy among all compared methods, with the largest gains usually appearing under aggressive compression.

Abstract:
As users increasingly expect LLMs to align with their preferences, personalized information becomes valuable. However, personalized information can be a double-edged sword: it can improve interaction but may compromise objectivity and factual correctness, especially when it is misaligned with the question. To alleviate this problem, we propose PersonaDual, a framework that supports both general-purpose objective reasoning and personalized reasoning in a single model, and adaptively switches modes based on context. PersonaDual is first trained with SFT to learn two reasoning patterns, and then further optimized via reinforcement learning with our proposed DualGRPO to improve mode selection. Experiments on objective and personalized benchmarks show that PersonaDual preserves the benefits of personalization while reducing interference, achieving near interference-free performance and better leveraging helpful personalized signals to improve objective problem-solving.

Abstract:
Large language models (LLMs) are increasingly used as proposal generators for evolutionary robot design, yet most loops remain memoryless: simulator results shape the next population but are not preserved as reusable design knowledge. We present Auto-Robotist, a self-evolving LLM agent that distills morphology-search traces into an explicit natural-language skill library. Each skill stores a structural archetype, evidence-grounded positive and negative rules, and the evaluated designs that support them, making design memory inspectable rather than implicit in a population. During search, the agent retrieves skills to condition LLM edits of elite bodies while retaining a Genetic Algorithm (GA) mutation path for exploration; after evaluation, it updates the library through Add, Diagnose, and Merge. Across seven EvoGym tasks spanning locomotion, traversal, and object interaction, Auto-Robotist improves cold-start 5×5 search and transfers learned skills to 10×10 design spaces, where reference-conditioned transfer outperforms or matches GA on every task when averaged across seeds. These results suggest that LLM agents can convert expensive physical evaluations into reusable, auditable design principles. Our code is publicly available at https://github.com/wangyf9/Auto-Robotist.

Abstract:
Language model (LM) agents are increasingly deployed to maintain codebases over extended periods, fixing streams of related defects while carrying context from one fix to the next. Yet existing software engineering (SWE) benchmarks evaluate models one bug at a time: the repository is reset, the codebase is re-read, and a single self-contained issue is graded in isolation. This setting collapses a continuous maintenance workflow into a series of independent sessions, ignoring the cumulative dependencies that make real-world bug fixing challenging. To bridge this gap, we introduce ChainSWE, the first benchmark for evaluating agents on sequential, dependent bug fixes within a shared codebase. We collect chronological chains of 304 issues across 54 Python projects, mined from six SWE-bench-family datasets. Our evaluation across a range of agents and models reveals a consistent performance drop by up to 70% as the chain length increases. Our data and code is available at [C-lister/ChainSWE](https://huggingface.co/datasets/C-lister/ChainSWE).

Abstract:
Long-context inference in large language models is limited by the linear memory growth of the key-value (KV) cache. Existing eviction methods typically retain tokens with high accumulated attention or remove semantic redundancy, but may discard low-frequency tokens that connect distant dependencies. We propose CentralKV, a training-free framework that models attention as a dynamic sparse causal graph. Its Stream-Influence algorithm updates token centrality online and preserves structurally important bridge tokens during KV compression. The attention graph is constructed directly from the attention operation without separate similarity computation. Experiments on LongBench show that CentralKV improves over H2O, SnapKV, and GraphKV under strict physical memory constraints, particularly on reasoning-intensive tasks, with minimal latency overhead.

Abstract:
Unified multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding? Existing evaluations provide mixed evidence, but confound task difficulty, reasoning paradigms, and the closed-loop interaction between generation and understanding. We introduce a fine-grained evaluation framework for vision generation-assisted understanding. It stratifies samples by difficulty, enables unified evaluation of multiple reasoning paradigms, and uses Oracle-Assisted Reference Protocols to decouple generation from understanding. Our analysis shows that generated visual aids help on easier instances but become unreliable as reasoning complexity increases. Oracle-assisted diagnosis further reveals that the main bottleneck often lies in visual understanding rather than generation, as current UMMs struggle to leverage even faithful visual aids. We also show that effective visual generation should target visual-understanding bottlenecks rather than add more reasoning steps, and identify a three-stage transition from task-irrelevant noise, to misleading plausible guidance, and finally to useful assistance. These findings lay a reliable foundation for the next stage of UMM development. The code is available at https://github.com/zyb1029/VGAU-Diag.

Abstract:
Tool-augmented vision-language agents can acquire external perceptual evidence through OCR, detection, segmentation, and other tools, but executing every proposed tool call is costly and sometimes unnecessary. We study the pre-call control problem: after a ReAct-style VLM agent proposes a perceptual tool call, should the call be executed or skipped before its output enters the context? Across five benchmarks, we find that the baseline agent exhibits poor local selectivity: helpful and harmful calls occur at similar rates (11.8% vs. 9.9%), while most calls do not change the immediate forced-answer prediction. We introduce ToolGate, a lightweight external controller that predicts execute/skip decisions from trajectory text and simple structural features. Across two Qwen3-VL backbones, ToolGate reduces token cost to 64–69% of the unrestricted ReAct baseline while preserving average accuracy in cross-domain settings. With matched-domain trajectory training on Qwen3-VL-30B, ToolGate additionally improves average accuracy by 1.65 points. These results suggest that tool-augmented VLM agents benefit not only from better perceptual tools, but also from explicit inference-time control over when tool outputs are worth paying for.

Abstract:
Deep Research agents tackle knowledge-intensive tasks through multi-round retrieval and decision-oriented generation. However, these agents suffer from severe overconfidence, making their expressed confidence unreliable for user trust and downstream abstention. To address this, we augment the Deep Research pipeline with step confidence elicitation after each retrieval, building on the commonly used post-answer verbalized confidence. Interestingly, we find that Evidence Confidence (E-Conf), elicited after the final retrieval step, provides a stronger uncertainty signal than Answer Confidence (A-Conf), elicited after answer generation, and that A-Conf is largely shaped by E-Conf. Based on these findings, we propose DualStake, a dual-path calibration method that applies margin-clipped, confidence-dependent stake rewards to jointly align E-Conf and A-Conf with answer correctness while limiting extreme confidence optimization. Experiments on Qwen2.5-7B, Qwen2.5-7B-Instruct, and Qwen3-4B across 8 QA benchmarks demonstrate that DualStake consistently improves calibration without sacrificing answer accuracy.

Abstract:
Civil litigation is inherently a life-cycle process: what a lawyer drafts on day one constrains what unfolds at trial months later. Yet existing legal benchmarks evaluate isolated subtasks, and prior legal-agent simulators reinitialize each scenario from shared ground truth, leaving cross-stage causal dependencies unmodeled. We present LegalWorld, a life-cycle interactive environment that models Chinese civil litigation as a causally connected state chain of five stages (seven sub-scenarios), grounded in 75,309 paired Chinese civil judgments. We pair it with reusable infrastructure (local memory, global case memory, a Skill/Tool library) and Reflective Legal Skill (RLS), which distills completed trajectories into reusable procedural skills. Building on this environment, we construct LongJud-Bench to evaluate agent capability across all five connected stages. 18,992 ratings from 217 legal-background evaluators confirm that LegalWorld trajectories are procedurally faithful and role-consistent; cross-model evaluation reveals stage-level divergences invisible to aggregate scores (backbones within 4 overall points can differ by up to 15 normalized points on individual stages), and RLS yields consistent gains on both reflected and held-out cases. Detailed resources will be provided upon acceptance.

Abstract:
With the rapid development of large language models (LLMs), role-playing dialogue systems have become increasingly popular for immersive and personalized human–AI interaction. However, in real-world role-playing scenarios, users often provide lazy queries—brief and ambiguous responses such as "oh," "hmm," or "?"—which lack explicit intent and emotional cues, making it difficult for role-playing LLMs to generate engaging replies. Existing query rewriting methods mainly target functional tasks such as information retrieval and are not tailored to role-playing dialogue, where rewrite quality depends on role-playing-specific factors such as character consistency and personalized user preferences. To address this, we first propose a listwise Generative Reward Model (GRM) trained from real user preference data collected through online regeneration behavior, which provides fine-grained process and final preference signals for training a Lazy Query Rewriter. Using rejection fine-tuning and reinforcement learning, the LQ-Rewriter learns to transform underspecified user inputs into richer expressions while preserving user intent and dialogue context. Experiments on real-world dialogue logs from a mainstream Chinese AI-powered romantic role-playing platform show that our GRM substantially outperforms general-purpose LLM judges. Moreover, our LQ-Rewriter achieves the highest GRM-based and human preference win rates among all rewriting baselines, and brings tangible improvements in online deployment.

Abstract:
Model editing aims to update large language models (LLMs) with new knowledge without full retraining, but existing methods mainly focus on fact-level knowledge. Rules that support mathematical and scientific reasoning must remain consistent across formulas, descriptions, and instances, making localized updates less effective for rule-level editing. In this paper, we construct RuleEdit-200, a manually verified benchmark of 200 mathematical and physical rules, each instantiated in formula, description, and instance forms. Using this benchmark, we conduct fine-grained causal tracing on four LLMs and identify a form-specific layer-wise separation of rule knowledge: formulas and descriptions are associated with earlier layers, whereas instances rely on a distinct set of middle-layer representations. This finding suggests that rule-level editing requires separate updates for different forms. Based on this insight, we propose Distributed Multi-Layer Editing (DMLE), which updates formulas, descriptions and instances at their corresponding layer ranges. Experiments on RuleEdit-200 show that DMLE remains competitive on standard editing metrics while substantially improving rule-level editing, with average gains of 18.22% and 61.00% in instance portability and rule understanding over the best baseline on each model.

Abstract:
Speech-based applications pass spoken queries through automatic speech recognition (ASR) before any retrieval module, so ASR errors enter the pipeline as a fixed upstream constraint. We empirically test whether two extensions to standard retrieval-augmented generation (RAG), entity-graph linking and iterative reformulation, absorb or amplify these errors. Using four English accents synthesized through neural TTS, we evaluate four RAG configurations on three multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA and MuSiQue) against a clean-text oracle. Although the structurally richer configurations generally retain higher absolute F1 under ASR input, both extensions amplify the error: the F1 gap from clean text to the highest-WER accent is 36-67% larger under their combination than under naive dense retrieval, on all three benchmarks. The dominant failure mode is corruption of one or more query entities, accounting for 87-96% of degradation cases on 2WikiMultiHopQA across all four methods. Two lightweight surface-form mitigations leave most of the gap intact, indicating that downstream retrieval structure amplifies remaining entity errors. We release code and data at https://github.com/Continuum-AI-Corp/spoken-multihop-rag.

Abstract:
Multi-turn tool calling is essential for LLMs to function as autonomous agents, yet synthesizing the training data required for these capabilities remains a fundamental challenge. Existing synthetic data generation pipelines often produce unrealistic dialogues for two reasons: they chain tools that are only superficially compatible rather than aligned with meaningful user tasks, and they generate dialogues in one shot, which often introduces arguments that were neither provided by the user nor produced by prior tool calls. These issues also lead to a severe underrepresentation of multi-step tool interactions. We introduce ToolWeave, a structured framework for synthesizing realistic multi-turn tool-calling dialogues. ToolWeave supports realistic multi-step workflows (or tool sequences) by constructing tools with built-in dependencies and filtering candidate workflows based on their alignment with user goals. It reduces parameter hallucinations through a fine-grained planning stage that explicitly tracks parameter provenance. As a result, 44.87% of conversational turns in ToolWeave-generated data are multi-step, and the data contains fewer hallucinations in parameters and tool names. Consequently, LLMs fine-tuned on ToolWeave consistently outperform those fine-tuned on prior datasets across three public benchmarks. Notably, Llama-3.1-70B fine-tuned on ToolWeave achieves 39.75% on BFCL-V3, compared to 23.50% when fine-tuned on SOTA ToolFlow data.

Abstract:
Recent work uses LLM agents to reproduce empirical social science results with access to both the data and code. We broaden this scope by asking: Can they reproduce results given only a paper’s methods description and original data? We develop an agentic reproduction system that extracts structured methods descriptions from papers, runs reimplementations under strict information isolation—agents never see the original code, results, or paper—and enables deterministic, cell-level comparison of reproduced outputs to the original results. An LLM-based attribution step traces discrepancies through the system chain to identify root causes. Evaluating four agent scaffolds and four LLMs on 48 papers with human-verified reproducibility, we find that agents can largely recover published results, but performance varies substantially between models, scaffolds, and papers. Root cause analysis suggests that failures stem both from agent errors and from underspecification in the papers themselves.

Abstract:
Instruction tuning is commonly assumed to endow language models with a domain-general ability to follow instructions, yet the underlying mechanism remains poorly understood. Does instruction-following rely on a universal mechanism or compositional skill deployment? We investigate this through diagnostic probing across nine diverse tasks in three instruction-tuned models. Our analysis provides converging evidence against a universal mechanism. First, general probes trained across all tasks show selective rather than uniform deficits relative to task-specific specialists, indicating that representational sharing is partial and structured rather than global. Second, cross-task transfer is weak and clustered by skill similarity. Third, causal ablation reveals sparse asymmetric dependencies rather than shared representations. Tasks also stratify by complexity across layers, with structural constraints emerging early and semantic tasks emerging late. Finally, temporal analysis shows that the constraint signal becomes decodable only once generation is under way, and remains so throughout the response. These findings indicate that instruction-following is better characterized as skillful coordination of diverse linguistic capabilities rather than deployment of a single abstract constraint-checking process.

Abstract:
Agentic retrieval-augmented generation (RAG) requires language models to decide when to continue searching and when to answer. Existing RL-based methods rely on external supervision and overlook the agent’s internal belief about whether the current evidence is sufficient. To address this problem, we reformulate the search decision quality as belief-action alignment and propose MetaRAG, a belief-action aligned policy optimization framework for agentic RAG. MetaRAG uses Verify-first Action Generation to elicit an explicit verification process before each actual action, and Internal Belief Probing to estimate the policy model’s own answerability belief from the same question-history context. Based on these, MetaRAG derives a consistency reward that is further gated by answer correctness, avoiding reinforcement of internally consistent but incorrect trajectories. The belief probe is used only during training and introduces no inference-time overhead. Experiments on seven public QA benchmarks show that MetaRAG consistently improves the accuracy-efficiency trade-off over strong RL-based agentic RAG baselines, with gains that transfer to deep research settings, different optimizers, and multiple model backbones.

Abstract:
A benchmark score credits final answers, but not the route by which an item can be answered. In medical multimodal multiple-choice questions (MCQs), this distinction matters because a correct answer can be supported by the intended image finding or by benchmark-preserved cues in the wording of answers, non-visual clinical text, visible image text, artificial annotations, or device/context artifacts. We call the resulting score-level overinterpretation reasoning inflation. Here, a route is an observable input path that can support answer selection, not a claim about the model’s hidden cognition. Across six medical multimodal MCQ datasets, we separate candidate cues from behavioral evidence through prompt- and image-side audits, modality ablations, and matched repairs that preserve the medical target and answer key. In a 13-configuration open-model panel, full-input accuracy is 62.63%, while text-only and options-only settings achieve 53.96% and 29.71%, respectively. Removing length-gap, absolute/conspicuous, and spatial/prepositional cues lowers accuracy by 6.58, 3.50, and 4.77 percentage points. We also construct MedQA-MM, a 1,000-item shortcut-mitigated subset, where text-only and options-only accuracy fall to 5.21% and 12.33%. This does not imply that models never use images; it shows that medical image-reasoning claims require route-level evidence.

Abstract:
Early failure prediction is important for long-horizon agents, as it enables timely intervention and can reduce inference and tool-use costs. Uncertainty quantification, such as verbal confidence and perplexity, offers a promising approach to detecting agent failures; however, it has not been explored whether these signals retain their discriminative power during the intermediate stages of long-horizon execution. We evaluate mainstream uncertainty signals on deep-research tasks and find that verbal confidence reliably distinguishes failures at trajectory completion, achieving a mean AUROC of 0.85, whereas all evaluated signals offer limited predictive value earlier in execution, with none exceeding a mean AUROC of 0.60 at 50% trajectory progress. We identify an underlying mechanism explaining this gap: path switching, where agents frequently abandon their current search direction in-trajectory, breaking the link between early signal and final outcome. These findings challenge the assumption that intermediate uncertainty can reliably guide early intervention. They also motivate a practical recommendation for agent harnesses in deep-research settings: use final-step confidence to decide whether to restart, an approach that our experiments find more effective than in-trajectory intervention.

Abstract:
Oblique retrieval, recently introduced by OBLIQ-Bench, asks a retriever to surface documents whose relevance is determined by a latent attribute (an implicit stance, an analogous reasoning technique, an authorial fingerprint, or a vague tip-of-the-tongue recollection) that has little or no surface expression in the document. State-of-the-art dense encoders and agentic search pipelines built around frontier language models leave a large first-stage bottleneck on these tasks, while the same language models reliably verify relevance when shown candidates. We address this with OBLIQ-IR, a single-vector dense retriever whose training mixture combines per-mechanism synthetic queries with a new form of cross-model supervision: kNN-graph distillation from a frozen authorship encoder, which transfers a style-versus-topic inductive bias into the student. A 3B retriever fine-tuned reaches 0.211 NDCG@10 on Writing-Style, 0.171 on Math, 0.177 on Twitter, and 0.281 on Congress, improving over the GPT-5.2 Multi-Hop Agent by 0.010 to 0.098 NDCG@10 and over Gemini-2-Embedding by 0.027 to 0.222 NDCG@10 on every reported task.

Abstract:
Parallel test-time scaling treats each rollout as an independent candidate answer. For long-horizon search agents, however, these rollouts also produce rich traces of tool calls, retrieved evidence, and partial reasoning states. Intermediate failures such as reasoning drift are difficult to self-correct within a single rollout, yet useful corrective signals often already exist in other live rollouts before termination. This creates an opportunity to turn cross-trajectory information into online correction during execution, rather than harvesting it only after rollouts terminate. However, simple broadcast is unsafe because partial states can be redundant, irrelevant, or risky for the receiving trajectory. We propose OPTA, a live aggregation framework based on checkpoint-level synchronization: each rollout maintains a private report, while a controller distills a typed global shared state, and injects only relevant and useful feedback into each trajectory, preserving its exploration identity. On BrowseComp-Plus and DeepSearchQA across four LLM backbones, OPTA is consistently the strongest test-time scaling method, lifting Qwen3-30B on BrowseComp-Plus from 37.9% to 59.6%. Our results show that OPTA is a practical route toward timely and selective test-time scaling for long-horizon LLM search agents.

Abstract:
In-group favoritism refers to the phenomenon of favoring members of one’s in-group over out-group members and is widely observed in numerous social cooperative behaviors. Recently, in-group favoritism biases have also been identified in generative language models. However, whether persona agents exhibit in-group favoritism when faced with conflicting information, including misinformation, and how to mitigate its adverse effects remain understudied. To address these problems, we propose a Truth or Tribe simulation framework to study agent cooperation in the spread of conflicting information through a triadic interaction paradigm, and conduct controlled trials to evaluate the primary moderating factors. Extensive results showcase that persona agents display strong in-group favoritism, accepting incorrect answers from identity-similar peers at much higher rates than from dissimilar peers. In-group favoritism continues to emerge in defeasible reasoning contexts where no absolute truth exists, and it intensifies as cognitive complexity increases. Furthermore, three intervention strategies—Identity-Blind Instruction, Structured Counterfactual Reasoning, and Heterogeneous Perspective Ensemble—are proposed to mitigate in-group favoritism.

Abstract:
Deep research agents often rely on agreement among retrieved documents when deciding whether a claim is reliable. This heuristic breaks when the agreeing documents are not independent: several independent reports can support a claim, but several copies of the same false claim should not. We study evidence laundering, a content-only attack that publishes one false claim across paraphrased pages that appear to provide support from multiple sources. We evaluate this failure with deterministic matched corpora that present the same false claim and supporting propositions as a single source, a paraphrased correlated cluster, or an apparent independence control, with matched retrieval exposure. Experiments show that agents often assign laundered clusters credence comparable to the apparent independence control and become more willing to publish the false claim. Within the matched evaluation, a structural provenance gate substantially reduces this failure, suggesting that source independence modeling should precede answer synthesis.

Abstract:
LLMs are increasingly used for interpersonal advice and as tools for studying social behavior across languages and cultures. A common shortcut for eliciting language- or culture-related variation is to ask a model to answer as a native speaker. We test whether this native-speaker persona reproduces the outputs obtained when models instead generate advice in the target language and translate the response back into English. Using 600 interpersonal advice questions across 13 languages and eight LLMs, we compare native-language generation followed by translation (NL) with native-speaker persona prompting (NP), measuring linguistic style, behavioral scaffolding, and forced-choice action recommendations. We find that NP and NL are not interchangeable. Compared to NL, NP often increases lexical social cues, including affiliation and positive tone, while reducing qualities such as concreteness and social attunement; NP also provides less actionable scaffolding in open-ended advice. In forced-choice scenarios, NP changes which action the model selects, favoring confrontation over redirection, with effect sizes varying across languages, topics, and models. Our results show that cross-lingual elicitation strategy is a consequential methodological choice that can change both how advice is framed and which actions models recommend.

Abstract:
Generative Engine Optimization (GEO) has emerged as a novel paradigm for transforming content to increase visibility in Large Language Model (LLM) responses. Traditional GEO methods, however, select rewriting strategies in isolation, ignoring a critical externality: as adoption of content optimization grows, optimal strategies for rewriting content change. We formalize GEO as a competitor-aware strategy selection problem and propose a two-phase pipeline to solve it: (1) We use Bayesian Optimization of Combinatorial Structures (BOCS) to efficiently search the space of rewriting strategies, (2) We generate preference pairs and grounded reasoning traces from the BOCS black-box observations to fine-tune a language model to analyze a document corpus and propose optimal rewriting strategy combinations. We achieve state-of-the-art performance across several impression metrics over existing agentic and single-heuristic methods on both geo-bench and our synthetically augmented competitive dataset geo-bench comp . Our method also transfers to multiple out-of-distribution datasets, proving effective across domains, queries, and document types.

Abstract:
Supervised fine-tuning (SFT) trains a base language model to imitate target responses, and these targets may require knowledge the base model has not robustly internalized. We study this as a source of hallucinations and frame a group of mitigation methods as knowledge-aligned SFT: constraining SFT training targets to the base model’s parametric knowledge. Under a unified setup, we compare existing generation-based and estimation-based knowledge-alignment methods and introduce two new variants: Evidence Rewrite, which verifies base-model generations using external evidence, and Recall Rewrite, which retains claims only when they can be consistently recalled by the base model. Experiments with Qwen 3 4B and OLMo 3 7B show that knowledge-aligned SFT can reduce factual hallucinations on WildHalu and Biography while largely preserving general capabilities. Recall Rewrite yields the strongest factuality gains and improves refusal behavior on UnknownBench. It thereby confirms that SFT targets beyond the base model’s knowledge drive hallucination behavior.

Abstract:
Key–value (KV) caching accelerates autoregressive decoding but makes long-context serving memory-intensive. Most KV quantizers minimize local reconstruction distortion, although this proxy ignores how perturbations propagate through downstream computation. We introduce Beyond, a loss-aware non-uniform quantizer that learns representation levels and decision thresholds from next-token negative log-likelihood while freezing the backbone. Training and inference use the same hard quantizer, with straight-through and boundary-localized pseudo-gradients updating only its parameters. With 4-bit codes and group size 32, Beyond uses 5.0 effective bits per KV scalar, providing 3.2× compression relative to a 16-bit cache. Across three instruction-tuned LLMs, it lowers held-out Pile perplexity below the full-precision baseline by 2.2–7.0% and attains the highest LongBench Avg21 among full-precision Native, KV-FP8, and TurboQuant for each model. On an NVIDIA B300, its fused packed-cache kernel achieves a peak 1.50× speedup over FlashAttention-4, while the resulting vLLM path improves end-to-end throughput by up to 1.17×. Our code is available at https://github.com/wsxhjnb1/Beyond-KV-Cache.

Abstract:
Proprietary Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet aligning their outputs with diverse user preferences remains challenging. Existing personalization approaches for black-box LLMs often rely on user-specific scoring heads, causing the number of personalized parameters to grow linearly with the number of users and requiring additional adaptation for unseen users. To address these limitations, we propose Mixture-of-Facets (MoF), a scalable personalization framework for black-box LLMs that models user preferences as compositions of shared latent preference facets rather than dedicated user-specific parameters. MoF performs personalization through history-conditioned routing over shared facet heads, enabling personalization for users unseen during training without additional parameter updates. Across diverse personalization tasks, MoF delivers stronger personalization performance while maintaining a more scalable and parameter-efficient design than prior approaches. Additional analysis indicates strong generalization to unseen users.

Abstract:
Harmful meme detection requires a stable understanding of how the image and the text work together. Since the text is already given as external input, a natural test is whether predictions remain stable when the corresponding visible text is removed from the image. Motivated by test-time multi-view inference, we build a text-erased reference view by keeping the external text unchanged and removing the visible text. If the model uses the external text in a stable way, this change should not greatly affect its harmfulness prediction. However, we observe clear score shifts and prediction flips, showing cross-view instability under fixed external text. We further find that the cross-view change after neutral-probe filtering is related to error risk, so it can serve as a test-time diagnostic signal. Based on this finding, we propose TIDC, a test-time framework that uses the filtered signal to calibrate the original-view score. We also provide theoretical analysis to support the diagnostic signal and calibration design. Experiments on HarM, FHM, and MAMI show that TIDC improves zero-shot performance across frozen multimodal backbones, without parameter updates or reasoning chains.

Abstract:
Multimodal retrieval is becoming a crucial component of modern AI applications, yet its evaluation lags behind the demands of more realistic and challenging scenarios. Existing benchmarks primarily probe surface-level semantic correspondence (e.g., object–text matching) while failing to assess the deeper reasoning required to capture complex relationships between visual and textual information. To address this gap, we introduce MR 2 -Bench, a reasoning-intensive benchmark for multimodal retrieval. MR 2 -Bench presents the following critical values: 1) all tasks are reasoning-driven, going beyond shallow matching to effectively assess models’ capacity for logical, spatial, and causal inference; 2) it features diverse multimodal data, such as natural images, diagrams, and visual puzzles, enabling comprehensive evaluation across content types; 3) it supports complex queries and documents containing multiple images and covers diverse retrieval scenarios, more accurately reflecting real-world applications. Our benchmark contains 1,309 curated queries, derived either from manual collection and annotation or from selective consolidation of public datasets. Despite achieving strong results on existing benchmarks, current state-of-the-art models still struggle on MR 2 -Bench: for example, the leading Seed1.6-Embedding model attains a Recall@1 of 77.78 on MMEB, but only 9.91 on MR 2 -Bench. This substantial performance gap highlights both the increased challenge posed by our benchmark and the pressing need for further advances in reasoning-intensive multimodal retrieval.

Abstract:
The increasing prevalence of texts generated by large language models (LLMs) drives a critical demand for reliable LLM text detection, where text watermarking has emerged as a promising proactive solution. Among existing methods, sentence-level watermarking is more robust to semantic-preserving adversarial attacks, but it still faces two key limitations: generation often requires iterative rejection sampling, and detection treats all sentence-level signals as equally reliable. To address these limitations, we propose CAD-Mark, a context-aware distributional sentence-level watermarking framework. During generation, CAD-Mark partitions a batch of candidate sentences with a context-derived adaptive boundary and samples the output sentence from the subset corresponding to the secret bits, embedding multiple bits per sentence without iterative rejection sampling. During detection, it estimates sentence-level reliability from re-sampled candidates and aggregates bit-level signals with a diversity-weighted z -test. Experiments on two benchmarks demonstrate that CAD-Mark consistently outperforms strong baselines in TPR@1%FPR while preserving generation quality with comparable cost.

Abstract:
Aligning Large Language Models (LLMs) with human values often degrades their general capabilities, known as the alignment tax. Existing methods mitigate this by balancing dual objectives, which heavily rely on massive general-purpose data or auxiliary reward models. In this paper, we argue that because safety features are inherently sparse within the output distribution, alignment requires localized modifications rather than global trade-offs. To this end, we propose SafeSteer, which performs on-policy distillation confined to safety tokens. First, we construct a safety teacher via activation steering. Based on this teacher, we develop a safety token selection algorithm. Consequently, SafeSteer restricts the reverse KL penalty to these tokens during training to preserve general capabilities. Experimental results across diverse models show that our SafeSteer significantly outperforms existing methods on 7 safety benchmarks, with only minimal degradation on 5 general capability benchmarks. Notably, SafeSteer requires only 100 harmful samples without using any general-purpose data, less than 1% of what previous baselines used, considerably reducing alignment cost.

Abstract:
Text Style Transfer (TST) aims to rewrite text in a target style while preserving its semantics. However, existing methods rely on style-specific supervision and implicit generation, leading to poor generalization and semantic degradation given the inherent scarcity of parallel data. We propose Style-Generalizable Trajectory Reasoning (SGTR), a novel framework that formulates TST as the joint modeling of reasoning trajectories and outputs, decomposing the process into three interpretable stages: style understanding, deviation localization, and reflection refinement. To further enhance stylistic generalization, we introduce a latent contrastive learning objective, enabling the model to disentangle stylistic attributes from semantic content directly within its hidden representations. We distill these sophisticated analytical capabilities into a lightweight student model. Experiments demonstrate that SGTR consistently outperforms strong baselines in both style quality and semantic preservation, while exhibiting superior zero-shot generalization to unseen styles.

Abstract:
Skills help LLM agents reuse procedural knowledge in recurring workflows, but broad transferability alone does not guarantee workflow fit: a reusable skill may be semantically relevant yet fail under local tools, files, formats, and failure modes. We propose SkillMe, a workflow-situated skill evolution framework that adapts agent skills through repeated use in matched workflow recurrences. A Publisher induces and validates an initial skill from a successful execution, while Executors use the skill in agent-native sessions and provide trace-grounded feedback. SkillMe activates the matched skill on demand as workspace files, allowing agents to inspect and adopt guidance progressively without permanently installing it as a local skill. On the 71-task skill-needed subset of PinchBench with OpenClaw, SkillMe improves the overall mean score of all eight Executors over their paired no-skill baselines, with mean gains of +25.7 points. Comparisons show that evolved task-scoped skills are more stable than static skills and more reliable than broadly reusable skills. SkillMe thus provides an agent-native path to evolve reusable skills into effective workflow guidance through situated use.

Abstract:
Equipping Large Language Models (LLMs) with human-like personas is crucial for agentic applications, such as role-play and user simulation. Traditional prompt-based methods rely on descriptive conditioning by injecting static textual profiles, which often makes agents show generic behaviors due to a lack of realistic life memory. To fill this gap, we introduce memory-based conditioning, a paradigm inspired by the cognitive psychology, which replaces abstract profiles with an autobiographical memory base, enabling frozen LLMs to dynamically retrieve situation-relevant memory to guide their behaviors. We formalize its enabling task as customized lifelong memory synthesis and propose MemoryForge, a novel framework to synthesize such lifelong memory from brief target personas. MemoryForge has three key components: a context generator for socio-historical grounding, a life organizer for developmental coherence toward the target identity, and a multi-resolution simulator that balances broad temporal summaries with high-fidelity episodic experiences. Experiments on PersonaGym for role-play and SimulatorArena for user-simulation, show that the synthesized memory base by MemoryForge enables frozen LLMs to exhibit more human-like behaviors than strong descriptive conditioning baselines across multiple metrics and LLM backbones. Code is available here.

Abstract:
Rubric-based text evaluation increasingly relies on large language models (LLMs) as scalable judges, yet frozen black-box models can interpret the same criteria inconsistently, produce score attributions that are difficult to audit, and map judgments poorly onto human scoring scales. We define this challenge as criteria transfer: translating human rubric intent into a stable, auditable inference-time scoring protocol. We introduce RULERS, which locks a task-level rubric specification, executes it through structured, evidence-grounded judgments, and calibrates the resulting signals to human score boundaries. Across four rubric-governed benchmarks and multiple frozen backbone models, RULERS achieves stronger agreement with human scores in most evaluated settings, while better matching empirical score distributions and remaining more stable under semantically equivalent rubric perturbations. Calibration controls and component ablations show that these gains cannot be attributed to post-hoc alignment alone, but depend on the combination of fixed criteria, traceable evidence, and calibrated score interpretation. These findings suggest that reliable LLM judging requires faithfully operationalizing human evaluation standards rather than relying on prompt-level scoring alone. Our code is available at https://github.com/LabRAI/Rulers.git.

Abstract:
The embedding table is a quiet ceiling of nearly every neural network model. Its size grows with the vocabulary, its rows are tied to a single modality, and unseen inputs require adding parameters. This makes the table prohibitively large at scale, impossible to share across modalities, and unable to represent new tokens without retraining. We remove this ceiling with SpinHash, an O(1) encoder in the vocabulary size that stays constant at about 0.2M parameters regardless of how many tokens or modalities it serves. Instead of storing a vocabulary dependent table, SpinHash hashes each token into a compact spin code and decodes it with one shared feedforward network. Our hash is not merely a compression mechanism, but also defines the geometry of the representation space, realized by a similarity preserving Spin-SimHash and an antiferromagnetic Ising Spin-Repulse. Across text, DNA, and clinical codes, SpinHash matches embedding tables with 17 to 283x fewer parameters, reaching parity on IMDB, AG-News, Yelp, and both clinical fusion tasks. For k=22 DNA, where 4^22 table entries are infeasible, SpinHash achieves 98.3% promoter accuracy, outperforming same-size HyenaDNA-tiny and the 1000x larger Nucleotide Transformer. This work shows that representations need not be tied to a vocabulary, and that we can compute them once and reuse them across vocabularies, modalities, and domains.

Abstract:
Large language models (LLMs) have achieved remarkable progress, yet their massive storage and memory-bandwidth demands still hinder efficient deployment. Weight binarization is a promising solution, but existing binarization-based post-training quantization (PTQ) methods usually far exceed the nominal 1-bit storage target due to hidden overhead. To address this gap, we propose All for 1-Bit (AF1), a genuine 1-bit PTQ framework for LLMs. AF1 comprises two complementary components: (1) Null-space-Aware Binary Factorization (NABF) for improving binary reconstruction through Hessian-aware surrogate reparameterization, null-space-aware binary factorization, and scale-only global reconstruction; and (2) Hierarchical Shapley Allocation (HiSA) for assigning structural capacity using hierarchical Shapley sensitivity. Together, they preserve model accuracy under a strict 1.0-BPW budget in the PTQ setting. Experiments on LLaMA, Qwen, and Gemma families show that AF1 consistently outperforms existing binarization-based PTQ methods in perplexity and zero-shot accuracy. Compared with BF16, AF1 achieves an average 2.5× inference speedup and over 90% memory reduction across evaluated models, providing a practical path toward deployable genuine 1-bit compression for LLMs. The code for reproducibility is available at AF1.

Abstract:
We introduce OntologyBench, a tiered biomedical retrieval benchmark comprising 471,854 training and 125,744 evaluation query–document relevance pairs across concept grounding, relational retrieval, and compositional phenotype-based retrieval. Although these tasks can be tractable using ontology-aware reference methods, across task tiers, embedding performance is generally lower on relational and compositional tasks than on concept-grounding tasks. Fine-tuning on ontology-derived supervision improves performance on several relational and compositional tasks, whereas the evaluated reranking and LLM-based candidate-scoring methods provide little or no end-to-end improvement. Errors frequently reflect diseases matching only subsets of the phenotype evidence. These findings indicate that the evaluated embedding and reranking configurations do not reliably recover the compatibility encoded by the selected ontology relations and phenotype combinations and motivate retrieval systems that better integrate learned representations with structured biomedical knowledge.

Abstract:
Visual inputs are often assumed to improve language understanding in multimodal models. We examine this assumption by asking whether vision-language models (VLMs) can distinguish useful visual evidence from incidental image context in lexical judgments. We use human concreteness and imagery ratings because they span words with varying expected visual relevance, from abstract and low-imagery words to concrete and high-imagery words. We find that real-image contexts do not yield consistent gains and often hurt alignment with human ratings, most sharply when visual evidence is least relevant. Through probing and canonical correlation analysis, complemented by an attribution case study, we find that real-image contexts are associated with representational shifts and greater sensitivity to spurious visual cues, coinciding with weaker recoverability of the targeted lexical properties. We further show that instructing models to focus solely on textual content at inference time can reduce this degradation, with the clearest gains on these vulnerable subsets. Our findings suggest that current instruction-tuned VLMs need better calibration of when visual context should inform lexical judgments.

Abstract:
Current Large Language Models (LLMs) excel at solving complex mathematical problems, yet this proficiency does not inherently translate into effective tutoring. While advanced LLM tutors may leverage multi-agent frameworks or fine-tuning, most still lack a mechanism to systematically accumulate and reuse pedagogical experience over time, limiting their adaptability to diverse student needs during fluid, multi-turn interactions. To bridge this gap, we propose the Reflective Experience-Augmented Tutoring (REAT) framework, which couples experience distillation from historical dialogues with real-time adaptive retrieval. Driven by a multi-agent Observer-Critic-Mentor (OCM) distillation pipeline, REAT reviews past conversational trajectories and distills raw interactions into structured, problem-agnostic pedagogical experiences. During live tutoring, a state-aware retrieval module injects these curated experiences to provide adaptive scaffolding based on the student’s cognitive state. Experiments demonstrate that the proposed framework significantly outperforms both prompt-only and supervised fine-tuning (SFT) baselines, particularly in improving complex, low-scoring teaching scenarios. Crucially, the distilled experiences exhibit robust generalization across diverse model architectures and mathematical datasets.

Abstract:
Visual instruction tuning adapts pre-trained Multimodal Large Language Models (MLLMs) to follow human instructions for real-world applications. However, rapidly growing instruction datasets introduce substantial redundancy and computational costs. Existing data selection methods aim to prune this redundancy, but predominantly rely on computationally demanding proxy-based inference or training-based metrics, often exacerbating the efficiency bottlenecks they are intended to resolve. To address this challenge, we identify a critical yet overlooked factor: anisotropy in visual feature distributions. We find that this anisotropy induces a Global Semantic Drift that distorts similarity estimates and limits efficient data selection. Motivated by this insight, we devise PRISM, a training-free framework for efficient visual instruction selection. PRISM mitigates the dominant global drift through implicit re-centering, allowing intrinsic visual semantics to provide a reliable redundancy signal without proxy models or additional training. Empirically, PRISM reduces end-to-end GPU time by about 70% while simultaneously enhancing performance, surpassing models fine-tuned on the full dataset across multimodal and language understanding benchmarks and reaching 101.7% relative performance against the full-data baseline.

Abstract:
Large reasoning models improve problem-solving performance by generating extended Chain-of-Thought, but often suffer from overthinking, producing unnecessarily long and redundant reasoning traces. Prior works mainly focus on directly compressing reasoning length, or selecting reasoning modes globally based on task difficulty, without accounting for the dynamic and heterogeneous nature of the reasoning process itself. This lack of consideration for inference-time diversity limits their ability to adapt reasoning strategies flexibly. To address this gap, we propose Interleaved System-1/2 Reasoning (InterSR), a cognitively inspired framework that dynamically alternates between System 1 (fast) and System 2 (slow) reasoning modes during generation. Central to InterSR is a learnable system switching signal that reflects the necessity for deeper reasoning at each intermediate step and enables fine-grained control over cognitive mode transitions. Experiments across mathematical and cross-domain benchmarks show that InterSR effectively captures internal reasoning signals to reduce token usage while maintaining strong and robust accuracy. We release our code at https://github.com/buaadh/InterSR.

Abstract:
Large Language Models (LLMs) have recently shown strong reasoning capabilities, motivating their use in complex decision-making environments. StarCraft II (SC2), with its massive state-action space and partial observability, is a challenging testbed. However, existing LLM-based SC2 agents primarily focus on improving the policy itself, leaving the integration of a learnable, action-conditioned dynamics model into the decision loop largely unexplored. In this work, we propose StarWM and StarWM-Agent, and conduct the first systematic study of the learnability and decision utility of player-view, action-conditioned textual world models for SC2. StarWM predicts short-horizon future observations under partial observability. StarWM-Agent integrates StarWM into a lightweight Generate-Simulate-Refine loop for foresight-driven policy refinement. Extensive experiments show that StarWM substantially outperforms zero-shot baselines across multiple dimensions, while StarWM-Agent achieves consistent win-rate gains of 30%, 15%, and 30% against the SC2 built-in AI at Hard (LV5), Harder (LV6), and VeryHard (LV7), respectively. Additional analyses show that our method is complementary to existing history-summarization SC2-agent approaches, with their combination achieving the strongest online performance in our evaluation. Code is available at https://github.com/yxzzhang/StarWM.

Abstract:
Recent advances in autonomous LLM agents demonstrate their ability to improve performance through iterative interaction with the environment. We define this paradigm as Test-Time Improvement (TTI). However, the mechanisms under how and why TTI succeed or fail remain poorly understood, and existing evaluation metrics fail to capture their task optimization efficiency, behavior adaptation after erroneous actions, and the specific utility of working memory. To address these gaps, we propose Test-time Improvement Diagnostic Evaluation (TIDE), a lightweight, agent-agnostic and environment-agnostic framework that decomposes TTI into three comprehensive and interconnected dimensions. The framework measures (1) the overall temporal dynamics of task completion and (2) identifies whether performance is constrained by recursive looping behaviors or (3) by burdensome accumulated memory. Through extensive experiments, TIDE highlights that improving agent performance calls for explicitly optimizing the interaction dynamics between the agent and the environment.

Abstract:
Complex 3D spatial text to image generation requires models to convert natural language into stable visual geometry, not merely semantic appearance. Existing prompt-driven or layout-conditioned methods improve controllability, but often lack an optimizable and verifiable spatial intermediary before visual sampling. As a result, object relations, occlusion, visibility, and camera constraints can decay during multi-round generation. This paper presents SpatialGuard, a structured layout-guided framework for complex 3D spatial text-to-image generation. SpatialGuard parses prompts into image synthesis-oriented 3D layouts through a Spatial Layout Architect, realizes them as visual conditions and candidate images through a Visual Realizer, and uses a Visual Alignment Critic to validate consistency among prompt, layout, and image. To keep constraints stable across iterations, SpatialGuard introduces a Layout Harness that organizes rule constraints, tool invocation, shared knowledge, and feedback loops around the editable layout state. This design turns complex spatial generation from implicit prompt following into a verifiable process of planning, realization, validation, and repair. Comprehensive experiments show that SpatialGuard achieves state-of-the-art performance in complex 3D spatial layout generation and improves spatial faithfulness over existing text-to-image and layout control baselines.

Authors: Mohamed Anwar, Abed Alhakim Freihat, George Ibrahim, Mostafa Awad, Abdelrahman Sadallah, Gurpreet Gosal, Gokulakrishnan Ramakrishnan, Sarath Chandran, Biswajit Mishra, Rituraj Joshi, Ahmed Frikha, Etienne Goffinet, Abhishek Maiti, Ali El Filali, Sarah AlBarri, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Awantika Shukla, Sajid Siddiki, Samta Kamboj, Onkar Pandit, Sunil Kumar Sahu, AbdelRahman Elbadawy, Amr Mohamed, Ahmad Chamma, Evan Dufraisse, Abdelaziz Bounhar, Dani Bouch, Hadi Abdine, Guokan Shang, Fajri Koto, Yuxia Wang, Zhuohan Xie, Ali Mekky, Rania Elbadry, Sarfraz Ahmad, Momina Ahsan, Omar El Herraoui, Daniil Orel, Hasan Iqbal, Kareem Elzeky, Mervat Abassy, Kareem Elozeiri, Saadeldine Eletter, Farah Atif, Nurdaulet Mukhituly, Haonan Li, Xudong Han, Aaryamonvikram Singh, Zainul Abedien Ahmed Quraishi, Neha Sengupta, Larry Murray, Avraham Sheinin, Joel Hestness, Natalia Vassilieva, Hector Xuguang Ren, Zhengzhong Liu, Michalis Vazirgiannis, Preslav Nakov

Abstract:
Jais 2 is a family of open-weight, Arabic-centric large language models designed for strong Arabic performance and broad cultural coverage. The family includes, to our knowledge, the largest open Arabic-centric LLM trained from scratch, with 70B parameters, alongside an efficient 8B model. A custom Arabic-centric vocabulary supports efficient training and inference, while an optimized architecture and training recipe improve compute efficiency. Jais 2 achieves the highest scores on the Open Arabic LLM Leaderboard v2 (OALL2) and AraGen v1 among the evaluated open models, while remaining competitive on English benchmarks. It also obtains strong results in Arabic poetry, dream interpretation, translation, and summarization. On the IslamicQA domain, Jais 2-70B achieves the second-highest overall average among the evaluated large open models, while attaining the highest scores on PalmX and the held-out in-house IslamicQA benchmark. We release both model sizes as open-weights on Hugging Face under a commercially permissive license. Jais 2-70B is also available as a chat application on the Web, iOS, and Android, reaching inference speeds of up to 2,000 tokens per second on Cerebras hardware.

Abstract:
Evaluating Retrieval-Augmented Generation (RAG) systems requires assessing not only end-to-end correctness but also how individual components interact and how errors propagate through the pipeline. We introduce a Bayesian evaluation framework that jointly models retrieval success, abstention behavior, and answer correctness, factorized according to the pipeline’s information flow. The model distinguishes task success. whether the user received a correct answer, from generator success, whether the generator behaved appropriately given the retrieval outcome. We apply the framework to 27 RAG configurations across three datasets, three retrievers, and three generators, and show that the conditional decomposition reveals substantial behavioral differences between systems that appear equivalent under marginal metrics. We further analyze the annotation allocation problem, demonstrating that retrieval-success annotations are more informative than task-success annotations for estimating policy adherence, and provide an information-theoretic explanation for this asymmetry. Finally, we extend the model to incorporate LLM-as-a-judge annotations as calibrated noisy observations, enabling practitioners to combine limited human judgments with cheaper automated assessments within a unified probabilistic model.

Abstract:
Federated Retrieval-Augmented Generation (FedRAG) is attractive for privacy-sensitive applications because full local corpora remain on clients. As a result, routing must rely on client-provided semantic profiles, creating a new opportunity for manipulation. We introduce Routing Hijacking, a routing-stage attack in which a malicious client forges its profile to attract target queries despite having irrelevant underlying data. We show that this vulnerability is severe. Across three representative FedRAG routing architectures, Routing Hijacking consistently misroutes target queries and leads to downstream disruptions and failures, including missing evidence, poisoning, incorrect answers, and hallucinations. In a controlled MedQA-USMLE stress test, we further show that poisoned retrieved evidence can mislead models across scales, leading to incorrect answers, hallucinations, and sycophantic failures. Existing defenses do not close this gap: encrypted routing preserves the exploited ranking, and Byzantine-robust Federated Learning (FL) rules transfer poorly to heterogeneous routing profiles. To address this gap, we propose a trust-aware post-routing framework that reweights clients using returned-evidence feedback, including retrieval relevance, profile consistency, and cross-client agreement; online experiments show that it suppresses persistent hijacking over recurring queries and transfers to a learned neural router. Our findings establish routing integrity as a security challenge in FedRAG and highlight the need for stronger defenses for secure federated retrieval.

Abstract:
Large reasoning models (LRMs) produce reasoning traces with non-linear structures, such as backtracking and self-correction, that complicate the evaluation and monitoring of the reasoning process. We introduce ReasoningFlow, a framework that captures the discourse structures of LRM reasoning traces into fine-grained directed acyclic graphs (DAGs). We develop and validate our annotation schema through careful manual annotation of 31 traces (2.1k steps), achieving high inter-annotator agreement, then scale to automatic annotation of 1,260 traces (247.7k steps) spanning three tasks (math, science, argumentation) and five models (Qwen2.5-32B-Inst, QwQ-32B, DeepSeek-V3, DeepSeek-R1, GPT-oss-120B). By analyzing ReasoningFlow graphs, we find: (1) LRMs exhibit structurally similar traces, despite being trained from different base models and potentially non-overlapping post-training data. (2) ReasoningFlow reveals diverse fine-grained reasoning behaviors (e.g., local verification, self-reflection, and assumptions) that can be used for better reasoning trace monitorability. (3) In LRMs, most of the erroneous steps are not used to derive final answers. (4) Mechanistic causal dependencies between steps do not reflect the language-level discourse structure.

Abstract:
Molecular dynamics (MD) is the canonical in-silico method for atomistic molecular science, simulating molecular behavior from first-principles physics. Designing an MD pipeline for a new system requires substantial expert knowledge: running it on even one molecule is expensive, ruling out trial-and-error. We automate this expert pipeline-design process with an LLM agent. Unlike existing MD agents that orchestrate a predefined tool set, we treat pipeline design as open-ended code generation in which the agent’s behavior is reshaped online by verbal reward. Specifically, we build MDForge, an LLM agent whose in-context update rule densifies the sparse reward via a multi-agent debate among physics experts. On three SAMPL host–guest binding free-energy benchmarks (CB[7], OAH, CBClip), MDForge automatically designs MD pipelines competitive with human experts. Deployed on a library of unseen candidate guests, its CB[7] pipeline discovers a novel binder that wet-lab competition NMR confirms is a high-affinity, picomolar CB[7] binder ( K a ≈ 8 × 10 12 M -1 ). Our data and code are available at https://github.com/Zehong-Wang/MDForge.

Abstract:
Despite the significant advancement of LLMs in conversation summarization, their evaluation remains limited by insufficient scenarios, input lengths, and sample sizes. Furthermore, existing benchmarks often omit frontier reasoning systems and efficient small models, or lack fine-grained, multi-dimensional assessments. To bridge these gaps, we propose OmniCSEval, a unified benchmark comprising 1,800 diverse conversations across six real-world scenarios, featuring context lengths ranging from 128 to 32k tokens. For fine-grained evaluation, we employ a bidirectional fact-checking framework that integrates key fact matching to assess completeness and conciseness, alongside summary fact verification to evaluate faithfulness. To ensure reliable assessment, we establish a human-LLM collaborative pipeline for key fact extraction and a multi-LLM consensus verifier for summary fact decomposition. Leveraging this framework, we evaluate 28 LLMs across four distinct categories grouped by reasoning capability and model scale. Our extensive empirical study reveals critical insights regarding the cross-scenario challenges current LLMs continue to face, the impacts of reasoning and scale, and the efficiency and adaptability of reasoning models. We also provide guidance for system selection in real-world deployments.

Abstract:
Systematic dialectal performance gaps in language models (LMs) are well documented, but the source of these disparities within the modern language modeling pipeline remains unclear. Our study traces this "dialect tax" across the natural language processing pipeline. Using parallel English dialect corpora that hold meaning fixed while varying surface form, we first confirm that LMs recognize matched Standard American English (SAE) and dialectal texts as semantically equivalent. However, we discover further representational gaps corresponding to downstream performance gaps. Across model families and generations, modern LMs still encode dialectal texts unequally during tokenization, pre-training, post-training, and inference. Strikingly, bypassing traditional subword segmentation via a character-level counterfactual tokenizer removes neither input and output asymmetries nor dialectal accuracy gaps. During pre-training, dialect pairs induce more divergent gradient updates than pairs of entirely unrelated SAE documents, indicating that models find semantically equivalent dialectal content harder to learn from than unrelated SAE documents. During post-training, reward models show contextual, unstable dialect preferences, assigning higher values to isolated AAVE-exclusive tokens than to SAE-exclusive tokens, while full reasoning contexts receive task- and model-dependent dialect penalties. Overall, our findings suggest that the dialect tax is encoded and accumulated not by any one step in isolation, but at every step of the language modeling process.

Abstract:
Two recent studies (Jones et al. (2026); Zeng et al. (2026)) reach apparently contradictory conclusions about whether LVLMs can coordinate on efficient referring expressions. We control for task differences between the studies while directly comparing their prompting styles. We replicate the finding that models can coordinate efficient referring expressions when explicitly prompted to do so, suggesting that other task differences are not responsible for divergent results. However, we also find that the same models fail to infer the need for communicative efficiency from a more implicit prompt, highlighting critical differences between how humans and AI systems communicate.

Abstract:
Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now approximate human responses at scale, they remain tuned to population-level consensus, often erasing the individuality of reasoning styles and belief trajectories. To advance the vision of more human-like reasoning in machines, we introduce HugAgent (HUman-Grounded AGENT Benchmark), which rethinks human reasoning simulation along three dimensions: (i) from averaged to individualized reasoning, (ii) from behavioral mimicry to cognitive alignment, and (iii) from vignette-based to open-ended data. The benchmark evaluates whether a model can predict a specific person’s behavioral responses and the underlying reasoning dynamics in out-of-distribution scenarios, given partial evidence of their prior views. HugAgent combines structured questionnaires with semi-structured think-aloud interviews to collect ecologically valid belief states, belief updates, and reasoning traces from human participants. Our experiments reveal a clear asymmetry: models recover a person’s belief state from their own context reasonably well, but struggle to predict belief updates under intervention. Cross-person and cross-domain controls trace this gap to associative matching within a topic rather than identity-consistent reasoning, suggesting that progress requires better-calibrated change detection, not simply more context. We scope the benchmark to self-reported belief reasoning in three policy domains: healthcare, surveillance, and zoning. The benchmark, along with its complete data collection pipeline and companion chatbot, is open-sourced as HugAgent (https://github.com/jajamoa/HugAgent) and TraceYourThinking (https://github.com/jajamoa/trace-your-thinking).

Abstract:
German text simplification requires reducing lexical and structural complexity while preserving meaning, yet standard fine-tuning optimizes reference imitation without directly controlling the predictive distribution where lexical choices are made. We propose VIB-Simp, a lightweight output-logit regularization framework inspired by the Variational Information Bottleneck (VIB) literature. Instead of estimating mutual information or adding latent variables, VIB-Simp regularizes the mean and variance of output logits using a closed-form Gaussian Kullback–Leibler (KL) surrogate, providing a lightweight way to stabilize lexical choices during fine-tuning. An optional unlikelihood-style lexical penalty further suppresses probability mass assigned to difficult lexical candidates. Experiments on TextComplexityDE and 20Min show consistent gains across LLaMA-3.2-3B, Qwen-2.5-3B, mBART, GerPT-2, and mGPT, with consistent SARI improvements. Additional baselines, length-controlled analysis, ablations, expert ratings, and logit diagnostics show that the gains are not explained by shortening or dictionary-only effects. The results support an IB-style view of simplification: preserve meaning-critical content while suppressing unnecessary lexical complexity.

Abstract:
Standard reasoning benchmarks conflate two distinct failure modes into a single accuracy metric: absent internal answer representations versus flawed output sequence scoring. In this work, we show these failure modes can be disentangled. Across controlled logic, ProofWriter, ANLI, and FOLIO, we compare internal hidden-state probes against output candidate-answer string scores. We find that in post-trained language models, correct answers are often strongly encoded in internal representations even when raw sequence scoring collapses to chance or incorrect labels. To expose this hidden evidence, we apply a lightweight diagnostic correction: adjusting per-label output log-probabilities via two additive offsets fitted from as few as 25 unlabeled examples to match a prespecified label prior. This target-label-free correction recovers between 9 and 34 accuracy points (with matching macro-F1 gains) across larger Qwen3.5 checkpoints, and successfully recovers performance on OLMo-2 and Llama-3.1. The gains survive on examples a TF-IDF control misses and significantly exceed label-count permutation nulls. Our findings demonstrate that many LLM logical reasoning failures are output scoring artifacts rather than internal reasoning deficits, showing that benchmark accuracy can systematically underestimate a model’s representational capacity.

Abstract:
Large Language Models (LLMs) achieve strong reasoning performance, but their robustness to realistic lexical corruption remains poorly understood. We evaluate four open-weight instruction-tuned models and frontier models across four reasoning benchmarks under keyboard noise, character swaps, and filler insertion. Character-level perturbations substantially degrade accuracy, especially on multi-step reasoning tasks, while filler insertion has little effect. We trace this asymmetry to Attention Diversion: lexical corruption fragments subword tokenization, and the resulting fragments attract disproportionate attention mass, concentrated in middle and final transformer layers. Length-matched controls confirm that fragmentation, not prompt length, drives the loss. A factorial intervention then shows why the damage is hard to undo: fragmentation corrupts token content and attention allocation together, and the two are coupled. Restoring clean attention while the content remains corrupted is actively harmful, restoring content alone is insufficient, and only restoring both recovers a substantial share of the gap. This coupling explains why inference-time strategies, including chain-of-thought prompting, spell-checking, self-repair, and stronger repair models, fail to consistently recover performance: each addresses one channel at a time. Code and data are available at https://github.com/Jiaqian-Janelle/Attention-Diversion.

Abstract:
Rubric-based RL is a promising route for extending reinforcement learning beyond verifiable rewards, yet existing methods optimize rubrics while treating the query distribution as fixed. We identify a structural bottleneck: rubric quality is constrained by query structure. Open-ended queries yield vague rubrics; naively narrowing them introduces fabricated references that no model can verify, so all responses fail and training receives no reward signal. We present QUBRIC, a framework that co-designs queries and rubrics. Teacher-derived key points ground the rewriting of open-ended prompts into scenario-based, evaluable questions. Contrastive rubric generation then turns teacher–policy gaps into query-level criteria, and learnability filtering retains only informative query–rubric pairs for GRPO training. QUBRIC achieves a +5.5 point gain on ArenaHard over the SFT baseline. Trained only on instruction-following data, it further transfers to three held-out benchmarks spanning legal, moral, and narrative reasoning (+6.3 points on average), with improvements concentrated in reasoning-related dimensions. These results provide evidence that co-designing queries and rubrics can make rubric-based RL a practical complement to RLVR beyond strictly verifiable tasks.

Abstract:
Adoption and diffusion of neologisms in online communities have received renewed attention in recent years. As internet slang terms such as APT, referring to a K-pop song, and phrases such as Canon Event meaning an embarrassing but pivotal event, go viral online, it becomes increasingly important to understand the mechanisms that contribute to their success. Prior studies have often explained slang diffusion either from the perspective of social interaction or from the linguistic properties of the slang itself, but rarely from both perspectives together. One major obstacle has been the high cost of annotating slang usage in large-scale online communication. Recent advances in large language models (LLMs), however, make it possible to use them as scalable annotators for such tasks. In this study, we first curate a human-annotated benchmark to evaluate LLM performance in detecting slang usage in real Reddit communication. We then leverage LLM-based annotations to model slang adoption and diffusion. Our results show that slang diffusers with higher bridging capital are associated with increased subsequent adoption, whereas diffusers with higher bonding capital are associated with reduced adoption. We also find that wider contextual usage of a slang term is associated with a longer time before new users officially adopt it. Together, these findings suggest that both social-network structure and linguistic context shape the diffusion of neologisms in online communities.

Abstract:
Skills, as a useful abstraction for the procedural capabilities of large language models (LLMs), capture how models perform structured, multi-step reasoning and program execution. Existing approaches typically treat skills as explicit, surface-level constructs specified through prompts or programs, leaving open the question of how such procedural capabilities are represented inside the model and whether they can be manipulated as structured objects in latent space. In this empirical study, we investigate whether procedural LLM skills can be represented as directions in activation space and whether vector-space operations over these directions can express skill-level behaviors. We find that procedural skills admit a vector-space representation: individual skill directions can be activated to shift model behavior; independently extracted directions can compose to form higher-level skills. Contrastive directions yield context-conditioned algorithmic personalization and optimization trajectories over skill directions evolve non-monotonically, with intermediate states often surpassing fully optimized solutions. These results support a representation-level view of procedural LLM skills: they admit a latent vector-space organization that allows direct manipulation through internal interventions.

Abstract:
LLM-based forecasting systems have improved on real-world tasks like financial markets and sports outcomes, largely by strengthening search, but the final prediction step remains crude: systems typically ask an LLM to read all collected evidence and emit one answer. We call this approach Monolithic Prediction, which fits forecasting poorly: probabilities over competing outcomes carry information valuable for downstream decisions, but a holistic answer collapses the supporting evidence, discarding this information. To address the above issue, we propose LEAP Likelihood Elicitation and Aggregation for Probabilistic forecasting) to keep the LLM out of the final prediction. LEAP treats evidence items as local signals, uses likelihood parameters to estimate how each supports candidate outcomes, and aggregates them through a deterministic probabilistic model into a posterior over candidate outcomes, thereby preserving per-item contributions, yielding comprehensive forecasts, and simplifying audits. We build a benchmark covering real-world prediction, information-seeking, and browsing tasks, evaluating LEAP on our custom agent loop and practical agent CLI frameworks. Given the same evidence, LEAP outperforms Monolithic Prediction in most settings and metrics across models.

Abstract:
Large Language Model (LLM) agents increasingly rely on domain-specific skills, yet manually authoring such skills does not scale, and skills generated purely from parametric knowledge often miss critical operational pitfalls. We introduce Trace2Skill, a framework that consolidates broad execution trajectories in parallel into a unified skill directory through inductive reasoning over agent experience. Trace2Skill supports both deepening existing human-written skills and creating useful skills from weak LLM-generated drafts. Experiments demonstrate the effectiveness of Trace2Skill across diverse domains, including office workflows, math reasoning, and vision QA. Importantly, the evolved skills are not merely memorized artifacts of the trajectories used to create them: they often transfer across model scales, across model families, and to out-of-distribution settings. For example, skills evolved from Qwen3.5-35B trajectories improve a Qwen3.5-122B agent by up to 57.65 percentage points on WikiTableQuestions. Further analyses show that Trace2Skill outperforms sequential skill editing and ReasoningBank-style retrieval memories, compresses recurring failures and workarounds into standard operating procedures (SoPs), and yields portable skills that can be reused without parameter updates or test-time retrieval.

Abstract:
Large Language Model agents increasingly solve tool-use tasks, yet often fail to determine what users actually want when goals are underspecified, incremental, or indirectly expressed. We introduce UserBench, a controlled Gymnasium environment for evaluating proactive preference elicitation in multi-turn, tool-grounded interactions. Instantiated in travel planning, UserBench hides user preferences behind implicit natural-language cues and requires agents to ask targeted clarification questions, search structured option databases, and select user-aligned options. Evaluating leading open- and closed-source models reveals a large gap between tool execution and user understanding: models achieve high valid-search rates but elicit only a minority of hidden preferences, and even stronger models remain far below a human oracle. Further analysis shows that many additional interaction turns are broad or non-informative rather than preference-relevant. UserBench provides a reproducible testbed for studying and training user-centric agent interaction while intentionally abstracting away full travel negotiation and infeasible-constraint recovery.

Abstract:
Memory has become an increasingly important component of agentic systems, as these systems are expected to reason over long-term experience. However, prior work has largely focused on unimodal memory, leaving multimodal memory relatively underexplored despite its central role in real-world applications. Compared with unimodal settings, multimodal memory introduces additional challenges, including heterogeneous input integration, person-centric information alignment, and evidence aggregation across different granularities. We present PyraVid, a hierarchical multimodal memory framework inspired by Event Segmentation Theory from cognitive science. PyraVid organizes long videos into a coarse-to-fine pyramid structure, enabling structured memory access and effective evidence aggregation. It further supports structure-guided memory expansion with pruning, allowing the retrieval of related events with strong causal connectivity but low semantic similarity while reducing noise. Experiments on multiple long-video understanding benchmarks show that PyraVid consistently improves performance across datasets, model scales, and question types, highlighting the effectiveness of hierarchical multimodal memory for long-horizon reasoning.

Abstract:
Reinforcement learning (RL) has shown promise for training agentic models that engage in dynamic, multi-turn interactions. However, the value of such agents ultimately depends on their ability to assist users with diverse goals, preferences, and evolving interaction patterns. We introduce UserRL, a reproducible and extensible testbed for training and evaluating user-centric agents through standardized gym environments paired with simulated users. UserRL supports flexible reward specification by decoupling turn-level reward shaping from trajectory-level scoring, enabling controlled studies of credit assignment in multi-turn RL. Across Qwen3 models, our experiments show that SFT cold start provides an effective initialization for user-centric RL; trajectory-level reward-to-go scoring improves both effectiveness and interaction efficiency; and stronger simulated users improve training, while open-source simulators remain a cost-effective alternative with positive cross-simulator and preliminary real-user transfer. These results highlight reward design and user simulation as key to user-centric agent training, positioning UserRL as a practical foundation for interactive agent RL.

Abstract:
As Large Language Models (LLMs) develop stronger multilingual capabilities, their sensitivity to culturally diverse entities becomes increasingly important. Prior work by Naous et al. (2024) has shown that LLMs often favor Western-associated entities in Arabic. Due to the lack of entity-centric multilingual benchmarks, it remains unclear if such biases also manifest in various non-Western languages. In this paper, we introduce Camellia, a benchmark for evaluating entity-centric cultural biases in nine Asian languages, spanning six Asian cultures. Camellia includes 19,530 manually annotated entities associated with the covered Asian or Western cultures, as well as 2,173 masked contexts for these entities derived from social media posts. Using Camellia, we evaluate cultural biases in four recent multilingual LLMs across three tasks: cultural context adaptation, sentiment association, and entity extractive QA. Our analyses show that LLMs struggle with cultural adaptation across these languages, with performance differing across models developed in different regions. We further observe that different LLM families can hold distinct biases, reflected in the ways they link cultures to particular sentiments. Lastly, we find that LLMs can struggle with context understanding in some Asian languages, creating performance gaps between cultures in entity extraction.

Abstract:
Watermarking has emerged as a crucial technique for detecting and attributing content generated by large language models. While recent advancements have utilized watermark ensembles to enhance robustness, prevailing methods typically prioritize maximizing the strength of the watermark at every individual layer. In this work, we identify a critical limitation in this "stronger-is-better" approach: strong watermarks significantly reduce the entropy of the token distribution, which paradoxically weakens the effectiveness of watermarking in subsequent layers. We theoretically and empirically show that detectability is bounded by entropy and that watermark ensembles induce a monotonic decrease in both entropy and the expected green-list ratio across layers. To address this inherent trade-off, we propose a general framework that utilizes weaker single-layer watermarks to preserve the entropy required for effective multi-layer ensembling. Empirical evaluations demonstrate that this counter-intuitive strategy mitigates signal decay and consistently outperforms strong baselines in both detectability and robustness.

Abstract:
Multi-agent reinforcement learning for human-AI interaction typically relies on a single large language model to simulate user behavior. We show that this approach systematically fails to generalize, and trace the failure to simulator collapse: because the simulator LLM is mode-collapsed, a LLM policy trained against it overfits to narrow strategies that exploit the simulator’s mode, and such a policy transfers poorly to unseen simulators and real users. We formalize this collapse theoretically and propose two solutions. The inference-time solution, Verbalized Sampling, broadens the simulator’s behavior by sampling from a verbalized response distribution, reducing mode collapse. The training-time solution, Co-Training, jointly optimizes the policy against a population of trainable simulators, preventing it from overfitting to any single simulator’s mode. We validate both solutions on three multi-turn benchmarks: Persuasion for Good, 𝜏 2 -bench, and CooperBench. Verbalized Sampling improves held-out success by up to 9% over single-simulator RL, and Co-Training pushes gains further to 14%; the human study shows similar gain on real users in both. Both solutions preserve the policy diversity that collapses under single-simulator RL. Broadly, our results %challenge the common practice of a frozen simulator and suggest that the diversity of the training environment is critical to the generalization of multi-agent RL to real users.

Abstract:
Long-context inference with large language models (LLMs) is constrained by the linear growth of the key–value (KV) cache to sequence length. While pruning offers mitigation, prevailing methods determine query-specific token importance that cannot be reused across unseen queries. In contrast, we introduce TaskPress, a framework for task-guided, query-agnostic KV cache eviction. Instead of optimizing the cache for a single query, TaskPress constructs a reusable memory representation conditioned on a high-level task guide. The guide functions as a "meta-query" during prefill to filter irrelevant tokens before downstream queries are issued. In addition, TaskPress leverages quantization scale factors as a zero-cost signal for detecting influential representation outliers, providing an efficient proxy for token importance. Experiments on conducted on various tasks with long context input demonstrate that TaskPress efficiently creates a compact, reusable cache across diverse queries.

Abstract:
While Large Reasoning Models (LRMs) show strong performance in English, they often fail to reason in the language of the query, a phenomenon known as language collapse. Existing RL-based fixes typically add a binary language fidelity reward to the accuracy objective, yet still incur trade-off in accuracy, mid-trace code-switching, and excessive token usage. In this work, we propose AdaMame, a two-stage training recipe for multilingual mathematical reasoning that addresses these limitations by adaptively aligning the reasoning language to the query language without compromising accuracy. The first SFT stage fine-tunes on non-MT reasoning traces across five languages to establish multilingual reasoning capability. In the subsequent RL stage, we introduce AdaMame-GRPO, an adaptation of Group Relative Policy Optimization (GRPO) in which a query-conditioned alignment factor grows progressively during training, guiding the model to first explore diverse reasoning languages before exploiting reasoning in the query language. Evaluated across two benchmarks, two LRMs, and 12 languages, AdaMame-GRPO achieves Pareto-optimal performance across reasoning accuracy, language fidelity, and token efficiency over all baselines, with the strongest gains on out-of-domain, lower-resource languages.

Abstract:
Mixture-of-experts variants of parameter-efficient fine-tuning enable per-token specialization, but they introduce additional trainable routers and expert parameters, increasing memory and training costs. This undermines the core goal of parameter efficient finetuning. We propose Monkey Jump, which brings MoE-style specialization to PEFT without adding extra trainable parameters for experts and routers. Instead of introducing new PEFT adapters as experts, Monkey Jump treats the PEFT adapters already present in each Transformer block (e.g., query, key, value, up, and down projections) as implicit experts and routes tokens among them. Routing is performed via k-means clustering with EMA-updated centers—no gradients, no learned parameters. We theoretically show that token-wise routing increases expressivity and can outperform shared adapters by avoiding cancellation effects. In multi-task experiments spanning 14 text, 14 image, and 19 video benchmarks, Monkey Jump achieves competitive performance with MoE-PEFT methods while using 5.3–9.2× fewer trainable parameters than the MoE variant of the same base method, up to 48% lower memory, and 1.5–2× faster training. Monkey Jump is architecture-agnostic and can be applied to any adapter-based PEFT method.

Abstract:
Large Language Model (LLM) agents are rapidly emerging as powerful systems for automating tasks across domains. Yet progress in the open-source community is constrained by the lack of high quality permissively licensed tool-agentic training data. Existing datasets are limited in diversity, realism, and complexity, particularly regarding multi-tool and multi-turn interactions. To address this gap, we introduce Toucan, which contains 1.5 million trajectories synthesized from nearly 500 real-world Model Context Protocols (MCPs). Unlike prior work, Toucan leverages authentic MCP environments to generate diverse, realistic, and challenging tasks with trajectories involving real tool execution. Our pipeline first produces a broad spectrum of tool-use queries using five distinct models, applies model-based quality filtering, and then generates agentic trajectories with three teacher models using two agentic frameworks. Rigorous rule-based and model-based validation ensures high-quality outputs. We also introduce three extensions to further diversify tasks and simulate multi-turn conversations. Models fine-tuned on Toucan outperform larger closed-source counterparts on the BFCL V3 benchmark and establish a new Pareto optimum on MCP-Universe Bench.

Abstract:
Multilingual natural language inference (NLI) benchmarks are routinely constructed by translating English datasets into target languages, based on the assumption that inference relations between premise–hypothesis pairs are preserved under translation. This assumption, however, has rarely been tested empirically at scale. We investigate this by conducting a large-scale human annotation study across nine typologically diverse languages: Arabic, German, Spanish, Portuguese, Thai, Urdu, Turkish, Mandarin, and Cantonese. We translated 1,000 English premise–hypothesis pairs, sampled across four linguistically challenging phenomena, and annotated each target-language version by a native-speaker linguist. Additionally, a native English speaker annotated the source pairs under the same protocol, allowing us to distinguish source-label disagreement from translation-induced change. While agreement between translated-language annotations and the original dataset labels ranges from 55.1% to 67.4%, agreement with the native-English annotation ranges from 57.8% to 64.2%. Entailment and Contradiction frequently shift toward Neutral, but the English control shows that some apparent drift is already present in the source items. These results call into question using transfer of English labels and highlight the need for native-language annotation in multilingual NLI evaluation.

Abstract:
Puns are a challenging form of figurative language that exploit phonetic similarity and semantic ambiguity to convey multiple meanings. Although large language models (LLMs) demonstrate strong language understanding capabilities, they still struggle with pun reasoning due to limited phonetic modeling and uncontrolled end-to-end generation. We propose PunGraph, a retrieval-enhanced knowledge graph framework for pun understanding. PunGraph constructs a phonetic-semantic lexical graph using the Unisyn phonetic dictionary, IPA and G2P representations, and WordNet definitions, and retrieves candidate words or senses to constrain LLM reasoning within a structured candidate space. We further introduce WebPun, a new large-scale dataset containing 5,730 annotated heterographic and homographic puns. Experiments on SemEval-2017 and WebPun show that PunGraph consistently improves the performance of small-scale LLMs and achieves competitive results against strong proprietary models. Further analysis shows that retrieval-guided phonetic and semantic constraints effectively reduce common reasoning errors in pun interpretation, highlighting the benefits of integrating structured knowledge with LLMs.

Abstract:
Across various non-verifiable tasks, human evaluation is reliable but expensive, while automatic metrics are more scalable but often biased. Building on prediction-powered inference (PPI), we propose prediction-powered evaluation, a framework that combines limited human judgments with large-scale automatic scores to obtain data-efficient system comparisons that are provably unbiased. We develop parametric and non-parametric procedures, analyze the efficiency trade-off between paired and unpaired designs, and validate the framework on six WMT datasets. We further introduce the Prediction-Powered Saving Ratio (PPSR), a meta-metric that measures how much human annotation an automatic metric can save when used within prediction-powered evaluation. PPSR directly targets metric utility for prediction-powered evaluation and yields more discriminative and stable metric rankings than existing system-level meta-metrics. Overall, our new paradigm reframes automatic metrics as tools for reducing human annotation cost rather than replacing human judgment, and applies broadly to non-verifiable tasks.

Abstract:
LLM-based agents increasingly tackle long-horizon tasks with interdependent decisions, where each action reshapes future constraints and intermediate errors can cascade. Existing RAG and agent memory systems organize histories by semantic similarity, retrieving content-relevant entries at decision time. We argue that this design mismatches execution-state dependencies: it fragments decision trajectories and mixes valid and erroneous traces, hindering coherent state reconstruction and error isolation. We propose MAGE (Memory as Agent-Guided Exploration), an active execution-state manager that stores interactions in a hierarchical state tree. The agent derives its state from the active root-to-current path, combining subgoal summaries, recent traces, and hints from prior branches. Four coupled operations maintain the tree: Grow records new traces, Compress summarizes completed subgoals, Maintain validates summaries, and Revise restores a target boundary and resumes on a new branch. This design bounds context growth while preserving state integrity and isolating flawed segments from the active path. Experiments on MemoryArena show that MAGE improves the average task success rate by 7.8–20.4 pp over baselines, while reducing token consumption by 55.1%. Code is available at: https://github.com/Komorebi660/MAGE.

Abstract:
Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus stopping commits that error to automated action with no recourse. We introduce Conformal Social Choice, a post-hoc layer that converts debate outputs into calibrated act-versus-escalate decisions. Verbalized probability distributions from heterogeneous agents are aggregated via a linear opinion pool and calibrated with split conformal prediction, yielding fixed-round marginal coverage: at any pre-specified round, the correct answer is included with probability ≥1-𝛼 , without assumptions on individual model calibration. A hierarchical action policy maps singletons to autonomous action and larger sets to human escalation. Across eight MMLU-Pro domains and two additional benchmarks, fixed-round coverage stays close to target. The key finding is not that debate becomes more accurate, but that the conformal layer makes its failures actionable: at the final round, 81.9% of wrong-consensus cases are flagged by non-singleton sets at 𝛼=0.05 . Separately, among cases that reach a singleton before the final round, accuracy reaches 90.0–96.8% on MMLU-Pro (up to 22.1pp above consensus stopping): a selection effect, not a reasoning improvement. Simple probability- and entropy-threshold rules attain comparable interception; conformal’s distinct value is a finite-sample fixed-round coverage guarantee together with explicit candidate sets for escalation. Adaptive first-singleton stopping is evaluated empirically and not covered by this guarantee.

Abstract:
Despite persistent efforts to develop low-resource language technologies (LRLTs), little attention has been paid to LRLT practitioners’ experiences, challenges, and opportunities. We employ a qualitative approach and conduct 15 semi-structured interviews with LRLT practitioners working on three Sino-Tibetan low-resource languages. Through our analysis, we identify not only technical challenges such as a lack of textual data and reliable language processing tools, but also linguistic, social, and financial challenges that cause and exacerbate them. Fortunately, we also find that participants are generally optimistic about LRLT development and mitigate these challenges by fostering and seeking support from diverse sources and leveraging large language models. We discuss the complex nature of the challenges in relation to prior literature and provide recommendations for practitioners, governments, and the broader NLP community.

Abstract:
Understanding when language diffusion models memorize training data versus generalize remains an open challenge. We address it by framing Uniform-based Discrete Diffusion Models (UDDMs) as Associative Memories (AMs) where both memorization and creativity coexist. Unlike Hopfield networks, which require an energy function to produce attractors, we show that attractors can be formed via conditional likelihood maximization. We observe that UDDMs treat both training and unseen test sequences as fixed points for denoising while retaining the capacity to synthesize novel texts. We demonstrate that, as training set size increases, attractor basins around training samples shrink while those around unseen test samples expand until they become indistinguishable. This transition can be detected via the conditional entropy of tokens, which vanishes during memorization. Our framework provides both a theoretical foundation and a practical metric to elucidate the generative regimes of UDDMs.

Abstract:
Conducting reinforcement learning (RL) in simulated environments offers a cost-effective and highly scalable way to enhance language-based agents. However, previous work has been limited to semi-automated environment synthesis or tasks lacking sufficient difficulty, offering little breadth or depth. In addition, the instability of simulated users integrated into these environments, along with the heterogeneity across simulated environments, poses further challenges for agentic RL. In this work, we propose: (1) a unified pipeline for automated and scalable synthesis of simulated environments associated with high-difficulty but easily verifiable tasks; and (2) a multi-environment user-centric RL algorithm that not only effectively mitigates user instability but also performs advantage estimation at the environment level, thereby improving training efficiency and stability. Comprehensive evaluations on agentic benchmarks, including 𝜏 -bench, 𝜏 2 -Bench, and VitaBench, validate the effectiveness of our proposed method. Further in-depth analyses underscore its out-of-domain generalization.

Abstract:
Long-context inference remains challenging for large language models due to attention dilution and out-of-distribution degradation. Context selection mitigates this limitation by attending to a subset of key-value cache entries, yet most methods allocate a fixed context budget throughout decoding despite highly non-uniform token-level contextual demands. To address this issue, we propose Uncertainty-Triggered Adaptive Context Allocation (UT-ACA), an inference-time framework that dynamically adjusts the context window based on token-wise uncertainty. UT-ACA learns an uncertainty detector that combines semantic embeddings with logit-based confidence while accounting for uncertainty accumulation across decoding steps. When insufficient evidence is indicated, UT-ACA selectively rolls back, expands the context window, and regenerates the token with additional support. Experiments show that UT-ACA substantially reduces average context usage while preserving generation quality in long-context settings. Code and dataset are available at https://github.com/Tommy307/UT-ACA.

Abstract:
Next-word prediction (NWP) trains language models against a single observed continuation, even though many contexts admit multiple plausible next words. Recent RL-based next-word reasoning methods make this tension explicit: they reward a model for producing a rationale that supports one context-conditioned continuation, which can turn a pre-existing preference into a confident, self-justifying trajectory. We introduce BOW, an RL framework that instead trains models to produce self-contained, neutral, and comprehensive descriptions of the plausible next-word space. The policy generates a next-word reasoning trajectory from the full context, but a frozen scorer computes the core reward from that trajectory alone, without separately receiving the context. BOW-Reg adds a lightweight breadth regularizer to this core reward to discourage premature collapse. On two model backbones, BOW remains competitive with the original models and often outperforms trained baselines across ten general reasoning benchmarks. On benchmarks testing ambiguous references and word meanings, BOW-Reg achieves the highest SharedRef correctness and the lowest HoWN-Simple single-sense collapse on both backbones. Human evaluation further shows that BOW-Reg produces broader next-word reasoning trajectories, while direct next-word-prediction evaluation shows that these trajectories remain predictive.

Abstract:
Sparse autoencoder (SAE)-based steering has been widely used to address knowledge conflicts by guiding LLMs to be more faithful to the contextual knowledge. Existing methods usually perform mass steering, which modifies a large batch of SAE features identified via correlation-based methods. However, due to the inaccurate correlation and the neglected feature interactions, mass steering methods fail to precisely identify the features that play the key roles in steering and introduce a large number of redundant ones, which add noise and weaken the steering effects. Our empirical studies reveal that steering only a small subset of the identified features can achieve comparable or even better performance. Motivated by this finding, we propose Key Path Identification (KPI), a novel method that identifies key steering features characterized by strong causal dependencies with both upstream and downstream features. From these features, KPI constructs key paths and steers through less feature modifications. In this way, KPI advances SAE-based steering from quantity-driven to quality-focused, offering a perspective for more precise and interpretable model editing. Experiments in RAG tasks with knowledge conflicts show that our method improves the accuracy by 18% on average compared to the best baseline of mass steering, effectively filtering redundant features, alleviating side effects and demonstrating the core role of key paths in steering.

Abstract:
Compared with typical vision–language tasks, document parsing places stronger demands on fine-grained visual perception. Existing vision–language model (VLM)–based parsing approaches rely on globally compressed visual tokens, where fine-grained details are entangled within a single representation and repeatedly accessed during decoding. However, we observe that the visual evidence for each prediction is typically localized and conditioned on the current decoding state, whereas such representations must be accessed in full at every decoding step, resulting in inefficient computation. To address this mismatch, we formulate perception as state-conditioned visual evidence retrieval (SCVER) during autoregressive decoding. The model operates on a compact global representation for coarse structure and retrieves a small set of relevant high-resolution regions conditioned on the current token state. This coarse-to-fine design enables on-demand access to fine-grained visual cues, relieving globally shared representations from encoding all fine-grained details. We further find that learning such state-conditioned retrieval in VLMs is challenging and unstable. To stabilize this process, we introduce a Spatially-Guided Learning Objective (SGLO) to guide the retrieval process. Experiments on document parsing benchmarks show that SCVER improves robustness under reduced input resolution and achieves a better accuracy–efficiency trade-off, demonstrating the effectiveness of on-demand visual evidence retrieval for fine-grained perception.

Abstract:
Rubric-based evaluation is a promising paradigm for judging large language model (LLM) outputs, yet self-generated rubrics lag human-annotated criteria on hard instances. We argue this discriminative gap reflects an objective mismatch: self-generated rubrics describe good responses, whereas effective criteria must discriminate between close candidates. To close this gap, we introduce SVR (Support Vector Rubrics), a framework that recasts rubric construction as max-margin boundary learning over preference data. SVR mines contrastive features from preference pairs into a rubric bank, learns a prompt-conditioned selector together with global rubric weights, and iteratively refines the bank through support-pair selection and adversarial probing of hard negatives. At inference, given only the prompt, SVR retrieves the top- k rubrics from the bank and scores responses. On RubricBench, SVR narrows the gap to human reference rubrics from 24.1 to 0.3 points and outperforms strong self-rubric and judge baselines, and the learned bank transfers across judges without retraining. On RewardBench 1 2, and RM-Bench, it remains competitive with dedicated reward models, demonstrating broader reward modeling capability. Overall, boundary-defining rubrics offer a principled route to closing the discriminative gap in LLM evaluation.

Abstract:
Large language models are increasingly used for social decision-making situations that require balancing cultural norms with personal preferences. For example, a user preferring honesty might ask whether to correct a coworker publicly when local norms favor indirect feedback. Yet existing research studies cultural alignment and personalization largely separately. We introduce PACT, the (Personal-Preference and Cultural-Norm Trade-off) framework, which evaluates whether models choose to follow a cultural norm or allow personal preferences. We find that LLMs vary in how rigidly they enforce cultural norms, with behavior shifted more by country context (7.8%) than age (1%) and gender (0.7%) and shifting non-uniformly after instruction tuning. Furthermore, our five-country human study on PACT shows that culture-following in humans is mainly driven by scenario country, with the lowest agreement when participants judge their own cultural contexts, showing within-culture pluralism. Finally, human-LLM alignment experiments show that models can match majority choices, but fail to capture response distributions and uncertainty (with best correlations reaching only 0.24). Together, these findings motivate alignment evaluations that go beyond majority to capture cultural pluralism and disagreement in social judgment.

Abstract:
As access to high-quality, domain-specific data grows increasingly scarce, multi-epoch training has become a practical strategy for adapting large language models (LLMs). However, autoregressive models often suffer from performance degradation under repeated data exposure, where overfitting leads to a marked decline in model capability. Through empirical analysis, we trace this degradation to an imbalance in learning dynamics: predictable, low-entropy tokens are learned quickly and come to dominate optimization, while the model’s ability to generalize on high-entropy tokens deteriorates with continued training. To address this, we introduce EntroDrop , an entropy-guided token dropout method that functions as structured data regularization. EntroDrop selectively masks low-entropy tokens during training and employs a curriculum schedule to adjust regularization strength in alignment with training progress. Experiments across model scales from 0.6B to 8B parameters show that EntroDrop consistently outperforms standard regularization baselines and maintains robust performance throughout extended multi-epoch training. These findings underscore the importance of aligning regularization with token-level learning dynamics when training on limited data. Our approach offers a promising pathway toward more effective adaptation of LLMs in data-constrained domains.

Abstract:
Refusals are often treated as face-threatening acts in pragmatics because they can challenge the requester’s socially claimed self-image. Large language models (LLMs) are increasingly trained to refuse unsafe and inappropriate requests, and these refusals may harm users when models fail to manage this interactional cost properly. While existing work has mainly approached LLM non-compliance as a safety-alignment outcome, it does not provide a way to evaluate whether LLMs refuse appropriately across different harmful contexts. To study this question, we propose (to our knowledge) the first taxonomy of LLM refusals that is grounded in pragmatic theory. Applying this taxonomy to responses from 16 modern LLMs across 14 harm categories, we find that although models differ in how they refuse, their refusals are overall explicit and strongly morally evaluative, with interactional repair occurring mainly through offering or providing safer alternatives instead of interpersonal facework. This pattern is especially consequential in sensitive harm contexts, where overuse of negative framing may make users feel shamed or provoked, undermining the purpose of safe non-compliance. We therefore call for alignment evaluation that considers not only whether models refuse harmful requests, but also whether they refuse in ways that are contextually adaptive and socially accountable for the interactional consequences of saying no.

Abstract:
Introducing reasoning models into Retrieval-Augmented Generation (RAG) systems enhances task performance through step-by-step reasoning, logical consistency, and multi-step self-verification. However, recent studies have shown that reasoning models suffer from overthinking attacks, where models are tricked to generate an unnecessarily high number of reasoning tokens. In this paper, we reveal that such overthinking risk can be inherited by RAG systems equipped with reasoning models, by proposing an end-to-end attack framework named Contradiction-Based Deliberation Extension ( CODE ). Specifically, CODE develops a multi-agent architecture to construct poisoning samples that are injected into the knowledge base. These samples 1) are highly correlated with the user query, such that they can be retrieved as inputs to the reasoning model; and 2) contain contradiction between the logical and evidential layers that cause models to overthink, and are optimized to exhibit highly diverse styles. Extensive experiments on two datasets across five commercial reasoning models demonstrate that the proposed attack causes a 5.32∼24.72× increase in reasoning token consumption, without degrading task performance. Finally, we also discuss and evaluate potential countermeasures to mitigate overthinking risks.

Abstract:
Cross-lingual alignment (CLA) aims to align the representations of large language models (LLMs) across languages, enabling cross-lingual transfer to improve multilingual capabilities. Previous CLA methods often ignore language-specific information encoded in representations and only consider sentence-level alignment, which may lead to suboptimal performance and input-output language mismatch. We propose CAROT (Cross-Lingual Alignment of Representations in a Language-Agnostic Space via Optimal Transport), which consists of two steps: identifying language-specific representations in LLMs’ internal states and aligning language-agnostic representations across languages at the token level by optimal transport, while explicitly preserving language-specific representations. Inference-time steering experiments show that the representations computed by CAROT are effective alignment targets, improving multilingual performance by up to 11.2 points in accuracy while maintaining input-output language consistency. We further use the representations obtained by CAROT as training targets, internalizing the aligned representations. The trained models outperform existing CLA methods in 11 of 18 evaluation settings (3 models × 3 tasks × ID/OOD languages). Our work provides insights into what constitutes effective alignment targets for CLA in LLMs. Code is available at https://github.com/ynklab/CAROT.

Abstract:
Building fine-grained cross-modal alignment via pure radiograph-report pairs has been a long-standing challenge in the task of Radiology Report Generation (RRG). An intuitive solution towards this issue lies at introducing extra contexts to assist the report generation process, yet clinical contexts are normally unavailable in most real-world scenarios. In this paper, we present an asymmetric context self-distillation framework for RRG, namely ConDistill. To facilitate radiograph-only report generation, it refines the basic radiograph-report alignment, by progressively injecting additional contexts via multi-stage curriculum. Since these contexts are not available in inference, we develop a self-distillation strategy to transfer the bridge states from the teacher branches with extra contexts to the student one with solely radiograph inputs. For evaluation, we conduct extensive experiments on both radiograph-only and multi-context benchmarks, i.e., MIMIC-CXR and MIMIC-RG4, respectively. Experimental results validate our promising performance and further ablations offer potential insights for the context learning in LLM-based RRG frameworks.

Abstract:
Video editing has become essential in digital media creation, yet existing automated systems are restricted to short segment processing and domain-specific tasks. They face two critical limitations: i) inability to handle diverse video comprehension and editing operations, and ii) lack of long-video understanding for coherent narrative creation. We propose VideoAgent, an all-in-one agentic framework addressing these challenges through two key innovations. First, we develop automated video shot creation with shot planning agents for coherent narratives and cross-modal retrieval for aligned visual content. Second, we design a multi-agent orchestration framework integrating over thirty specialized editing agents. Intent parsing filters relevant tools while textual-gradient graph optimization assembles complex editing pipelines. Extensive experiments on our newly proposed VideoEdit benchmark and public datasets demonstrate VideoAgent’s superiority over existing multimodal LLMs and agentic systems. VideoAgent achieves 87-95% orchestration success rates while reducing API costs by 60%. Human evaluation across six video categories shows that VideoAgent produces professional-quality content approaching human-level performance, with ratings only 4% below human-created videos.

Abstract:
Existing LLM unlearning methods struggle with robustness: unbounded forget losses degrade model coherence, fixed-weight balancing cannot adapt as retain difficulty shifts mid-training, and methods that work on one benchmark falter under scaling or repeated application. We propose BLADE, a constrained bilevel framework whose three mechanisms give smooth, predictable control over the optimization landscape: a clamped-entropy forget loss whose gradient is exactly zero once a token reaches sufficient uncertainty; an asymmetric augmented Lagrangian that permanently ratchets retain protection after any violation; and a bilevel structure confined to LoRA adapters that repairs retain damage before each forgetting step. BLADE dominates across three benchmark families, improving average composite scores over the strongest baselines by 6% on TOFU, 9% on MUSE Books, and 7% on KnowUndo, and it remains stable under 4× scaling and 4 sequential unlearning steps on MUSE News where the best competing method collapses entirely.

Abstract:
Real-world embodied tasks require agents to act under physical constraints while tracking evolving object and task states. Because tools mediate many everyday and professional activities, understanding these tasks requires reasoning about affordances, hand–tool–object geometry, procedural progress, and causal effects. Tool-centric embodied reasoning therefore calls for real-world egocentric training data and diagnostic benchmarks that connect these abilities. We introduce EgoTools, the first comprehensive suite for egocentric tool-use understanding. It combines EgoTools-Data, approximately 100 hours of tool-centric egocentric recordings with synchronized audio, dense captions, reasoning-heavy narrations, and supplementary 3D information, with EgoTools-Bench, a diagnostic benchmark of 1,000 QA pairs across four tracks spanning perception, geometry, procedure, and causal reasoning. EgoTools-Bench reveals a substantial gap between current models and human experts. Under strict source-video separation, supervised fine-tuning of Qwen3-VL-8B-Instruct on instruction data derived from EgoTools-Data improves accuracy from 50.0% to 60.9% on the full 1,000-question benchmark. These results establish EgoTools as a unified training and diagnostic evaluation resource for real-world egocentric tool-use understanding.

Abstract:
Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms, social morality, and policy. While computational legal research has advanced in applying established rules to cases, inducing legal rules from judicial decisions remains understudied across jurisdictions. The advent of Large Language Models (LLMs) offers unprecedented potential to automate the extraction of such latent principles, yet progress is stymied by the absence of formal task definitions and benchmarks. To address this gap, we formalize Legal Rule Induction (LRI) as the task of deriving concise, generalizable doctrinal rules from analogous precedents, distilling their shared preconditions, normative behaviors, and legal consequences. We further propose a reproducible pipeline for LRI dataset construction and, instantiating it on Chinese law as a representative jurisdiction, introduce the first LRI benchmark comprising 5,121 case sets (38,088 court cases) for model tuning and 216 expert-annotated gold test sets. Experimental results reveal that: 1) State-of-the-art LLMs struggle with over-generalization and hallucination; 2) Training on our dataset markedly enhances LLMs’ capabilities in capturing nuanced rule patterns across similar cases.

Abstract:
Textual Emotion-Cause Pair Extraction in Conversations (TECPE) requires identifying emotion utterances, their categories, and the textual cause spans that explain them. Although recent systems improve aggregate scores, clean supervision can induce evidence bias: models exploit explicit emotion cues, cause–emotion proximity, and speaker/discourse priors rather than evidence sufficient for causal explanation. We introduce SAAM-EC , a sufficiency-aware adversarial evidence masking framework for span-grounded TECPE. SAAM-EC adversarially masks high-impact conversational evidence units during training, but treats masking as a semantic intervention rather than a label-preserving perturbation. When masking removes the cause span or necessary bridge context, the original span may no longer be extractable or justified. SAAM-EC therefore combines adversarial masking with a sufficiency gate and an extractability guard, training masked examples to recover the original span only when the remaining dialogue still supports the emotion–cause relation and preserves the annotated span; otherwise, it trains the model with an unanswerability objective. All auxiliary components are discarded at inference, leaving the backbone unchanged. Experiments on span-level TECPE and utterance-level ECPE benchmarks show consistent performance gains over clean backbones and masking-based alternatives. Diagnostics further show better recovery on distant and cross-speaker causes and stronger abstention under destructive masks.

Abstract:
The rapid advancement of code large language models (LLMs) has sparked significant research interest in systematically evaluating their code generation capabilities, yet existing benchmarks predominantly assess models at a single structural granularity and focus on limited programming languages, obscuring fine-grained capability variations across different code scopes and multilingual scenarios. We introduce M 2 G-Eval, a multi-granularity, multilingual framework for evaluating code generation in large language models (LLMs) across four levels: Class, Function, Block, and Line. Spanning 18 programming languages, M 2 G-Eval includes 17K+ training tasks and 1,286 human-annotated, contamination-controlled test instances. We develop M 2 G-Eval-Coder models by training Qwen3-8B with supervised fine-tuning and Group Relative Policy Optimization. Evaluating 30 models (28 state-of-the-art LLMs plus our two M 2 G-Eval-Coder variants) reveals three main findings: (1) an apparent difficulty hierarchy, with Line-level tasks easiest and Class-level most challenging; (2) widening performance gaps between full- and partial-granularity languages as task complexity increases; and (3) strong cross-language correlations, suggesting that models learn transferable programming concepts. M 2 G-Eval enables fine-grained diagnosis of code generation capabilities and highlights persistent challenges in synthesizing complex, long-form code.

Abstract:
We introduce OceanGym, the first comprehensive benchmark for underwater embodied agents, designed to advance AI in one of the most demanding real-world environments. OceanGym features eight realistic task domains and a unified agent framework driven by Multimodal Large Language Models (MLLMs) that integrates perception, memory, and sequential decision-making. Agents must follow natural language instructions, interpret optical and sonar inputs, autonomously explore complex environments, and achieve long-horizon goals. Extensive experiments reveal a significant performance gap between state-of-the-art MLLM-driven agents and human experts, underscoring persistent challenges in perception, planning, and adaptation underwater. By offering a high-fidelity, rigorously designed platform, OceanGym establishes a testbed for robust embodied AI and represents a foundational step toward sim-to-real transfer on Autonomous Underwater Vehicles (AUVs), marking progress toward agents capable of operating in one of Earth’s last unexplored frontiers.

Abstract:
Model merging offers a training-free way to combine multiple post-trained expert models, but merging experts obtained through reinforcement learning (RL) remains challenging. Existing spectral merging methods often assume that leading singular directions contain the main task signal, while lower-energy residual components can be compressed, selected, or attenuated to reduce interference. We find that this assumption does not hold for RL task vectors: after decomposing each task vector into a leading spectral head and a residual component, both parts can independently recover substantial behavior knowledge, while exhibiting different merging properties. The head is highly concentrated and informative but more prone to sharp cross-expert conflicts, whereas the residual component is more dispersed and provides a more stable basis for aggregation. Based on this observation, we propose ResMerge, a residual-based spectral merging framework for RL experts. ResMerge first constructs a stable residual backbone with Spherical Residual Consensus Adaptation, which estimates a reliability-weighted consensus direction on the Frobenius sphere. It then reintroduces leading-head information through a Lightweight Head Correction module gated by positive cross-expert agreement. Experiments across multiple RL expert groups and capability domains show that ResMerge better preserves expert capabilities than representative task-vector and spectral merging baselines.

Abstract:
Vision-Language Pretraining (VLP) for 3D CT scans is severely constrained by the massive GPU memory required for high-resolution volumes and large contrastive batches. We propose ECT-CLIP , a resource-efficient CT VLP framework that achieves negative scaling—scaling up both the quantity and quality of negative samples—under limited computational resources. To scale the quantity, we exploit the memory asymmetry between CT and text encoding to decouple the image and text batch sizes, thereby enlarging the negative pool. To scale the quality, we design two complementary strategies: (1) deterministically separating false negatives from informative hard negatives via UMLS concept normalization, and (2) fusing multiple frozen medical text encoders to enrich semantic diversity without extra training cost. Trained on a single consumer-grade RTX 4090 GPU, ECT-CLIP achieves state-of-the-art zero-shot classification AUCs of 83.4% on CT-RATE and 75.0% on the external RAD-ChestCT benchmark, outperforming prior methods that require 4–8 high-performance GPUs. Extensive ablation studies validate the effectiveness of each negative-scaling component.

Abstract:
Low-bit activation quantization remains a major bottleneck in efficient large language model (LLM) deployment. The difficulty is not only that activations contain outliers, but that their transformed distributions often remain poorly matched to a low-bit uniform quantizer. Existing post-training quantization (PTQ) methods suppress peaks, balance channels, or minimize reconstruction error, yet they rarely specify what activation distribution is actually easy to discretize. As a result, activations may appear numerically smoother while still incurring large quantization error because the quantization range remains wide or most values collapse into a few levels near the mean. We recast activation transformation as quantizer-facing distribution design and analyze quantization error from an information-theoretic perspective. Our analysis shows that quantization-friendly activations should jointly have a smaller numerical range and sufficient dispersion within that range. Guided by this analysis, we propose InfoQuant, a train-free method that employs Peak Suppression Orthogonal Transformation (PSOT) to shape activations into more quantization-friendly distributions. We further introduce adaptive outlier-token selection to improve the robustness of PSOT during optimization. Across multiple LLM families, InfoQuant consistently outperforms prior PTQ and end-to-end training baselines. Under W4A4KV4, InfoQuant preserves approximately 97% of the floating-point zero-shot accuracy on average. Code is available at: https://github.com/LLIKKE/InfoQuant.

Abstract:
Multi-agent LLM systems delegate through natural language, creating trust boundaries that single-agent safety hierarchies were not designed to preserve. We identify provenance erasure as a multi-agent safety failure mechanism: even in the minimal two-agent case, delegation strips trust metadata—98% of format markers, 100% of authority entities, and 44% of source attribution across 460 pairs—while often preserving the actionable payload. This lossy transform enables Trust Cascade, amplifying system-channel attacks from 0% to 33–63% cascade ASR across eight open-weight models; a delegation-as-user control isolates channel trust escalation rather than content reformulation. Because the failure occurs at the delegation boundary, we propose the Trust Boundary Auditor (TBA), a zero-inference provenance diagnostic achieving 86–100% conditional detection with 0% false positives across four domains. Training, scale, and closed-source evaluations show that stronger models can interrupt different cascade stages but do not remove the boundary problem, suggesting that multi-agent safety requires delegation gating plus provenance auditing.

Abstract:
Hierarchical semantic classification is essential for taxonomy-driven applications, yet existing classifiers often optimize flat label likelihoods and therefore weakly control parent-child validity, semantic error severity, coarse-to-fine coherence, and out-of-scope rejection. This paper aims to align LLM-based classifiers with the full structured utility of hierarchical prediction rather than treating taxonomy labels as independent targets. We propose TaxoAgent, a utility-aligned policy optimization framework that formulates taxonomy prediction as a verified parent-to-child decision trajectory. Given an input, TaxoAgent first predicts coarse parent labels, expands them into fine-grained child decisions, and verifies the resulting trajectory with a composite utility that jointly measures outcome correctness, hierarchy consistency, semantic proximity, and trajectory coherence; group-relative policy optimization then reinforces high-utility trajectories over low-utility alternatives. Across IOS-APP-ZH, Google-APP-EN, RCV1, WOS, and CLINC150, TaxoAgent consistently improves Micro-F1 and Macro-F1, reduces hierarchy inconsistency, strengthens long-tail classification, and improves OOS-F1 by up to 3.68 points over the strongest prior baseline. These results show that treating taxonomies as structured decision spaces offers a practical and extensible path toward more reliable, adaptable, and utility-aware semantic classifiers.

Abstract:
Video-to-audio (V2A) generation aims to produce a single soundtrack that is simultaneously semantically faithful, temporally synchronized, aesthetically natural, and—for stereo output—spatially accurate. These perceptual axes are judged on the same waveform yet often pull generation in locally conflicting directions, while the dominant post-training recipe—reinforcement learning (RL) from an aggregated scalar reward—supervises them only at the final waveform with one entangled signal that gives little indication of which axis failed or where, so competing axes interfere through a single advantage value. We present MosaicAudio, a V2A framework that replaces terminal scalar rewards with dense, per-axis supervision distilled on-policy. MosaicAudio first trains four specialist flow-matching teachers, one per perceptual axis, then consolidates them into a single student: the student samples its own denoising trajectories and, at each visited state, receives a full teacher velocity field routed to one axis at a time, so that incompatible corrections are not averaged together. A perceptual manifold anchor keeps the student on a high-fidelity audio prior and curbs reward hacking during alignment. On the in-domain VGGSound test set and the out-of-domain AudioCanvas benchmark, MosaicAudio achieves state-of-the-art performance among compared V2A systems across objective metrics and human ratings, with especially consistent gains in complex multi-event scenes.

Abstract:
Large Language Models (LLMs) serving as long-term personal companions require memory systems that support complex reasoning over extended dialogues. However, existing Retrieval-Augmented Generation (RAG) frameworks treat memory as passive storage, suffering from semantic collapse—where subjective beliefs and objective facts become indistinguishable—and rigid navigation that fails to connect dispersed information. We introduce CogMem, a cognitive memory architecture that reconstructs dialogue context through explicit structure. CogMem organizes memory via the PEC²F (Person-Event-Concept-Claim-Fact) schema, using dedicated Claim nodes to separate subjective attributions from objective records and prevent information contamination. Rather than static retrieval, CogMem employs a deterministic controller with four cognitive operators (anchoring, traversal, intersection, evidence grounding) that dynamically navigate the graph based on query intent. Comprehensive evaluation on LoCoMo and LongMemEval shows that CogMem achieves state-of-the-art performance across all task categories, surpassing neural memory systems and graph retrieval methods. These results demonstrate that structured cognitive modeling significantly improves long-term memory accuracy and robustness compared to passive vector retrieval.

Abstract:
Recent advances in persona-centric memory demonstrate the promise of multi-agent systems for managing persona memory, especially in conversational scenarios. However, their complex pipelines often incur information loss and poor cross-scenario robustness, leading to suboptimal performance. In this paper, we propose DeltaMem, an agentic memory management system that formulates persona-centric memory management as an end-to-end task within a single-agent setting. To further improve the performance of our agentic memory manager, we draw inspiration from the evolution of human memory and synthesize a user-assistant dialogue dataset along with corresponding operation-level memory updating labels. Building on this, we introduce a novel Memory-based Levenshtein Distance to formalize the memory updating reward, and propose a tailored reinforcement learning framework to further enhance the management capabilities of DeltaMem. Extensive experiments show that both training-free and RL-trained DeltaMem outperform all product-level baselines across diverse long-term memory benchmarks, including LoCoMo, HaluMem, and PersonaMem.

Abstract:
Emotional support conversation (ESC) requires responses that recognize users’ feelings and situational meaning while balancing relational support with practical orientation. Yet it remains unclear whether improvements from synthetic post-training reflect structured empathy signals or merely stronger rewritten responses. We introduce structured empathic supervision, representing each turn with Feeling/Meaning understanding targets and Subjective/Interpersonal/Objective (S/I/O) response plans. On a coverage-oriented diagnostic suite of 219 turns from 20 ESConv dialogues, Full-SIO-SFT significantly outperforms a decoding- and budget-matched Rewrite-SFT control on aggregate support quality, perceived empathy, counseling quality, and cognitive depth. Controlled perspective analyses show target-aligned specialization: SI favors relational support, whereas IO favors informational support. Starting from Full-SIO-SFT, balanced DPO further improves the aggregate six-dimensional support objective, while directional objectives induce distinct support profiles. Blinded comparative ratings from two psychology-background annotators corroborate the structured-supervision gain and the directional profile shift. These results support structured supervision and preference weighting as complementary mechanisms for controllable ESC post-training.

Abstract:
Recent LLM benchmarks have tested models on a range of phenomena, but are still focused primarily on natural language understanding for extraction of explicit information, such as QA or summarization, with responses often targeting information from individual sentences. We are still lacking more challenging, and importantly also multilingual, benchmarks focusing on implicit information and pragmatic inferences across larger documents in the context of discourse tracking: integrating and aggregating information across sentences, paragraphs and multiple speaker utterances. To this end, we present DiscoTrack, an LLM benchmark targeting a range of tasks across 12 languages and four levels of discourse understanding: salience recognition, entity tracking, discourse relations and bridging inference. Our evaluation shows that these tasks remain challenging, even for state-of-the-art models.

Abstract:
To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing long-term memory benchmarks mainly evaluate language-centric retrieval and question answering, while embodied benchmarks often focus on short-horizon task execution without testing long-term memory use in dynamic environments. We introduce WorldLines, a project-driven benchmark for long-horizon embodied household assistance. It constructs temporally extended household traces with dialogues, actions, execution feedback, object and device state changes, and converts them into evidence-linked samples for Memory QA and Embodied Task Planning. We further propose ObsMem, an observer-grounded memory framework that maintains visibility-aware memories and action-native state trails for state-aware decisions. Experiments reveal persistent challenges in partial observability, overwritten world states, and translating long-term memory into embodied plans, while ObsMem offers a stronger reference architecture for this setting.

Abstract:
Large language model agents are expected to continuously adapt to new tasks and environments over their lifetime by reusing past experience. However, existing memory-based agents struggle to transfer reusable experience across environments and suffer from catastrophic forgetting as experience accumulated. To address these challenges, we propose LifeMem, a lifelong learning framework that enables agents to transfer knowledge across multiple environments. During learning, LifeMem clusters accumulated interaction trajectories based on underlying workflows to extract reusable skills. When solving a new task at inference time, the agent recalls relevant skills and trajectories to guide actions. To validate our method, we conduct experiments across 10 environments and over 13k tasks with 2k newly annotated interaction trajectories. Results show that LifeMem enables effective experience reuse in lifelong learning, achieving both reduced forgetting on learned tasks and superior cross-task transfer. Further analysis reveals that task streaming impacts learning, while consolidating structurally similar trajectories within memory boosts performance.

Abstract:
Emotion recognition benchmarks predict one emotion per text, missing many real-world scenarios where two people arrive at opposing emotions from a single shared trigger. For example, a child kicks her seat in excitement while the passenger ahead feels mounting irritation. We introduce Chiaro, a 1,000 human-annotated sentence benchmark for contrastive emotion inference grounded in appraisal theory. Each scene describes one causal trigger eliciting a positive emotion in one person and a negative emotion in the other, drawn from a ten-class taxonomy. We benchmark seven frontier LLMs and four off-the-shelf emotion classifiers, where the strongest LLM reaches 67.3 macro-F1, well below human agreement, while existing emotion classifiers score near chance. Beyond evaluation, Chiaro also serves as a training signal: combined with an existing emotion corpus, the resulting downstream classifier improves on Chiaro itself and on six of ten external emotion benchmarks, positioning Chiaro as a complementary signal for emotion recognition.

Abstract:
Retrieval-augmented generation (RAG) improves factual question answering by grounding LLMs in external evidence, but it becomes brittle when retrieved context conflicts with the parametric memory of the model. Existing training-free methods often address this problem by globally comparing two knowledge sources or structurally aligning them, yet they still struggle with three coupled failure axes: question complexity, long misleading evidence, and off-target parametric recall. We first quantitatively analyze the individual and joint effects of these three axes, showing that their interaction makes knowledge conflict resolution harder than any single axis alone. Motivated by this finding, we propose TRIAD, a training-free framework for knowledge conflict resolution in RAG. TRIAD decomposes a question into minimal subquestions, performs subquestion-level evidence calibration to retain concise and sufficient evidence, and uses indirect clues from calibrated evidence to guide parametric knowledge elicitation. Across three benchmarks and six LLMs, TRIAD achieves state-of-the-art performance and is the only method that surpasses a setting with additional retrieval in most cases. Additional experiments further demonstrate its generalization and worst-case robustness.

Abstract:
Long-term conversational agents often face time-sensitive queries, requiring evidence that is both contextually relevant and valid for a specific time, event, or factual update. However, noisy temporal-entity cues, weak coupling between temporal anchors and retrieval paths, and sparse long-tail evidence often lead memory systems to retrieve semantically similar but temporally invalid or outdated evidence. We propose TSHMem, a Temporal-Semantic Hierarchical Memory Graph for long-term conversational agents. TSHMem addresses noisy conversational evidence by organizing dialogue turns, self-contained statements, hierarchical summaries, and entities into a heterogeneous memory graph. To connect temporal constraints with retrieval paths, it parses temporal intent, activates query-relevant entities, resolves event-referenced constraints, and injects temporal relevance into graph transition probabilities. It then combines time-aware structural propagation with semantic matching and hierarchical ranking to retrieve compact evidence that is temporally valid, contextually relevant, and robust to sparse long-tail memories. Experiments on LongMemEval S and LoCoMo show that TSHMem consistently outperforms strong retrieval and memory-agent baselines, achieving 66.3% accuracy on LongMemEval S, with an absolute gain of 18.6% over the strongest baseline while retrieving fewer evidence tokens.

Abstract:
Contrastive decoding (CD) provides an effective inference-time strategy for audio-visual speech recognition (AVSR) by contrasting audio-visual and audio-only predictive distributions without retraining the underlying model. Existing approaches apply a fixed contrastive term at every decoding step, despite substantial variation in how informative the contrastive signal is across local decoding contexts. In AVSR, a fixed adjustment that benefits some decoding steps may provide little value or introduce instability in others, making uniform CD sensitive to the contrastive coefficient 𝛼 and limiting its reliability across varying acoustic conditions. We propose Gated CD (GCD), a training-free framework that adapts the contrastive term according to local candidate geometry at each decoding step. Since contrastive adjustment primarily affects ranking among nearby competing tokens, GCD estimates local reliability through two complementary quantities — a margin support term and a dispersion risk term — that together capture whether the contrastive signal is directionally consistent and locally stable. These are combined into a continuous gate that adaptively modulates the contrastive term during decoding. Experiments on LRS3 and LRS2 across multiple noise conditions and model scales demonstrate that GCD consistently improves over both standard AVSR decoding and uniform CD, while substantially reducing sensitivity to 𝛼 , improving robustness across acoustic conditions.

Abstract:
Latent memory, which stores past knowledge fragments as per-layer hidden states, has emerged as a promising paradigm (e.g., MemoryLLM and M+) for long-term memory in large language models (LLMs). However, the paradigm suffers from significant performance degradation during memory updates, due to positional encoding misalignment and the absence of any tracing mechanism to distinguish target memory fragments from irrelevant ones. To discover such a tracing mechanism, we probe the layer-wise attention density over stored memory fragments, and find that a small set of middle transformer layers consistently concentrates the highest density on the target fragment — exposing an inherent tracing signal. In light of this, we propose MemDefrag, a training-free and model-agnostic framework that (1) uses a middle-layer tracing signal to conduct memory defragmentation (rank, reorder, and filter memories), and (2) applies an informativeness-guided proportional forgetting mechanism once capacity is exceeded. Experiments show that MemDefrag substantially outperforms MemoryLLM and M+ on knowledge retention (e.g., 43.0% vs. 17.4%/17.6% after 50 memory updates) and long-context benchmarks, and generalizes well across various LLMs and latent-memory variants.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capabilities of large vision-language models (LVLMs). However, standard on-policy RLVR algorithms face a critical optimization bottleneck in preserving and reinforcing visually grounded reasoning behaviors: valuable visually-grounded reasoning trajectories are discarded after a single update, while uniform token advantage allocation prevents the model from reinforcing critical perception or reasoning steps. To bridge this gap, We propose PIVOT, a dual-level learning framework that anchors policy optimization around informative visual reasoning signals. Specifically, PIVOT introduces a self-calibrated experience replay mechanism, which selectively collects and replays visually-grounded historical experiences as stable reference anchors for policy optimization. Building upon this, we further design a vision-guided advantage allocation mechanism to allocate additional vision-aware advantages to tokens based on their local visual support and impact on downstream reasoning. Extensive experiments across diverse benchmarks demonstrate that PIVOT achieves highly competitive performance in enhancing the multimodal reasoning capabilities of LVLMs.

Abstract:
Search Agents face a severe reliability crisis during reinforcement learning (RL) fine-tuning. Heuristic Top-K retrieval often causes critical evidence loss or noise inclusion, while overconfidence induced by progressive RL leads to hallucinated answers and redundant searches. To build highly reliable agents, we introduce Conformal Prediction (CP) and propose Conformalized Agentic Search (CAS). This framework establishes reliability guarantees on both the retrieval and training sides: on the retrieval side, an Adaptive Prediction Set (APS), a specific CP realization, translates statistical coverage into dynamic document truncation to construct prediction sets that are adaptive in size; on the training side, Adaptive Conformal Inference (ACI), a dynamic CP algorithm, dynamically constructs prediction sets with controllable coverage to quantify answer confidence, which is then used to penalize low-confidence trajectories within the Group Relative Policy Optimization (GRPO) objective, ensuring the model learns only from reliable ones. Experiments across single-hop and multi-hop QA datasets demonstrate that our framework significantly improves reasoning accuracy while drastically reducing redundant tool invocations, establishing a highly reliable and efficient agent paradigm.

Abstract:
Automated Writing Evaluation has shifted from score-centered assessment toward revision-oriented feedback. However, current Large Language Models (LLMs) often generate generic comments that overlook essays’ explicit structure and discourse-level flaws. We propose Graph-Reasoned Automated Feedback and Evaluation (GRAFE), which uses genre-specific structural graphs to guide score-feedback generation. GRAFE encodes discourse-level priors, including argumentation logic, evidence integration, and narrative progression, as continuous representations injected into the LLM. Furthermore, we introduce a training pipeline that combines two-stage Supervised Fine-Tuning for score-calibrated initialization with Group Relative Policy Optimization to align the model toward structural diagnostic accuracy, scoring consistency, and format compliance. We construct a graph-augmented ASAP++ feedback dataset with an expert-calibrated evaluation set. Experiments show that GRAFE outperforms text-only baselines on Structure-Aware Diagnosis and Structure-Grounded Actionability, demonstrating the value of explicit structural modeling.

Abstract:
Reliable deployment of large language models (LLMs) requires accurate uncertainty estimation. Existing answer-first methods produce confidence only after generating an answer, thereby measuring the correctness of a specific response and limiting their practical usability. We study a confidence-first paradigm, where the model outputs its confidence before answering, interpreting this score as the model’s probability of answering the question correctly under its current policy. We propose CoCA(Co-optimized Confidence and Answers), a GRPO reinforcement learning framework that jointly optimizes confidence calibration and answer accuracy via segmented credit assignment. By assigning separate rewards and group-relative advantages to confidence and answer segments, CoCA enables stable joint optimization and substantially mitigates the observed reward-hacking failure mode. Experiments across math, code, and factual QA benchmarks show improved calibration and uncertainty discrimination while preserving answer quality, thereby enabling a broader range of downstream applications.

Abstract:
Linguistic steganography hides secret messages in natural language text. Large language models (LLMs) have reshaped the field, but a systematic account of how these scattered advances collectively reshape the field in this new era is still missing. We provide one along four axes: 148 steganographic methods, 60 linguistic steganalysis countermeasures, 23 evaluation metrics, and 9 open challenges, each with taxonomies, reviews, and adoption analyses. Cutting across these axes, we identify five specific paradigm shifts in the LLM era: (1) from covertext modification to prompt-only generation, (2) from heuristic to provable security, (3) from white-box symmetric LMs to black-box or asymmetric access, (4) from security-centric designs to joint optimization, and (5) from text-quality concerns to engineering issues. The survey aims to serve as both a reference and a roadmap for practical and responsible linguistic steganography in the LLM era.

Abstract:
We introduce SocialArena, a benchmark for systematically defining and evaluating the social intelligence of large language models (LLMs). Although LLMs are increasingly deployed in socially interactive applications such as mental health counseling, companionship chatbots, and educational tutoring, existing evaluations remain fragmented—focusing on isolated capabilities or single scenarios—and evaluation reliability is largely unverified. This work makes three main contributions to address these limitations. (1) We first propose a unified three-tier framework that defines social intelligence along three capability dimensions: role-playing ability, mental state reasoning, and behavioral reasoning. To operationalize this definition, we design a role-playing dialogue paradigm with explicit character thinking for evaluation, enabling simultaneous assessment of these capabilities in a manner that mirrors realistic human interactions. (2) To ensure diversity across social scenarios and thereby prevent evaluation conclusions from being confined to specific scenarios, we introduce the U-TRIMF framework, which guides us in synthesizing social contexts with broad coverage. (3) To obtain reliable evaluation, we invest 132.4K to recruit and train human annotators and adopt a human-annotated arena-style protocol, where annotators compare model responses and provide preference labels along with natural-language critiques. Experiments on a wide range of frontier LLMs reveal substantial gaps in their social intelligence capabilities. We hope that SocialArena can provide valuable insights and guidance for future LLM improvements.

Abstract:
Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear Key-Value (KV) cache growth. Prior approaches mitigate this via post-hoc selection or eviction but overlook the root inefficiency: indiscriminate token admission. In this paper, we formalize KV management as a causal system of three primitives: KV Admission, Selection, and Eviction. We instantiate KV Admission via Write-Gated KV (WG-KV), a lightweight mechanism that learns to predict token utility before cache entry. By filtering out redundant states early to maintain a compact global cache alongside a sliding local cache, WG-KV significantly reduces memory usage and accelerates both prefill and decode phases. Our results demonstrate that learning what to write is a principled and practical recipe for efficient long-context inference. Code is available at https://github.com/EMCLab-Sinica/WG-KV.

Abstract:
As autonomous agents increasingly interact, they inevitably attempt to influence one another. While prior work in text-only settings has explored the dynamics of Agent-to-Agent (A2A) persuasion, the rise of Vision-Language Models (VLMs) introduces a more complex challenge: multimodal content conveys richer information while integrating subtle, hard-to-detect persuasive cues. To study this vulnerability, we present MMPersuade, a unified framework and dataset for A2A multimodal persuasion. We model interactions between a persuader agent, which leverages images and psychological strategies, and a persuadee VLM. Our benchmark spans commercial, subjective and behavioral, and adversarial contexts, and evaluates persuasion via function-calling that capture behavioral shifts beyond verbal responses. Experiments on six VLMs reveal three findings: (1) multimodal inputs consistently outperform text-only persuasion, with raw visual signals uniquely increasing susceptibility in adversarial settings by bypassing text-activated safety defenses; (2) persuadee vulnerability is highly domain- and format-dependent, with realistic and community-style formats driving susceptibility in commercial settings while different formats dominate in adversarial ones; and (3) psychological strategy efficacy varies with context and model architecture, as more capable models resist benign persuasion yet become more susceptible under adversarial multimodal inputs. Our framework provides a foundation for building more robust and aligned VLMs in multi-agent environments.

Abstract:
Unstructured discharge notes in Electronic Health Records (EHRs) often carry signal complementary to structured medical codes, holding patient-specific evidence that standardized cohort-level codes alone cannot capture. However, this evidence in notes is frequently buried in lengthy, noisy text that is not intentionally written with any specific clinical prediction in mind. Summarization is an obvious mitigation, but generic summaries, tuned for fluency rather than the outcome, routinely omit decisive evidence while retaining plausible but uninformative detail. To this end, we propose RASPER, a Reward-Aligned Summarizer for Prediction in EHR, that optimizes note summarization directly against the downstream clinical task. RASPER employs a tunable LLM-based summarizer to extract task-relevant evidence from discharge notes and trains it via reinforcement learning from prediction feedback, using a reward derived from the downstream predictor’s loss. To ground the summarizer, a longitudinal encoder converts structured codes into soft prompts that incorporate each patient’s clinical context into note summarization. By rewarding the quality of the resulting multimodal prediction, RASPER encourages the summarizer to retain patient-specific evidence that complements, rather than duplicates, information captured by structured codes. RASPER consistently outperforms strong baselines on both readmission prediction and medication recommendation across MIMIC-III and MIMIC-IV.

Abstract:
LLM-based multi-agent systems (MAS) increasingly use latent collaboration to avoid the information loss and repeated encoding-decoding overhead of natural-language communication. However, directly forwarding all sender latents makes the receiver-side context scale with both the number of agents and the reasoning length, increasing computation, memory usage, and collaboration latency. A natural solution is latent compression. But we find that cross-agent redundancy remains unresolved in existing latent compression approaches, which typically compress each sender independently and then concatenate the results. We propose LatCom, a cross-agent latent compression framework for efficient multi-agent latent collaboration. LatCom maps multiple sender latents into a fixed number of receiver-readable and task-relevant slots. Rather than reconstructing all sender hidden states, it optimizes the compressed latents for receiver-side task utility. LatCom trains the compressor in two stages: single-sender readability learning first establishes a latent interface interpretable by the frozen receiver, and multi-sender fusion learning then trains the compressor to fuse complementary evidence and remove redundancy across agents. Experiments on multiple benchmarks with the Qwen-4B model show that LatCom improves average inference speed by 2.46x over LatentMAS, reduces token usage by 70.3%, and increases average accuracy by 0.46%.

Abstract:
Post-training pretrained autoregressive models (ARMs) into masked diffusion models (MDMs) provides an efficient route to diffusion language modeling, but it remains unclear whether the resulting models reuse inherited autoregressive computation or reorganize it for non-autoregressive generation. We compare two 7B ARM–MDM families across four controlled diagnostic tasks and find a task-dependent mechanism shift. On prefix-dominant tasks, MDMs largely preserve inherited high-attribution pathways or exhibit only modest changes in where computation occurs. On globally constrained tasks, the reorganization is substantially stronger, with task-relevant computation shifting toward earlier layers. This depth-wise pattern persists across prompt resampling, circuit budgets, and tested inference budgets, while targeted ablations support the functional importance of the identified structures under the tested intervention protocols. At the component level, diagnostic probes suggest that ARMs rely more strongly on sharply specialized components, whereas MDMs exhibit weaker single-component specialization and more diffuse output-space alignment. Together, these results suggest that diffusion post-training selectively preserves or reorganizes inherited computation according to task structure, rather than uniformly replacing autoregressive mechanisms.

Abstract:
Large language models often answer structurally unanswerable questions, such as computing \cot (-540∘) or evaluating (1).startswith("1"), instead of abstaining. We ask whether this failure reflects missing recognition or failed routing from recognition to abstention. Across instruction-tuned models from 1.7B to 70B parameters, a single linear direction in the hidden state separates answerable from structurally impossible math and code prompts, showing that models represent impossibility before generation. Yet this recognition direction is nearly orthogonal to the canonical safety-refusal direction that mediates trained harmful-content refusal. An in-domain behavior-defined invalidity-aware direction is closer to recognition, but only partially aligned with it, and remains near-orthogonal to safety refusal. Generation-time steering along the recognition direction changes invalidity-aware behavior bidirectionally and dose-responsively on structural math and code cells, while random directions do not. Base/instruct comparisons further show that the low-cosine geometry is already present at the pretraining endpoint. The confident-on-impossible failure is therefore better explained as a routing failure than as an encoding failure: the model has a usable “no admissible answer” signal, but the safety-refusal pathway is not aligned to use it.

Abstract:
In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the language mixture ratios. Multilingual scaling laws can predict the test loss under different language mixture ratios and can therefore be used to estimate the optimal ratios. However, the current approaches to multilingual scaling laws do not explicitly quantify cross-lingual transfer between individual languages, resulting in suboptimal mixture-ratio optimization. In this paper, we consider multilingual pretraining as a cooperative game in which each language acts as a player that jointly contributes to pretraining, gaining the resulting reduction in test loss as the payoff. Consequently, from the perspective of cooperative game theory, we quantify the cross-lingual transfer from each language by its contribution in the game, and propose a game-theoretic multilingual scaling law called ShapleyLaw. Our experiments show that ShapleyLaw outperforms baseline methods in model performance prediction and language mixture optimization. Our code and data are available at [https://github.com/YcTtXyLLM/ShapleyLaw](https://github.com/YcTtXyLLM/ShapleyLaw).

Abstract:
LLMs are increasingly used to arbitrate among multiple candidate outputs and compress them into a single final result. The final answer may not be incorrect, yet information compression can still make it suboptimal. We define this problem as source-resolution distortion (SRD), the irreversible loss or incorrect mapping of task-relevant source-side structure in the final output during arbitration. We model pairwise arbitration as a compression process; we then present a source-bounded SRD measurement framework that aligns the reference structure fixed before arbitration with the final output and uses six metrics to measure its preservation and incorrect mapping. To reduce SRD, we further propose A²R (Auditable Arbitration for Resolution), a reference resolution method based on a decision ledger. Motivated by anti-aliasing in signal processing, A²R extracts query-relevant source units and records the local resolution relations among them in a decision ledger before using it to constrain final generation. Across 520 pairwise cases, compared with Checklist Prompt, A²R-Decision lowers CER from 0.23 to 0.16 and raises Prov-H from 0.45 to 0.67. Its answer-preference scores against three structured baselines range from 51% to 53%, showing that similarly preferred answers can still differ markedly in source recoverability. A 200-case three-candidate extension preserves the system ordering.

Abstract:
Jailbreaks against multimodal large language models (MLLMs) typically operate through image-text inputs. After each target response, the attacker must decide how the new feedback should shape the next image-text query: which clues should be stated in the text, which should appear as visual evidence, and how the text should refer to the image.Existing multimodal attacks typically rely on fixed image-text pairs or candidate search, limiting their ability to adapt across turns.Text-only multi-turn attacks can adapt to prior responses, but they do not decide how response-derived clues should be expressed across image and text.In this paper, we formulate this missing step as trace-to-query realization and propose ̲M emory- ̲A ugmented ̲Vi sual ̲Q uerying (MAViQ), a black-box framework that converts response-derived clues into visual and textual constraints, refines the query after observing the generated image, and propagates feedback to subsequent turns.Extensive experiments on two benchmarks across six target MLLMs show that MAViQ achieves average attack success rates (ASRs) of 70.60% on MM-SafetyBench and 82.13% on JailbreakV-mini, outperforming the strongest baseline by 5.90% and 8.32%, respectively. Our code is available at https://github.com/jixingtian/Maviq.

Abstract:
Prompt injection is listed as the #1 threat to AI agents. When an agent accesses external data from websites, files, or emails, an attacker may inject a prompt into the data, saying, "Ignore all prior instructions and perform ." To prevent arbitrary manipulation of agents, defenders try to train secure LLMs, which, however, still suffer from near 100% attack success rates (ASRs) against adaptive prompt injections. We note that this is because existing defensive fine-tuning recipes rely on sequence-level feedback signals (in DPO or GRPO). Treating an entire output equally prevents the model from learning precisely which output tokens are insecure. In this paper, we propose Secure On-Policy Distillation (SecOPD) that provides token-level feedback to guide defensive fine-tuning. The LLM receives an injected sample and produces a rollout, whose tokens are scored by the initialization model given the corresponding clean input. With more fine-grained training signals, our defended Qwen3.6-27B achieves a 9.0% ASR against the SoTA PISmith adaptive prompt injections, compared to 94.0% for the prior SoTA, Meta-SecAlign. The obtained security generalizes to domains completely unseen in training: in agentic tool calling, SecOPD achieves a 4.7% ASR compared to 5.5% for Meta-SecAlign.

Abstract:
Large reasoning models achieve strong performance on complex tasks by generating extended chain-of-thought (CoT) traces via reinforcement learning with verifiable rewards (RLVR). While current RLVR methods have achieved strong results with correctness-based reward signals, they provide limited guidance on the quality of the reasoning process itself, leaving the internal reasoning structure largely unoptimized. Through empirical analysis across multiple model families, we identify a consistent pattern: correct reasoning traces exhibit more frequent and larger token-level entropy drops within the thinking phase than incorrect ones. We propose ERR+, a two-phase RLVR framework grounded in this observation. The first phase trains with the Entropy Relief Reward (ERR), a bonus proportional to cumulative token-level entropy drops in the thinking phase, log-normalized by response length. Unlike prior methods that suppress entropy, ERR rewards the resolution of uncertainty while leaving exploratory high-entropy states unconstrained. The second phase introduces the Robust Relative Efficiency Reward, which scores each response’s length against co-generated peers via a tanh-transformed within-group z-score. We provide a formal analysis showing that joint optimization of the two objectives induces gradient conflict in early training, motivating the sequential design (Theorem 1). Experiments on five datasets demonstrate consistent improvements in both accuracy and response conciseness across model backbones.Our code is available at https://github.com/XrkArul/err_response.

Abstract:
Voting-based leaderboards such as LM Arena are central to open-ended LLM evaluation, but their reliability depends on response anonymity. Prior work shows that surface features such as bag-of-words and TF-IDF can reveal model identity from the response, yet such signals are unreliable when models are stylistically similar or closely matched, and are easily down-weighted during vote aggregation. To examine the remaining anonymity risk, we introduce INTERPOL, a model-driven identification framework that learns target-specific response signatures from interpolated preference data. INTERPOL synthesizes controllable hard negatives by interpolating a target-mimicking copy model with its initial backbone, and trains a target-specific detector via curriculum-based triplet preference learning. Across three frontier target LLMs and two query sources, INTERPOL yields average relative gains of 32% in Accuracy and 25% in AUROC over prior methods. Moreover, in arena-style promotion simulations, INTERPOL attains the same ranking gain with about 19% fewer interactions than existing identification baselines; with competitor-aware voting, interactions are further reduced by about 69.9% relative to target-only voting.

Abstract:
Peer review is the primary mechanism for evaluating scientific contributions, yet prior studies have mostly examined paper features or external metadata in isolation. The emergence of open platforms such as OpenReview has transformed peer review into a transparent and interactive process, recording not only scores and comments but also rebuttals, reviewer–author exchanges, and meta-reviewer decisions. In this paper, we present a large-scale empirical study of ICLR 2017–2025, encompassing over 28,000 submissions, integrating four dimensions: paper structure and language quality, submission strategies and external metadata, author–reviewer interaction dynamics, and reviewer disagreement patterns. Our results show that factors beyond scientific novelty significantly shape acceptance outcomes. In particular, the rebuttal stage emerges as a decisive phase: timely, substantive author–reviewer communication strongly increases acceptance likelihood, often outweighing initial reviewer skepticism. Clearer writing, earlier submission, and effective disagreement resolution also correlate with higher acceptance probabilities. Based on these findings, we propose data-driven guidelines for authors, reviewers, and meta-reviewers to enhance transparency and fairness in peer review.

Abstract:
Large language models (LLMs) are being increasingly used to answer subjective, information-seeking questions, where users are sensitive to how responses are communicated, not just whether the answers are correct. Existing LLM evaluations for subjective cultural queries largely focus on factual correctness, ignoring how the response is framed. To this end, we introduce FRANZ, an automated FRAmework for respoNse characteriZation to conduct a communicative audit of LLM responses along four dimensions: cultural positioning, use of generalizing language, anthropomorphic cues, and adherence to conversational maxims. To enable this evaluation, we contribute SQUARE - a corpus of 376k subjective questions sourced from 57 subreddits and mapped to 7 countries and 19 question categories. We demonstrate FRANZ’s applicability by scoring responses from three open-weight LLMs. We observe that LLMs show statistically significant differences in the frequency with which they employ each response characteristic. Unlike single-dimensional audits, FRANZ reveals that insider positioning and anthropomorphism are positively coupled, with the degree of coupling varying by country, providing a diagnostic lens for identifying framing divergences.

Abstract:
Medical Image Quality Assessment (IQA) serves as a critical safety gate for clinical AI, yet existing approaches rely on scalar metrics that fail to capture the descriptive, expert-like evaluation process central to clinical practice. We introduce MedQ-Bench, a comprehensive benchmark for language-based medical IQA evaluation with MLLMs, establishing a perception–description paradigm across 5 imaging modalities and over 40 quality attributes. The benchmark comprises two complementary tasks: MedQ-Perception probes low-level visual attribute discrimination via 2,600 expert-annotated perceptual queries; MedQ-Description covers both no-reference and comparison description sub-tasks with 708 expert-annotated samples, evaluated through a four-dimensional judging protocol. Comprehensive evaluation of 23 state-of-the-art MLLMs reveals that current models exhibit only preliminary and unstable capabilities, and highlights the urgent need for targeted optimization of MLLMs in medical IQA. Project Dataset: https://github.com/liujiyaoFDU/MedQ-Bench

Abstract:
Large-table semantic annotation aims to assign ontology-grounded semantic labels to columns and column pairs in large relational tables, while considering the efficiency bottleneck caused by the increasing scale of table contexts. However, as relational tables continue to grow in scale, existing methods face a critical effectiveness–efficiency trade-off: directly modeling the full table preserves rich context but introduces substantial computational overhead, while aggressive filtering may discard useful semantic cues. To address this issue, we propose TRACER, an information entropy-grounded framework for efficient large-table semantic annotation. TRACER ranks table rows according to ontology-related semantic uncertainty and constructs compact evidence-oriented contexts through hybrid compression, enabling a lightweight semantic reader to perform column type and column property annotation efficiently. From an information-theoretic perspective, TRACER formulates large-table compression as a conditional entropy minimization problem, providing a principled basis for preserving key semantic evidence while filtering redundant information. Experiments on several benchmarks show that TRACER achieves a strong balance between annotation effectiveness and inference efficiency compared with task-specific and large language model baselines. The source code and data are available at https://anonymous.4open.science/r/TRACER-4884.

Abstract:
Mobile Graphical User Interface (GUI) agents powered by multimodal large language models have demonstrated promising capabilities in automating complex smartphone tasks. However, existing approaches face two critical limitations: the scarcity of high-quality multilingual datasets, particularly for non-English ecosystems, and inefficient history representation methods. To address these challenges, we present SecAgent, an efficient mobile GUI agent at 3B scale. We first construct a human-verified Chinese mobile GUI dataset with 18k grounding samples and 121k navigation steps across 44 applications, along with a Chinese navigation benchmark featuring multi-choice action annotations. Building upon this dataset, we propose a semantic context mechanism that distills history screenshots and actions into concise, natural language summaries, significantly reducing computational costs while preserving task-relevant information. Through supervised and reinforcement fine-tuning, SecAgent outperforms similar-scale baselines and achieves performance comparable to 7B-8B models on our and public navigation benchmarks. Our dataset and benchmark are available at https://huggingface.co/datasets/alibabagroup/CMGUI.

Abstract:
Long-video understanding remains challenging for multimodal large language models (MLLMs), as informative visual evidence is often sparsely distributed across many redundant frames. Although sampling more frames can improve coverage, it also increases visual encoding overhead and consumes more visual tokens, making compact frame selection essential. However, existing methods often sample candidate frames uniformly before selection, which may require decoding most frames of the video. Moreover, simply selecting frames by query relevance may overlook the global temporal structure needed for coherent video understanding. To address these limitations, we propose VISA , a VI deo S keleton- A ware framework for efficient long-video frame selection. VISA exploits sparse I-frame anchors from the native GOP structure to avoid dense candidate decoding and constructs a video skeleton that preserves the coarse temporal progression. Query relevance is aggregated over this skeleton to select informative anchors, followed by cross-GOP refinement to obtain final frames. Extensive experiments show that VISA consistently improves long-video understanding, achieving up to 8.8% relative gain over uniform sampling. Meanwhile, VISA reduces average end-to-end latency by 32.9%–53.6% compared with recent frame selection baselines on LongVideoBench, demonstrating both effectiveness and efficiency for long-video frame selection.

Abstract:
Long-horizon tool-use agents need not only to search and plan, but also to decide when to finalize. We study late-stage pressure states, in which an agent is biased toward submitting a final answer that appears complete and polished while key constraints remain unresolved. We first train a linear probe to show that this pressure state is identifiable from the agent’s hidden states. Then, we use activation interventions along this pressure direction and find that shifting the hidden states changes both the pressure score and whether the agent continues tool use or submits early. Through controlled context manipulations, we further see that the pressure is mitigated by constraint clarity and action mapping. Based on these findings, we propose Probe-Sensed Pressure Relief (PSPR), a plugin that applies lightweight pressure relief direction under moderate pressure and moves to structured organization under high pressure risk. Experiments on multiple long-horizon benchmarks show that our method consistently strengthens existing agent methods.

Abstract:
Multi-hop question answering requires retrieving and integrating multiple pieces of evidence distributed across documents. Recent graph-based methods improve multi-hop reasoning through graph traversal over relational evidence structures. However, existing methods rely on independent path ranking during evidence organization, which often produces redundant and reasoning-incomplete evidence sets due to substantial structural overlap among selected graph paths. In this work, we argue that graph-based evidence organization may, to some extent, be formulated as a constrained evidence subgraph optimization problem. Based on this perspective, we propose CROG, a coverage-aware reasoning optimization framework that performs marginal-gain-based path selection to favor previously uncovered reasoning steps while suppressing redundant evidence, with node-budget and connectivity constraints to encourage compact and coherent evidence subgraphs. Experiments on multiple benchmarks demonstrate that CROG achieves superior reasoning performance while using fewer evidence units on average, and analysis based on exhaustive optimal search further validates the effectiveness of the greedy optimization strategy.

Abstract:
Existing reinforcement learning (RL) approaches treat large language models (LLMs) as a unified policy, overlooking their internal mechanisms. In this paper, we decompose the LLM-based policy into Internal Layer Policies and Internal Modular Policies via the Transformer’s residual stream. Our entropy analysis of internal policy reveals distinct patterns: (1) universally, internal policies evolve from high-entropy exploration in early layers to deterministic refinement in the top layers; and (2) Qwen exhibits an explicit progressive reasoning structure, contrasting with the abrupt convergence in Llama. Furthermore, we discover that optimizing internal layers induces feature refinement, forcing lower layers to capture high-level reasoning representations early. Motivated by these findings, we propose Bottom-up Policy Optimization (BuPO), a novel RL paradigm that reconstructs the LLM’s reasoning foundation from the bottom up by optimizing internal layers in early stages. Extensive experiments on complex reasoning benchmarks demonstrate the effectiveness of BuPO.

Abstract:
Large language models (LLMs) often produce incorrect answers with high confidence, yet the factors associated with such overconfidence remain insufficiently understood. We study this problem through the lens of knowledge popularity. Using entity-centric factual QA derived from Wikidata triplets, we characterize popularity through question entity popularity, answer popularity, and question–answer co-occurrence. We find two consistent patterns. First, hallucinated answers are far from random: compared with ground-truth answers, they tend to be more popular or more frequently associated with the question entity. Second, confidence is strongly tied to the popularity of generated answers: even among incorrect predictions, more popular answers or those with higher question–answer co-occurrence receive higher confidence. Together, these findings suggest that popular but wrong alternatives may contribute to overconfidence. We further show that popularity-related signals can mitigate overconfidence and improve overall confidence estimation. Across six models and three datasets, incorporating knowledge popularity reduces average confidence on incorrect answers from 0.765 to 0.254 and overall ECE from 0.356 to 0.050, while improving Alignment from 77.08% to 83.72%.

Abstract:
Mobile GUI agents often act confidently under ambiguous or incomplete user instructions-a common situation in real-world use. Even when explicitly offered a "call the user" action, open-weight agents ask for clarification in only 0.1% of cases on average, instead guessing the user’s intent and proceeding with actions. We name this failure Act-without-Asking. Existing step-level benchmarks largely miss this failure mode because they credit any ground-truth-matching action regardless of asking. We introduce MID-Bench, a step-level ambiguity benchmark constructed by rewriting clear instructions into ambiguous ones, with a strict clarification protocol: a user simulator answering only what is asked, and a scoring rule crediting a step only when the agent both asks an appropriate question and emits the correct final action. The Hard subset applies stricter masking to the action-critical detail in the instruction, making it harder to succeed without asking. Across four open-weight GUI agents, train-free cascades in which a gate decides whether to act or ask the user for clarification substantially improve ambiguous-task performance while preserving accuracy on clear instructions; LoRA fine-tuning on MID-Bench further combines effectively with these cascades.

Abstract:
Weak-to-strong (W2S) generalization is a promising framework for scalable oversight, yet existing evaluations often test students under matched train-test distributions. Therefore, we study W2S preference learning under zero-shot distribution shift and find that strong students trained on weak preference labels can appear successful in-distribution while failing to transfer across preference datasets. We provide evidence consistent with a representational failure mode in which weak-supervised fine-tuning may move the strong model toward source-domain features instead of maintaining broadly transferable preference representations. To mitigate this, we propose Representation Anchoring (Anchor), a simple yet effective regularizer that constrains excessive drift from the pretrained strong model’s representation space during fine-tuning, while still allowing task-relevant adaptation. Across preference domains, datasets, and model families, Anchor improves out-of-distribution transfer while maintaining competitive in-distribution performance. Together, our evaluation protocol, transfer-aware metrics, and method expose hidden brittleness in current W2S reward modeling and provide a practical path toward more robust preference transfer.

Abstract:
Large Language Models (LLMs) have demonstrated impressive progress in complex reasoning tasks, largely driven by the Chain-of-Thought (CoT) paradigm, which decomposes difficult problems into intermediate steps. However, CoT reasoning remains fundamentally constrained by the probabilistic nature of neural generation, leading to unfaithful reasoning chains that undermine reliability. Neuro-symbolic approaches attempt to address these issues by combining LLMs with symbolic solvers, yet they face persistent challenges, including hallucinated translations, the mismatch between natural language and formal logic, and the limited enhancement of the LLM’s intrinsic reasoning ability. To overcome these limitations, we propose Symbolic-Neural Soft Reasoning (SSR), a unified framework that integrates LLMs with symbolic reasoning and improves robustness by relaxing strict logical determinism while preserving verifiability. Our approach improves reasoning performance, automatically generates verifiable and human-readable logical thinking chains for training and fine-tuning, and facilitates cross-disciplinary applications such as AI for mathematics. Experiments across multiple models and benchmarks demonstrate that SSR consistently outperforms existing reasoning frameworks, highlighting its effectiveness in enhancing both the robustness and interpretability of LLM reasoning. Our code is available at https://github.com/1240148048/SSR.

Abstract:
Recent work on ASR for endangered languages has focused on fine-tuning a pretrained multilingual model to a small amount of data in the target language, but even with large and diverse pretrained models, error rates can be stubbornly high. Including additional data from the target language can yield improvements, but acquiring this kind of data for endangered languages is challenging. One commonly proposed alternative is to leverage data in a more highly resourced language from the same language family, but there has been remarkably little work evaluating the real-world utility of this practice. In this short paper, we explore the relevance of phylogenetic relatedness of transfer data for improving ASR for endangered languages with minimal training resources. We find, unexpectedly, that augmenting with unrelated donor language data improves word error rate (WER) more than related language data. In addition, the performance improvement observed via transfer of unrelated data is larger than that observed when nearly doubling the quantity of in-language training data. These surprising results hold even when controlling for factors such as acoustic similarity, audio quality, and subword overlap. Our findings point the way to practical new approaches for improving ASR for endangered languages.

Abstract:
Large language models (LLMs) default to figurative interpretations of idiomatic expressions when context supports them. We investigate whether this bias can be directly controlled via activation steering: asking whether idiomaticity is encoded as a manipulable direction in residual space. We introduce IdioSteer, a small controlled benchmark of 100 sentences across 20 idioms designed to expose interpretation-dependent generation. Using a steering vector applied to five LLMs from two model families, ranging from 2B to 12B parameters, we show that steering systematically shifts model outputs toward literal interpretations while preserving fluency. Three findings stand out: cross-layer steering, vectors from mid-to-late layers injected at early layers - consistently outperforms same-layer interventions; steering vectors transfer to held-out idioms; and effects replicate across architectures and model sizes using the same intervention protocol.

Abstract:
Tabular prediction can benefit from in-table rows as few-shot evidence, yet existing tabular models typically perform instance-wise inference and LLM-based prompting is often brittle. Models do not consistently leverage relevant rows, and noisy context can degrade performance. To address this challenge, we propose TabSieve, a select-then-predict framework that makes evidence usage explicit and auditable. Given a table and a query row, TabSieve first selects a small set of informative rows as evidence and then predicts the missing target conditioned on the selected evidence. To enable this capability, we construct TabSieve-SFT-40K by synthesizing high-quality reasoning trajectories from 331 real tables using a strong teacher model with strict filtering. Furthermore, we introduce TAB-GRPO, a reinforcement learning recipe that jointly optimizes evidence selection and prediction correctness with separate rewards, and stabilizes mixed regression and classification training via dynamic task-advantage balancing. Experiments on a held-out benchmark of 75 classification and 52 regression tables show that TabSieve consistently improves performance across shot budgets, with average gains of 2.92% on classification and 4.56% on regression over the second-best baseline. Further analysis indicates that TabSieve concentrates more attention on the selected evidence, which improves robustness to noisy context.

Abstract:
Wearable augmented reality (AR) assistants are moving toward continuous real-world interaction, where they perceive the user’s activity through first-person video and audio and provide timely spoken guidance without being explicitly asked. While proactive video assistants, spoken dialog systems, and egocentric task understanding have each advanced rapidly, existing systems do not address the joint problem of deciding when to speak and what to say from continuous first-person streams. We introduce EgoVoice, a framework for training and evaluating proactive egocentric spoken assistants. From HoloAssist video recordings of real human instructors, we construct clean audio streams through source separation and speech resynthesis, and convert each video session into a format where the model must decide at each moment whether to remain silent or provide spoken guidance. We fine-tune an omni-modal LLM with our data, and further improve its proactive intervention behavior with direct preference optimization. Experiments across closed and open-source models show that existing systems rarely produce well-timed, meaningful proactive interventions, while EgoVoice yields clear improvements in intervention timing, content relevance, and human preference over the zero-shot backbone. We will release the dataset, code, and model checkpoints.

Abstract:
Model merging provides an efficient paradigm for constructing multi-task large language models (LLMs) without full model retraining, yet it remains challenged by parameter interference. While existing methods aim to preserve the capabilities of individual expert models and mitigate interference, they generally do not directly learn from the potentially degraded behaviors exposed by naive merging. In this paper, we propose a conflict-driven preference optimization framework for model merging (CoMerge), which reformulates model merging as a preference optimization problem. The approach utilizes a self-supervised, conflict-driven strategy that leverages the defects of naive merging methods (e.g., task arithmetic) as hard negative samples to construct preference pairs without external annotations. By applying preference optimization to refine lightweight, tensor-wise merging coefficients, CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities. Extensive experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench, outperforming all evaluated data-free and data-driven model-merging baselines. Furthermore, on Llama-3.1-8B-Instruct, CoMerge yields marked improvements on conflict-sensitive tasks such as instruction following and safety, while remaining highly competitive with full-parameter fine-tuning despite optimizing only 1,445 scalar coefficients.

Abstract:
Distilling the tool-use capabilities of large language models (LLMs) into small language models (SLMs) is essential for their practical application. The predominant approach, supervised fine-tuning (SFT), is an off-policy distillation method that suffers from poor out-of-domain (OOD) generalization because it rigidly aligns with static teacher trajectories. While reinforcement learning (RL) offers an alternative, the capacity limitations of SLMs pose a severe dilemma: sparse outcome rewards provide insufficient guidance, whereas strict trajectory matching imposes overly restrictive constraints. To bridge this capacity-driven gap, we propose MENTOR, an on-policy distillation framework that introduces a flexible yet process-aware reward structure. Instead of enforcing rigid replication, MENTOR uses the teacher’s reference to guide tool-use behavior, balancing behavioral alignment with downstream performance. Extensive experiments on controlled executable-tool benchmarks demonstrate that MENTOR improves OOD tool-use performance compared to SFT and strict RL baselines. Our findings suggest that within verifiable tool-use environments, flexible tool-use alignment offers a more effective approach than strict trajectory replication for developing adaptable small models.

Abstract:
Long-context LLMs and Retrieval-Augmented Generation (RAG) systems expand accessible context, but still defer state tracking and evidence consolidation to query time. This passive paradigm is brittle in streaming settings where facts are updated and answers depend on latent states rather than explicit spans. We introduce Unified Memory Agent (UMA), an end-to-end reinforcement-learning framework that trains a single policy to maintain external memory and answer questions. UMA updates a structured memory bank via CRUD operations and combines it with raw-context retrieval for QA. To jointly optimize memory maintenance and question answering, we propose Task-Stratified GRPO, which normalizes heterogeneous trajectories separately for more effective credit assignment. We also introduce Ledger-QA, a controlled benchmark for long-horizon state tracking over accumulated updates. Experiments on Ledger-QA and 13 TTL/AR benchmarks show that UMA improves long-horizon tracking, achieves the best average TTL/AR score among compared methods, and remains competitive on retrieval-centric benchmarks. Our results highlight learned proactive memory management as a complement to longer context windows and query-time retrieval.

Abstract:
Model compression techniques such as pruning and quantization facilitate the efficient deployment and acceleration of Large Language Models (LLMs). However, recent studies show that weight sparsification methods, such as SparseGPT, can amplify existing biases in models, with outputs varying significantly depending on persona cues in the prompt. In this paper, we introduce Debias-SparseGPT, a post-training pruning method incorporating representational debiasing using a second-order term defined over demographically contrasting inputs. We perform empirical validation of our method over a wide range of generative LLMs. Across models and sparsity regimes (25%, 50%, and structured 2:4 sparsity), Debias-SparseGPT consistently reduces pruning-induced bias compared to SparseGPT while preserving model perplexity and zero-shot accuracy. Under the most restrictive 2:4 structured sparsity pattern, which most aggressively degrades model quality, augmenting the calibration set with long-context, content-rich examples further improves both downstream performance and fairness. Overall, Debias-SparseGPT advances the bias-performance trade-off while preserving the computational efficiency of sparse models.

Abstract:
Large reasoning models (LRMs) have led to new possibilities in terms of problem-solving, through the devising of a natural language thought process prior to answering a query. While their capabilities are well known across mathematics and coding tasks, their impact on the task of machine translation (MT) remains underexplored. In this work, we explore the benefits of the generation of intermediate tokens when performing MT across multiple language pairs of different levels of resourcedness and multiple setups. We first show that the "thinking tokens" of general-purpose LRMs do not help them better perform MT, motivating the question of what kind of intermediate tokens could help when building a dedicated thinking MT model via fine-tuning. We then show that distilling chains of thought (CoT) from a teacher model using CoT templates inspired by human translators’ step-by-step practices fails to outperform standard input-output fine-tuning. In contrast, constructing CoT traces with prompting strategies specifically designed for MT does yield improvements. A closer analysis reveals that these gains stem from the teacher’s translation attempts embedded in the traces, rather than from the surrounding reasoning. More broadly, our results suggest that leveraging a teacher to refine target translations or expand parallel corpora is more impactful than distilling its CoT explanations into "thinking" MT models.

Abstract:
Large language models are increasingly used in settings where information must be evaluated not only for content, but also for reliability. This paper introduces a human-grounded evaluation[All data and code used in this paper will be made available with the camera-ready version.] of source-sensitive calibration in Turkish and English. Across hearer-side uptake tasks, we manipulate source reliability while holding propositional content constant, and test whether humans and LLMs adjust action-readiness judgments according to source trustworthiness. Turkish human participants show robust source sensitivity: they are substantially more willing to act on information attributed to high-trust sources than to low-trust sources, while Turkish evidential morphology itself does not independently affect uptake once the source is overtly specified. LLMs show weaker calibration. In Turkish, models partially register explicit source reliability, but the effect is much smaller than in humans and there is no reliable effect of the -DI / -mIş contrast. In English, models show small effects of both source reliability and the lexical marker apparently , but no robust interaction between them. These results suggest that current LLMs encode source cues only partially and often fail to convert them into human-like judgments about when information is reliable enough to guide action.

Abstract:
While Large Language Models (LLMs) are increasingly deployed in long interactions, existing evaluations focus predominantly on retrospective memory (RM) via explicit queries. Prospective memory (PM), the ability to spontaneously recall and act on latent constraints without direct requests, remains largely unevaluated. We introduce TriggerBench, a five-dimensional PM benchmark across daily assistants and professional workflows. TriggerBench pairs scenarios with matched RM controls, contrastive positive/negative variants, and overloaded triggers, enabling fine-grained measurement of proactive recall, false-alarm rate, and attentional robustness. We find: (i) PM shows a precision-recall trade-off and attentional fragility. Though enhanced reasoning improves proactive recall, models can overfit to always-remind heuristics. Furthermore, PM accuracy degrades substantially under implicit constraints or triggers overloaded by concurrent requests, indicating that robust PM remains an open challenge. (ii) PM is notably harder than RM: on identical contexts, RM remains near-saturated up to 100K tokens, while PM decays sharply as context length increases. (iii) PM may serve as a behavioral probe of spare reasoning capacity. Pairing PM with AIME-2025 math problems reveals that successful trajectories yield higher PM accuracy than failed ones at the same context length, showing PM is a behavioral correlate of spare reasoning budget that token count obscures. Project page: https://github.com/KristenZHANG/TriggerBench-Official.

Abstract:
Reinforcement learning for long-horizon agents and multi-step reasoning models typically relies on outcome-level rewards, which provide only trajectory-level supervision and make it difficult to identify the contribution of individual intermediate actions. As a result, redundant actions in successful trajectories may be over-reinforced, while useful actions in failed trajectories may be undesirably penalized. To address this issue, we propose Dual Experience Pool Policy Optimization (DEPPO), a critic-free and experience-driven framework for adaptive step-level credit assignment. DEPPO maintains two structured experience pools for successful and failed trajectories, and estimates the relative quality of each action by comparing its state-conditional likelihood under the two pools. The resulting success–failure preference signal is used to adaptively rescale the advantage, amplifying actions aligned with successful patterns while suppressing those associated with failure modes. Furthermore, DEPPO introduces an asymmetric and dynamically evolving update mechanism: the success pool emphasizes both correctness and novelty through freshness-aware insertion, while the failure pool focuses on consistently risky behaviors, with decay and pruning to keep the memory compact and up-to-date. Without additional value models or process-level supervision, DEPPO provides a lightweight, interpretable, and effective solution for fine-grained credit assignment in long-horizon reinforcement learning. Code available at: https://anonymous.4open.science/r/DEPPO-1051/

Abstract:
This study employs LLM-based multi-agent simulation to investigate how fairness-related behavior evolves in repeated resource allocation. Thirty agents with five idealized value orientations participate in a survival-production cycle under equal, needs-based, contribution-based, and negotiation-based allocation mechanisms. Results show that negotiation achieves the highest productive return and subjective satisfaction with the strongest consensus in our main setting, and its subjective advantage holds across eight LLMs, suggesting that deliberative allocation can help navigate the fairness–efficiency tradeoff. Agents also exhibit systematic misalignment between subjective satisfaction and objective resource outcomes, shaped by value orientations and interaction conditions. Over repeated interaction, agents’ reflection-derived fairness positions shift away from their initial orientations toward more hybrid combinations of equality-, merit-, and needs-related reasoning. To provide human-grounded structural comparison, we further benchmark GPT-4o against published human choice distributions in 16 controlled allocation scenarios, finding moderate rank-level correspondence but substantially more concentrated LLM choices. These findings characterize value-conditioned reasoning and semantic change in LLM societies while remaining computational hypotheses rather than direct evidence of human fairness behavior.

Abstract:
To address the limitations of conventional ICL, recent task-vector-based methods, such as Implicit ICL (I 2 CL), encode demonstrations into a unified task vector that is injected during inference, yielding promising improvements in efficiency and stability. However, existing methods encode each demonstration in isolation, neglecting the rich interactions between demonstrations and between demonstrations and queries—a core capability of standard ICL. This results in task vectors with limited expressiveness. To bridge this gap, we propose Interactive Implicit In-Context Learning (I 3 CL), a simple yet effective framework that restores these essential interactive capabilities. I 3 CL introduces only two lightweight attention-based modules, preserving the efficiency advantages of I 2 CL while delivering significant performance improvements. Experiments across nine diverse tasks using four LLMs show that I 3 CL achieves an average performance improvement of 3.54% over prior task-vector-based baselines. Comprehensive analyses on efficiency, scalability and task vector properties further confirm the superiority of our approach.

Abstract:
Weakly-supervised temporal action localization (WS-TAL) remains challenging due to the severe temporal ambiguity inherent in video-level supervision. Existing methods often overfit to highly discriminative snippets, struggling to capture the complete action extent. In this paper, we reconceptualize actions not as isolated events, but as multi-stage action evolutions. To alleviate temporal ambiguity, we propose a novel framework driven by Linguistic Temporal Knowledge Transfer (LTKT) Network, comprising three cohesive modules. First, Temporal Stage Grounding (TSG) leverages explicitly deconstructed LLM descriptions as cross-modal priors, guiding unconstrained visual snippets to anchor their specific evolutionary stages. Second, Temporal Transition Gating (TTG) enforces forward causality on the timeline. By mandating that an execution stage is flanked by historical preparation and future completion, TTG strictly filters static noise to yield refined temporal action scores. Finally, Linguistic Knowledge Transfer (LKT) integrates knowledge distillation and contrastive learning to embed this structural knowledge into the visual backbone. Extensive experiments on THUMOS14, ActivityNet1.2 and ActivityNet1.3 demonstrate that our method achieves state-of-the-art performance.

Abstract:
Recent advances have significantly improved the performance of Abstract Meaning Representation (AMR) parsers, yet determining when their outputs can be trusted remains largely unexplored. This work presents the first study of uncertainty estimation for AMR parsers in a black-box setting using top- k beam-search candidates. Uncertainty is estimated from pairwise candidate similarities and evaluated using the Prediction Rejection Ratio. This work characterises the relationship between parsing performance and uncertainty, evaluates the reliability of beam candidate distributions, analyses the effect of evaluation metrics on uncertainty assessment, and identifies parsers that provide the strongest balance between performance and reliability. Experimental results show the value of uncertainty estimation as a complementary dimension for evaluating AMR parsers.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) can improve reasoning in large language models, but specialized non-English domains expose a data mismatch. Raw English domain text is often available, while target-language RLVR tasks with verifiable rewards are scarce. We introduce code-switched task synthesis for cross-lingual RLVR , a data-centric approach that derives target-language and code-switched RLVR task variants from otherwise unverifiable English domain text. Using fill-in-the-middle multiple-choice QA (FIM-MCQA) as a seed task format, we synthesize automatically verifiable tasks from domain passages and reshape them into cross-lingual variants. These variants preserve the same evidence-answer relation but change how English and target-language evidence are presented during RLVR. On Korean medical benchmarks and cybersecurity benchmarks across four target languages, code-switched training outperforms matched monolingual and bilingual-mixture baselines on average target-language performance while maintaining comparable English performance. Additional analyses provide evidence of better target-language input understanding and stronger cross-lingual alignment in both model outputs and internal representations. These results suggest that code-switched training helps target-language inputs access domain reasoning learned more strongly in English.

Abstract:
Lawyer-client consultation is a critical starting point for legal services. Effective legal assistance hinges on eliciting sufficient and truthful information from clients in order to devise strategies that best protect their interests. This task requires Large Language Models (LLMs) not only to perform robust legal reasoning, but also to strategically elicit material facts through multi-turn interactions and effectively guide clients with diverse personalities. Yet existing legal benchmarks overlook this interactive capability. To fill this gap, we introduce DLawBench, a diagnostic benchmark for real-world legal consultation. Drawing on realistic client behavior, we characterize lawyer-client interactions into four types: Cooperative, Dependent, Withdrawn, and Adversarial. Using dialogues grounded in real cases, DLawBench evaluates whether LLMs can effectively conduct legal consultation under realistic conditions. DLawBench comprises 461 cases from Chinese and U.S. law, 5,532 paired fact entries, 3,411 inquiry rubrics, and 3,348 issue-resolution rubrics, and evaluates 26 representative LLMs. Systematic experiments show substantial headroom: the best-performing model, GPT-5.5, achieves only 0.562 on consultation-grounded legal reasoning. More importantly, DLawBench exposes both sycophancy in legal consultation and a paradox: models perform worse when clients need guidance most.

Abstract:
Recent studies of language evolution with neural agents often use models where agents can either speak or listen, where there are only two agents in the population, or where agents can share internal representations. These choices prevent the study of how language emerges through cultural evolution, i.e. through the cumulative adaptation of individual behaviours to fit the communicative needs of the community. We tackle this issue with a new model that trains populations of bi-directional neural agents in a fully distributed fashion to solve a referential game. We demonstrate the effectiveness of this approach on multiple image datasets and show that it is able to scale to large populations. We also show that more conventional languages are easier to learn, providing a direct advantage for cultural transmission. Overall, this work provides a novel approach to neural emergent communication that opens up new directions for evolutionary linguistics.

Abstract:
Accurately assessing jailbreak success remains unsolved, as existing automatic methods often diverge from human perception. To address this, we propose JADES, a universal evaluation framework that decomposes a harmful query into weighted sub-questions, scores each sub-answer, and aggregates the scores into a final decision; an optional fact-checking module further mitigates hallucination-induced misjudgments. We validate JADES on JailbreakQR , a meticulously annotated benchmark of 400 prompt–response pairs. Under binary settings (success/failure), JADES achieves 98.5% human agreement, outperforming strong baselines by over 9%. Re-evaluating five popular attacks across seven LLMs reveals systematic overestimation of jailbreak. JADES also enables granular ternary evaluation (failure/partial success/success), uncovers that partial successes dominate current reported successes, and recalibrates the perceived severity of existing attacks. Overall, JADES reveals a major limitation of current jailbreak evaluation and, together with ternary assessment, enables a more accurate understanding of the true jailbreak effectiveness.

Abstract:
A situated query like "where is Lin Wei?" often encodes more than its literal content: the user may also want to know whether Lin Wei is free, in a good mood, or worth interrupting now. Standard tool-use agents answer the literal question and stop. AURA inserts an inference step between scene perception and tool use that produces an IntentFrame: a structured estimate of the implicit need with a scalar gap score that controls per-query probe budget and tool selection. On a 100-query four-scene implicit-intent benchmark, AURA improves implicit-need coverage over ReAct-style probing (Delta = +0.07, p < 10^-6); three of four scenes are individually significant, the gain reproduces on a second backbone, and a prompt ablation attributes the lift to gap calibration rather than answer memorisation. A component ablation localises the gain further: the gap-derived budget ceiling alone outperforms the full architecture (p = 0.02), and bin-wise calibration analysis (r = -0.33) identifies the gap score as a probe trigger rather than a calibrated value estimator. On factual lookup the controller trades raw accuracy for 82% fewer probes and zero forbidden-tool violations on a privacy-sensitive slice; scope conditions are detailed in Limitations. Code, simulator, and benchmark are released at https://github.com/innovation64/AURA, with the implicit-intent benchmark also on the HuggingFace Hub (innovation64/aura-implicit-intent).

Abstract:
Morals and culture are a crucial driver of our behavior and social perception: unsurprisingly, social psychology provides established tools to investigate them in individuals. NLP, on the other hand, has focussed on the interpretation side, with annotations (and models) that, at best, reconstruct the inference an annotator (the reader, in this paper) made of someone else’s (the writer) text, knowing little (or nothing) about the reader, nothing about the writer, and focussing on WEIRD countries. This is a dire resource gap, that we fill with the MCWR dataset, covering Mexican Spanish, Chinese, and Arabic, alongside English (US). An absolute novelty with respect to previous work, we adopt a two-step collection process, covering both generation and annotation, and collect value profiles for writers and readers. This enables us to investigate the impact of value mismatches in the annotation output: we find that readers’ perceptions of writers’ values are strongly shaped by their own values, causing misalignments which in turn raise concerns about generalizations made by models based on reader/writer agnostic data.

Abstract:
Large language models have demonstrated strong performance across a wide range of tasks, yet they remain prone to generating incorrect information with high confidence. To mitigate this reliability issue, it is necessary to properly assess model confidence before answer generation. However, existing methods either rely on post-hoc estimation based on generated tokens or sampling results, or require additional training based on supervised learning, which limits their computational efficiency and generalizability. In this paper, we propose PR eemptive I nternal C onfidence E stimator ( PRICE ), which estimates confidence before answer generation by leveraging internal signals of LLMs that can be extracted from a single forward pass. PRICE combines signals that capture pattern agreement across attention heads and the semantic clarity of hidden states. Through experiments on diverse models and benchmarks, we show that PRICE achieves competitive performance against classifier-based methods without requiring additional training. These results suggest that PRICE can contribute to improving both the efficiency of LLMs and their safety in high-risk domains.

Abstract:
Retrieval-augmented generation (RAG) can achieve strong answer accuracy on multi-hop questions, but outcome-level rewards often leave reasoning traces weakly grounded and difficult to audit. Under noisy retrieval, models may exhibit right-answer-wrong-reason failures, where the final answer is correct but the supporting rationale exploits shortcuts or unsupported evidence. We therefore ask whether faithfulness-guided alignment hurts answer accuracy in post-retrieval reasoning. To study this question, we propose CRAFT (Calibrated Reasoning with Answer-Faithful Traces), a reinforcement learning framework for the response-generation stage of retrieval-augmented multi-hop question answering. CRAFT trains models to produce structured reasoning traces with configurable auditability, while combining deterministic rewards for format compliance, answer correctness, and citation validity with a judge-based reward for semantic faithfulness. Experiments across model scales and benchmarks show that CRAFT unlocks task-specific reasoning capacity from 1.5B upward, improving both answer accuracy and Faithfulness; at 0.5B, performance remains sharply template-dependent. At 7B, CRAFT improves Faithfulness over the Base model in all evaluated settings and remains competitive with strong closed-source models. Code is available at https://github.com/Ameame1/CRAFT.

Abstract:
Large Language Models (LLMs) have demonstrated superior performance in listwise passage reranking task. However, directly applying them to rank long-form documents introduces both effectiveness and efficiency issues due to the substantially increased context length. To address this challenge, we propose a per-document summarization model SumRank, aligned with downstream listwise reranking, to compress long-form documents into concise rank-aligned summaries before the final listwise reranking stage. To develop our summarization model SumRank, we introduce a three-stage training pipeline consisting of cold-start Supervised Fine-Tuning (SFT), specialized Reinforcement Learning (RL) data construction, and rank-driven alignment via RL. This paradigm aligns the SumRank with downstream ranking objectives to preserve crucial relevance signals. We conduct extensive experiments on both in-domain (ID) benchmark datasets from the TREC Deep Learning tracks and out-of-domain (OOD) datasets from the BEIR benchmark. Results show that our lightweight SumRank model achieves state-of-the-art (SOTA) ranking performance while significantly improving efficiency by reducing both summarization overhead and reranking complexity.

Abstract:
In scientific research, analysis requires accurately interpreting complex multimodal knowledge, integrating evidence from different sources, and drawing inferences grounded in domain-specific knowledge. However, current artificial intelligence (AI) systems struggle to consistently demonstrate such capabilities. The complexity and variability of scientific tables and figures, combined with heterogeneous structures and long-context requirements, pose fundamental obstacles to scientific table figure analysis. To quantify these challenges, we introduce AnaBench, a large-scale benchmark featuring 63,178 instances from nine scientific domains, systematically categorized along seven complexity dimensions. To tackle these challenges, we propose Anagent, a multi-agent framework for enhanced scientific table figure analysis through four specialized agents: decomposes tasks into actionable subtasks, retrieves task-specific information through targeted tool execution, synthesizes information to generate coherent analysis, and performs iterative refinement through five-dimensional quality assessment. We further develop modular training strategies that leverage supervised finetuning and specialized reinforcement learning to optimize individual capabilities while maintaining effective collaboration. Comprehensive evaluation across 170 subdomains demonstrates that Anagent achieves substantial improvements, up to ↑13.43 in training-free settings and ↑42.12 with finetuning, while revealing that task-oriented reasoning and context-aware problem-solving are essential for high-quality scientific table figure analysis.

Abstract:
Industrial standard documents combine hierarchical section structures, table-based rules, and complex conditional statements; as a result, the same expression can refer to different rules depending on where it appears in a document. However, existing text-based approaches and conventional knowledge graph (KG) construction methods do not jointly preserve these structures, making them brittle on complex rule-based queries. In this work, we propose an ontology-based KG construction method that formalizes section structure as an ontology and integrates it with rule-aware triplets extracted from text and tables. To evaluate our approach, we construct IndusSpec-QA, a QA dataset based on three industrial standard documents, and compare the proposed method against a range of baselines. Experimental results show that the proposed Ontology-based KG-RAG outperforms existing methods, with particularly large gains on table-based rule questions and multi-condition reasoning. These results demonstrate the importance of knowledge representations that jointly preserve section and rule structures for QA over industrial standard documents. The code is publicly available at: https://github.com/Ontology-KG/ontology_based_kg_paper

Abstract:
Discovering stable neuron behavior across entire domains remains a challenge in mechanistic interpretability. Existing methods often rely on instance-level point estimates or computationally expensive procedures, which either obscure population-level variability or limit scalable domain-wide analysis. We present RACE (Residual Alignment for Consistency Estimation), a forward-pass statistical framework that evaluates the domain-wide functional consistency of Transformer neurons. Compared with gradient-based point estimates, RACE produces neuron rankings that yield more domain-specific effects under perturbation. Token-distribution shifts support the connection between the selected neurons and the target domain, while scoring requires roughly one-hundredth of the computational overhead of the gradient-based methods. Code is available at https://github.com/Nexround/RACE.

Abstract:
Enabling artificial intelligence systems, particularly large language models, to update knowledge and flexibly apply it during reasoning remains a central challenge. Existing knowledge editing approaches emphasize atomic facts, improving factual recall but often failing to integrate updated information into a coherent framework usable across contexts. In this work, we argue that knowledge update is fundamentally a reasoning problem rather than a memorization problem. Consequently, a model should be trained in situations where the new information is instrumental to solving a task, combined with pre-existing knowledge, and exercised through multi-step reasoning. Based on this insight, we propose a training strategy based on three principles. First, new knowledge is introduced as a coherent background story that contextualizes novel facts and explains their relation to existing knowledge. Second, models are trained using self-generated multi-hop questions that require multi-step reasoning involving the new information. Third, training is done using knowledge distillation, forcing a student model to internalize the teacher’s reasoning behavior without access to the novel information. Experiments show that models trained with this strategy effectively leverage newly acquired knowledge during reasoning and achieve remarkable performance on challenging questions that require combining multiple new facts.

Abstract:
LLMs perform strongly across NLP, but their ability to produce explicit grammatical analyses remains unclear. Arabic provides a challenging testbed due to its rich morphology and orthographic ambiguity, which create strong morphology–syntax interactions. We present a unified evaluation of LLMs on Arabic morphosyntactic tagging and dependency parsing, covering pre-tokenized, raw-text, and cascaded settings. We compare zero-shot prompting with retrieval-based in-context learning. Relevant demonstrations substantially improve performance. The strongest LLMs approach supervised tagging and parsing systems; however, they require substantial annotated data for demonstration retrieval and considerable computational resources. We make all code and data used in this paper publicly available.

Abstract:
Recovering an intended word from a noisy spelling is a general word-retrieval problem relevant to language learning, spelling correction, and cross-script search. We study this problem in Arabic, where learners produce variable transcriptions in Arabic or Roman script that may preserve only part of the target form. Using ArabScribe, we evaluate four large language models (LLMs) on ranked intended-word recovery under three cue conditions: the noisy form, an English meaning gloss, and both. Combining form and meaning consistently outperforms either cue alone across models and scripts, with particularly strong benefits for distorted and Roman-script input. An expert-human evaluation shows the same benefit from glosses and places the strongest models near individual expert performance on the sampled items. Analyses of edit distance, learner proficiency, and target frequency show that semantic information improves recovery across difficulty conditions but does not eliminate sensitivity to form noise and lexical rarity. These findings establish gloss-grounded intended-word recovery as a useful framework for studying how LLMs integrate form and meaning under noisy, cross-script input.

Abstract:
Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants ( N = 123 ) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.

Abstract:
Image retrieval has traditionally been formulated as a point-wise matching problem, where each candidate image is scored in isolation. However, this atomic paradigm fails to capture the complexity of human search intent within personal photo collections, where users often seek compact visual stories bound by structural relations rather than isolated snapshots. To address this limitation, we introduce Image Bundle Composition (IBC), a novel paradigm that shifts the objective from ranking individual images to dynamically composing cohesive image bundles from a massive, unstructured photo pool. Since target bundles are not predefined, IBC presents a severe combinatorial explosion challenge and demands modeling non-decomposable joint relevance. To establish this paradigm, we construct IBCBench, the first IBC benchmark dataset containing 109,467 images and 667 verified queries, built via a semi-automated verification pipeline. Furthermore, we propose BundleWeaver, an agentic framework that reformulates IBC as query-conditioned incremental hyperedge discovery. By employing a Large Language Model to adaptively search for missing relational roles and utilizing a Vision-Language Model for whole-bundle verification, BundleWeaver effectively navigates the combinatorial space. Extensive experiments demonstrate that while state-of-the-art embedding models and static decompose-and-rerank paradigms suffer from relational blindness, BundleWeaver achieves substantial performance gains, highlighting the necessity of shifting from atomic scoring to dynamic relational composition. Our dataset and code are available.

Abstract:
Memory is essential for LLM agents to accumulate task experience and reuse task-specific execution strategies. However, real-world deployment over boundary-agnostic and evolving task streams exposes a fundamental stability-plasticity dilemma. External retrieval-based memory can rapidly absorb new evidence, but it often fails to internalize recurring execution patterns and incurs inference-time retrieval overhead. Parametric memory enables stable and efficient execution once learned, but typically relies on explicit task boundaries and fixed parameter budgets. Inspired by the human brain, which balances plasticity and stability through complementary episodic storage and gradual consolidation, we propose UniMem, a self-routing framework for autonomous memory management. UniMem uses learnable routing tokens as memory controllers, enabling adaptive coordination between complementary memory pathways: novel or sparse tasks are retained in an episodic buffer for retrieval-augmented execution, while recurring and reliable patterns are consolidated into expandable parametric memory. By decoupling task identification from task execution with routing tokens and parametric memory blocks, UniMem expands memory on demand without task labels during deployment or uncontrolled parameter growth. Experiments on long-horizon streaming task sequences show that UniMem typically achieves strong aggregate results while maintaining routing and execution stability, with an average gain of 4.0 EM over Replay LoRA.

Abstract:
SQL query rewriting reformulates queries into more efficient forms while preserving equivalence. Most existing methods rely on predefined rules that struggle to generalize beyond fixed query patterns and cannot express complex rewriting strategies. Large language models (LLMs) can reason over complex structures and generalize to diverse patterns, but their limited execution awareness and semantic grounding often result in non-equivalent or inefficient rewrites. To this end, we propose E3-Rewrite, an LLM-based framework that generates executable, equivalent, and efficient SQL rewrites. It features two key designs: (1) an execution-aware context construction that integrates query plans and retrieved examples to expose structural bottlenecks; and (2) a reinforcement learning optimization with executability, equivalence, and efficiency rewards, trained under a correctness-to-efficiency curriculum for stable convergence. Experiments on multiple benchmarks show that E3-Rewrite-Rewrite reduces execution time by up to 25.6% over leading baselines and delivers 7.3% more equivalent rewrites, extending SQL rewriting to query patterns previously beyond rule-based methods.

Abstract:
Legal Judgment Prediction (LJP) models are typically trained on documents that describe facts from a prosecutorial perspective. Existing datasets further exhibit severe label imbalance toward guilty outcomes. Consequently, these models suffer from "Guilty Bias", blindly accepting the prosecution’s narrative as objective truth. Previous studies employing three-step reasoning structures or training on synthetically generated innocence data improve overall accuracy, but they still fail to mitigate bias at inference time. In this paper, we introduce OBJECTION, an inference-time pipeline that integrates an Adversarial Lawyer Agent into each 3-step reasoning of offense, unlawfulness, and culpability. Unlike generic critics, our agent actively challenges the model’s presumptions of guilt by injecting legal defense arguments at each reasoning stage. To thoroughly evaluate this, we present a new "Natural Innocent" dataset including 3.4k real-world cases, overcoming the limitations of synthetic innocence benchmarks. Test results show that OBJECTION drastically reduces the False Guilty Rate (FGR) from 82.93% (SOTA baseline) to 16.69%, proving its capability to perform substantive legal reasoning. This work denotes a key progress toward aligning Legal AI with the presumption of innocence.

Abstract:
Taxonomy induction aims to organize concept sets into coherent hierarchical structures. Recent LLM-based methods can induce taxonomies directly from flat term lists, avoiding the need for corpora, but degrade sharply as concept sets scale up. We argue that this degradation stems not only from context length limitations, but also from structural failures in hierarchical reasoning. To address this, we adopt a divide-and-merge paradigm that partitions concepts into smaller subsets, induces local taxonomies, and merges them into a global hierarchy. However, we identify two structural failure modes inherent to this paradigm: Structural Fragmentation, where partitioning weakens local hierarchical signals, and Parent Displacement, where locally plausible relations are misplaced in the global hierarchy. To address both, we propose SPARROW, a scalable taxonomy induction framework that combines structure-preserving spectral partitioning to retain hierarchical connectivity within each block, and constraint-guided incremental fusion that treats block-level relations as structural constraints rather than ground truth for global placement. Experiments on large-scale benchmarks show that SPARROW consistently achieves the strongest global structural quality, as measured by Ancestor F1, across backbones. The code is available at https://github.com/rebeccazyr/SPARROW.

Abstract:
Block diffusion speculative decoding accelerates LLM inference by predicting all tokens within a block simultaneously for the target model to verify in parallel. Predicting an entire block at once requires a sufficiently capable draft model and effective utilization of the target model’s internal knowledge. However, the state-of-the-art method DFlash constrains all draft layers to share a single fused representation derived from only a few target layers, limiting per-layer expressiveness and hindering further scaling of draft capacity. In this paper, we present DFLare, which flares out the narrow conditioning bottleneck of DFlash through a lightweight layer-wise fusion mechanism: each draft layer attends to its own learnable combination of a broad set of target layers at negligible overhead, simultaneously injecting richer target knowledge and providing every draft layer with a distinct input. This enhanced per-layer expressiveness enables scaling the draft model to deeper architectures with consistent gains. We further scale training data from 800K to 2.4M samples to fully exploit the enlarged capacity. On six benchmarks spanning mathematical reasoning, code generation, and conversation, DFLare attains average wall-clock speedups of 5.52x on Qwen3-4B, 5.46x on Qwen3-8B, and 3.91x on GPT-OSS-20B, improving over DFlash by roughly 11%, 8%, and 5% respectively.

Abstract:
Although large language models (LLMs) possess powerful general capabilities, their insufficient domain expertise limits their application in high-reliability scenarios. The sparse activation patterns of neurons in LLM feed-forward layers suggest that structured fine-tuning strategies for domain adaptation can be constructed by identifying important parameters, thereby enhancing domain-specific performance while preserving general capabilities. However, due to high computational costs or weak scalability, current studies struggle to support large-scale and fine-grained domain-relevant parameter analysis. To address this, we investigate the distribution patterns and hierarchical evolution patterns of domain knowledge within the hidden state space of LLMs, and propose a novel parameter evaluation method: Subspace Alignment Score. Intervention experiments across three models and six domains illustrate that, compared with baseline methods, our method demonstrates superior performance in locating domain-relevant important parameters. Furthermore, structured fine-tuning strategies based on the important parameters achieve performance comparable to full fine-tuning while substantially reducing training costs, which is helpful for understanding efficient and interpretable domain adaptation. Our code is available at https://github.com/lewu0723/subspace-alignment-score.

Abstract:
AI-based Visually Impaired Assistance (VIA) remains challenging, largely due to the high cost of human evaluation. The VLM-as-a-Judge paradigm may offer a promising alternative, although it has mostly been studied in general domains. We therefore ask whether such judges can be trusted for VIA tasks. To investigate this question, we introduce VIABLE (Visually Impaired Assistance Benchmark for VLM-as-a-Judge Evaluation), the first benchmark for VLM-as-a-Judge evaluation in VIA. VIABLE contains over 300K judgment samples across three scenarios and introduces an Effectiveness–Impartiality–Stability framework with a 12-mode failure taxonomy. Based on VIABLE, our systematic study of seven judges across different model scales shows that existing models are largely unreliable across all evaluation axes. The strongest judge, GPT-5.4, achieves only 52.6% single-failure diagnostic accuracy, yet exhibits the highest self-preference rate at 94.2%; while open-source judges are strongly biased and adversarially fragile. To address these issues, we propose VIA-Judge-Agent, a model-agnostic inference-time harness that augments judges with visual evidence extraction and a taxonomy-guided workflow. It enables positive improvements in diagnostic accuracy and downstream VIA responses more preferred by BLV users. Data and code are available at:https://github.com/YiyiyiZhao/VIABLE.

Abstract:
Large Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records (EHRs) navigation remains constrained by a reliance on curated inputs and simplified retrieval tasks. To bridge the gap between idealized experimental settings and realistic clinical environments, we present AgentEHR. This benchmark challenges agents to execute complex decision-making tasks, such as diagnosis and treatment planning, requiring long-range interactive reasoning directly within raw and high-noise databases. In tackling these tasks, we identify that existing summarization methods inevitably suffer from critical information loss and fractured reasoning continuity. To address this, we propose RetroSum, a novel framework that unifies a retrospective summarization mechanism with an evolving experience strategy. By dynamically re-evaluating interaction history, the retrospective mechanism prevents long-context information loss and ensures unbroken logical coherence. Additionally, the evolving strategy bridges the domain gap by retrieving accumulated experience from a memory bank. Extensive empirical evaluations demonstrate that RetroSum achieves performance gains of up to 29.16% over competitive baselines, while significantly decreasing total interaction errors by up to 92.3%.

Abstract:
Deep Research Agents have shown strong capability in multi-step information retrieval, reasoning, and long-form report generation, but existing benchmarks and systems remain predominantly text-centric, with limited evaluation of whether visual elements are factually reliable and well aligned with the surrounding analysis. To address this gap, we introduce TVIR (Text–Visual Interleaved Report Generation), which includes TVIR-Bench, a benchmark of 100 expert-curated multimodal deep research tasks that require visual elements to serve specific analytical sub-goals, and TVIR-Agent, a hierarchical multi-agent framework for constructing structured outlines, retrieving images, generating charts with traceable sources, and composing reports through context-aware sequential writing. We further develop a dual-path evaluation framework that combines Textual Assessment and Visual Assessment. Experiments across nine deep research systems show that TVIR-Agent achieves strong overall performance, underscoring the importance of explicit multimodal design and evaluation for evidence-driven report generation.

Abstract:
Parameter-efficient fine-tuning is usually framed as a question of how many parameters to update. Under a severe trainable-state budget, however, where those coefficients act is equally consequential. We study this choice through frozen-core adaptation: a calibration pass fixes left and right bases for each weight matrix, and fine-tuning optimizes only an r× r core. This removes the ability of trainable factors to repair a poor initial span and makes subspace quality directly observable. We introduce FCCA, which estimates the signed input–error cross-covariance, whitens it with diagonal Fisher moments, truncates it in the resulting local metric, maps the selected directions back, and applies thin QR to obtain stable core coordinates. Under a matched r 2 budget, we compare eight basis constructors on 11 tasks, four model settings, and three seeds. On Qwen2.5-3B, FCCA reaches an 83.0 macro-average, 2.3 points above the next-best matched-budget constructor, and exceeds its unwhitened RawGrad control on all 11 tasks. It ranks first at all three Qwen scales and finishes within 0.13 points of the best method on Llama-3.2-1B. Controlled ablations show gains of 2.7–17.2 points from whitening and identify QR as necessary for stable core optimization in the tested regime. Finally, FCCA comes within 0.32 and 0.23 average points of LoRA and DoRA while optimizing 36.9K rather than roughly 7.4M parameters. These results show that a carefully selected fixed span can recover most of the benefit of movable low-rank factors at a much smaller trainable and optimizer-state cost.

Abstract:
Large language models (LLMs) are increasingly used as primary knowledge sources, yet their epistemic diversity — defined as the diversity of real-world claims in their outputs — has never been measured. The dominant paradigm is that overall LLM diversity is low, but this is always with respect to a single point in time, with no reference baseline or consideration for variation across countries. We address this by performing the first systematic study of epistemic diversity in LLMs across time and cultural context, testing 27 LLMs on 155 topics covering 12 countries, resulting in 1.7M responses and 70M individual claims. We find that epistemic diversity has increased substantially over the past three years, a positive counter to recent diversity pessimism. However, despite progress, we find that every LLM is less diverse than a search baseline. The gap is not uniform: RAG can improve diversity, while larger models are counterintuitively less diverse than smaller ones. Moreover, LLMs systematically reflect English-language knowledge over local-language knowledge for country specific topics. Together, these results demonstrate that while progress on epistemic diversity is tangible, it is insufficient and unevenly distributed.

Abstract:
Large language models (LLMs) remain vulnerable to jailbreak attacks that conceal harmful intent within complex adversarial prompts. Existing defenses primarily rely on input perturbation or harmful-output suppression, but they rarely model where malicious intent resides, resulting in brittle protection and excessive over-refusal. We propose SENTINEL, a plug-and-play, generation-time jailbreak defense that reframes mitigation as an intent extraction problem. Our key insight is that instruction-tuned LLMs exhibit strong input–output semantic consistency: regardless of jailbreak complexity, generated outputs tend to align with the attacker’s true intent. SENTINEL exploits this property by matching semantically aligned input–output regions to extract intention-revealing subsequences, scores these subsequences using refusal-direction projections to estimate harmfulness, and halts generation when necessary. Experiments on HarmBench across multiple LLMs show that SENTINEL reduces jailbreak success rates to close to 5% while maintaining low over-refusal. We further demonstrate robustness to adaptive attacks and provide a mechanistic interpretation: SENTINEL re-distributes jailbreak features from alignment blind spots to aligned regions.

Abstract:
Conversations with chat assistants increasingly span many topics in a single long-running thread, challenging memory systems. Existing long-context and memory benchmarks often expose session or topic boundaries, or probe direct personal-memory questions. These settings understate a harder assistant-memory regime: a flat mixed-topic thread where the system must infer which earlier episode makes a later task decision valid. We introduce SCALE-QA, a constraint-grounded task QA benchmark for flat unsegmented threads targeting episode integrity failure. The dataset contains 3,000 audited questions across 10 domains, uses deterministic four-way multiple-choice grading, and includes a deterministic runtime builder; experiments use all 3,000 questions through 128k and a stratified 400-question diagnostic at 1M. SCALE-QA questions are ordinary task-oriented requests whose correct answer depends on causally related evidence introduced earlier in the conversation. We also propose Temporal-Semantic Interleaved Memory Reconstruction (TSIM), which segments the turn stream into coherent episodes and indexes them through a hierarchical multi-view memory stack with deterministic episode-level summary and cluster-routing views. Experiments show that SCALE-QA challenges strong RAG baselines and long-context LLMs alike; across three open-source and proprietary LLM backends, TSIM achieves the highest accuracy in every backend setting, gaining 5.6–17.6 accuracy points over the strongest corresponding baseline.

Abstract:
Retrieval-based speculative decoding accelerates large language model inference without training an additional draft model, but its effectiveness is often limited by retrieval ambiguity. Existing future-guided methods typically rely on a fixed future-token position, such as a proxy for the next-next token, as the retrieval anchor. However, this fixed-anchor strategy may be suboptimal across different contexts. We propose InfoSpec, a training-free framework that formulates retrieval-anchor selection as an optimization problem performed at inference time. At each decoding step, InfoSpec constructs a lightweight candidate anchor pool from the target model, estimates the retrieval utility of each candidate via hash-index probing, and selects anchors based on model confidence, ambiguity reduction, locality, and continuation consistency under a fixed draft-tree budget. Experiments on generation, reasoning, code, and long-context benchmarks show that InfoSpec improves mean accepted tokens and end-to-end speedup over strong retrieval-based baselines. Further analyses show that InfoSpec reduces high-ambiguity matches and uses the verification budget more effectively while preserving plug-and-play deployment.

Abstract:
Multi-agent LLM systems often fail not for lack of candidate answers, but because they have no persistent mechanism for deciding which incompatible claim should currently be trusted. Majority vote, debate, and judge-based selection choose an output without recording which claim wins, which is contested, or why a later update supersedes it. We present LatticeMind , a conflict-aware structured memory that handles contradiction at write time. It maintains explicit item status, applies cheap symbolic conflict checks, and routes the flagged cases to an LLM reconciler. On a label-blind ConflictBank evaluation that removes source-name hints, over a uniform draw of 200 examples from the full benchmark pool, LatticeMind reaches 0.965 accuracy versus 0.525 for the strongest baseline, with every gap significant at p<10 -24 by paired McNemar test. Ablations show that removing the checker or the reconciler costs 12 to 14 points. Our claims are scoped to credibility conflicts, where one claim should defeat another: the coordination branch of the design is a safety override that we do not validate empirically, and all prompt-based comparisons are conditioned on a single backbone. On three secondary planning benchmarks and one synthetic scheduling diagnostic the picture is mixed: LatticeMind beats naive merge on three of four, but does not replace deliberation methods on tasks rewarding iterative search.

Abstract:
In test-time reasoning, pairing a weak local generator with a strong but expensive teacher makes deployment cost hinge on when additional teacher computation is worthwhile. We propose Cascade-SC, a containment-gated cascade that accepts the teacher’s independently solved answer when it matches the generator’s candidate set and draws additional teacher samples otherwise. The cross-model gate fires roughly 5× less often than within-model unanimity, concentrating teacher compute on harder cases. Across GSM8K, MATH500, and AIME-2024, Cascade-SC improves over self-consistency by 11–37%. On GSM8K, it matches ModelSwitch within 0.46% at roughly one-third the teacher-inference cost and beats matched-compute Teacher-only SC by 1.71%. On low-recall AIME-2024, it reaches 40.00% versus the candidate-restricted Oracle@8 ceiling of 11.11%. It also outperforms trained 7B process-reward rerankers by 4–11% without verifier training. Self-consistency and Teacher-greedy arise as special cases, while a teacher-only versus union-vote ablation shows that aggregation is a separate design choice. Source codes are available at https://github.com/SUGAR-NLP/Cascade-SC.

Abstract:
Verifiers play a central role in large language model reasoning by providing reliable supervision for post-training methods such as reinforcement learning. In code generation, however, verification is still predominantly based on fixed test cases, offering a narrow but underexploited form of verifiable supervision. We explore how existing test cases can be reformulated into more diverse and challenging sources of verifiable training signals. We propose Code-TVT (Test-Case-Driven Verifiable Tasks), a data synthesis framework that transforms program test cases into intrinsically verifiable code reasoning tasks. Using test cases already available in programming problems, we design two complementary tasks: Edge Test-Case Generation, which constructs edge-case inputs that expose subtle correctness failures, and Input Reconstruction via Program Output Inversion, which reasons backward from outputs to plausible inputs under execution-based verification. Experiments show that training with Code-TVT consistently improves code generation performance on three time-windowed LiveCodeBench splits (LCB-2408-2411, LCB-V5, and LCB-V6) under rejection-sampling fine-tuning and Group Relative Policy Optimization.

Abstract:
Conversational shopping assistants now serve hundreds of millions of customers, yet no existing benchmark jointly evaluates the open-ended multi-turn reasoning, domain expertise, and criterion-level quality that real shopping conversations demand. Shopping reasoning is unique among language model applications. Unlike factual question answering or verifiable code generation, it requires balancing subjective preferences, budget constraints, and cross-product trade-offs across multi-turn dialogue, capabilities absent from previous e-commerce and general-purpose benchmarks. We introduce the Shopping Reasoning Bench, an expert-authored benchmark of 525 missions (232 single-turn, 293 multi-turn) with 10863 importance-weighted binary rubrics authored by retail domain experts. These criteria are organized under a taxonomy of five reasoning categories and fifteen subcategories covering diverse demands such as preference refinement, trade-off analysis, and compatibility assessment. An evaluation of nine models across three families (GPT, Claude, Gemini) shows that pass rates reach only 57–77% overall. On multi-turn missions, all models score 13–29 points lower on optional above-and-beyond criteria than on required ones, and performance degrades 4–18 points as conversations progress. These gaps show that current models handle basic shopping assistance but fall short of expert-level advice, making Shopping Reasoning Bench a challenging testbed for future shopping assistant development.

Abstract:
Text embedding models are central to information retrieval and deep research tasks, and their behaviors are strongly shaped by the training objective. While recent systems have largely converged on InfoNCE-style loss functions, there have been continued attempts to design additional loss functions beyond this core objective. One such component is spread-out loss, a batch-level penalty on high pairwise similarity that encourages embeddings to occupy the representation space more evenly. However, in existing systems, this design is usually combined with many other design choices and has not been sufficiently ablated. To isolate its effect, we use spread-out loss as a controlled lens for studying geometric regularization in text embedding training. Across 13 retrieval benchmarks, spread-out improves fine-tuned retrievers over contrastive baselines across architectures and model sizes. We then diagnose where these gains arise and find that they concentrate on short queries, where spread-out reduces query-side directional crowding. Based on this diagnosis, we propose QUARTO, a Query-length Adaptive spread-out loss that uses batch composition to scale the full regularizer more strongly for short-query-dominated batches and less strongly for longer-query-dominated batches. QUARTO further improves retrieval performance over fixed-weight spread-out. Overall, our findings point to the potential of embedding-space regularization and suggest that text-embedding training should explore a broader design space of objectives beyond InfoNCE-style contrastive learning. Project page: https://github.com/JoeZhou-encoding/Spreadout_QUARTO

Abstract:
CoT-style LLM recommenders generate a natural-language preference note as a chain-of-thought prefix to ranking, but the note is trusted without verification, and the ranker never consults behaviorally similar users. We propose ReflectRec, organized around a preference-to-history round-trip verifier whose recall score serves as an operational signal of preference-to-history recoverability, enabling conditional regeneration for low-recoverability preferences and selection among regeneration attempts. Around this verifier, we wire a semantic Stage-1 prompt, conditional regeneration, and Forward-Look retrieval that injects future watches of behaviorally similar users as a collaborative hint. On MovieLens-100K and MovieLens-1M, the full pipeline outperforms CoT4Rec-ICL by +16.25 / +4.46 pp Hit@10, with positive gains across all ValScore buckets and three LLM backbones (Qwen3-30B, Qwen2.5-72B, and Llama-3.3-70B). We also observe a similar verifier-guided regeneration pattern on Amazon Electronics.

Abstract:
Commercial text-to-image systems silently revise user prompts before generating images, a step users typically cannot disable or even see. Existing audits of cultural bias examine only the final images and treat generation as a single pipeline, so they cannot tell where the bias originates. We introduce WORLDVIEW, a multilingual benchmark of 8,960 prompts across 15 languages and 31 language–context pairings. Using it, we audit the revision layer in three systems (DALL-E-3, Imagen-4, GPT-Image-1.5) through a three-step analysis of how heavily it marks each cultural context, whether it flattens that context into a narrow vocabulary, and whether that vocabulary is stereotypical. Relative to a no-context English baseline, the US is the least-marked context, while non-Western and non-Anglophone contexts are marked far more heavily, flattened into narrow vocabularies applied across topically diverse prompts, and reduced to recognizable cultural stereotypes. Comparing images from original versus revised prompts on models without a revision layer, we identify the layer itself as a previously undocumented, causal source of this stereotyping. To locate cultural bias, and fix it, we must audit the system as deployed, not the model alone.

Abstract:
Audio language models (LMs) are now integrated into high-stakes, decision-oriented settings: hiring pipelines, oral assessments, and language proficiency tests. Prior work in natural language processing has primarily focused on how LMs respond to socially-driven textual variation. With the growing use of LMs that natively process audio, similar robustness and fairness issues arise for speech variation. In our work, we examine how accent variation in English inputs affects audio LMs’ assessments of speakers. We evaluate six audio LMs on speech across five accent groups: American, British, Chinese, Indian, and Nigerian English. We find that several LMs assign lower speech delivery scores to Chinese- and Nigerian-accented speakers, regardless of content quality. At the individual level, delivery scores correlate negatively with acoustic distance from American English across three widely-used models (Spearman ρ = −0.35 to −0.66, all p < .05), suggesting that English speech is scored against an implicit American acoustic norm. Even though models are able to transcribe the speech in our studies, their assessment and perceptions of speakers may result in allocational harm. We release our framework and corpus to support future analyses of bias in audio LMs.

Abstract:
Synthetic data has become essential for training code generation models, yet it introduces significant noise and hallucinations that are difficult to detect with current metrics. Existing data selection methods like Instruction-Following Difficulty (IFD) typically assess how hard a model generates an answer given a query (A|Q). However, this metric is ambiguous on noisy synthetic data, where low probability can distinguish between intrinsic task complexity and model-generated hallucinations. Here, we propose QAQ, a novel data selection framework that evaluates data quality from the reverse direction: how well can the answer predict the query (Q|A)? We define Reverse Mutual Information (RMI) to quantify the information gain about the query conditioned on the answer. Our analyses reveal that both extremes of RMI signal quality issues: low RMI indicates semantic misalignment, while excessively high RMI may contain defect patterns that LLMs easily recognize. Furthermore, we introduce a selection strategy based on the disagreement between strong and weak models to identify samples that are valid yet challenging. Experiments across three datasets spanning code generation (WarriorCoder, Magpie-Qwen2.5-Coder-Pro-300K) and math reasoning (OpenR1-Math-220k) demonstrate that selecting just 25% of data using stratified RMI matches full-data performance while being consistently competitive with or better than existing data selection methods. Our approach highlights the importance of bidirectional semantic coherence in synthetic data curation, offering a scalable pathway to reduce computational costs without sacrificing model capability. Code is available at https://github.com/XXSg559/QAQ.

Abstract:
Hidden-state probes effectively detect LLM hallucinations, but the geometry of the signal remains poorly characterized, driving increasingly complex probe architectures. Across three 7B-scale models and three datasets in a paired-example paradigm, we find the signal overwhelmingly dominated by a single mean-shift component, and removing this direction collapses detection to chance. Shrinkage linear discriminant analysis closes about 73% of the gap between 1D and full-dimensional classifiers, so apparent architectural complexity largely reflects high-dimensional covariance estimation difficulty rather than exploitable non-linearity. A simple L2-regularized logistic regression (0.952 AUROC) bounds or outperforms twelve controlled architectural alternatives, and our multi-layer aggregation exceeds CLAP cross-layer attention probing under matched paradigm. Because the signal spans a contiguous layer band, LayerMix aggregates it to match oracle-layer performance without oracle access. Our claims characterize the geometry within the controlled paired-example paradigm. Our code is available at https://github.com/js-lee-AI/LayerMix.

Abstract:
Some properties of languages, e.g., subject-object-verb (SOV) word order, are more prevalent than others among the thousands of attested natural languages (NLs). Such typological commonality is often attributed to learning biases. Computational simulations, recently with language models (LMs), have facilitated the exploration of this theory. In this paper, we extend existing analyses of the relationship between LMs’ learning biases and typological commonality on both data and model sides, focusing on: (i) cross-serial dependencies, the upper limit of attested syntactic complexity, and (ii) stack-based LMs (SLMs), potentially facilitating learning of hierarchical patterns. We first evaluate generalization of SLMs on cross-serial dependencies across diverse artificial languages and confirm that they struggle with such constructions. However, SLMs with limited working memory generalize better suggesting a possible basis for such inductive bias and thus the typological commonality of some word order configurations.

Abstract:
Large language models (LLMs) remain unreliable when reasoning about the future. Existing models often overlook the difference among future states, committing to specific answers even when the queried facts are inherently volatile. To capture this challenge, we introduce ChronoRegime, a future-oriented temporal reasoning task that requires models to determine whether historical temporal evidence justifies commitment to a future answer. Furthermore, we propose Confidence-Aware Temporal Reasoning (CTR), a framework that estimates query predictability from temporal dynamics and regulates answer commitment accordingly. Rather than performing unconditional forecasting, CTR commits only when the reconstructed evidence supports the target future state. Across multiple LLMs, CTR improves strict overall accuracy by 6.7 percentage points and reduces hallucination rates by 18.4 percentage points on average over the plain QA baseline. These results demonstrate that explicit predictability assessment is an effective mechanism for safer and more reliable temporal reasoning under future uncertainty.

Abstract:
The morphological form of a word can often give cues to its meaning, but purely relying on these mappings can lead to overgeneralization in high-stakes domains. In the medical domain, for instance, LLMs can confidently reason about fictitious drugs from their affixes alone (e.g., wugcillin) and generate plausible-looking clinical content. We present a behavioral and mechanistic study of LLM ”affix heuristics” in pharmacology. Using fictitious drug names built from real affixes, we show that affix signals alone elicit class-level pharmacological responses. We introduce a framework for identifying whether a model’s drug semantics are driven mainly by the affix, the stem, or the drug name as a whole. Applied across 653 drugs, our framework reveals that models often induce drug meaning primarily through affix cues, yet rarely explicitly indicate this reliance, and sometimes incorrectly conflate properties among affix-sharing drugs. Activation patching across models further localizes this behavior to early-mid layers. These findings show that morphological shortcuts pose a subtle but measurable risk to safety.

Abstract:
LLM chatbots increasingly serve as a first source of support for people in psychological distress, including those whose distress is entangled with delusional beliefs. Prior work on LLM mental-health safety largely evaluates general therapeutic quality or single-turn crisis detection, leaving unclear how models behave when distress is intertwined with delusion over sustained conversations. We address this gap with matched multi-turn simulations, across clinically grounded personas and six LLMs, that pair each delusional conversation with a distress-only control to isolate the effect of an operationalized delusional presentation. This reveals a recognition-intervention gap: models detect distress at comparable rates regardless of framing, yet some fail to act on it once distress is embedded in delusion, with safety interventions suppressed by up to 4.5x in the most affected model. The failure is correlated with accumulated premise acceptance whereas emotional validation suppresses intervention in both contexts alike. We term this candidate explanation as narrative debt. Worse, the intuitive fix of prompting models to assess user distress backfires under delusional framing; only delusion-aware prompting with explicit response guidance narrows the gap, and even this depends on a delusion classifier that is itself unreliable on the most vulnerable models. Safe deployment therefore requires treating delusional framing as a distinct risk signal that overrides conversational accommodation.

Abstract:
Vision-Language Models (VLMs) have catalyzed the development of Graphical User Interface (GUI) agents for desktop automation. However, reinforcement learning (RL) for GUI agents remains bottlenecked by two challenges: existing RL frameworks still leave substantial idle time inside the rollout stage, and the learning signal from long-horizon, sparse-reward interactions is highly imbalanced. In this paper, we introduce DART, a fully Decoupled Agentic RL Training framework for GUI agents. Whereas prior RL designs only separate the trainer from the rollout, DART splits the pipeline into four independent modules: environment cluster, rollout service, data manager, and trainer, which brings two extra efficiency benefits: rollout-wise trajectory scheduling and per-worker model synchronization. This yields a 1.6× increase in GPU utilization, a 1.9× boost in training throughput, and a 5.5× increase in environment utilization over a non-decoupled baseline. We further propose a multi-level adaptive data curation scheme operating at the task, trajectory, step, and token levels: it reallocates rollout budget by task difficulty, bootstraps hard tasks with a pre-collected experience pool, restricts policy updates to high-entropy critical steps, and applies truncated importance sampling to absorb the distribution gap between th rollout and training engines. On the OSWorld benchmark, our DART-GUI-7B achieves a 42.13 task success rate, a 14.61 absolute improvement over its base model and a new state of the art among open-source 7B-scale agents. Code, data, and model checkpoints are open-sourced at https://dart-gui-anonymous.github.io (anonymous).

Abstract:
We study how language models aggregate conflicting evidence in controlled multi-juror settings. As a null control, symbolic majority voting, where jurors provide only YES/NO votes, yields near-zero minority override across models, confirming this is not an artifact of the prompt family. Under semantic aggregation, where jurors instead provide short evidence statements paired with local verdicts, override becomes strength-dependent and position-sensitive: rates rise when a minority statement is operationalized as stronger than a weak majority, while a strong-majority/weak-minority control remains low-override. Authority cues applied to a minority statement can modulate this effect in some contexts, but do not universally dominate evidence strength. We validate these patterns with seed replications, cross-domain templates, phase-transition mapping across majority strength, minority strength, and position, and rationality-vs-salience controls. We further confirm the pattern on a larger open-weight model (Llama-3.3-70B-Instruct) and the GPT-4.1 flagship model: Llama-3.3-70B stays close to the smaller-model baseline, while GPT-4.1 shows a substantially larger override magnitude, indicating the effect’s size is model-dependent even as its qualitative shape persists. We emphasize that evidence “strength” here is a synthetic, operationalized manipulation rather than real-world ground truth, and that minority-following can be rational when minority evidence is genuinely decisive.

Abstract:
Current AI agents compellingly describe slides. However, AI-assisted slide editing requires more than understanding: the output must retain layout, style, component structure, and native editability. Towards, AI-assisted slide editing, existing agents operate on screenshots or weak document representations and often fragment coherent visual units, rasterize editable content, or break layout. In contrast, for controllable slide editing we introduce an agentic framework, SlideForge, which builds a Deck State Graph, an executable slide state that links visual decomposition, native PPTX object structure, and perceptual organization. By recovering human-referable components while retaining fine-grained editable structure, SlideForge supports theme-preserving reconstruction through slide-native operations and rendered-state verification. We further introduce an evaluation paradigm for controllable slide transformation that jointly measures component recovery, preservation, restyling consistency, visual quality, and native editability. Experiments show that SlideForge outperforms direct prompting, screenshot-based agents, and generic code-agent baselines across these dimensions.

Abstract:
Scaling the interaction horizon—the maximum number of environment interactions per episode—improves LLM agents on long-horizon tasks, and curriculum-based methods that progressively expand the horizon outperform fixed-horizon alternatives. However, existing schedules are open-loop: they monotonically increase the horizon until a manually specified maximum, with no mechanism to detect when further expansion stops helping. We propose the effective interaction frontier hypothesis: a dynamic boundary beyond which additional interactions yield diminishing returns while cost grows linearly. We then introduce Elastic Horizon, a closed-loop controller that tracks this boundary via the 90th percentile of successful trajectory lengths. On AppWorld and BFCL, fixed-horizon sweeps reveal clear saturation plateaus; Elastic Horizon stabilizes the horizon inside the saturation band from both under- and over-capacity initializations, attains the best success rates across 7B and 14B backbones, and saves up to 25% of per-step trajectory tokens. Our work shifts the paradigm from how to scale interaction horizons to when to stop scaling.

Abstract:
Hateful video detection is essential for maintaining safe and trustworthy online platforms. Current state-of-the-art methods are predominantly trained in a supervised manner, which incurs substantial annotation costs as online video content continues to grow rapidly. Although recent label-free approaches have shown promising progress, they still suffer from an effectiveness–efficiency trade-off, lagging behind supervised methods while requiring considerable computational resources. To address this limitation, we propose TRIAGE, a coarse-to-fine label-free detection framework that first localizes a boundary region and then progressively refines the cases within it. The framework introduces two core designs. The Boundary Mapper uses a lightweight MLLM to efficiently localize a boundary region, finalizing cases outside this region while retaining ambiguous hard cases for further reasoning. The Adaptive Boundary Resolver then queries diverse stronger MLLMs sequentially, resolves cases within the boundary region through Bayesian sequential updates, and terminates once sufficient evidence is accumulated. Across four public datasets, improves accuracy over the best label-free/few-shot baseline by 2.5–8.7 points, reduces parameter-weighted inference cost by at least 2.0, and outperforms the best supervised baseline on three datasets. Code is available at https://github.com/JunyiChen-ai/TRIAGE .

Abstract:
Lifelong model editing seeks to continuously incorporate new knowledge into Large Language Models (LLMs) without costly retraining. Memory-augmented editors preserve pretrained weights to protect unrelated capabilities, but struggle to balance Generalization to rephrased edit queries and Locality on non-edited knowledge. This challenge becomes particularly severe under dense editing settings, where semantically related edits induce substantial memory interference. To tackle this problem, we propose MH-HoReN, a multi-head Hopfield retrieval framework that performs retrieval across complementary subspaces and combines head-wise matching signals for more reliable memory selection. We further extend Hopfield-based editing into a clustered batched scheme that combines shared codebook memories for related edits with token-level pooling for efficient knowledge storage and query routing. Extensive experiments across diverse LLMs and editing benchmarks demonstrate that MH-HoReN achieves better locality-generalization trade-offs as editing scales increase, improving the sum of Locality and Generalization by more than 12% over existing methods while preserving almost perfect Reliability in sequential editing, alongside 7.7% gains in Overall Performance under batched editing. Our code is available here.

Abstract:
While repeated sampling against a verifier is the standard test-time compute strategy for code generation, measured by pass@ K , drawing K independent samples from a single distribution often wastes the compute budget on redundant, near-duplicate reasoning paths. This failure is costly in competitive programming, where many problems admit multiple distinct algorithmic strategies and passing pass@ K requires only one correct attempt. We propose Coordinated Pass@ K Policy Optimization (CPPO), which trains a joint planner–solver policy for pass@ K : a planner emits a tuple of K=4 alternative high-level methods, and a shared solver attempts one solution per method. We train it with a multiplicative planner reward, R plan = J 𝜓 ⋅ R out assigning credit only to valid strategy tuples that lead to verifier-confirmed pass@ K success. Across APPS, CodeContests, and LiveCodeBench-v6, CPPO improves pass@4 over direct sampling, planning baselines, planner-only SFT, and pass@ K -oriented RL under the same K=4 solver-attempt budget, with statistically significant gains on six of nine model–benchmark cells. For instance, it improves Qwen3.5-9B on LiveCodeBench-v6 from 0.588 with PKPO, the strongest baseline, to 0.728, an absolute gain of 0.14 under a hierarchical bootstrap test with p<0.05 .

Abstract:
Multilingual machine translation for endangered and severely underrepresented languages faces two interacting challenges: scarce parallel data and uneven subword vocabulary allocation across writing systems. We study these factors for seven Indian tribal languages, Gondi, Mundari, Bhili, Santali, Kui, Garo, and Kokborok, spanning four language families and four scripts. We release a speaker-curated Hindi–English–X tri-parallel corpus of approximately 20k sentences per language across six community-relevant domains, with all development and test references human-curated or independently speaker-verified. We evaluate prompted LLMs, fine-tuned IndicTrans2 and NLLB-600M, and byte-level ByT5 under corpus-Native, Devanagari-Unified, and Latin-Unified script conditions. Fine-tuning improves CHRF++ by 16-25 points over prompting, but script unification is strongly model-dependent: the same Latin conversion improves Kui under NLLB-600M while substantially degrading performance under IndicTrans2. We introduce Script Tokenization Parity and Script Sensitivity to analyse script–tokenizer interaction; byte-level modelling reduces Script Sensitivity from 14.2 to 2.1 CHRF++, localizing most cross-script variance in learned vocabulary allocation. Our findings show that script choice in endangered-language MT is not routine preprocessing, but a community-sensitive modelling decision that directly shapes linguistic representation, translation quality, and equitable digital access.

Abstract:
Transcoder attribution graphs are usually trained to explain why a model assigns high probability to a particular next token. We introduce Concept-Targeted Attribution (CTA), which instead trains attribution graphs with respect to a linear probe direction. CTA therefore yields probe-specific circuits that explain why an internal concept representation arises in a prompt, independently of whether it is expressed in the generated token. Using Cross-Layer Transcoders, we show that these probe-targeted graphs contain predictive structure: graph-level features predict probe accuracy across four widely studied concept categories ( 𝜌 = 0.91 , R 2 = 0.84 ), while local features identify the sparse components driving per-prompt classification. This connects probe performance to interpretable circuit structure, allowing us to ask not only whether a probe works, but which internal computations make it work. Causal ablations further show that probe-targeted and logit-targeted graphs capture functionally distinct mechanisms. Removing probe-relevant features reduces internal concept scores while largely preserving generated tokens, whereas removing logit-relevant features changes the generated token in 92% to 100% of cases with near-zero effect on probe scores. CTA provides a framework for moving from behavioral probe accuracy to mechanistic explanations of probe performance, enabling more detailed audits of internal concept representations, including safety-critical ones.

Abstract:
Existing emotional support conversation (ESC) systems often react directly to surface-level distress expressions, generating reassurance or advice without adequately modeling the underlying psychological needs driving the distress. However, effective emotional support requires an interpretive process that connects what is expressed with why it is expressed and ultimately what support is needed. We argue that current ESC systems lack such an interpretive layer between observation and supportive intervention. To address this gap, we propose HierINT, a psychologically grounded hierarchical interpretive schema for emotional support conversations. HierINT distinguishes observable communicative intentions from the underlying psychological interpretations that drive them, including cognitive appraisals and unmet fundamental psychological needs. Building on this schema, we construct HierINT-Dlg, a dataset with hierarchical interpretive annotations and need-aligned support strategies. We further propose HierINT-CoRe, a collaborative reasoning framework that operationalizes HierINT by jointly modeling psychological interpretation and support strategy planning prior to response generation. Experiments on ESConv and HierINT-Dlg demonstrate consistent improvements over surface-level and intention-aware baselines, particularly on empowerment and personalization metrics. Our results suggest that effective emotional support requires moving beyond reactive response generation toward psychologically grounded need-informed intervention. Our data and code are available at https://github.com/43zxj/HierINT.

Abstract:
Typological features are widely used in multilingual NLP, and the prediction of such features holds downstream utility. However, existing methods to predict missing values lack interpretable justifications for predictions, while their performance across resource levels and feature types remains underexplored. Given LLMs’ abilities in meta-linguistic reasoning and in providing rationales, we investigate LLMs’ performance in typological feature prediction via an in-context learning approach with linguistic data from URIEL+ and Glottolog. We find that zero-shot prompting is insufficient, but when given phylogenetic and geographic neighbour evidence, LLMs substantially outperform all baselines without disadvantaging low-resource languages. We further find that most LLM rationales are consistent with the provided evidence, offering a step toward explainable typological feature prediction.

Abstract:
We study learning a mixture of k Plackett-Luce models from multi-way ranking responses from annotators that may represent heterogeneous underlying preferences. This problem has many applications in AI alignment and preference optimization. Prior work has studied mixtures of Bradley–Terry models from pairwise comparisons. However, estimating a mixture of multi-way ranking models can become theoretically unidentifiable when k exceeds m/2 , where m is the ranking length. We design an efficient algorithm to address this issue by first augmenting the rankings to a larger size (e.g., generating comparisons from a base model), followed by a gradient-based estimation to reduce inference cost (in the input embedding space). With this procedure in mind, we then fit a mixture of Plackett-Luce (PL) models via an expectation-maximization-style iteration, or MoPLEx in short. We conduct extensive experiments to verify this algorithm. First, we find that the gradient-based approximation estimates true probabilities with less than 5% error on models with up to 34 billion parameters. Second, MoPLEx improves clustering and ranking accuracy by an average of 43.7% and 15.2% over baselines using a single PL model or a mixture of Bradley-Terry models, on UltraFeedback and PERSONA datasets. These results demonstrate the effectiveness of MoPLEx for tackling multi-way rankings following heterogeneous preferences through measuring alignment via gradients.

Abstract:
As language models take on agentic roles that call APIs, read tool outputs, and act on third-party content, their attack surface expands beyond what users type. Whether they treat a malicious instruction the same way regardless of where it arrives has not been studied systematically. We introduce the Safety Asymmetry Score (SAS), measuring how a model’s susceptibility to adversarial content shifts depending on whether it arrives in the user message, tool metadata, or tool output, using matched payload pairs that hold the malicious text identical and vary only the channel. Across 10 production LLMs and three attack families, general-purpose models sharply discount instructions arriving as tool metadata relative to identical instructions in the user message, while agent-native models discount them far less. This differential survives an affordance-matched control equalizing tool availability and scoring, and a size-controlled mixed-effects analysis. Within the tool surface the pattern reverses: the same content is far more instruction-like in a tool’s description than in its output, with affordances identical. Models treat tool metadata as instructions and tool results as data. In Llama 3.3 70B and GPT-OSS 120B this signal is causally present at mid-to-late depths but non-linearly encoded, so linear probes miss what activation patching recovers.

Abstract:
Retrieval plays a central role in multi-hop question answering (QA), where answering complex questions requires gathering multiple pieces of evidence. We propose PRISM, an agentic retrieval framework that leverages large language models (LLMs) in a structured loop to retrieve relevant evidence with high precision and recall. PRISM decomposes retrieval into three specialized agents: a Question Analyzer that breaks complex queries into sub-questions, a Selector that identifies the most relevant context for each sub-question (focusing on precision), and an Adder that brings in any missing evidence (focusing on recall). The iterative interaction between the Selector and Adder produces a compact yet comprehensive evidence set, avoiding both brittle error propagation and noisy context accumulation. It achieves higher retrieval accuracy while filtering out distracting content, enabling downstream QA models to surpass full-context answer accuracy while relying on significantly less irrelevant information. Experiments on four challenging multi-hop QA benchmarks, including HotpotQA, 2WikiMultiHopQA, MuSiQue, and MultiHopRAG, demonstrate that our approach consistently outperforms strong baselines.

Abstract:
Post-training quantization (PTQ) is essential for deploying LLMs under memory and bandwidth constraints. However, extreme low-bit quantization remains highly sensitive to activation outliers and anisotropic weight curvature. Existing incoherence-based PTQ methods mitigate this issue with fixed randomized Hadamard transforms (RHTs), which improve quantization robustness but cannot adapt the rotated basis to the layer, calibration distribution, or quantizer. We introduce HARP (Hadamard-preconditioned Adaptive Rotation Processor), a learnable structured two-sided orthogonal processor that replaces fixed Hadamard mixing while preserving exact full-precision equivalence. HARP represents each rotation as a product of sparse butterfly-like block-orthogonal stages, supports non-power-of-two dimensions through Mixed-Radix schedules, and initializes to the RHT processor up to a fixed permutation. Fitted only on calibration data, HARP adapts the quantization basis to each layer and backend. Across 2-4 bit settings on Llama models from 1B to 70B, HARP consistently improves perplexity and yields its clearest zero-shot gains at 2 bits; a 2-bit Qwen3-8B experiment shows the same transfer beyond the Llama family. HARP also preserves deployment efficiency: on Llama 2 7B at 2 bits, it reaches 128 tok/s, retaining 90% of RHT throughput (142 tok/s) and running approximately 2.1× faster than FP16 (61 tok/s).

Abstract:
Media bias in news articles operates through subtle linguistic cues—loaded language, selective framing, and strategic omission—that resist single-model detection and have traditionally required large annotated corpora for supervised training. We ask whether structured multi-agent deliberation can serve as a principled, training-free alternative to supervised classification for this task. We introduce MABPD (Multi-Agent Bias Probing Detection), a pipeline in which three specialized LLM agents analyze an article from complementary perspectives and resolve disagreements through a Structured Argument Debate (SAD) protocol. SAD implements a domain-motivated asymmetric burden of proof—biased claims without grounded textual evidence carry zero weight—combined with role-weighted voting and post-consensus verification, replacing task-specific supervised decision boundaries with explicit deliberative structure. Ablation confirms that this structured deliberation, not mere agent parallelism, drives performance: removing the debate module reduces F1 by up to 10.6 points. On the BABE benchmark (4,121 expert-annotated sentences), MABPD achieves 83.4% macro F1 on the held-out test split—within 0.7 percentage points (pp) of the supervised SOTA (MAGPIE, 84.1% macro F1; Horych et al., 2024)—without any task-specific training or threshold tuning on annotated data. Cross-dataset evaluation on the SemEval 2019 HyperPartisan corpus (644 articles) yields 75.0% zero-shot accuracy, within 7.2 pp of the supervised SOTA accuracy (82.2%; Kiesel et al. 2019), confirming transfer across annotation regimes. We release the full pipeline and evaluation code.

Abstract:
In open-ended generation, LLMs frequently fall into the "likelihood trap", characterized by repetitive degeneration and vocabulary dullness, resulting in a discrepancy between machine-generated and human-written text. While post-hoc tail truncation (e.g., Top-p, Min-p) avoids sampling from the unreliable tail, it can misalign generation with human lexical preferences by over-sampling from the uncalibrated head; fixed scalar repetition penalties, in turn, ignore how the scale of the logit distribution varies across inference steps, which can disrupt semantic coherence. To address both shortcomings, we propose Variance-Calibrated Modulation (VCM), a training-free pre-decoding intervention. VCM directly reshapes the probability distribution prior to truncation via two dynamic mechanisms: (1) Contextual Searchlight via PMI, which naturally suppresses global stopwords and elevates context-evoked tokens, and (2) Adaptive Self-Debiasing, which utilizes real-time logit standard deviation to provide scale-invariant penalization. In experiments across open-ended generation, factual QA, and mathematical reasoning, we show that VCM consistently mitigates the likelihood trap. With negligible computational overhead, VCM integrates with existing decoding strategies, improving diversity and coherence and, particularly at higher decoding temperatures, reasoning accuracy. Our code is publicly available on GitHub: https://github.com/AetherDing/VCM

Abstract:
Large Language Models (LLMs) have recently shown strong potential for time series (TS) analysis by integrating numerical signals with rich text prompts. In this process, although TS are encoded into token sequences and fed into LLMs in a manner similar to text prompts, TS tokens differ from prompt tokens in their underlying structure and semantics, a distinction that remains underexplored in existing LLM-based TS systems. In this work, we investigate both TS tokens and prompt tokens from a new perspective. First, we re-examine TS tokens in the frequency-domain and reveal that token-level information contribution is highly uneven: large groups of tokens share redundant spectral patterns, while a small subset acts as critical frequency anchors. Motivated by this observation, we introduce a frequency-based token merging strategy for TS tokens, which preserves essential information while eliminating redundancy. Second, we uncover a unexplored pyramidal decay effect in prompt tokens, showing that their influence rapidly diminishes with model depth. Building on these findings, we propose TokenDecouple, a unified framework that decouples the compression of TS tokens and prompt tokens. Extensive experiments across diverse TS tasks show that TokenDecouple achieves up to 7.68× inference speedup while delivering performance improvements in 78% of evaluated scenarios, establishing an efficient paradigm for LLM-based TS analysis.

Abstract:
Steering vectors have emerged as a lightweight and effective approach for aligning large language models (LLMs) at inference time, enabling modulation of model behaviors by shifting intermediate representations toward a target behavior. Many existing steering methods apply interventions at globally fixed locations in an LLM, effectively treating the optimal intervention location as input-invariant. However, representations relevant to a target behavior can be encoded at different LLM layers depending on the input. Theoretically, we show that different inputs may require steering at different layers to achieve alignment with a given behavior. Empirically, we observe that the most effective layer for steering varies substantially between inputs. Motivated by these observations, we introduce Where to Steer (W2S), a general framework that adaptively selects intervention layers conditioned on the inputs by learning a mapping from input representations to steering layers. Across multiple LLMs and alignment behaviors, W2S consistently outperforms fixed-layer baselines in both in-distribution and out-of-distribution settings. Our findings suggest that input-dependent layer selection is a key design dimension complementary to methods for steering vector extraction.

Abstract:
Conversational Recommender Systems (CRS) have improved personalized recommendation through dialogue, but most studies and datasets remain limited to single-session settings. Such single-session settings make it difficult to capture long-term preference evolution and the interplay between latent behavioral patterns and explicit conversational styles, partly because real-world conversations aligned with users’ long-term behavioral histories are scarce. To address this gap, we introduce TRACE, a long-term CRS dataset built from behavioral histories that reflect persistent user preferences. Our analysis shows that TRACE provides reliable and diverse behavioral personas, as well as higher-quality CRS interactions than existing CRS datasets. Building on TRACE, we introduce EvoCRS, a persona-based CRS framework that models accumulated and evolving preferences through updated behavioral and dialogue personas. Experiments show that both persona types improve recommendation performance, and that persona updates are important for capturing long-term preference evolution. Our code and datasets are available online.

Abstract:
Garden path sentences present a processing difficulty for humans— the sentence prefix leads the listener towards one interpretation, until the listener hears a critical word that shows that the initial interpretation was wrong. Lexical surprisal, a measure that usually predicts sentence processing difficulty quite well, fails to provide good predictions for garden path sentences. We propose an alternative that actively predicts a distribution over syntactic trees (its syntactic belief) and updates that distribution after each new word. If a processor is led down a garden path, syntactic beliefs will be wrong, requiring a large update at the critical word. The magnitude of the update is measured with a Rényi divergence. Crucially, this metric is dependent on lexical items, but is distinct from lexical probability and measures a degree of belief update over non-lexical syntactic structure only. This Syntactic Belief Update provides a better fit to human reading time data on garden path sentences, suggesting a new research direction examining non-lexical alternatives to surprisal for psycholinguistics.

Abstract:
The EU Artificial Intelligence Act now requires providers to mark generated text so its origin can be checked. In regulated industries the duty is sharper: a marker must never flag third-party writing, survive routine review edits, and work on outputs often a sentence long. We call these the regulated-provenance triad and prove that no bounded-distortion watermark perturbing a model’s token distribution can satisfy all three below a critical length, the provenance horizon. The proof combines the Le Cam two-point method with a strong data-processing inequality for the edit channel; a matching achievability bound for the green-list family under a memoryless resampling channel makes the scaling order-tight. The barrier rises at negligible-distortion budgets, not at deployed ones. The contraction factor entering the bound is a proven upper bound for the memoryless resampling channel and a calibrated estimate for realistic paraphrase, under which we estimate most sentence-length regulated outputs fall below the horizon. Relocating provenance to a tamper-evident log read through a conformally calibrated retrieval test sidesteps the barrier: under exchangeable calibration its false-positive rate is at or below the target at every length. Coverage carries no guarantee and degrades under an adaptive editor.

Abstract:
Causal thinking enables humans to understand not just what is seen, but why it happens. To bring this ability to modern AI systems, we introduce visual causal discovery, the task of inferring visually verifiable direct causal dependencies among entities in a single static image. We construct the Visual Causal Graph dataset (VCG-32K), a large-scale collection of over 32,000 images annotated with entity-level causal graphs, and develop Causight, a causally aware vision-language model for this task. Our training recipe combines data curation from VCG-32K, Tree-of-Causal-Thought (ToCT) reasoning synthesis, and reinforcement learning with a graph-structured causal reward. Experiments show that Causight outperforms GPT-4.1, achieving over a threefold performance boost (21% absolute gain).

Abstract:
User behavior simulation with large language models (LLMs) is increasingly used to support multi-agent ecosystem simulation. Existing simulators typically rely on static user profiles inferred from historical observations, which become inadequate in socially intensive environments such as live streaming where interaction dynamics continuously reshape user behavior. We propose LiveSim , an LLM-based framework for live-stream ecosystem simulation. It represents users as editable behavioral hypotheses and progressively refines them through trajectory-grounded interactions, where discrepancies between simulated and observed trajectories reveal missing environmental shaping effects. These signals are further extracted as transferable environment-behavior patterns and accumulated in a collective behavioral memory to improve user-level behavioral fidelity and support ecosystem-level simulation. Experiments on real-world live-stream risk-control data validate the effectiveness of LiveSim in improving user-level behavioral fidelity and enabling ecosystem-level analysis of risk evolution and platform intervention effects.

Abstract:
Diffusion Large Language Models (DLLMs) have recently emerged as a parallel alternative to autoregressive models, offering bidirectional context modeling and parallel decoding. However, developing highly performant DLLMs is hindered by lifecycle challenges, including suboptimal timestep weighting in pre-training, train-inference distribution gaps in post-training, and underutilized caching during inference. To address these obstacles, we introduce ReDLLM, a 30B-parameter Mixture-of-Experts DLLM. It incorporates a time-aware weighting strategy during continual pre-training that scales gradients according to denoising difficulty. Furthermore, we propose two block-oriented refinements: a terminal-block treatment to narrow the train-inference gap, and a block-aware inference pipeline featuring inline block prefill and block-level scheduling to maximize KV-cache reuse and parallelization. Extensive evaluations demonstrate that ReDLLM surpasses existing DLLMs at the same active-parameter scale, achieves generation quality competitive with 100B DLLMs, and attains the highest inference throughput among evaluated models.

Abstract:
Emergent generative AI technologies, while remarkably beneficial across several tasks and domains, can suffer from unforeseen vulnerabilities and difficult-to-anticipate failure modes. Such failure modes can disparately affect vulnerable communities. This paper investigates two well-known vision language models (VLMs) and inspects how these models encode protected identities while generating images. We show that when VLMs are requested to transform human face images to reflect race identities (e.g., Hispanic, Black, White, or Asian) and sexual orientation (e.g., gay or straight), VLMs often comply with such requests. We demonstrate that compliance (and refusal) is non-deterministic, group-sensitive, and generated images are statistically distinguishable, often accentuating deep-seated stereotypes. We further show that representation biases introduced during image generation can re-emerge in textual reasoning and characterization tasks. Finally, we show that this worrisome finding is not limited to racial identity or sexual orientation, image generation can perpetuate bias around criminality, raising concerning possibilities of AI-assisted profiling.

Abstract:
Web search agents powered by Large Language Models (LLMs) show strong promise, but deep research tasks expose a recurring failure mode: once an agent has produced a query, plan, or intermediate conclusion, it becomes less objective when later judging the consequences of that same action. We term this phenomenon inertia bias. To make it measurable, we introduce the IBIS benchmark, which controls the search observations while varying whether the model is evaluating the outcome of its own prior action. We find that models are substantially worse when they "own" the preceding search step, showing that self-authored action history can systematically distort subsequent judgment. We further show that this bias propagates into two forms of system-level degradation: search noise at the worker level and contextual noise at the manager level. To address this problem, we propose NIS-Agent, which applies context isolation at the two decision points most vulnerable to inertia bias: webpage triage and final-answer validation. Across GAIA, WebWalkerQA, BrowseComp, and BrowseComp-zh, NIS-Agent achieves competitive performance while reducing token cost by 33% compared to our baseline. We further train an 8B model to be intrinsically more resistant to inertia bias; under the same NIS-Agent framework, it attains average performance comparable to GPT-4o on deep research benchmarks. Our code is publicly available at https://github.com/PangSMPang/NIS-Agent.

Abstract:
Document ranking aims to order candidate documents based on their relevance to a given query. However, identifying relevant documents in complex real-world scenarios requires deep reasoning beyond surface-level matching, driving a shift from semantic matching to reasoning-intensive reranking. Since the latter requires integrating information across documents for complex reasoning, it presents two key challenges: (1) varying reasoning difficulty, where assessing document relevance to the query requires multi-step reasoning with different complexity; and (2) strong contextual sensitivity, where the relevance of a document is heavily influenced by other documents within the context window. To address these challenges, we propose CuRLRank, which consists of (1) CuRL, an adaptive curriculum reinforcement learning approach that enables LLMs to develop complex reasoning capabilities across varying levels of difficulty, and (2) IFRS, an iterative filtering ranking strategy to achieve efficient and robust ranking. Extensive experiments on BRIGHT and R2MED benchmarks demonstrate that our CuRLRank significantly outperforms baseline methods while achieving lower ranking latency.

Abstract:
As LLMs become reusable infrastructure for evidence access, NLP evaluation must move beyond verifying whether generated outputs are faithful to their sources. We study source-framing propagation: whether the directional effect of institutionally authored evidence is preserved or altered through LLM-mediated representations. We introduce Evidence-to-Action Propagation (E2A), an evaluation protocol that traces how fixed evidence affects reasoning, belief, and action across role-conditioned LLM receivers. In a controlled primary evaluation over 94 post-knowledge-cutoff earnings events and 24,816 LLM-agent decision traces, supplemented by targeted mechanism and scope diagnostics, raw disclosures produce a strong positive-return skew. Source-faithful canonical extraction produces no detectable change in this skew (80.1% vs. 80.0%, p=0.889); no-style synthesis shifts action away from buy, while its belief and quality effects remain unresolved. Shared summaries synchronize evidence paths and rationales. A separate diagnostic shows that randomizing evidence order reduces the skew by 4.30 percentage points, although its quality effect remains unresolved. These findings reveal a faithfulness paradox: reducing hallucination risk through source-faithful representations may still preserve a source’s downstream directional influence. More broadly, the paper argues that NLP evaluation should examine not only whether models reproduce sources accurately, but how language interfaces shape the path from evidence to reasoning and action.

Abstract:
Summaries of real-world events can become outdated as contexts evolve and new information arrives. A common response is to generate a new summary from the updated context, but full regeneration discards the previous draft, can obscure what changed, and may be unnecessary when only a few claims are unsupported. We study localized faithfulness repair: updating outdated spans in an existing summary while preserving supported content. We propose Detect-Remask-Repair, a diffusion-based framework that identifies, remasks, and repairs outdated regions with masked diffusion language models. To evaluate evolving-context summarization, we introduce StreamSum, a benchmark of synthetic event timelines. Experiments on DialogSum and StreamSum show that localized diffusion repair provides a controllable alternative to full rewriting: faithfulness-steered repair improves early drafts, one-step repair reduces repair cost to under half a second, with the framework enabling faithfulness–speed–preservation tradeoffs across datasets. We also find that the framework can provide a post-hoc correction step that improves faithfulness for autoregressive systems.

Abstract:
Supervised fine-tuning (SFT) data selection is commonly formulated as instance ranking: score each example and retain a top- k subset. However, effective SFT training subsets are often produced through ordered curation recipes, where filtering, mixing, and deduplication operators jointly shape the final data distribution. We formulate this problem as fixed-pool data recipe search : given a raw instruction pool and a library of grounded operators, the goal is to discover an executable recipe that constructs a high-quality selected subset under a limited budget of full SFT evaluations, without generating, rewriting, or augmenting training samples. We introduce AutoSelection , a two-layer solver that decouples fixed-pool materialization based on cached task-, data-, and model-side signals from expensive full evaluation, using warmup probes, realized subset states, local recipe edits, Gaussian-process-assisted ranking, and stagnation-triggered reseeding. Experiments on a 90K instruction pool show that AutoSelection achieves the strongest in-distribution reasoning average across three base models, outperforming full-data training, random recipe search, random top- k , and single-operator selectors. Additional held-out reasoning, instruction-following, graph-reasoning, search-stability, structural-ablation, and 1.5B-to-7B transfer results further show that recipe structure matters beyond individual selection operators. Code is available at https://github.com/w253/AutoSelection .

Abstract:
In this work, we identify three representation-level dynamics shared across diverse large language models (LLMs) during the processing of few-shot prompts. These structural behaviors are not explicitly trained, yet they emerge reliably across model families and scales to systematically shape internal representations. Using Sparse Autoencoders (SAEs) as an analytical lens, we uncover three primary phenomena: (1) a decimal periodicity effect, where specific features exhibit a repeating modulo-10 activation pattern; (2) local feature homeostasis, a compensatory push–pull dynamic in which current representations are inversely modulated by prior exemplars; and (3) adaptive feature redistribution, wherein specific features flexibly shift their semantic profiles in response to contextual constraints. We validate these mechanisms across multiple model families and prompt formats. Together, they offer fresh insights into the internal dynamics of in-context learning and raise important caveats for future SAE-based interpretability studies.

Abstract:
Verification agents are increasingly used to judge candidate answers in web search, yet their own reliability remains underexplored. Existing verification methods often rely on annotated ground truths or manually crafted rubrics, limiting scalability under high annotation costs and the dynamic nature of the web. Hence, we propose the self-derived verification task, where an agent derives validation rules from a search query and verifies whether a candidate answer satisfies them using external tools, without human-provided rubrics or ground-truth answers. We further introduce BrowseComp-Verify, a benchmark of 3298 query-answer pairs constructed from 453 BrowseComp questions with reliable correctness labels. Using BrowseComp-Verify, we evaluate a broad suite of LLMs as self-derived verification agents. Most models complete the task with high coverage, but accurate judgment remains challenging. In particular, many models exhibit a conservative bias toward rejecting candidate answers: GPT-v5.2-Thinking-high achieves 99.04% accuracy on incorrect answers but only 20.39% on correct answers.

Abstract:
Tables in real-world documents, including scientific papers, financial reports, and statistical summaries, frequently use merged cells, multi-level headers, and sectioned layouts. Yet the lightweight formats LLMs handle best (Markdown, CSV) cannot express these structures, while expressive formats (HTML, LaTeX) impose excessive token cost and parsing complexity. To enable systematic analysis of this trade-off, we introduce a taxonomy decomposing table structure into six atomic features (F1–F6) and corresponding complexity levels (L1–L6), and construct STB (Synthetic Table Benchmark), a controlled benchmark with 1,089 tables systematically distributed across complexity levels. Using this framework, we propose LTEM (LLM-friendly Table-Enhanced Markdown), a formally specified, minimal extension of Markdown that explicitly encodes spans and hierarchical headers. Experiments across five state-of-the-art LLMs show that LTEM achieves accuracy comparable to HTML, offers a favorable accuracy–token trade-off relative to HTML and LaTeX, and produces substantially fewer invalid responses than LaTeX; for table generation, LTEM is comparable to MultiMarkdown though HTML is best. We release our taxonomy, benchmark, and format specification to support future research on LLM-table interaction.

Abstract:
Knowledge-based visual question answering (KB-VQA) lets vision-language systems answer questions that exceed their parametric knowledge by conditioning a reader on passages retrieved from a Wikipedia-derived knowledge base. In pure-text long-context LLMs, retrieved-context use follows the U-shaped lost-in-the-middle effect of Liu et al. (2024): information at the start and end of context is used, the middle is lost. Whether this transfers to deployed multimodal KB-VQA is open. To close this gap, we design the first controlled probe of reader-side position dependence in multimodal KB-VQA: a gold-position protocol in which only the gold passage’s prompt slot varies within question. We run it on three open-source 7B/8B VLM readers and two KB-VQA benchmarks with up to 20 retrieved passages. The shape flips from U to primacy: gold-at-first beats gold-at-last by 16 to 26 points on all six combinations of reader and benchmark, an effect we call Lost at the End; the gap holds at every scale we test, 3B to 32B, attenuating at 32B. Three targeted ablations narrow the cause. A text-only control that removes the image and changes nothing else shows the primacy is already present in text mode and does not depend on the image. Image-position and distractor-shuffle ablations trace the effect to prompt slot 0 of the instruction-tuned reader, where a second answer-bearing passage placed later is largely wasted. On a frozen reader, three retrieval-side fixes (MMR, oracle reranking, rank-based reordering) all fail to improve on the deployment default. Our findings indicate that recall@k is the wrong metric for deployed KB-VQA and that the remaining headroom sits on the reader side; we release our protocol as a controlled instrument for evaluating reader-side interventions.

Abstract:
Large Language Models (LLMs) are increasingly deployed in education, powering tutoring systems, automated feedback, and curriculum support. Most alignment techniques optimize toward a single aggregated notion of human preference, a mismatch for education, where legitimate disagreement is the norm. Students, parents, teachers, and policymakers often hold deeply conflicting views on assessment, discipline, curriculum, and institutional authority, reflecting genuine value pluralism rather than noise. Pluralistic alignment, which seeks to represent and navigate this diversity, offers a principled path forward. Despite growing interest, no benchmark evaluates how well LLMs handle value-laden disagreement in educational contexts. To address this, we introduce EQUAL (Eliciting QUality sAmples of pLuralistic alignment in education), comprising 14,270 samples—5,824 value-laden scenarios and 8,446 multiple-choice questions, curated from six sources, including over two decades of Phi Delta Kappa polling data, via a lexical filtering pipeline and human validation. EQUAL supports evaluation across Overton, Steerable, and Distributional paradigms. We benchmark nine LLMs under four approaches and additionally evaluate three frontier reasoning-capable models. No single method consistently satisfies pluralistic objectives. These findings provide a foundation for future research on pluralistic alignment in educational AI.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) successfully aligns autoregressive models in rigorous domains like mathematical reasoning and code generation. However, applying RLVR to Diffusion Large Language Models (dLLMs) is hindered by intractable sequence-level likelihoods, forcing existing methods to rely on unstable, high-variance approximations. We propose Sequence Likelihood-Free Policy Optimization (LFPO), an efficient framework that circumvents this bottleneck by formulating policy alignment as discrete velocity alignment. Inspired by continuous flow matching, LFPO eschews trajectory-level estimation and directly optimizes denoising logits via contrastive token-level updates, providing a stable, low-variance surrogate. Furthermore, predicting terminal solutions from intermediate states implicitly straightens the generation trajectory, enabling high-quality sampling with significantly fewer denoising iterations. Extensive experiments on reasoning and coding benchmarks demonstrate that LFPO matches or exceeds the accuracy of likelihood-based baselines while substantially accelerating inference.

Abstract:
Automated LLM workflow design often relies on expensive per-task search, which is computationally expensive and does not explicitly reuse structural knowledge across tasks. We propose SWIFT (Synthesizing Workflows via Few-shot Transfer), which reuses prior search trajectories and demonstrations to synthesize target-task workflows without iterative target-task optimization. SWIFT distills compositional heuristics and output-interface contracts from source-task trajectories, then conditions a single LLM generation pass on these priors and cross-task workflow demonstrations to produce an executable workflow. Across 5 benchmarks, SWIFT outperforms the SOTA search-based baseline while reducing marginal per-task optimization cost by three orders of magnitude given an existing source bank. It also improves performance on 4 additional held-out benchmarks and transfers from GPT-4o-mini to Grok, Qwen, and Gemma execution backbones. These results suggest that prior workflow search can expose reusable structural priors that reduce optimization cost while preserving or improving accuracy on related tasks.

Abstract:
Algorithmic reasoning—internalizing a procedure from small examples and applying it at arbitrary scale—is a hallmark of human intelligence: a child who learns arithmetic on a few digits can, given time and paper, evaluate expressions of unbounded length. Modern language models cannot: Qwen3.5-27B, despite 100k-token reasoning contexts, degrades to below 10% accuracy on 30-digit multiplication. Departing from reasoning-through-scale, we introduce RACE (Recursive Arithmetic Control Encoder), a 3.6M-parameter Transformer trained from scratch that generalizes far better by recursively decomposing problems while keeping its working memory bounded. Two mechanisms drive this: token-level editing, which erases intermediate scratchwork, and self-calling, which recurses on subproblems. Within a single model, RACE improves multiplication length generalization 4.7 times over the Arithmetic Transformer and, for the first time, achieves length generalization on subtraction and division (×47.1 and ×17.6), while also solving nested, multi-operator expressions. Code is available at https://github.com/Pusheen-cat/RACE_2026.

Abstract:
Document Layout Analysis (DLA) pipelines provide structured page representations for retrieval-augmented generation, long-document question answering, and related applications. Yet their robustness evaluation remains largely area-centric. We identify this Footprint Bias and propose ProSA, a lightweight output-level auditing framework that decouples controlled probing, policy-driven targeting, and structure-aware diagnosis. ProSA combines Block-level Structural Loss Rate (B-SLR), granularity-aware exposure descriptors, and pathway attribution to analyze where structural identity is lost, at what exposure granularity failures emerge, and how failures propagate. Across MinerU and PP-StructureV3 on 1,000 pages, affected area weakly tracks perturbation-induced OCR instability ( R 2 = 0.384/0.110 ), whereas B-SLR aligns much more closely with it ( R 2 = 0.727/0.916 ). Exposure descriptors further separate occlusion- and topology-dominant pathways, while matched-footprint structural probes cause much larger downstream QA/retrieval drops than area-matched erasure. These results shift DLA robustness evaluation from footprint-based measurement toward structure-aware vulnerability auditing.

Abstract:
Vision-language retrieval with CLIP-style dual encoders achieves strong cross-modal performance, yet practical accuracy often hinges on localized semantic distinctions where top-ranked near misses differ from the true match by a single critical detail. Hard-sample mining can select confusable candidates but cannot construct corrected counterparts; synthetic augmentation can generate novel samples but, without conditioning on actual model failures, targets irrelevant dimensions of hardness. We observe that a top-ranked false positive is a counterfactual scaffold—sharing most of the query’s semantics while differing in a localized failure-causing residual. Minimally correcting this residual yields a hard positive of the ground truth in the same modality; the corrected and unedited versions form a hard negative pair that straddles the decision boundary, producing complementary pull–push supervision. We introduce RePair, guided by three principles—Validity, Minimality, and Locality—which mines false positives bidirectionally, applies LLM-guided counterfactual editing, and trains with a local hard-pair contrastive objective. On Flickr30K and COCO30K, RePair outperforms controlled augmentation baselines with only 107K synthetic samples—26%–75% fewer than comparable methods—confirming failure-conditioned repair is more data-efficient than error-agnostic augmentation.

Abstract:
Large vision-language models used as listwise rerankers must jointly process visual tokens from tens of candidates per query, making token pruning essential for practical deployment. Existing pruning methods retain tokens by attention saliency, yet we show that saliency is systematically misaligned with ranking contribution: visually prominent tokens often capture order-neutral patterns shared across candidates. This mismatch is layer-dependent: saliency becomes informative only where attention is concentrated, and normalized attention entropy diagnoses the reliability shift (Pearson r=0.87). We propose RaDiCal (Rank-Discriminative Calibration), a training-free framework that uses normalized attention entropy to decide when saliency can be trusted, fusing it with an attention-free rank discriminative prior and se-lecting pruning layers from the same trust landscape. Across three retrieval benchmarks and multiple VLM architectures, RaDiCal matches Dense MRR@10 on Flickr30K and surpasses it on MSCOCO at a 20% token budget, ranks first among all pruning methods on FashionIQ, and holds within 1.2 pp on Flickr30K and MSCOCO at 10% retention. It cuts FLOPs by 39–45% and delivers 1.28–1.45× measured speedups across two VLM architectures without dataset-specific retuning.

Abstract:
In this paper, we analyze the reasoning trajectories of a variety of DRAs and show that existing agents often suffer from reasoning stagnation: the majority of iterations contribute little or no improvement to final performance, while agents lack awareness of their trajectories and are therefore ineffective at adapting their search strategies or determining when to terminate. To address this issue, we introduce a set of unsupervised signals and a Retrieval-Aware Agent Controller (RAAC), which assists the agent in selecting optimal actions at each stage of the research process. RAAC incorporates key information retrieval principles, namely search novelty and information coverage, resulting in more effective reasoning trajectories that improve overall performance while reducing unnecessary iterations, and consequently cost and latency. Specifically on BrowseComp-Plus and across a large set of DRAs, adding RAAC reduces the number of search calls by an average of 14, significantly improves the best-performing DRA on recall and accuracy, and achieves an accuracy gain of up to 10% (3% in average).

Abstract:
Long-form video question answering hinges on locating brief, sparse evidence buried in minutes of dense action. Flattening the entire video into a single caption stream forces the LLM to sift through thousands of tokens, where decisive cues are easily drowned out by surrounding context. We present MSTree, a training-free framework that organizes captions into a Multi-Scale Evidence Tree. The tree is built by a coordinated dual-modal segmentation method that combines visual scene transitions and textual coherence under temporal contiguity, producing segments that are both visually and semantically consistent. Inference follows a mine-then-reason workflow: the LLM first scans coarse segment summaries to locate the most relevant event, then reasons over its fine-grained captions. When the first-round prediction falls below a confidence threshold, Contrastive Targeted Reflection (CTR) activates. The model articulates a discriminative gap that specifies what evidence is still missing to separate the top candidate answers, and an answer-anchoring constraint requires any newly retrieved evidence to both support the revised prediction and contradict the original. On EgoSchema, NExT-QA, and Video-MME, MSTree reaches 68.4%, 77.5%, and 55.7% accuracy, outperforming all training-free baselines.

Abstract:
Machine unlearning aims to make a model forget specific data, yet unlearned LLMs often fail to stay unlearned: brief fine-tuning can revive removed knowledge. Existing robustness predictors rely on global weight-space displacement, but distance alone can be misleading when random or destructive updates collapse performance. We argue that relearning robustness depends on update structure: robust unlearning should affect forget-critical weights while sparing retain-critical ones. We introduce the Forget-Retain Alignment Gap (FRAG), a training-free predictor that scores an update’s forget-retain alignment without running a relearning attack, and separates selective from dense updates more reliably than global distance. Building on the forget-critical, retain-sparing principle, Forget-Retain Pruning (FRP) improves relearning robustness. Our results suggest that weight selectivity better explains robustness than distance alone. Our code is available at https://github.com/Yi1-Chen/FRAG.

Abstract:
The rise of fully autonomous agents necessitates a rigorous evaluation of their reliability under real-world API complexity. Accordingly, moving beyond perfectly sanitized settings that disregard realistic obstacles, we introduce WildAGTEval, a configurable benchmark environment that accounts for two critical dimensions of realistic API complexity: API specification, which includes detailed documentation and usage constraints, and API execution, which captures runtime challenges. Consequently, WildAGTEval offers (i) an API system spanning 86 functions across 7 domains with 60 independently activatable complexity scenarios, yielding approximately 32K distinct test configurations, and (ii) 300 multi-turn conversations grounded in this system. Through systematic evaluation of 11 state-of-the-art LLMs, we find that: (1) individual complexities reduce accuracy by up to 23.1%; (2) compounding them amplifies this drop to 52.1%; and crucially, (3) failing agents fabricate success and distort intent rather than admitting limitations, exposing a critical reliability gap invisible to existing benchmarks.

Abstract:
Agentic RAG agents must generate effective queries across diverse retriever backends, yet current agents produce retriever-oblivious queries, leading to large performance gaps when the backend changes. We observe that different retrievers, despite dissimilar architectures, exhibit structured behavioral patterns in their document-return overlap, forming a continuous behavioral space that can guide retriever-adaptive query generation. We propose RAMP (Retriever-Adaptive Multi-retriever Policy), which probes a black-box retriever with shared queries, encodes the resulting overlap patterns into a compact behavioral embedding, and injects it as soft tokens to condition query generation. A single RAMP model replaces four retriever-specific specialists without per-retriever retraining and generalizes to unseen retrievers by interpolating in behavioral space. On four QA benchmarks, RAMP matches 96–98% of four retriever-specific specialists (+3.8 EM over unconditioned training) and reaches 91.8% of the retrained upper bound on unseen retrievers, outperforming routing, few-shot, and fusion alternatives.

Abstract:
Rubric-based reinforcement learning decomposes open-ended instructions into prompt-specific, flexible rubrics, making it better suited than reinforcement learning with verifiable rewards for post-training LLMs on open-ended tasks. However, static rubrics are inevitably hacked as the policy evolves, and existing dynamic approaches introduce new problems: undirected rubric extraction, unreliable hack detection, and unbounded rubric proliferation. We propose CARE ( C ontrastive A nchor-based R ubric E volution), which grounds every rubric evolution step in a high-quality anchor response generated by a frontier model conditioned on the prompt and its rubrics. At each training step, CARE contrasts the highest-scoring rollout against the anchor, enabling two complementary mechanisms: an Adaptive branch that reactively repairs reward misspecification; and a Chase branch that proactively converts frontier-level quality gaps into sharper rubrics. Together, the two branches maintain discriminative accuracy in the high-reward region —the precise region where reward over-optimization mostly originates. Experiments on WildChecklist-9K with Qwen2.5-7B-Base and Qwen2.5-7B-Instruct show that CARE achieves state-of-the-art performance on Arena-Hard-2.0, InfoBench, and FollowBench, and is the only method whose win rate against GPT-4.1 anchor responses shows sustained improvement throughout 300 training steps; additional results on Llama-3.1-8B-Instruct and Qwen3-8B further indicate that CARE generalizes across model families.

Abstract:
Human–LLM co-writing is increasingly used for open-ended text generation, but much prior work focuses on final outputs rather than the interactional dynamics through which stories are produced. We study turn-based collaborative storytelling across three matched conditions: Human–Human (HH), Human–AI (HA), and AI–AI (AA). Using a shared storytelling paradigm, we measure how agents align, introduce novel material, and influence narrative development through turn-level measures of valence adaptation, semantic novelty, transience, and resonance. Our results show that HA co-writing is not intermediate between HH and AA collaboration. Instead, it displays a distinctive asymmetry where humans tend to introduce more novel and persistent narrative material, while LLMs tend to elaborate and stabilize the existing context. These findings suggest that, in this setting, LLMs function less as human co-authors and more as adaptive narrative amplifiers that reshape how agency is distributed in collaborative writing.

Abstract:
Unified vision-language models (VLMs) that tokenize images through a vector-quantized (VQ) codebook routinely hallucinate objects on grounded yes/no benchmarks, yet existing decoding-time fixes treat this as generic miscalibration without an architectural account. Using activation patching across twenty-five models spanning eight LLM families, we identify an early-layer (L₀) attention routing circuit shared across VQ-tokenized VLMs and propose a three-gate diagnostic that distinguishes the models carrying it. The diagnostic isolates ten positive models (five natural unified-VQ VLMs across three LLM families and five induced variants) and rejects the remaining fifteen. A single-variable architectural swap (LLaVA-1.6 CLIP+MLP → VQ+Linear) installs the circuit, while a matched-compute MLP control does not, isolating vector quantization as the source of the pathological signal; the routing pathway that carries it is one that the backbone already provides. Against tuned baselines, DoLA wins on binary calibration, but only L₀ ablation reduces object hallucination in open-ended generation (CHAIRᵢ -31% relative; tuned DoLA and VCD leave it unchanged or worsen it). These results recast object hallucination in unified VQ VLMs as a property of architecture and pretraining, and yield a targeted intervention that mechanism-agnostic decoding cannot replicate.

Abstract:
We investigate how synonyms evolve over time, focusing on two apparently contradictory tendencies: the Law of Differentiation, whereby synonyms diverge in meaning, and the Law of Parallel Change, whereby they co-evolve. We develop a framework that breaks down the laws into a fine-grained taxonomy, which we use to construct a diachronic dataset based on the Oxford English Dictionary. We rely on definition embeddings and the use of LLM-as-a-Judge to link synonyms. We validate our methodology by performing a human evaluation on a subset of the constructed dataset. Finally, we populate the taxonomy and conduct extensive analyses across different time spans to investigate the predominance of each law in English over time. We find that while the Law of Differentiation is larger in absolute numbers, it is driven by the general tendency for meanings to disappear from the language rather than by a property unique to synonymy.

Abstract:
In persona-based dialogue generation (PDG), LLMs often overuse persona attributes by incorporating them regardless of dialogue context, resulting in unnatural responses. Despite its practical significance, the underlying causes remain unexplored, with no method to mitigate this problem or metric to assess the appropriateness of persona use. To address these issues, we first conduct a comprehensive analysis of LLM-based PDG, revealing that LLMs exhibit a systematic bias to incorporate all given persona attributes, and that existing metrics fail to capture contextual appropriateness. Building on these findings, we propose Self-CONtrastive Persona Overuse Suppression (SCONPOS) to mitigate overuse by directly intervening in LLMs’ internal representations at the prompt encoding stage, without requiring any response generation. We further propose the Persona Appropriateness Score (PAS), a novel metric that penalizes both overuse and underuse. Experimental results demonstrate that SCONPOS systematically reduces the overuse, and PAS captures the contextual appropriateness of persona use.

Abstract:
Multi-agent systems built from large language models are deployed widely, yet how much performance is lost when two LLMs must coordinate rather than act alone remains unclear. We formulate the collaboration tax as the team-decentralisation loss of a two-player cooperative game with private information, with two propositions characterising its sign and its equivalence to a max-superadditivity violation. We operationalise this definition on 32 solo-tractable tasks grouped by source of grounding friction and measure it on 11 models from 7 providers. The tax is structured along two no-exception axes: a category ordering across every model and a monotonic decrease with capability. The proximate mechanism is not a reasoning deficit but a four-stage conversational cascade in which agents make ungrounded claims, fail to query the partner, skip integrating both views, and accept the answer without re-derivation. The tax is mechanically predictable from conversation features and partly tractable: a prompt intervention targeting all four stages closes a substantial fraction of the gap, with the dominant bottleneck differing across categories. In heterogeneous pairs the tax is pulled toward the stronger partner rather than the additive midpoint, empirically realising the max-superadditivity violation predicted by our framework. Together these results recast collaboration in LLM systems as a measurable, predictable, and partly tractable cost.

Abstract:
Large language models (LLMs) have undergone rapid growth, forming a heterogeneous ecosystem whose model relationships and capability profiles lack comprehensive documentation. Organizing such an ecosystem requires reusable model representations that are comparable across architectures while still reflecting how each model processes shared inputs. We propose ABLE (Attribution-Based Large-model Embedding), a training-free framework that represents each model in an attribution space. Given a shared probing corpus, ABLE computes feature attributions, aligns token-level attribution scores into a tokenizer-agnostic word-level representation, and projects the resulting attribution patterns into compact model embeddings. These embeddings summarize model-level input-dependence patterns and can be reused across downstream tasks. We construct ABLE representations for 239 LLMs and evaluate them on model organization and relationship analysis, model routing, and benchmark score prediction. Across these tasks, ABLE captures meaningful model-level structure, achieves stronger relationship prediction than parameter- and output-based baselines, supports competitive model routing, and predicts benchmark scores from frozen representations, while theoretical analysis further supports the stability of the representation.

Abstract:
Domain specialization can improve LLM behavior, but often weakens the general capabilities inherited from the original model. Recent Multi-Teacher On-Policy Distillation (MOPD) pipelines recover model capabilities by supervising student-generated trajectories with teacher feedback, but typically assume teacher-aligned prompt coverage, requiring prompts to match the teachers’ training distributions. This assumption is difficult to satisfy when the general teacher is an open-source model whose post-training data are unknown. Instead of attempting to reconstruct this hidden distribution, we study general capability recovery with readily available proxy general prompts. We identify two failure modes of vanilla MOPD in this incomplete-coverage situation: recovery-preservation counteraction from mixing conflicting recovery and preservation gradients, and weak-signal flattening from uniformly averaging samples with unequal correction demand. We propose Counteraction-Aware Multi-Teacher On-Policy Distillation (CaMOPD), which addresses these issues with decoupled alternating training and gap-based sample selection. CaMOPD allocates dedicated updates to general recovery, periodically performs domain-preservation updates, and selects samples with larger averaged token-level teacher-student log-probability gaps to concentrate correction signals. Across role-play dialogue and medical reasoning QA scenarios, CaMOPD outperforms all baselines in general capability recovery while maintaining domain-specific behavior. Gradient coherence analyses further support the intended effect of CaMOPD in producing more coherent correction signals.

Abstract:
Explanations are often promoted as tools for transparency, but they can also foster blind trust; users may assume reasoning is correct whenever model outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in multimodal moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones. Specifically, we perturb reasoning chains generated by vision-language models (VLMs) with three error types (omissions, contradictions, hallucinations) and measure their effects on user trust and error detection through a study with 800 participants, alongside a parallel detection task with models. Our findings reveal two key effects: (1) users often equate trust with outcome agreement regardless of flaws in the model’s reasoning, and (2) confident tones suppress scrutiny while leaving agreement and self-reported trust largely unchanged. These results highlight how CoT explanations can simultaneously clarify and mislead in tandem, and point to user-side scrutiny, not just model-side faithfulness, as a critical dimension for calibrated trust in AI reasoning. All code will be released publicly upon acceptance.

Abstract:
Temporal Knowledge Graph (TKG) extrapolation seeks to infer future facts from time-varying relational histories. Recent diffusion-based approaches improve uncertainty modeling through generative denoising, but their aggregated conditioning on subject histories may insufficiently distinguish query-specific evidence from non-salient historical facts, thereby diluting target-discriminative signals. To bridge this gap, we propose FreqDiff, a Frequency-aware Diffusion framework for TKG extrapolation. Specifically, FreqDiff formulates future object prediction as query-slot denoising and develops a dual-stream denoiser that integrates temporal dependency modeling with context-aware spectral calibration. The spectral branch synthesizes history-conditioned filters from learnable bases to adaptively re-calibrate denoising representations, while a frequency-domain regularizer is proposed to align the denoised target with the gold object in spectral space. Experiments on four public TKG benchmarks demonstrate that FreqDiff achieves state-of-the-art performance.

Abstract:
Recent state-space models (SSMs) such as Mamba achieve language modeling performance comparable to transformers despite relying on fundamentally different architectures. This raises an important question: how do these structural differences influence the geometry and functional nature of their internal representations? We study this question through a multi-scale analysis of representations in transformers, SSMs, and hybrid architecture. First, we find that SSMs distribute their representational information evenly across all dimensions, whereas transformer representations are heavily dominated by a single principal direction. By evaluating hybrid architectures, we observe that the representation space becomes increasingly skewed toward a single dominant direction after each attention layer. Next, we explore how the different geometric spread of representations impacts representational capacity through compressibility. Surprisingly, we find that despite their contrasting geometric structures, both architectures exhibit tightly matched effective capacities. We further investigate whether this skewed geometry affects how concepts are encoded. Using rank-constrained probes, we demonstrate that both architectures encode concepts in subspaces of surprisingly similar dimensionality. Furthermore, we demonstrate that the transformers’ dominant principal direction does not inherently encode more conceptual information. Finally, we zoom in and examine the alignment between manifolds, either by analyzing representations of specific topics or by looking at the nearest neighborhoods of tokens, and find that they are highly aligned. Ultimately, our analysis suggests that while transformers and SSMs induce different usage of latent space, they display a striking functional convergence at the level of local semantic manifolds.

Abstract:
Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluation increasingly relies on generation tasks where outputs are scored continuously rather than marked correct/incorrect. We present a principled extension of IRT-based adaptive testing to continuous bounded scores (ROUGE, BLEU, LLM-as-a-Judge) by replacing the Bernoulli response distribution with a heteroskedastic normal distribution. Building on this, we introduce an uncertainty aware ranker with adaptive stopping criteria that achieves reliable model ranking while testing as few items and as cheaply as possible. We validate our method on five benchmarks spanning n-gram-based, embedding-based, and LLM-as-judge metrics. Our method improves ranking correlation by 0.13 𝜏 over random sampling and has 99% accuracy on confident predictions while using 2% of the items after a one-time calibration step

Abstract:
With recent advances in diffusion models, text-to-image (T2I) generation has achieved remarkable progress, producing high-quality images from simple text prompts. However, as these models have become more expressive, concerns regarding the generation of unsafe or inappropriate content have grown. To address this issue, we propose the Safety-Adjustable Framework for Ethics (Safe), a novel training-free, plug-and-play prompt optimization framework. Leveraging the reasoning capabilities of large language models (LLMs), Safe optimizes prompts to meet concept-specific safety requirements while preserving their original semantics. The framework operates in three stages: (1) generating an image from the original prompt using a pretrained T2I model, (2) analyzing the generated image with a vision-language model (VLM), and (3) conducting DAG-based reasoning over both the prompt and the image description to produce refined prompt. In addition, Safe uses a search module to incorporate external knowledge such as cultural norms and legal constraints, enabling adaptation to diverse user contexts. Extensive experiments demonstrate that Safe effectively complements existing safety methods and achieves strong performance in user-centric safety alignment.

Abstract:
LLMs typically linearize 2D tables into 1D sequences to fit their autoregressive architecture, which weakens row-column adjacency and other layout cues. In contrast, purely visual encoders can capture spatial cues, yet often struggle to preserve exact cell text. Our analysis reveals that these two modalities provide highly distinct information to LLMs and exhibit strong complementarity. However, direct concatenation and other fusion methods yield limited gains and frequently introduce cross-modal interference. To address this issue, we propose DiVA-Former, a lightweight architecture designed to effectively integrate vision and text information. DiVA-Former leverages visual tokens as dynamic queries to distill long textual sequences into digest vectors, thereby effectively exploiting complementary vision-text information. Evaluated across 13 table benchmarks, DiVA-Former improves upon the pure-text baseline by 23.9% and achieves consistent gains over existing baselines using visual inputs, textual inputs, or a combination of both. Our code is available at: https://github.com/Applied-Machine-Learning-Lab/EMNLP26_DiVA

Abstract:
Trustworthy clinical AI must use real evidence and avoid relying on surface-level artifacts. We evaluate 12 open-weight vision-language models (VLMs) on two clinical neuroimaging cohorts for binary classification of affective disorders and cognitive decline. Both cohorts include structural magnetic resonance imaging (MRI) acquired under their original research protocols. Prior work does not establish the included neuroimaging inputs as reliable stand-alone diagnostic evidence for the present tasks. Nevertheless, when neuroimaging context is introduced, smaller VLMs gain up to 0.66 F1 under the evaluated augmented conditions, becoming competitive with models an order of magnitude larger. Confidence estimation shows that most of the calibration improvement for the analyzed smaller models occurs after the MRI reference is added to the prompt, before any image is supplied. Our preliminary expert case study finds that faithfulness remains low in every condition examined, with the reviewed model introducing unverified clinical details. Finally, in our single-model intervention, preference alignment suppresses MRI-referencing behavior but reduces the augmented-condition advantage, leaving the underlying issue unresolved. These results caution against reading surface metric gains as evidence of true multimodal integration, with direct implications for clinical VLM deployment

Abstract:
Existing methods for test-time reinforcement learning (TTRL) derive rewards from answer-level self-voting on unlabeled test-time tasks with canonical answers, but this breaks down for code generation because programs cannot be compared by surface form and therefore do not directly provide a usable training signal. To make TTRL applicable to code generation, we propose probe-driven TTRL, which constructs output-free probe inputs from the problem statement, executes candidate programs on these probes, and defines a Probe Consensus Reward (PCR) from the resulting behavioral agreement. PCR provides a behavioral training signal for open-vocabulary programs, but it is not a fully reliable verifier and remains susceptible to reward hacking through spurious consensus. We therefore introduce Entropy-Regularized Rank-Masked Policy Optimization (ERPO), which converts low PCR into conservative negative updates through rank masking and controls policy drift with an entropy ceiling. On coding benchmarks, ERPO substantially improves pass@1 and pass@k in both in-domain adaptation and zero-shot transfer.

Abstract:
Interactive large language model agents are often specialized to a single environment and transfer poorly across others. We study training-free consolidation of such specialists into a single generalist checkpoint. A key difficulty in this setting is that, in multi-turn agents, small merge errors on role-critical outputs can trigger long-horizon failure cascades. We therefore propose Agent-Role Merging (ARM), a neuron-level training-free framework that selects a merged backbone with an answer-free activation-overlap signal and then repairs residual gaps through conflict-aware transplantation of a small set of role-salient neurons. Our analyses further show that role-conditioned tracing yields more benchmark-specific neuron sets, enabling more localized repair. Across Qwen3-8B and Qwen2.5-7B expert pools, ARM produces the strongest balanced merged generalist among compared training-free baselines, improving both average performance and worst-suite robustness while preserving out-of-domain generalization. These results suggest that targeted neuron-level edits offer a practical route to robust multi-turn agent merging without end-to-end retraining.

Abstract:
Demographic cue-based evaluation is widely used to study how large language models (LLMs) adapt their responses to signaled demographic attributes within and across groups. This approach typically relies on a single cue (e.g., names) as a proxy for group membership, implicitly treating different cues as interchangeable operationalizations of a single underlying identity-conditioned behavior. We test this assumption in realistic advice-seeking interactions spanning 14.8 million prompts, focusing on race and gender in a U.S. context. We find that cues for the same group induce only partially overlapping changes in model responses, yielding inconsistent conclusions about personalization, while bias conclusions are unstable, with both magnitude and direction of group differences varying across cues. We further show that these inconsistencies reflect differences in cue–group association strength and linguistic features bundled within cues that shape model responses. Together, our findings suggest that demographic conditioning in LLMs is not a cue-invariant category-level parameter but depends fundamentally on how identity is cued, reflecting responses to linguistic signals rather than stable demographic categories. We therefore call for multi-cue, mechanism-aware evaluations as a foundation for robust and interpretable claims about demographic variation in LLM responses.

Abstract:
Dense retrievers exhibit positional bias, favoring documents whose query-relevant information appears near the beginning and degrading retrieval performance when the information appears later. While prior work on positional bias in dense retrievers has largely focused on architectural explanations, we study how the positional distribution of evidence in training data affects retrieval-level bias direction. To test this, we construct synthetic position-targeted training sets in which query-relevant evidence appears at the beginning, middle, or end of documents, and fine-tune eight architecturally diverse pretrained models under position-skewed and balanced training distributions. At the ranking level, we observe a strong directional pattern across the examined models: skewed training distributions favor evidence at the corresponding positions. Position-balanced training reduces positional sensitivity by 56–87% on position-aware benchmarks, with competitive mean retrieval performance in our controlled setting. Representation-level analyses further suggest that fine-tuning often reshapes learned positional preferences, although pre-existing architectural or pretraining-specific tendencies persist in some models. These results identify training-position distribution as a major controllable factor in retrieval-level position bias and suggest balanced data curation as a practical mitigation strategy. Our code is available at https://github.com/sionic-ai/position-bias-dense-retrievers.

Abstract:
We introduce Stateless Bernoulli Watermarking (SBW), a new statistical watermark for Large Language Models that determines green list membership through independent per-token Bernoulli trials. Unlike KGW’s vocabulary permutation or SynthID’s multi-layer tournament, SBW requires only a single comparison per token against a counter-based random number generator, reducing membership complexity to O(1) and enabling single-kernel execution with zero intermediate allocations. We prove that this formulation preserves the same detection guarantees as fixed-size green lists: the z-score test remains 𝒩(0,1) under the null. The stateless architecture enables capabilities unavailable to existing methods: full-vocabulary self-salt watermarking (over 6000 × faster than KGW’s self-salt and 2 × faster than SynthID despite biasing the entire vocabulary with candidate-dependent seeding) and architectural compatibility with distributed inference. In end-to-end generation benchmarks, SBW adds less than 1% overhead at all batch sizes. We additionally identify hash function design as a previously unexplored axis for watermark quality, showing that a GPU-native Jenkins hash improves null calibration by 1.8 × while producing more diverse text. Experiments across two seeding schemes and eight (𝛾, 𝛿) configurations confirm statistical equivalence with ROC-AUC differences below 0.01.

Abstract:
Standard on-policy distillation (OPD) for large language models estimates the reverse-KL objective using student-sampled tokens, yielding an unbiased single-sample Monte Carlo estimator that avoids vocabulary-wide computation. However, we show that this estimator suffers from severe training pathologies in practice: sample inefficiency, unstable generation dynamics, and a substantial performance gap compared to exact full-vocabulary OPD. Reward-level diagnosis traces these pathologies to the log-ratio reward, which is unbounded by construction, producing extremely high-variance gradients concentrated at early positions and persisting throughout training; standard post-hoc scaling fail as they operate only after this distortion occurs. To solve this problem, we propose PowerOPD: a family of natively bounded, sign-consistent rewards from the Box-Cox power transformation, parameterized by 𝛼 > 0 , of which the log-ratio is the degenerate 𝛼 → 0 limit. Across six mathematical reasoning benchmarks and four Qwen3 teacher–student pairs, PowerOPD achieves benchmark-averaged Avg@8/Pass@8 gains of up to +6.37/+5.71 over vanilla OPD, +3.01/+3.54 over post-hoc stabilization, and +2.59/+8.90 over full-vocabulary OPD, while reducing wall-clock time by 59.2% and peak GPU memory by 23.1%. Larger 𝛼 generally improves accuracy, consistently shortens responses, and keeps gradient norms more than 3,000× smaller than vanilla OPD.

Abstract:
Visual document retrieval (VDR) bridges language-based systems with visually grounded knowledge. Recent VDR models adopt multi-vector late interaction, representing each document with multiple visual token embeddings and scoring relevance with MaxSim. However, existing late interaction engines struggle to support multi-vector VDR at scale due to the higher dimensionality and larger number of visual tokens per document. We propose VISTA, a VDR-specific late-interaction engine that exploits the correlation among visual tokens within each document to optimize both compression and search. To the best of our knowledge, VISTA is the first VDR-specific late-interaction engine for scalable multi-vector visual document retrieval. Experiments on ViDoRe V2 and ViDoRe V3 show that VISTA achieves comparable or even better retrieval quality than compressed baselines, while consistently reducing latency by 1.2–6.7 × and index size by 1.4–6.1 × .

Abstract:
Subword tokenization hinders low-resource language processing by imposing frequency patterns from dominant languages onto script-sharing variants. Byte-level models bypass this issue by processing raw UTF-8 characters, yet they create a granularity mismatch for word-level tasks in non-Latin scripts. Hierarchical byte-level architectures address this mismatch by grouping bytes into word-aligned chunks. However, these architectures require massive training data and suffer from representational misalignment when paired with frozen subword-based language models. In this paper, we propose an adapted hierarchical network framework that bridges this modality gap without extensive training. Our method initializes byte embeddings directly from the subword representations of a frozen base model. We apply a chunk alignment loss to project dynamically grouped byte chunks toward precomputed subword targets, and interleave lightweight part-of-speech (POS) supervision to guide boundary detection. Experiments across six languages demonstrate that our tokenizer-free approach improves performance for word-level morphological tasks, yielding up to a 13.3% improvement on POS tagging.

Abstract:
Large Language Models struggle with memory demands from the growing Key-Value (KV) cache as context lengths increase. Existing compression methods homogenize head dimensions or rely on attention-guided token pruning, often sacrificing accuracy or introducing computational overhead. We propose FourierAttention, a training-free framework that exploits the heterogeneous roles of transformer head dimensions: lower dimensions prioritize local context, while upper ones capture long-range dependencies. By projecting the long-context-insensitive dimensions onto orthogonal Fourier bases, FourierAttention approximates their temporal evolution with fixed-length spectral coefficients. Evaluations on Llama and Qwen models show FourierAttention achieves the best long-context accuracy on LongBench and Needle-In-A-Haystack (NIAH). Besides, a custom Triton kernel, FlashFourierAttention, is designed to optimize memory via streamlined read-write operations, enabling efficient deployment without performance compromise.

Abstract:
Latent communication provides an efficient alternative to text-based exchange in multi-agent LLM systems by transmitting continuous hidden representations instead of natural-language messages. However, opacity is not privacy: latent messages may still carry private prompts, agent memory, or task-irrelevant sensitive information recoverable by downstream agents or adversaries. We study this risk through communicated latent messages , the edge-level representations that cross agent boundaries and define the privacy interface of latent multi-agent systems. We identify two coupled leakage mechanisms: instantaneous leakage from individual messages and propagation leakage accumulated across multi-round transcripts. We propose Information-Constrained Latent Communication (ICLC), a model-agnostic communication-layer framework that constrains latent information exposure without modifying backbone LLM parameters. ICLC combines a receiver-conditioned latent bottleneck, adversarial regularization over messages and transcripts, and topology-aware gates that suppress unnecessary communication edges. Across private-context question answering, collaborative reasoning, and tool-use planning, ICLC preserves the utility benefit of latent communication while reducing average leakage from 0.747 to 0.398 compared with raw latent communication. These results suggest that privacy in latent multi-agent systems should be treated as a dynamic information-flow problem over messages, edges, and interaction rounds.

Abstract:
Activation steering often averages activation differences between positive and negative prompts to construct a steering vector. This difference-in-means (DiM) estimator is simple, but a contrast pair can coactivate the intended concept together with topic, style, length, refusal, or truthfulness-related nuisance features. We model each pair as revealing the target vector through a quotient that removes its active nuisance span. The resulting polysemantic information operator determines which target components are identifiable and the number of contrast pairs needed to recover them. We establish identifiability, sample-complexity bounds, a coactivation phase diagram, and sparse-dictionary recovery guarantees. Synthetic and controlled experiments support the predicted quotient-recovery behavior. Public-data and Pythia-410M pilots instead evaluate recovery of the quotient estimand induced by chosen nuisance supports, not an externally validated semantic target. A controlled TruthfulQA counter-confounder illustrates the main NLP lesson: raw behavioral margin can reward a style-entangled DiM vector even when quotienting removes the injected style component.

Abstract:
Large Language Model (LLM) agents are increasingly expected to interact with external environments rather than only generate text. A key challenge in this process is to accurately and efficiently ground high-level agent intents into valid executable action sequences, especially when the environment consists of multiple interacting modules. To address this challenge, we introduce executable action programs (EAPs), a restricted Python-based intermediate representation between textual agent intents and structured environment-module action calls, which leverages the expressive power of Python to explicitly represent complex behavioral sequences, their branching relationships, and cross-module data dependencies. Based on this representation, we implement EAPRouter , which generates, validates, executes, and reuses EAPs through an agent-EAP interface and a lightweight environment module integration protocol. We evaluate EAPRouter on real agent-environment interaction traces from social simulation, a demanding application scenario with multi-agent, multi-module, and repeated interaction patterns. Experiments show that EAPRouter achieves a 60.0% grounding success rate with Kimi-K2-Instruct, outperforming the SOTA function-calling baseline by 31.6%. It also improves over the strongest baselines by 11.5%-25.4% on GLM-4.7, Claude Opus 4.5, and Gemini 2.5 Flash, with smaller gains on less code-capable models. On the cross-domain NESTFUL v2 benchmark, EAPRouter matches ReAct’s executable accuracy (83.8% vs. 84.9%) while reducing LLM calls, tokens, and execution time by about 80%. EAP pre-generation further handles over 70% of interaction requests, cache reuse covers 66.5% of the remaining ones, and safety validation blocks all tested unsafe payloads and prompt-injection attempts. The code is available at https://github.com/tsinghua-fib-lab/EAPRouter.

Abstract:
Significant progress has been made in aligning LLMs with target value functions. We argue that, even when an LLM has been well aligned in (post-)training, it may still fail to maximise the aligned value in reasoning. We mathematically formalise this gap as rational value risk: the utility discrepancy between a model’s deployed reasoning strategy and its rational counterpart whose responses maximise utility in the steepest direction. The estimation error of rational value risk is further decomposed into three components from bounded prompts, bounded responses, and imperfect verifiers. Extensive experiments are conducted, covering models Llama-3.1, Qwen-2.5, Tülu-3 families (7B-72B), GPT-5.2, GPT-5.5, and DeepSeek-V4, and benchmarks UltraFeedback, AlpacaEval, GSM8K, MATH, HumanEval, and MathArena. The results validate that (1) rational value risk is widespread; (2) value alignment can reduce, but cannot avoid, it; (3) self-consistency can improve rationality; and (4) a longer chain of thought improves rationality but with diminishing returns. The code is at https://github.com/EVIEHub/LLM-Rationality .

Abstract:
Does fine-tuning on a user’s social-media history make an LLM a better simulator of that user? We study this question using QLoRA, conditioning models on a user’s chronological action history and a reply thread before generating that user’s reply. Across 12 matched baseline–QLoRA pairs from four model families (0.8B–32B), plus a baseline-only 122B model, LLM-as-judge believability evaluations show that QLoRA improves several smaller models, has mixed effects on stronger models, and does not lift a sub-10B model into the strongest large-model tier. However, a real-versus-generated classifier gives a different result: QLoRA makes Qwen, OLMo, and Llama replies much less detectable, while Gemma remains readily detectable. Contextual believability and detector AUC are uncorrelated across systems. We further explore LLM evaluations: strong LLM judges agree on broad quality tiers, and a blind 200-comparison audit finds human–judge agreement of 66% when three strong judges agree and 47% when they split. The measured effect of QLoRA therefore depends on the evaluation axis.

Abstract:
Generative retrieval offers a unified paradigm for personalized search by autoregressively decoding items as Semantic IDs. However, personalized search hinges on reasoning about query-history relevance, in which explicit textual reasoning incurs heavy cost and high latency, whereas purely implicit reasoning does not provide a structured intermediate latent interface for retrieval. To resolve this tension, we propose LaPR, an adaptive latent retrieval reasoning framework that turns explicit retrieval reasoning into compact latent signals. Specifically, LaPR constructs verified step-wise retrieval traces, induces step-conditioned semantic slots via vector quantization, and distills textual reasoning into these slots through a text-latent to pure-latent curriculum. At inference, LaPR adaptively predicts the reasoning depth and corresponding slots from the query and user history, providing latent reasoning signals for efficient Semantic-ID generation. Extensive experiments show that LaPR consistently surpasses all baselines, achieving SOTA personal retrieval performance while substantially reducing reasoning-token cost and inference latency.

Abstract:
Large Action Models (LAMs) select and execute actions from external tool spaces. Traditional relevance-based retrieval selects actions that satisfy semantic relevance but violate execution constraints, parameter requirements, or action preconditions. Reactive execution discovers infeasible action orderings only during rollout, which increases replanning cost. Reliable multi-step execution therefore requires feasibility-aware selection and dependency-aware ordering. We propose ActionRAG, a training-free retrieval framework that combines Constraint Induction, Constraint-Aware Retrieval, and Action-of-Thought to select and order feasible actions before execution. Constraint Induction extracts environment-level and task-level constraints from the environment state and the user goal. Constraint-Aware Retrieval ranks actions by semantic relevance and feasibility coverage, filtering infeasible actions before trajectory construction. Action-of-Thought then constructs feasible action trajectories over the filtered action set using inter-action dependencies and preconditions. We evaluate ActionRAG on StableToolBench and MCP-Bench across four backbones under single-tool, multi-step, and cross-category tool-use settings. On StableToolBench, ActionRAG improves the solvable pass rate by 40.3–43.4%p over CoT/ReAct, reaching 60.7%p on cross-category tasks, and reduces token cost by up to 6.2× over DFSDT.

Abstract:
Chain-of-thought (CoT) distillation transfers reasoning behaviors from a strong teacher to a smaller student, but prior work reports a capacity gap: distillation may fail when the teacher-student capability mismatch is large. We revisit the capacity gap from a practical perspective by re-examining commonly used experimental settings. Notably, we find that CoT distillation often degrades performance compared to the student’s pre-distillation baseline, and that some settings used in prior work, while suitable for establishing the capacity gap as a phenomenon, do not reflect realistic deployment scenarios. Complementing prior work that establishes the capacity gap, we evaluate its practical impact under more realistic settings and find that it does not consistently dominate; stronger teachers tend to be preferable when candidate teachers differ substantially in performance. Our results offer practical guidance for selecting teacher-student pairs in CoT distillation.

Abstract:
Remote sensing has emerged as an important domain for multimodal large language models (MLLMs), where reliable understanding requires locating small, dense, and spatially subtle visual evidence. However, our diagnostic analysis shows that current MLLMs often fail because their reasoning is not grounded to the correct visual evidence, leading to wrong-region attention or missed task-relevant objects. To address this issue, we introduce RsEvi-8B, an evidence-aware remote sensing MLLM that reasons with parseable bounding-box-grounded evidence. To enable this capability, we construct RsEvi-248K, a large-scale supervised dataset spanning VQA, visual grounding, object detection, object counting, and scene classification. The dataset provides verified bounding-box-grounded reasoning traces, where each intermediate visual claim is aligned with supporting evidence regions and filtered for both answer correctness and evidence faithfulness. To improve evidence-faithful reasoning, we further propose the evidence-aware GRPO, which uses an evidence verifier to regulate learning on generated evidence spans while preserving task-level outcome optimization. Extensive experiments across five remote-sensing task families show that RsEvi-8B improves the dataset-balanced average score by 17.15 points over Qwen3-VL-8B-Instruct, establishing explicit visual grounding as a key ingredient for reliable remote-sensing reasoning.

Abstract:
Large Reasoning Models (LRMs) show remarkable capabilities in complex reasoning tasks through Chain-of-Thought (CoT) mechanisms. However, their reasoning processes frequently suffer from overthinking, generating inefficient patterns and substantial overhead without improving quality. Existing pruning approaches typically rely on computationally expensive external teacher models or aggregated sequence-level metrics, failing to capture the fine-grained, intrinsic cognitive state of individual reasoning steps. In this paper, we propose ThinkLeap, a novel confidence-guided optimization framework that identifies and retains only critical cognitive leaps within reasoning chains. Our approach is grounded in the observation that a model’s internal confidence dynamics serve as a fine-grained structural signal: rising confidence often reflects evidence accumulation, while declining confidence signals doubt or redundancy. By leveraging these dynamics, ThinkLeap autonomously replaces inefficient steps with ellipsis markers, preserving the logical flow while signaling omitted deliberation. Extensive experiments across three challenging reasoning benchmarks demonstrate that ThinkLeap establishes a new paradigm for efficient reasoning, achieving an average 19.4% reduction in reasoning length while maintaining comparable accuracy.

Abstract:
Multimodal Large Language Models (MLLMs) have achieved impressive progress in vision-language understanding, yet they remain susceptible to object hallucination, which refers to generating descriptions of objects that do not exist in the input image. Through comprehensive analysis, we uncover a fundamental trade-off in MLLMs: strengthening visual attention enhances grounding completeness but also increases hallucinations, especially additive hallucinations that appear in the trailing part of the generated sentence. To alleviate this trade-off, we propose GLVAC, Global-to-Local Visual Attention Correction, a training-free dual-path inference framework that adaptively combines global conservatism and local enhancement in visual attention. The global path maintains stable and grounded predictions under moderate visual attention, while the local correction branch is dynamically activated for high-uncertainty tokens identified via entropy-based estimation. A confidence-guided gating mechanism then determines whether to adopt the correction output, ensuring safe and reliable refinement. GLVAC can be seamlessly integrated into existing MLLMs without retraining or external modules, effectively reducing object hallucinations while maintaining perceptual completeness across diverse benchmarks and model families. Our code is available on https://github.com/Trusted-LLM/GLVAC.

Abstract:
Chain-of-thought reasoning improves large language models by spending more tokens at test time, but this couples accuracy to generation length. Activation steering offers a lightweight alternative, yet most steering methods apply a fixed vector throughout generation and cannot adapt to different stages of a multi-step solution. We propose STIR (Self-Distilled Tools for Internal Reasoning), a test-time framework that builds a compact library of state-conditioned correction vectors from the model’s own successful and failed rollouts. At inference time, STIR retrieves candidate corrections at structural reasoning checkpoints, validates them with a short lookahead probe, and abstains when the current trajectory already matches a high-quality state. Across six mathematical and knowledge-intensive reasoning benchmarks, STIR improves average accuracy by 1.9% to 7.5% over Vanilla decoding while reducing average token consumption by up to 35%. These results suggest that selective latent interventions can improve the accuracy-cost trade-off of reasoning without forcing every problem to generate longer chains. Our code is available at https://github.com/sznnzs/STIR.

Abstract:
Large language models (LLMs) deployed in real-world applications process heterogeneous inputs, including system prompts, user requests, and external content, exposing them to prompt injection attacks. Existing defenses, including prompt-level safeguards, external filters, and source-aware instruction handling, remain brittle against diverse direct and indirect prompt injection attacks. We propose CBC (Consistency Before Complying), a task-consistency alignment framework to defend against task-redirection prompt-injection attacks. CBC characterizes task consistency through semantic conflicts along action, constraint, and domain dimensions, then uses structured reasoning to determine whether newly introduced instructions are consistent with the original task before acting on them. Using targeted synthetic supervision, we train CBC to reason about task consistency and further strengthen this capability through dual-objective online alignment while preserving task utility. Across three model families and a diverse set of direct and indirect prompt-injection benchmarks, CBC improves defense success rates by up to 13.2 percentage points over the strongest baseline while better preserving task utility under attack. Our code and models can be found at: https://github.com/XOTaichi/CBC-PI.git

Abstract:
Transformer-based large language models often suffer from inter-layer parameter redundancy, where functional transformations are redundantly learned across network depths. We propose CS-MoE, a novel Transformer architecture featuring cross-layer expert sharing to address this inefficiency. Deviating from the widely used Mixture-of-Experts (MoE) architecture that terminates each Transformer block with layer-isolated experts, CS-MoE combines layer-independent experts with concurrent access to a centralized, globally shared expert pool. This Global Experts Sharing mechanism enables elastic control over token-level parameter activation and computational consumption (FLOPs). Experiments demonstrate that CS-MoE achieves lower perplexity than equal-scale dense Transformers while activating only 55% of parameters. Furthermore, its performance scales monotonically with an increased number of activated experts and approaches MoE counterparts that consume more FLOPs by expanding the shared pool without increasing FLOPs. CS-MoE also establishes a flexible Pareto frontier between computational cost and model capacity, offering an efficient alternative for computation-constrained environments.

Abstract:
Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control latency, deployments routinely compress inputs, but this degrades task accuracy. Speculative decoding (SD) accelerates generation losslessly, yet it assumes the drafter and verifier share an identical context, preventing SD from resolving the accuracy–overhead trade-off. We propose AsymSpec, an asymmetric speculative decoding framework that breaks this symmetry: a lightweight drafter reads the full input while the large verifier operates on the compressed view. The drafter steers the verifier via a contrastive 𝛿 -fusion of logits, modulated by a divergence-aware acceptance gate that preserves verification stability and high draft acceptance rates. Evaluated across four agentic capabilities and two end-to-end agent benchmarks, AsymSpec reaches ≈ 90% of full-context accuracy on average, delivering throughput speedups of 1.3× – 1.7× at 0.2× – 0.3× the compute cost on isolated text capabilities. These results show that asymmetric context access yields substantial gains precisely when compression discards critical reasoning signals.

Abstract:
Standard semi-autoregressive decoding for full-attention masked diffusion language models (MDLMs) computes predictions for future blocks but discards them until those blocks become active. When introduced as soft-future context, these predictions reveal a stability contrast: hypotheses later revised during decoding exhibit larger distributional shifts than those eventually retained. We introduce FUSA (Future-aware Semi-Autoregressive Decoding), a training-free decoder that exploits this contrast by alternating the next block between masked and soft representations and committing active-block tokens that remain stable across the two views, without additional model forward calls. On LLaDA-8B-Instruct, FUSA improves scores across reasoning and code-generation benchmarks while delivering 2.10– 3.94× wall-clock speedups over Top-1 decoding, with gains of up to 5.1 points. Additional results show that FUSA transfers to other MDLMs and remains effective under KV-cached inference and in long-form instruction following. Our code is available at https://github.com/yangryeol/FUSA.

Abstract:
In speech language modeling, two architectures dominate the frontier: the Transformer and the Conformer. However, it remains unknown whether their comparable performance stems from convergent processing strategies or distinct architectural inductive biases. We introduce Architectural Fingerprinting, a probing framework that isolates the effect of architecture on representation, and apply it to a controlled suite of 24 pre-trained encoders (39M–3.3B parameters). Our analysis reveals divergent hierarchies: Conformers implement a "Categorize Early" strategy, resolving phoneme categories 29% earlier in depth and speaker gender by 16% depth. In contrast, Transformers "Integrate Late," deferring phoneme, accent, and duration encoding to deep layers (49-57%). These fingerprints suggest design heuristics: Conformers’ front-loaded categorization may benefit low-latency streaming, while Transformers’ deep integration may favor tasks requiring rich context and cross-utterance normalization.

Abstract:
Autoformalization translates informal mathematical theorems into code for proof assistants such as Lean. A central challenge is that current evaluation metrics can accept type-correct but misaligned statements or reject correct statements written in a different formulation. Inspired by Pass@ k , we propose SA-PASS (Semantic Alignment Pass), which tests formal statements using auxiliary statements called shadows that characterize the intended statement. A generated statement receives full credit only when it compiles, implies each shadow (forward check), and is implied by their conjunction (backward check). We instantiate SA-PASS in ShadowBench, a Lean 4 full autoformalization benchmark of 178 postgraduate- to research-level problems spanning eight mathematical areas. Claude Code (Opus 4.8) with Numina-Lean-Agent reaches 61.8% compile rate and 11.2% SA-PASS. Across outputs generated by six agentic configurations, SA-PASS achieves 98.8% binary agreement with expert judgments. An early version of ShadowBench served as the benchmark for Track 4 of the ICML 2026 AI4Math Challenge.

Abstract:
Large language models are increasingly used through multi-turn refinement loops, but most evidence for iterative improvement comes from settings with informative feedback, external tools, full conversation history, or structured self-critique. Less is known about a configuration, uninformative memoryless refinement, where a model repeatedly revises only its previous output under vague instructions such as “improve it”. In this paper, we establish a framework to analyze when this uninformative memoryless refinement configuration improves LLM responses across ideation, coding, and mathematical reasoning. We evaluate seven LLMs on a 450-task pool under vague and domain-specific feedback. We score every turn and measure both task outcomes and trajectory dynamics, including semantic drift, volatility, lexical novelty, and output growth. This trajectory view reveals a consistent discovery–retention gap: models often produce better intermediate outputs before overwriting them. The effect is most severe in coding, where correctness falls from 40.7% at T1 to 2.5% at T12, while best@12 exceeds final@12 by 42.5 percentage points. Math shows model- and prompt-dependent behavior, with GPT-OSS-20B improving under elaboration but several models drifting downward. Ideation benefits from explicit steering, but vague feedback produces rhetorical inflation and feasibility loss.Our analysis stresses the importance of turn-wise, trajectory-aware evaluation of uninformative memoryless refinement and shows why verified intermediate outputs should be retained rather than automatically replaced by later revisions.

Abstract:
Long-form chain-of-thought reasoning can improve LLM performance on complex tasks, but models often continue generating unnecessary reasoning after a correct answer has emerged. We call this behavior overthinking. We study this phenomenon from the perspective of GRPO-style reinforcement learning (RL) post-training, framing it as a training-time credit-assignment problem rather than merely a decoding-time stopping problem. In rollouts sampled at the onset of GRPO training, we observe that successful trajectories can exhibit a slightly higher degree of overthinking than unsuccessful trajectories for the same prompts. This early imbalance provides a starting point for an undesirable feedback loop: because GRPO assigns sequence-level credit, it cannot distinguish the solution-reaching prefix from the unnecessary continuation that lengthens a successful trajectory. Both receive positive update signal, allowing the initial imbalance to grow into more severe overthinking during training. To address this issue, we introduce Dynamic Rollout Editing (DRE), a training-time intervention for successful trajectories that continue thinking after answer emergence. DRE preserves the accepted verified prefix, edits the remaining thinking, and prefers the edited trajectory within the same RL group, weakening the preference signal for unnecessary thinking without penalizing the reasoning needed to reach the answer. Experiments across diverse tasks show the effectiveness of DRE.

Abstract:
Continued pre-training on domain-specific corpora has become the prevailing approach for adapting pre-trained language models (PLMs) to materials science. However, existing strategies typically treat scientific literature as independent samples, thereby overlooking the cumulative nature of scientific knowledge where novel discoveries systematically build upon prior research. To address this limitation, we propose Chronology-grounded continued pre-training for materials science (Chronos), which explicitly considers the chronological knowledge in terms of data construction and training schemes. Specifically, our framework traces the relevant prior articles by using citation-based information and constructs a chronological curriculum to progressively learn the materials knowledge. The comprehensive experiments demonstrate that Chronos outperforms state-of-the-art (SOTA) baselines on various materials science benchmarks and backbone models, achieving improvements of 4.7% on the Macro-F1 score. In-depth analysis shows that considering the cumulative nature of knowledge enhances both the quality of the corpus and the effectiveness of training schemes.

Abstract:
Full-duplex voice agents can now listen and speak at the same time, yet no benchmark asks whether they can further exploit this capability—proactively interrupting a user mid-utterance to correct a content error. We introduce the Model Interruption Bench (MIB), the first benchmark for real-time, content-aware error correction in voice agents. MIB frames the task as proactive correction of spoken chain-of-thought math: ~600 step-annotated solutions derived from GSM8K, played to the model under a streaming protocol that scores whether it interrupts in the right window with the right correction. To localize failure, we pair the full-duplex streaming evaluation with turn-based ablations that isolate error detection, and provide an automated LLM-judge pipeline with backchannel filtering and a fine-grained FP/FN sub-type taxonomy to analyze failure cases. Evaluating six voice agents, Freeze-Omni, Gemini 2.5/3.1 Flash Live, GPT-4o Realtime, Moshi, and PersonaPlex, we uncover a striking detection–interruption gap: the three most detection-capable models recognize 59–67% of errors in the turn-based condition but reach below 10% mid-speech TP% when forced to interrupt in real time. The bottleneck is not understanding errors but acting on them in time. Three behavioral archetypes emerge from the sub-type breakdown—passive, post-hoc responder, and aggressive—exposing real-time error interruption as a concrete open challenge for full-duplex voice agents.

Abstract:
Steering vectors (SVs) offer a lightweight way to control large language models (LLMs) at inference time by shifting hidden activations, providing a practical middle ground between prompting and fine-tuning. Yet SVs can be unreliable in practice. Some concepts are unsteerable, and even when steering helps on average it can backfire for a non-trivial fraction of inputs. Reliability also degrades in long-form generation and multi-attribute steering. We take a geometric view of these failures. A static SV applies the same update vector everywhere in representation space, implicitly assuming that the concept-improving direction is constant across contexts. When the locally effective direction varies with the current activation, a single global vector can become misaligned, which yields weak or reversed effects. Guided by this perspective, we propose Steering Vector Fields (SVF), which learns a differentiable concept scoring function whose local gradient defines the steering direction at each activation, making interventions explicitly context-dependent. This formulation supports coordinated multi-layer interventions in a shared, aligned concept space, and enables efficient long-form and multi-attribute control within a unified framework. Across multiple LLMs and steering tasks, SVF delivers stronger and more reliable control, improving the practicality of inference-time steering. The code is available at https://github.com/lab-flair/SVF.git.

Abstract:
We find that transformer concept representations systematically anti-concentrate in the spectral tail of the unembedding covariance, encoding word-level concepts in low-variance directions across a 17-model core suite and an expanded set of 22 semantic concept categories, with convergent replications from three independent extraction methods. Residual-stream difference-of-means vectors anti-concentrate in all 17 models (model-level one-sample t-test, p = 3.8e-9), and remain more tail-aligned than norm-matched random directions in 13 of 17; convergent support comes from sparse autoencoder (SAE) features (p = 4.5e-19 across concepts within a model) and linear probes on Llama and Qwen. We identify a dual geometry: activation-space concept directions anti-concentrate while static unembedding-row contrasts concentrate in high-variance directions (p < 10^-4). This investigation arose from testing whether the causal inner product of Park et al. (2024) aids cross-lingual concept transport; a matched-spectrum randomization across 17 models and four language pairs finds no evidence that Whitened Causal Alignment improves over spectral regularization alone (p = 0.95). Split-injection interventions, restricted to steering strengths at which both arms remain interpretable, show the predicted interference asymmetry in four of five models (paired Cohen’s d_z up to 1.19) with no significant reversal inside that regime, and POS-tag probing across eight models shows syntax preferentially encoded in the high-variance subspace in six of eight architectures, with a significant reversal in the Qwen 2.5 family. These results suggest transformers rotate semantic content into spectrally quiet regions during contextualized processing, where, in some architectures, interventions may reduce grammatical disruption relative to high-variance steering.

Abstract:
In real-world multimodal applications, different modalities often exhibit weakly aligned adaptation requirements, making it challenging for a shared adapter to effectively capture heterogeneous signals. Existing approaches typically use a unified low-rank adapter for concatenated multimodal representations, leading to competition among modality-specific adaptations and diminishing returns as the adapter rank increases. To address this limitation, we propose msLoRA, a modality-specific adaptation framework for efficient multimodal fine-tuning. msLoRA assigns independent low-rank adapters to different modalities, allowing them to capture specialized adaptation directions while maintaining efficient parameterization. Our theoretical analysis introduces the concept of cumulative information span, demonstrating that modality-specific adapters can expand the effective adaptation space and avoid the cross-modal disagreement typical of shared adapters. We evaluate msLoRA using multiple prominent open-source LLMs, including the Llama, Qwen, and Mistral families, across four multimodal tasks, showing consistent improvements in scalability and effectiveness for low-rank adaptation in multimodal LLMs. The source code is available at https://github.com/XiuzeZhou/msLoRA/ .

Abstract:
Annotator disagreement is widespread in NLP, particularly for subjective and ambiguous tasks such as toxicity detection and stance analysis. While early approaches treated disagreement as noise to be removed, recent work increasingly models it as a meaningful signal reflecting variation in interpretation and perspective. This survey provides a unified view of disagreement-aware NLP methods. We first present a domain-agnostic taxonomy of the sources of disagreement spanning data, task, and annotator factors. We then synthesize modeling approaches using a common framework defined by prediction targets and pooling structure, highlighting a shift from consensus learning toward explicitly modeling disagreement, and toward capturing structured relationships among annotators. We review evaluation metrics for both predictive performance and annotator behavior, and noting that most fairness evaluations remain descriptive rather than normative. We conclude by identifying open challenges and future directions, including integrating multiple sources of variation, developing disagreement-aware interpretability frameworks, and grappling with the practical tradeoffs of perspectivist modeling.

Abstract:
Clinical language and vision-language models increasingly generate diagnostic and treatment rationales, but those claims remain difficult to audit. Existing benchmarks usually score final answers over fully revealed records, without testing whether each claim is grounded in valid patient evidence or whether models can seek missing information efficiently. We introduce Provena-Med , a benchmark for evaluating provenance-faithful multimodal clinical reasoning over linked EHR tables, clinical notes, chest radiographs, and guideline rules. Models must produce a staged artifact comprising a problem list, differential, workup, therapy, and monitoring, in which each claim cites typed evidence units and may abstain when information is insufficient. Provena-Med supports both full-information evaluation and an interactive evidence-seeking track. We further propose MM-AIS, which measures citation validity, objective support under evidence-removal tests, salient-evidence recall, and, for open-weight models, sensitivity to cited evidence. Across ten open-weight models and five cohorts, the best attribution yield is only 0.42, citation hallucination is common, and diagnostic accuracy, safety, and provenance do not reliably co-vary. Provena-Med provides an auditable testbed for evaluating grounded clinical reasoning beyond final-answer accuracy.

Abstract:
Diffusion language models (LMs) enable parallel decoding, but still lag behind autoregressive (AR) LMs in accuracy, and behind multi-token prediction (MTP) methods in efficiency. This raises three questions: should diffusion LMs compete with AR LMs or be harmonized with them; can diffusion provide a stronger acceleration mechanism than MTP; and does diffusion decoding have sufficient long-term potential to justify further exploration? We study these questions through Nemotron-Labs-Diffusion, a tri-mode LM that unifies three decoding modes—AR, diffusion, and self-speculation (diffusion drafts, AR verifies)—within a single model to sustain high throughput across all concurrency levels. Our study shows that (1) AR and diffusion objectives are mutually beneficial rather than competing, (2) diffusion-based self-speculation achieves higher acceptance lengths and better system vs. per-user throughput trade-offs than MTP baselines, and (3) diffusion decoding shows strong potential for parallel decoding under an optimal sampler in a speed-of-light analysis. Based on these insights, we develop the Nemotron-Labs-Diffusion family at 3B, 8B, and 14B scales. Our models outperform state-of-the-art open-source AR and diffusion LMs in both accuracy and speed, e.g., achieving 6x tokens per forward over Qwen3-8B with comparable benchmark accuracy, translating to 4x throughput on SPEED-Bench measured with SGLang on an NVIDIA GB200 GPU.

Abstract:
LLMs are increasingly being used by users from diverse demographics with pluralistic values and distinct backgrounds. Maintaining safety while respecting value-laden personas is uniquely challenging because persona-conditioned requests are often not overtly malicious, unlike those in existing safety benchmarks. Alignment methods aimed at adversarial requests therefore may miss the harm that emerges from persona use, leaving models to either refuse legitimate value expression or blindly follow unsafe personas. We construct a dataset of 11k controversial topics paired with value-based personas mapping to value dimensions from the Schwartz Value Survey. Through an agentic pipeline that iteratively refines the topic and persona, we surface and collect harmful responses from well-aligned commercial LLMs with a more than 47% success rate. We use UNESCO’s global citizenship principles as a minimal common ground for distinguishing valid value expressions from harmful behaviors, and introduce a controllable alignment model that lets users balance persona adherence and safety at inference time. The resulting model is safer and more persona-adherent than baselines on most samples.

Abstract:
Continual VideoQA with multimodal LLMs remains challenging because sequential adaptation induces task interference, while storing task-specific prompts becomes impractical as task sequences grow. We introduce DynaTokens, a transformer-based token generator that dynamically produces fine-tuning tokens on demand, enabling task-adaptive prompt updates through shared generation weights. To mitigate forgetting, we introduce meta-learning-inspired regularisers that look ahead to avoid task-specific sharp update directions while anchoring the evolving generator to prior-task behaviours. We theoretically connect this objective to sharpness-aware optimisation, showing how it favours flatter cross-task minima and improves retention. DynaTokens combines gradient-free routing based on robust pretrained token and visual embeddings with lightweight auxiliary multimodal supervision, reducing router drift during continual adaptation. Across standard continual VideoQA benchmarks, DynaTokens achieves higher average accuracy and substantially lower forgetting than strong baselines. It also improves zero-shot generalisation and remains effective in longer domain-incremental sequences with extended task shifts. Finally, we introduce a challenging ImageQA->VideoQA protocol and show that DynaTokens enables robust cross-modal continual transfer.

Abstract:
Speech is a primary communication interface for humans, and thus a crucial modality for large language models (LLMs). Despite this, the vast majority of open speech-capable LLMs are developed for and evaluated on a small set of <20 high-resource languages, adding to the already massive inequity of modern language technology. Massively multilingual speech understanding is primarily hindered by the limited amounts of speech training data for most languages and tasks. For the lion’s share of the world’s languages, speech datasets for tasks other than automatic speech recognition (ASR) simply do not exist. In this work, we bypass the sparsity of task-specific speech data by leveraging text – much more available for many languages – to enable an LLM to understand speech input. Our approach, dubbed Text4Speech, leverages state-of-the-art multilingual pre-aligned text and speech encoders: we first align the multilingual text encoder to an LLM using abundant text data and then further align the speech encoder to the LLM using a limited amount of speech. In both steps, we train the alignment parameters via self-distillation on the LLM. Our evaluation on three multilingual speech understanding benchmarks that encompass 90 languages renders Text4Speech highly effective, yielding average improvements of up to 6% on Fleurs-SIB over a very strong ASR-based cascade baseline. Crucially, we show that our text-based alignment unlocks a highly sample-efficient alignment of the speech encoder to the LLM.

Abstract:
Eye-Tracking (ET) data provide valuable insights into human language processing and have recently been employed across a wide range of fields, from psycholinguistics to NLP. In this work, we introduce CERVINO, the largest Italian eye-tracking-while-reading dataset to date, comprising data from more than 200 participants reading legal texts, recorded at 120Hz. Our analyses show that the dataset captures established psycholinguistic effects: reading times are influenced by word length, lexical frequency, and surprisal. We further evaluate the usefulness of the dataset in downstream tasks by fine-tuning BERT on ET measures. Results show improved alignment between model attention and human reading behavior while preserving downstream task performance. Moreover, combining our data with higher-frequency ET resources yields the strongest alignment results, suggesting that lower-frequency ET data provide useful cognitive signals. Overall, these findings indicate that lower-frequency ET data retain sufficient cognitive information to provide useful supervision for attention alignment and complement existing high-frequency resources.

Abstract:
Retrieval Augmented Generation (RAG)’s success depends on the utility the LLM derives from the content used for grounding. Quantifying content utility does not have a definitive specification and existing metrics ignore model-specific capabilities and/or rely on costly annotations. In this paper, we propose Grounding Generation Utility (GroGU), a model-specific and reference-free metric that defines utility as a function of the downstream LLM’s generation confidence based on entropy. Despite having no annotation requirements, GroGU is largely faithful in distinguishing ground-truth documents while capturing nuances ignored by LLM-agnostic metrics. We apply GroGU to train a query-rewriter for RAG by identifying high-utility preference data for Direct Preference Optimization. Experiments show improvements by up to 18.2 points in Mean Reciprocal Rank and up to 9.4 points in answer accuracy.

Abstract:
Recent work reports large gains for LLM agents on low-budget molecular optimization, with 2-3 times AUC improvements over reinforcement-learning and genetic-algorithm baselines on the PMO benchmark and similar gains on protein-targeted docking benchmarks. We investigate whether these gains reflect genuine optimization or benchmark artifacts, by re-evaluating a representative LLM agent under controlled blinding protocols and against a trivial random-walk baseline. We find that on PMO at the 50-call budget, the agent’s apparent advantage collapses below classical baselines once task names and curated reference compounds are removed from its prompt; a 2 times 2 ablation shows that retrieval from a small reference dictionary, not iterative optimization, is the dominant driver of the headline number. On a new protein-targeted lead-optimization benchmark with an AutoDock Vina composite oracle, a single-edit halogen/methyl random-walk mutator outperforms the LLM agent, a graph-based GA, and a non-iterative GPT baseline on every aggregate metric—a property of the oracle landscape under tight budgets, not of LLM agents per se. We release both protocols (name-blinded PMO scoring; random-walk lower bound for Vina-based protein benchmarks) as standard sanity checks for future LLM-agent papers in molecular optimization.

Abstract:
Embedding-based code retrieval is a core component of coding agents and retrieval-augmented code generation, where retrieving correct code matters more than retrieving lexically similar code. Existing code-retrieval benchmarks do not plant controlled, execution-verified single-edit variants of each query’s canonical implementation in the search pool, leaving the question of whether embeddings can functionally discriminate correct from near-clone-but-incorrect code unanswered in a retrieval setting. Resolving this requires a benchmark whose search pool itself contains the relevant counterfactuals — execution-verified buggy variants near-identical to each canonical — so that a retriever’s rank ordering can be directly tested for functional discrimination rather than topical or identity overlap. We introduce ExecRetrieval, 939 Python tasks each paired with one execution-verified canonical implementation and up to four execution-verified buggy distractors, each generated by a mechanical mutation making a single targeted edit, and evaluate 23 dense embedding configurations plus BM25 under provider-native invocation with paired McNemar tests and query-level bootstrap intervals. With near-clone counterfactuals in the pool, the top hosted system reaches exec@10 = 1.00 but only exec@1 = 0.331; rank-1 misses are paired buggy variants 91.5–99.4% of the time across the four leading systems, and the canonical scores below at least one of its four paired distractors in 67–78% of queries on the leading systems. The full dataset, execution oracle, embedding matrices, environment snapshot, and pairwise statistical tests are released at the URL in Appendix D.

Abstract:
Phonological competence - knowledge of how surface characters encode underlying pronunciation - underpins applications from poetry generation to speech recognition. Although text-only Large Language Models (LLMs) receive no direct phonetic supervision, prior work shows that they nonetheless develop such competence in English. It remains, however, unclear (1) whether this generalizes across languages and scripts and (2) how this text-based phonological knowledge interacts with the audio processing competencies of speech-augmented LLMs (SLLMs). In this work, we introduce mPhonBench, a massively multilingual benchmark that probes text-only and speech-augmented LLMs for phonological knowledge, spanning 10 tasks, 58 diverse languages, and 19 scripts. While mPhonBench is text-centric, we complement its nine text-only tasks with word-level phone recognition (PR) to jointly analyse text- and speech-based phonological knowledge in SLLMs. We evaluate a wide selection of open-weight LLMs on mPhonBench, and find that: (i) mid-sized LLMs match or surpass expert systems on grapheme-to-phoneme conversion (G2P) and generalize beyond memorization; (ii) underrepresented languages incur a performance penalty; (iii) while this penalty can be partially offset via in-context learning with few task examples, models struggle to exploit, in-context provided, general phonological descriptions of languages; and (iv) frontier SLLMs are competitive with expert models on word-level PR, but the capability is not linked to model size, suggesting that training regimes play a key role in developing audio-based phonological competence.

Abstract:
Large language models (LLMs) are increasingly deployed on mobile devices, making energy efficiency a key deployment constraint, yet the energy impact of prompt design remains underexplored. This paper aims to understand how two prompt properties, cognitive load and phrasing pattern, shape the energy behavior of on-device LLM inference. We conduct a broad empirical study covering prompt properties, datasets, models, and devices, with phase-level profiling that separates prefill and decode energy. We find that cognitive load primarily affects the energy cost per token, while phrasing pattern affects energy largely through token usage. Our energy-quality analysis further shows that prompt design reshapes the attainable frontier differently across models, highlighting the need for model-aware prompt design in energy-efficient on-device LLM inference. Code, datasets, and scripts are available at https://amai-gsu.github.io/PromptProperty/.

Abstract:
How do learners avoid overgeneralizations such as Tom laughed me without explicit negative evidence? Constructionists have posited two proposals that describe indirect negative evidence against overgeneralizations: preemption (which privileges exposure to near-synonymous construction—e.g., she made him laugh) vs. entrenchment (all exposures to a verb’s grammatical usages, including cases like He laughed). We disentangle these hypotheses by running controlled rearing experiments on LMs trained on child-caregiver conversations, where we systematically remove preemptive vs. non-preemptive evidence. We find that while LMs avoid overgeneralizations, they do not show preemption at a verb-specific level, instead showing weak but non-zero evidence of abstract preemption. Combined with results from analyzing the LMs’ training dynamics, we find that LMs treat competing structures as indirect positive—as opposed to negative—evidence in the verb-specific condition. Insofar as preemption is the more plausible route to avoiding overgeneralizations in humans, our results point the need for there to be sensitivities to indirect negative evidence in neural network learners, and suggest new human experiments to test abstract preemption.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) enhances reasoning in large language models (LLMs) but typically depends on human-annotated answers or curated reward specifications. Label-free alternatives such as majority voting or LLM-as-a-Judge remove this cost but introduce false positives that destabilize training. We introduce JURY-RL, a label-free RLVR framework that decouples answer proposal from reward disposal: rollouts matching the plurality-voted answer receive positive reward only when that answer is verified in Lean. When verification is inconclusive, ResZero (Residual-Zero) discards the unverified plurality and redistributes a zero-mean, variance-preserving signal over the residual answers, keeping the optimization gradient stable without reinforcing unverifiable consensus. Across three backbones trained on math data, JURY-RL outperforms label-free baselines on mathematical reasoning, transfers to code generation and general benchmarks, and surpasses a majority-vote baseline in the test-time RL (TTRL) setting; it matches supervised ground-truth training on pass@1 and beats it on pass@k and response diversity.

Abstract:
Coding tasks are typically complicated and require multiple capabilities, ranging from high-level planning to low-level implementation. While coding agents are optimized for the joint capabilities, individual capabilities may have different optima, and high-level planning remains a major bottleneck. To address this challenge, we train a separate critic model that is specialized in high-level planning to steer the coding agent in inference. We construct SFT and DPO data to train the critic model to identify errors made by the coding agent and provide correct and clear high-level guidance without generating concrete actions. Experiments show that our fine-tuned 4B and 8B critic models significantly improve the performance of 6 larger coding agents (e.g., improving the resolved rates of GLM-4.7-Flash-30B-A3B and GPT-OSS-120B by 16.0% and 14.4% on SWE-Bench Verified). The critic model also reduces the total inference costs for some coding agents by solving tasks in fewer steps (e.g., reducing the per-example inference cost for GPT-OSS-20B from 0.07 to 0.03). We provide the code in https://github.com/shubhamrgandhi/critic-training.

Abstract:
An empirical bottleneck for Natural Language Generation tasks such as summarization is the lack of accurate and robust evaluation metrics for many languages, which hinders progress. While Large Language Models (LLMs) are increasingly adopted as evaluators with the advent of LLM-as-a-Judge approaches, their agreement with human judgments remains low across many languages. We take inspiration from recent studies suggesting that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance during text generation. Motivated by this hypothesis, we ask whether steering the activations of neural metrics with respect to an English pivot can improve their correlation with human judgments in multilingual summarization evaluation. We experiment with encoder- and decoder-based metrics across five languages. We find that activation steering is effective for neural metrics, and that a single, language-independent steering vector improves metric–human agreement. Steering also results in improvements when applied to COMET, a small encoder-based metric, showing notable human agreement despite its small scale.

Abstract:
Speech Language Models (SLMs) are increasingly deployed in multi-speaker environments, yet their ability to attribute speech to the correct speaker and reason over speaker identities remains unclear. Hence, we introduce HEAR, a conceptually hierarchical benchmark diagnosing the foundational capabilities of speaker-attributed reasoning, comprising 2.4K human-verified samples from 887 diverse multi-party audio clips. Evaluating 20 leading SLMs on HEAR reveals they struggle with these foundational tasks, often relying on semantic priors rather than actual vocal cues. To address this, we present A2R, a 30B model optimized on Counterfactual Audio with Speaker-level Hard negatives (CASH), a dataset designed to guide the model to prioritize acoustic vocal cues over linguistic signals. A2R achieves strong performance on HEAR and exhibits zero-shot generalization to diverse multi-speaker downstream tasks, demonstrating that learned speaker attribution unlocks the model’s latent capacity for speaker-aware reasoning. All resources are available on our https://attributetoreason.github.io/AttributeToReason/

Abstract:
Recent progress in vision-language models has enabled processing of increasingly long video sequences, but handling extended token streams does not translate to understanding complex narrative structure in long videos. Existing long-video benchmarks focus on needle-in-a-haystack retrieval rather than evaluating how low-level actions form events, interact across time, and drive narratives, for example whether a model can connect an early job loss to a later relationship breakup despite intervening scenes or flashbacks. We introduce NEST (Narrative Event Structures in Time for Long Video Understanding), a dataset of 1,005 full-length movies (avg. 98 minutes), each annotated with about 103 multimodal narrative events grounded in visual content, dialogue, and audio. NEST links these events through relations that reflect narrative structure, including explicit temporal ordering, hierarchical composition, and long-range dependencies. We introduce baselines for event trigger detection (ETD), event localization (EL), event argument extraction (EAE), and event relation extraction (ERE). The benchmark is highly challenging for grounded event discovery, with the best performance across evaluated zero-shot models on main evaluation subsets remaining below 8% for ETD, 6% for EL, and 11% for EAE. In contrast, ERE is more tractable once events are given, reaching 34.23% F1 zero-shot and 49.20% F1 after fine-tuning.

Abstract:
Multimodal foundation models edit natural photographs at production quality, yet the same models struggle with structured visual content such as infographics. Unlike photographs, infographics encode information through logical relations; editing one element often requires surrounding elements to be adapted. We refer to this global layout reasoning capability as reflow. Existing image-editing benchmarks neither provide a dedicated setting for structured visual content nor evaluate the reflow capability. We introduce InfoEdit, a novel benchmark of 1,000 infographics across eight logical-relation families, paired with 4,000 editing instructions across four editing tasks, and a reflow-aware evaluation protocol. Across eight frontier editors, only GPT-Image-2 clears 60% average success rate; most models fall below 7%, and no editor exceeds 36% on Swap-Block task even with perfect target localization. We further show that code-level editing can match the strongest pixel-level editor, revealing complementary strengths across tasks. InfoEdit identifies reflow as a central challenge in structured visual content editing and provides a diagnostic benchmark to facilitate future progress.

Abstract:
Agentic serving can consume orders of magnitude more tokens than chatbot workloads, stressing both KV-cache capacity and decode-time bandwidth. Most KV-eviction methods score cached keys against representative queries drawn from the most recent tokens, assuming future attention resembles recent attention. We show that agentic generation violates this assumption: future queries form a mixture over think, act, tool, and others phases, and principal-angle analysis shows these components occupy measurably different query subspaces, so recency representatives systematically undervalue keys that upcoming phases will need. We propose AgentKV, which maintains a small query buffer per phase and scores cached keys against their union. We further implement AgentKV in a persistent multi-turn serving path that carries compressed KV state across turns and compacts retained KV pages online. Across two models, six task domains, and three KV budgets each, AgentKV improves task score by 5.5 points on average over R-KV and 5.3 over Tri-attention. Relative to upstream full-KV SGLang, AgentKV improves output-token throughput by up to 1.80× . Code: https://github.com/LiuTaowen-Tony/agentkv.

Abstract:
Large language models (LLMs) deployed as decision agents in regulated settings—loan approval, hiring, clinical triage—must pass pre-deployment fairness audits that report a single fairness number per model under one implicit configuration. We argue this framing misses the dominant source of variation: configuration itself. Sweeping five axes (persona, interview, deliberation, aggregation rule, order) across three frontier LLMs (Claude Sonnet 4.6, GPT-4o-mini, Grok 4.20 Reasoning) on a 1,000-applicant Adult Census Income sample and a 500-applicant ACSIncome (2024 ACS data) sample, we find that within a single model, persona alone swings positive-prediction rate by up to 91 percentage points and collapses decision rules on two of three models; within a Conservative/Balanced/Progressive panel, replacing the panel chair on identical deliberation transcripts moves the disparate-impact ratio from adverse-impact to near-parity; and reversing the speaking order flips up to 34% of applicants on the same panel. We identify a median-voter effect: the Balanced agent matches the majority-vote verdict on 95.8–100% of applicants across all three models—deliberative aggregation reduces, mechanically, to single-agent decision by the median. The audit unit is the configuration, not the model: a system can pass an audit under one configuration and be silently reconfigured into a non-compliant one through a prompt edit—a change that current regulations do not treat as requiring re-audit. The code to reproduce the experiments in this paper is available at https://anonymous.4open.science/r/agent-decision-bias-emnlp-F059/

Abstract:
Fine-tuning large language models on domain data is a standard practice, yet vanilla pipelines rely on a costly data mixture sweep to avoid catastrophic forgetting of general capabilities. We propose Dynamic Anchor Annealing (DAA), a full-parameter fine-tuning framework that decouples general and domain learning at the gradient level rather than at the data level. DAA estimates a global gradient anchor once on general data through zero learning rate training, anchors the domain gradient toward this anchor during fine-tuning, modulates step sizes by the cosine similarity between the two gradients, and combines this with annealing learning. Under this design, fine-tuning relies solely on domain data and does not lose general capability. Across four domains and three base models, DAA improves the joint domain and general harmonic mean by 12.5 points on average over direct fine-tuning and by 3.8 points over vanilla data mixture fine-tuning, and the margin widens to 66.9 points on a temporal out-of-distribution setting where direct fine-tuning collapses. Our headline 91% reduction in GPU hours is a workflow level saving from removing the mixture ratio sweep rather than a per run speedup, since the per run cost of DAA is comparable to direct fine-tuning.

Abstract:
Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, creating a need for fact-checking systems that can verify claims directly from speech. We introduce VERISPEAK, a probe benchmark for studying speech-based fact verification in Large Audio Language Models (LALMs). VERISPEAK contains 3,879 spoken claims spanning temporal, geographical, and relational facts, with balanced true and false labels. The benchmark is designed to examine whether factual verification ability transfers from text to speech, and whether retrieval-augmented LALMs can use textual evidence to correctly support or refute spoken claims. Our experiments reveal a consistent text–speech modality gap: LALMs that verify written claims reliably often fail on the same claims when spoken. Moreover, retrieval alone provides limited gains because models frequently conflate retrieved evidence with the spoken claim. In contrast, retrieval combined with explicit reasoning improves claim–evidence comparison, with a thinking-tuned LALM reaching 86.1% accuracy. VERISPEAK highlights that effective speech misinformation detection requires not only speech understanding, but also grounded reasoning over retrieved evidence.

Abstract:
Despite substantial progress in safety alignment, aligned Large Language Models (LLMs) remain fragile under small parameter-space and activation-space perturbations, such as downstream fine-tuning and jailbreak attacks. To explain why local perturbations can trigger broad safety failures, we analyze safety-task neuron circuits from the perspectives of activation flow and circuit topology. We identify a critical topological fragility in safety alignment: In aligned models, safety-task neuron circuits exhibit a low-redundancy, isolated single-chain topology, in sharp contrast to the highly redundant dense-mesh topology of general-task circuits. This brittle topology creates rigid serial dependencies, whereby perturbing a small set of upstream neurons can disrupt downstream safety neurons and cascade into broad alignment failures. To mitigate this topological fragility, we propose Dense Safety Mesh (DSM), a topology-aware alignment framework that transforms brittle single-chain safety circuits into more redundant and resilient mesh-like structures. DSM employs structural adversarial masking to suppress dominant safety pathways, forcing safety circuits to develop diverse parallel pathways. Extensive experiments across seven models and multiple evaluations show that DSM improves robustness against perturbations attacks while maintaining general capabilities.

Abstract:
Language-model agents increasingly rely on skills: reusable procedural knowledge for reasoning, tool use, and interaction. Existing work studies how skills are acquired, retrieved, compressed, or composed, but often assumes that once a skill is selected, its interface to the agent is fixed. We argue that this overlooks a key source of skill utility: the same skill can help, distract, or mislead depending on how it is exposed. We propose SkillAlign, a provider-agnostic framework that represents candidate skills as multi-view procedural cards and renders them through alternative exposure interfaces, including full instructions, hints, compressed summaries, workflows, or no exposure. This enables counterfactual evaluation where the task, agent, and candidate skills are fixed while only the exposure interface varies. Across ALFWorld and SkillsBench, we show that exposure form substantially affects task success and rendered context cost, and that compact top-k exposure can outperform full-library injection. We further conduct a replay-based policy-learning analysis on ALFWorld, showing that adaptive exposure contains learnable signal but remains far from oracle selection. Our results suggest that skill-augmented agents should optimize not only which skills to use, but also how those skills are presented.

Abstract:
Group Relative Policy Optimization (GRPO) significantly enhances the reasoning performance of Large Language Models (LLMs). However, this success heavily relies on expensive external verifiers or human rules. Such dependency not only leads to significant computational costs and training latency, but also yields sparse rewards that hinder optimization efficiency. To address these challenges, we propose Latent-GRPO, a framework that derives intrinsic rewards directly from latent space geometry. Crucially, our empirical analysis reveals a compelling geometric property: terminal token representations of correct reasoning trajectories form dense clusters with high intra-class similarity, whereas incorrect trajectories remain scattered as outliers. In light of this discovery, we introduce the Iterative Robust Centroid Estimation (IRCE) algorithm, which generates dense, continuous rewards by mitigating magnitude fluctuations via spherical projection and estimating a robust “truth centroid” through iterative aggregation. Experimental results on multiple datasets show that our method maintains model performance while achieving a training speedup of over 2x compared to baselines. Furthermore, extensive results demonstrate strong generalization ability and robustness.

Abstract:
A model that refuses once may not refuse again. We introduce MAZE (Multi-turn Adversarial Zone Exploration), a black-box jailbreak framework that adapts its prompts to the target model’s responses across a dialogue. Across ten state-of-the-art models, MAZE achieves 50% attack success on commercial systems and 74% on open-source systems, with the highest observed average ASR and average rank among evaluated baselines. Ablations show that accumulated conversational context is the primary driver: removing target-side history reduces ASR by 32.9% on open-source models and 56.1% on commercial models, revealing strongly path-dependent safety behaviour. When refusals occur, adaptive recovery helps restore conversational progress through reframing. We identify three recurring interaction patterns: legitimacy construction, context-dependent risk calibration, and helpfulness–safety tension, that help explain why follow-up requests can receive different safety treatment from isolated requests. Claude-Sonnet-4.5 is a notable exception, maintaining refusals across most reframing attempts. These findings motivate evaluating alignment across full multi-turn trajectories rather than at the point of initial refusal.

Abstract:
Measuring the value orientations of LLMs is helpful for diagnosing model risk and understanding their misalignment behavior. However, existing value benchmarks are often AI-generated and lack validity verification, leading to noisy and unreliable results. Inspired by Psychometrics, we propose a novel Subset Item Response Theory (S-IRT) model for assessing LLMs’ values under any value theories. S-IRT directly models the subset of values jointly expressed in each LLM response, rather than reducing responses to binary outcomes as in traditional IRT-based ability assessment. Meanwhile, it jointly estimates the LLM’s latent values and captures the relative priorities among them. Based on it, we further design a tailored information reward that estimates the information gain of each item in reducing uncertainty about the evaluated model’s value, and thus helps select a small set of high-quality items from existing benchmarks. Extensive experiments on 23 popular LLMs and three value systems (21 value dimensions) demonstrate S-IRT can be applied to multiple value systems simultaneously and improve evaluation reliability and validity. The information reward also helps reduce the number of questions to as few as 200 while preserving comparable evaluation effectiveness.

Abstract:
Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context windows do not necessarily translate into effective comprehension and representation of long-context multimodal inputs, which remains a critical bottleneck for real-world deployment. To address the lack of systematic evaluation in this setting, we introduce MMLongEmbed, the first comprehensive benchmark for evaluating MEMs in long-context scenarios. MMLongEmbed comprises four retrieval tasks spanning multiple context-length ranges, covering text, document, and video modalities. Through extensive evaluation of state-of-the-art models, we find that current architectures rely heavily on superficial feature matching and struggle to capture deep semantic and structural dependencies. We further observe that performance degradation varies systematically with context length and key information placement. Moreover, models exhibit substantially different robustness to redundant contextual information across modalities.

Abstract:
Table–Text question answering asks systems to answer natural-language questions whose evidence is split between structured tables and linked unstructured passages. Many realistic questions in this setting are coverage-sensitive: the answer depends on a set of evidence items distributed across table rows and linked passages, and the same failure mode that appears in aggregation also surfaces in joins, multi-row filters, list questions, and universal or existential queries. Existing hybrid systems retrieve and read such evidence, yet leave passage-derived facts as strings in the LLM context rather than as typed, joinable, and repairable relational values that SQL can operate on. We propose TRACE (Table–Text Relationalization And Completion of Evidence), a completeness-aware query-processing framework built on two relational state transformations. Schema-Guided Relationalization (SGR) turns linked passages into typed, foreign-keyed side tables, exposing textual evidence as SQL-visible relational values; Question-Conditioned Relational Completion (QCRC) then repairs this state per question by adding missing attributes and filling incomplete cells. SQL performs set-oriented operations over the completed state, while a lightweight passage fallback handles residual gaps. On the coverage-sensitive SPARTA benchmark, TRACE more than triples the strongest baseline F1 (60.5 vs. 19.2 with gpt-4o-mini); on lookup-heavy HYBRIDQA, it stays competitive while substantially reducing the drop on questions that require many evidence items.

Abstract:
Medical knowledge is continuously evolving. This creates a need to update or selectively forget information encoded in already-trained medical LLMs. Machine unlearning aims to remove the influence of specific training data from a model without full retraining. Yet, existing unlearning benchmarks rely on synthetic or small-scale general data, leaving clinical unlearning understudied. We introduce AMNESIA, the first large-scale, open source benchmark for medical unlearning, with 70,560 question-answer pairs (QAs) from 8,820 patient notes across 11 disease categories. AMNESIA includes both factual questions testing direct recall and reasoning questions testing clinical inference. We use it to evaluate four widely used unlearning methods at both random patient and disease-level, and introduce a new metric for detecting leakage of medical terminology. We show that unlearning individual patients erodes knowledge of others with the same condition, calling for methods that can better separate patients from shared clinical knowledge.

Abstract:
Question answering (QA) systems should remain stable across meaning-preserving variations of the same problem. Earlier robustness work showed brittleness to local lexical perturbations, such as synonym replacement, word swaps, insertions, and deletions. Revisiting this robustness axis for LLMs, we find that high-performing models are now comparatively stable to such edits. However, this stability does not extend to discourse-level rewrites: answer-preserving transformations that alter a question’s structure and presentation while preserving its intent and gold answer. Across reasoning-focused QA benchmarks, these rewrites induce substantially larger performance drops than local perturbations. They also change how models fail: using a compact failure taxonomy and Total Variation distance, we show that discourse-level rewrites systematically redistribute error types, revealing distinct failure regimes. Our results suggest that local lexical perturbations remain useful, while discourse-level rewrites provide a complementary and stronger diagnostic signal for evaluating reasoning stability in QA LLMs.

Abstract:
Multilingual LLMs answer the same multi-hop reasoning question across languages, but we lack a mechanistic account of whether they share an internal circuit. We identify Bridge Routing Heads (BRH) in two large multilingual LLMs through a three-stage pipeline. The resulting language-specific head sets exhibit near-complete mutual exclusivity across the five languages, with a mean Jaccard similarity of only 0.017 for Llama 3.1 70B and 0.057 for Qwen 2.5 72B, revealing language-idiosyncratic circuits. Ablating general BRH increases two-hop Negative Log-Likelihood (NLL) by 39-89 × the random-head baseline, providing direct causal evidence of their role. Amplifying these heads in a failing target-language pass rescues up to 51.7% of cross-lingual failures, with no training. The two models share this dual-circuit pattern but allocate heads differently: Llama concentrates chaining in a large general pool, while Qwen leans on larger language-specific pools. Together these results show that activation-level intervention alone can recover correct answers from cross-lingual reasoning failures.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning using outcome-level supervision, but this supervision is temporally coarse: it indicates whether a trajectory ends with a correct answer, without revealing when the reasoning becomes reliable or silently fails. Existing verbalized-confidence methods typically calibrate a single confidence score after the final answer, leaving intermediate uncertainty unobserved. We propose SCOPE (Stepwise Calibration with Outcome-grounded Prefix Evidence), which trains reasoning models to emit confidence after each reasoning step. SCOPE interprets step confidence as the model’s prefix-conditioned belief in the gold answer: how likely the gold answer is under the reasoning prefix generated so far. During training, SCOPE constructs a soft target by scoring the gold answer conditioned on each reasoning prefix under the current policy. It then distills this signal with a two-stage masked RL objective: the first stage stabilizes the output format and final-answer accuracy, while the second stage performs calibration-only masked updates by applying stepwise and final-confidence rewards only to the numerical tokens inside the emitted confidence spans. Experiments on mathematical reasoning benchmarks show that SCOPE improves stepwise and final-confidence calibration while preserving final-answer reasoning performance. We further show that the resulting confidence signals support downstream inference-time control, enabling more reliable Learn-Then-Test risk control and stronger confidence-weighted aggregation.

Abstract:
Automatic Survey Generation (ASG) aims to produce comprehensive literature surveys by retrieving, organizing, and synthesizing academic papers. Despite rapid progress in specialized ASG frameworks and Deep Research agents, existing evaluations largely center on Computer Science or rely on generic criteria, leaving it unclear whether current systems satisfy the survey standards of diverse disciplines. We introduce SurveyLens, the first discipline-aware ASG benchmark. SurveyLens comprises SurveyLens-1k, a curated dataset of 1,000 human-written surveys across 10 disciplines, and a dual-lens framework that combines discipline-aware rubric scoring with reference-based alignment to human-written surveys. Evaluating 11 state-of-the-art systems spanning vanilla LLMs, specialized ASG systems, and Deep Research agents, we find that Deep Research agents perform most consistently across the 10 disciplines, some specialized ASG systems show particular strength in outline generation, and reference coverage and relevance remain a shared bottleneck across paradigms. These findings further provide practical guidance for discipline-specific system selection and future ASG design.

Abstract:
Effective spoken AI assistants must decide not only what to say, but also when to say it. Although appropriate AI turn-taking behavior is inherently scenario-dependent, current approaches fall short in complementary ways: human-human speech corpora capture natural timing phenomena but provide little AI-role grounding or scenario-specific norms, while heuristic or prompted synthesis methods inject turn-taking behaviors without grounding them in human preferences. We introduce DuplexGen, a human-calibrated framework that generates synthetic human-AI dialogues with scenario-adaptive turn-taking. DuplexGen transforms text dialogues into spoken-style transcripts, marks candidate intra-utterance action slots, and calibrates LLM predictions using a small set of slot-level human preference annotations. Across six tasks spanning cooperative and competitive interactions, DuplexGen aligns substantially more closely with human preferences than uncalibrated prompting and models trained solely on generic human-human data; furthermore, a full-duplex model trained on DuplexGen-generated data exhibits distinctive, human-preferred turn-taking behaviors across scenarios.

Abstract:
Large language models (LLMs) are increasingly deployed as personalized assistants that interact with users over extended periods of time. As conversations grow longer, relying on full interaction histories becomes increasingly inefficient and unreliable, motivating memory systems that selectively store and retrieve user-specific information. In realistic interactions, users often seek practical guidance such as recommendations, planning, and decision support. Unlike factual recall tasks, personalized guidance requires models to integrate information across multiple past conversations and reason about changing user preferences and experiences. However, existing conversational memory evaluations mainly focus on retrieval and factual recall. To study this challenge, we introduce PRAGMA, a benchmark for evaluating personalized guidance in long-term conversations. PRAGMA contains curated longitudinal conversation histories, evidence annotations, and guidance scenarios grounded in evolving user contexts and incorrect user assumptions. Experiments across retrieval systems, memory systems, and long-context models reveal that current systems struggle both to recover the appropriate conversational evidence and to effectively use it for personalized guidance. Our results highlight the need for memory architectures that support robust conversational retrieval and memory-grounded reasoning beyond evidence recall.

Abstract:
Mitigating social bias in large language models (LLMs) has become an increasingly important research objective. However, existing methods often rely on prompt-level control or optimize bias-aware guidance and final prediction within a single model, which may lead to coupled optimization and task interference. To address these limitations, we propose MARD, a Multi-Agent Reinforced Debiasing framework that decomposes debiasing into two role-specialized agents. The Bias-Guard Agent identifies stereotype cues and generates bias-control guidance, while the Evidence-Decision Agent produces evidence-grounded predictions without relying on identity-based assumptions. MARD further introduces retrieved group-related references as negative warning signals to expose potential stereotype cues, and adopts an alternating GRPO optimization strategy to improve role specialization. Experiments on five datasets across four task types demonstrate that MARD achieves strong overall debiasing performance. Further analyses highlight the effectiveness of role decoupling, alternating optimization, format control, and retrieved warning references.

Abstract:
Identifying the speaker of each dialogue instance is a key step in manga understanding, supporting applications such as translation, summarization, and voice generation. In zero-shot settings, the task is challenging because each new manga title introduces an unseen cast and provides no speaker annotations, predefined character identities, or per-title model adaptation. Existing methods, including iterative multimodal fusion, still organize speaker reasoning around manga-specific visual evidence such as characters’ appearance and local text-character relationships. However, such evidence is highly variable in manga, where speakers may be off-panel, character appearance changes across poses and styles, and local visual relations can be misleading. We propose Speaker-Seeded Identity Consolidation (SSIC), which reframes zero-shot manga speaker attribution as book-level identity consolidation driven by recurring dialogue consistency rather than local visual-region propagation. SSIC induces local anonymous speaker hypotheses from ordered dialogue context, aligns them into cross-window speaker seeds, constructs a global speaker identity registry, and performs registry-constrained attribution. In the main controlled evaluation on Manga109Dialog, SSIC achieves 74.62% speaker attribution accuracy, improving over iterative multimodal fusion by 7.02 points. The gains are largest on hard cases where local text-character relationships are unreliable, and ablations confirm that cross-window speaker seeds are central to registry construction.

Abstract:
Modern large language models (LLMs) achieve state-of-the-art machine translation performance, but they do so as broad generalists largely trained for many tasks and capabilities unrelated to translation. Thus, they are heavily overparameterized for this task, resulting in excessive memory and compute requirements. In this paper, we present a method for aggressively pruning experts from modern mixture-of-experts LLMs while incurring negligible degradation in translation quality. Our approach exploits expert specialization and the separability of multilingual capabilities in LLMs to identify experts irrelevant to translation. And because of the modular nature of MoEs, these can be easily pruned without any training. Without retraining, we are able to prune half of all experts with negligible degradation and 70% with only minor losses. With a very short SFT, we prune 75% of experts while recovering baseline performance, and in some settings remove nearly 90% while maintaining reasonable translation quality. Overall, our results show that translation requires only a fraction of the LLM, enabling substantial compression of the MoE blocks that contain over 90% of parameters.

Abstract:
Multi-agent systems (MAS) can substantially extend the reasoning capacity of large language models (LLMs). Most MAS frameworks aggregate agent outputs via simple majority voting, discarding the evidential structure of reasoning traces. Majority voting is brittle under confabulation consensus, where agents share correlated biases and converge on the same incorrect rationale. We introduce AgentAuditor, which moves beyond frequency-based aggregation by organizing agent traces into a Reasoning Tree that explicitly represents agreements and divergences in their reasoning. AgentAuditor resolves conflicts by comparing branch-level evidence at critical divergence points, turning global adjudication into efficient, localized verification. We further propose Anti-Consensus Preference Optimization (ACPO), which trains the adjudicator with evidence-verified preference supervision to reduce conformity to misleading majority cues. Across four MAS frameworks and multiple reasoning benchmarks, AgentAuditor consistently improves aggregation performance over majority voting, with gains of up to 5% absolute accuracy while remaining token-efficient.

Abstract:
When non-expert users ask LLMs for assistance, their queries can often have misconceptions (e.g., "How do I parse XML with regex?"). In such cases, often referred to as the XY-problem, LLMs must identify the misconception ("regex are fragile") and meaningfully direct the user toward a pragmatic solution that will address the root problem implicit in the request ("use an XML parser"). We introduce XYBench, a benchmark of 8,115 such queries, drawn from technical (StackOverflow/StackExchange) and everyday (WikiHow and a manually-curated subset) domains. We design an evaluation paradigm that assesses model responses along three criteria grounded in cooperative response theory: (a) presence and (b) emphasis on pragmatic solutions, and (c) identification of misconceptions. Our experiments show that even the strongest LLMs predominantly answer the literal request (0.75–0.92) and far less often the intended one (0.33–0.71), while substantially lagging behind humans at identifying misconceptions (at most 63% vs. 79–90%). Further, models overwhelmingly prefer pragmatic responses in a multiple choice setting yet consistently fail to generate them. Oracle ablation experiments show that providing explicit user intent at generation time helps; however a large gap remains, suggesting pragmatic redirection is a fundamentally underdeveloped capability in current LLMs.

Abstract:
In NLP, models trained via Empirical Risk Minimization (ERM) often exploit spurious correlations, such as dialectal markers, at the expense of robust linguistic features, thereby degrading their out-of-distribution (OOD) generalization. Invariant Risk Minimization (IRM) aims to suppress such shortcuts by identifying predictive relationships that remain stable across environments, but its practical utility for NLP remains debated. We use a causality-driven framework to evaluate IRM under three shortcut-generating mechanisms: label-generated features, confounding, and selection bias. Our results show that IRM successfully rejects shortcuts across the evaluated causal and anti-causal structures, including under limited environmental variation and increasing label noise. We then extend our analysis beyond controlled synthetic artifacts to two less constrained settings: gender-profession associations in Bias in Bios and semantically loaded actor-name shortcuts in ActorCorr. These experiments show that invariant learning can mitigate more realistic textual shortcuts, while semantically meaningful features require careful calibration of the invariance penalty.

Abstract:
Long-paragraph detailed audio captioning, which requires dense and transcript-faithful descriptions of fine-grained audio content, remains unsolved for current audio-visual multimodal language models. We attribute this failure to two structural problems. The first is data poverty, as no public corpus jointly provides long clips, paragraph captions, and verbatim-transcript fidelity. The second is generation-mode failure, evidenced by a 44.8 to 46.4 percentage-point gap between right-audio and shuffled-audio multiple-choice question (MCQ) accuracy. We address both within Self-Check Captioning (SCC), a unified framework that instantiates audio-grounded question answering as the verification primitive at every lifecycle stage. SCC yields three artifacts. Long-paragraph Audio Caption 50k (LACap-50k) is a 50,222-clip audio-visual corpus with 491.5-word captions and a post-hoc automatic speech recognition (ASR) audit. Layer-Curvature Supervised Fine-Tuning (LC-SFT) is the first on-policy supervised fine-tuning method to weight tokens by intermediate-layer evidence, motivated by our identification of Late-Layer Semantic-Entropy Collapse (SEC). SCC-Verifier arbitrates among caption rollouts via audio-grounded self-answering at inference. Across multiple benchmarks, our system attains state-of-the-art among open-source captioners and is competitive with proprietary baselines. We release LACap-50k to fill the resource gap for long-paragraph detailed audio captioning research.

Abstract:
The probabilistic nature of large language models (LLMs) has regularly invited inquiries into their consistency with regard to repeated sampling, input variations, and conceptually related prompts. However, unlike accuracy benchmarks, consistency evaluations cannot rely on standardized measurement tools. We conduct a comprehensive survey of current methodologies evaluating language model consistency across 85 peer-reviewed publications, organizing them into a novel three-level taxonomy: (1) sampling, (2) semantic, and (3) relational consistency. Throughout this process, we identify three fundamental threats to the construct validity of LLM consistency measurements: Valid evaluations must disentangle intrinsic consistency from correctness, probabilistic uncertainty, and artifacts of constrained generation. Our systematic analysis shows that none of the 85 publications accounts for all three factors jointly, allowing construct-irrelevant variance to obscure our understanding of consistency.

Abstract:
Large Language Reasoning Models have demonstrated remarkable success on static tasks, yet their application to multi-round agentic planning in interactive environments faces two fundamental challenges. First, the intractable credit assignment problem renders conventional reinforcement learning ineffective in sparse-reward settings. Second, the computational overhead of verbose, step-by-step reasoning histories is prohibitive. To address these challenges, we propose BPO, a three-stage framework (bootstrapping, extrapolation, and refinement) that establishes a self-improving data flywheel to develop robust reasoning models for long-horizon, sparse-reward environments. Our framework first bootstraps efficient reasoning using the proposed planning quaternions with long-short chain-of-thought fusion. It then extrapolates to out-of-distribution tasks through complexity-stratified curriculum learning. Finally, the model iteratively refines itself by learning exclusively on experiences selected via reward-gated rejection sampling. Experiments on ALFWorld, ScienceWorld, and WebShop demonstrate that our approach achieves state-of-the-art with significant token efficiency, providing a new recipe for reasoning models in agentic planning.

Abstract:
Although a growing body of research has begun to describe user-LLM interactions, the picture it paints is largely static; little is known about how individual users change their behavior over time. To address this gap, we analyze the conversational trajectories of ~12,000 randomly sampled Microsoft Bing Copilot users and compare these with data from WildChat-4.8M. While the Copilot data contains significant population-level trends, we find that trends in individual user trajectories are much weaker; user habits prove to be overwhelmingly sticky. We also find stark differences between users of different activity levels: more active users have more successful conversations and use the LLM for more complex and professionally oriented tasks. Some user trends also appear in WildChat-4.8M, but we find evidence that this dataset is significantly skewed towards highly proficient "power" users. Ultimately, our results suggest that existing user behavior is difficult to change and demonstrate the extent of user heterogeneity. Our comparison between datasets highlights that WildChat does not represent typical user-AI interactions, an important caveat for downstream uses of the data.

Abstract:
GUI grounding benchmarks ask whether agents can localize a target that is already visible in a static screenshot. Real interfaces often violate this assumption: a target may be off-screen, appear only after a state change or hover, remain occluded, or become actionable only after a delay. This leaves the ability to recover target visibility unevaluated. To measure this reveal step, we introduce Interactive GUI Grounding (IGG), a benchmark in which an agent must first make the target visible through interaction and then click it. IGG contains 601 tasks across 16 controllable GUI environments and seven visibility-constraint sub-types, with a compact action space of scrolling, moving, dragging, waiting, and clicking. In our human study, participants achieve 94.6% accuracy, while GPT-5.5 reaches 79.4% and the strongest evaluated open-source VLM reaches only 31.6%. To identify the source of this gap, we conduct two controlled interventions. Telling Qwen3-VL which interaction to perform raises its accuracy only from 30.8% to 33.9%, whereas performing the interaction for the model raises it to 81.9%. These results identify reveal-interaction execution, rather than target recognition after exposure, as the main bottleneck. We further show that this capability is learnable, demonstrating that visibility recovery can be strengthened consistently across model families.

Abstract:
Hate speech detection is inherently subjective: people from different demographic groups perceive the same content very differently. Collecting enough annotations from multiple demographic groups is costly and difficult to scale. Persona-conditioned Large Language Models (models prompted to adopt a specific demographic identity) have been proposed as a way to simulate diverse perspectives at scale. But do they actually reflect how different groups disagree? We evaluate three aspects of human social judgement: (i) whether personas from different groups disagree in human-like ways (inter-group disagreement), (ii) whether they become more sensitive when content targets their own identity (in-group sensitivity), and (iii) whether they can accurately predict how another group would react (vicarious prediction). Our results show that no model consistently captures all three dimensions, and performance is highly model-dependent and does not emerge reliably from minimal identity prompts alone. However, vicarious prompting with Llama 3.1 yields the highest cross-group agreement in most demographic axes and provides the closest overall approximation to human disagreement patterns, indicating that this configuration may provide a more reliable setting for automatic annotation aligned with human judgements.

Abstract:
Large language models have shown impressive capabilities in code generation, yet they often produce functionally incorrect code. Uncertainty quantification (UQ) methods have emerged as a promising approach for detecting hallucinations in natural language generation, but their effectiveness for code generation tasks remains underexplored. We systematically evaluate how UQ techniques transfer to code generation across three programming languages, five LLMs, and over 1,700 problems. We find that some token-probability-based methods generalize effectively without modification, while sampling-based methods relying on natural language inference (NLI) fail because NLI models cannot distinguish functionally different code, causing most responses to collapse into a single semantic cluster. To address this, we introduce functional equivalence methods , a family of code-specific methods that replace NLI-based semantic equivalence with an LLM-based functional equivalence assessment, including functional entropy, a code-specific analog of semantic entropy. Functional equivalence methods achieve top AUROC in 11 out of 15 model-benchmark combinations and the best calibration across most settings, consistently outperforming both NLI-based counterparts and all other methods evaluated.

Abstract:
Multimodal web agents that process both screenshots and accessibility trees are increasingly deployed to interact with web interfaces, yet their dual-stream architecture opens an underexplored attack surface: an adversary who injects content into the webpage DOM simultaneously corrupts both observation channels with a consistent deceptive narrative. Our vulnerability analysis on MiniWob++ reveals that attacks including a visual component far outperform text-only injections, exposing critical gaps in text-centric VLM safety training. Motivated by this finding, we propose Dual-Modality Multi-Stage Adversarial Safety Training (DMAST), a framework that formalizes the agent-attacker interaction as a two-player zero-sum Markov game and co-trains both players through a three-stage pipeline: (1) imitation learning from a strong teacher model, (2) oracle-guided supervised fine-tuning that uses a novel zero-acknowledgment strategy to instill task-focused reasoning under adversarial noise, and (3) adversarial reinforcement learning via Group Relative Policy Optimization (GRPO) self-play. On out-of-distribution tasks, DMAST nearly halves the attack success rate (41.2% → 21.4%) while raising task completion by over 60% relative (6.2% → 10.2%). Our approach outperforms established training-based and prompt-based defenses, demonstrating genuine co-evolutionary progress and robust generalization to complex, unseen environments. Code is available at https://github.com/huajianduzhuo-code/DMAST_official.

Abstract:
Natural language conveys information at varying levels of granularity, from fine-grained references to broad descriptions. While granularity is fundamental to human communication, existing measures mostly capture surface detail or sentence specificity. We introduce Granuscore, a reference-free measure of granularity that leverages structural properties of a hierarchical embedding space. Granuscore reliably recovers hierarchical orderings on the Granola-EQ dataset and captures expected differences in granularity across discourse contexts. Across domains, we further show that Granuscore explains non-linear variation in sentence specificity beyond sentence length. Finally, we apply Granuscore to four question-answering benchmarks and analyze how granularity differs for questions, gold answers, and model outputs across response outcomes. The analysis reveals consistent differences in model behavior and provides a principled lens for characterizing the difficulty of QA datasets. Together, the results position Granuscore as a scalable, broadly applicable tool for analyzing granularity in text.

Abstract:
Given a query, how can a QA system ask an informative clarifying question to accurately answer it? Open-domain conversational question answering requires not only detecting when a query is ambiguous but also asking clarifying questions that efficiently uncover the latent intent of the user. However, existing approaches either rely on prompt-only clarify-or-answer decisions with limited robustness or distill clarification trajectories without explicitly modeling intent convergence. In this paper, we propose D4CQ (Data-Efficient Diagnosticity based Dynamic Distillation for Clarifying Question), a data-efficient framework for learning diagnostic clarifying questions through dynamic distillation. Our approach distills a structured refinement protocol, prioritizes informative clarifications via diagnosticity-based reweighting, and improves sample efficiency through hindsight relabeling. Empirical results demonstrates that D4CQ consistently outperforms strong baselines, while maintaining comparable interaction budgets. Our code is available at https://github.com/ddddcq/d4cq.

Abstract:
Healthcare prediction using electronic health records (EHRs) is a cornerstone of modern clinical decision support. Conventional machine learning (ML) models achieve strong predictive performance but rarely provide knowledge-grounded, human-readable explanations. Large language models (LLMs) incorporated with medical knowledge graphs (KGs) offer advanced reasoning and explainability with encoded clinical facts, yet existing approaches typically perform monolithic reasoning over entire patient records, mixing task-irrelevant context and obscuring disease-specific attribution. To bridge these gaps, we propose CRU-Reason, a novel framework that integrates ML models, medical KGs, and LLMs through Clinical Rationale Units (CRUs) for knowledge-grounded and interpretable healthcare prediction. CRU is a localized structure that explicitly captures clinically meaningful communities among EHR concepts, thereby enabling LLMs to perform disease-centered, fine-grained reasoning. Experiments on two public clinical datasets demonstrate that CRU-Reason improves over the strongest evaluated EHR-prediction baselines by an average of +9.4% across drug recommendation and diagnosis prediction, while exhibiting improved rationale quality and cross-dataset robustness.

Abstract:
Recent studies show that the safety alignment of large language models (LLMs) can be easily compromised even by seemingly non-adversarial fine-tuning. To preserve safety alignment during fine-tuning, a widely used strategy is to jointly optimize safety and task objectives by mixing in the original alignment data, which is typically inaccessible even for open-weight LLMs. Inspired by generative replay in continual learning, we propose Generative Replay for Safety Alignment Preservation (GR-SAP), a unified framework that synthesizes domain-specific alignment data from LLMs and integrate them during downstream adaptation to preserve safety alignment. Theoretical and empirical analyses demonstrate this synthetic data serves as a reliable proxy for the original alignment data. Experiments across various models and downstream tasks show that GR-SAP substantially mitigates fine-tuning–induced safety degradation while maintaining comparable downstream performance.

Abstract:
Humans possess an innate ability to understand fine-grained interpersonal relationships, which is central to everyday social interactions. Although such reasoning is inherently multimodal, it remains largely unexplored by existing multimodal large language models (MLLMs). To address this gap, we introduce PIVOTS-Bench, the first benchmark built from Social-IQ 2.0 and YouTube data to evaluate MLLMs’ ability to predict bidirectional interpersonal relationship dimensions grounded in established psychology research. In addition, PIVOTS includes auxiliary tasks that assess models’ ability to identify and leverage the critical visual cues underlying such predictions. We evaluate both proprietary and open-source MLLMs and conduct detailed ablation studies to analyze the effects of visual modalities and explicit social role information in conversational utterances. We further examine how joint and pairwise prediction settings benefit MLLMs in scoring bidirectional PIVOTS dimensions. Project page: https://ciossayin.github.io/pivots-bench/

Abstract:
Disclaimer: Samples in this paper may be harmful and cause discomfort. Multimodal large language models (MLLMs) enable multimodal understanding but inherit toxic signals from weakly curated pretraining corpora, leading to explicitly toxic outputs, especially under adversarial triggers that late, opaque training-free detoxification methods struggle to handle. We propose SGM, a white-box neuron-level multimodal intervention that acts like safety glasses for toxic neurons: it recalibrates a set of toxicity-associated neurons via expertise-weighted soft suppression, neutralizing harmful cross-modal activations without any parameter updates. We establish MM-TOXIC-QA, a multimodal toxicity data framework, and compare SGM with existing detoxification techniques. Experiments on open-source MLLMs show that SGM mitigates explicit toxicity in standard and adversarial conditions, cutting average harmful rates from 45.0% to 4.5% while preserving fluency and multimodal reasoning. SGM is extensible, and its combined defenses, denoted as SGM, integrate with existing detoxification methods for stronger safety performance, providing an interpretable, low-cost solution for toxicity-controlled multimodal generation.

Abstract:
Large Language Model (LLM) evaluation is frequently challenged by prompt sensitivity, which limits the reliability of benchmarks relying on a single template. To address this, we systematically investigate a multi-prompt evaluation framework that aggregates performance across diverse templates, leveraging a Quality-Diversity (QD) prompt selection mechanism to ensure both variety and fidelity. Through extensive empirical analysis, we uncover a novel Evaluation Scaling Law: evaluation reliability scales consistently with template diversity. Specifically, our experiments demonstrate that evaluation variance diminishes as the scale of prompts increases, while ranking monotonicity exhibits a log-linear improvement relative to the number of templates. By decoupling a model’s intrinsic capability from template-specific fluctuations, our framework facilitates more reliable assessments of LLMs, thereby providing a solid foundation for benchmarking that enables more effective monitoring of training dynamics and rigorous comparisons among state-of-the-art models.

Abstract:
Scientific papers advance claims that later work supports, extends, or sometimes refutes. Yet existing methods for citation and claim analysis capture only fragments of this dialogue. In this work, we make these interactions explicit at the level of scientific claims. We introduce ClaimFlow, a claim-centric view of the NLP literature, built from 1,617 ACL Anthology papers (1979-2025) that are manually annotated with 5,689 claims and 4,871 cross-paper claim relations, indicating whether a citing paper supports, extends, qualifies, refutes, or references a cited claim as background. Building on ClaimFlow, we define a new task – Claim Relation Classification – which requires models to infer the scientific stance toward a cited claim from the text and citation context. Evaluating neural models and large language models on this task, we report baseline performance of 0.81 macro-F1, suggesting that the task is tractable while leaving room for improvement. We then scale this framework to ~13k NLP papers to study claim evolution across decades of NLP research. We show that 63.5% of claims are never reused; only 11.1% are ever challenged. Widely propagated claims are more often reshaped through qualification and extension than supported or refuted. Overall, ClaimFlow offers a lens for examining how ideas shift and mature within NLP.

Abstract:
Large language models are effective at code completion when the required evidence is local, but repository-level fill-in-the-middle completion remains challenging when it depends on code and structural relationships across files. We present ACAR, a framework that combines lexical and AST/LSP-based structural retrieval through line-level reciprocal rank fusion, then applies syntax-aware post-processing to align generated completions with the surrounding code. We evaluate ACAR on RepoEval, ReccEval, and CrossCodeEval across Python, Java, TypeScript, and C#, covering more than 3,000 real-world repositories. In the main Qwen2.5-Coder experiments, ACAR outperforms strong retrieval baselines in most settings, reaching 42.98 EM on ReccEval and 32.86 on CrossCodeEval at 3B, versus 36.25 and 28.40 for the strongest retrieval baseline, respectively. With the largest tested Qwen and DeepSeek backbones, ACAR improves EM over their corresponding no-retrieval baselines across all five evaluation settings. Ablations show that lexical and structural evidence are complementary, while removing syntax-aware refinement causes the largest drop in edit similarity. These gains come with 87–95 ms of non-decoding overhead under warm-cache conditions across the three benchmarks, supporting latency-sensitive code completion. To support reproducibility, we make our implementation and evaluation scripts publicly available at https://github.com/longstnguyen/ACAR.

Abstract:
Latent reasoning models (LRMs) replace explicit chain-of-thought with continuous thoughts. Recent work treats observable latent-state patterns, such as BFS-like frontiers and decodable arithmetic computation, as evidence for internal reasoning mechanisms. Evaluating two LRMs (Coconut and CODI) against controls lacking the proposed recurrence or curriculum, we find these patterns also appear in the controls and do not consistently affect behavior. Causal interventions reveal that latent-thought utilization is not binary but graded, scaling with a thought’s causal effect on model behavior. Geometric analyses reveal this effect concentrates in low-dimensional subspaces whose step-wise geometry grows more structured as their behavioral influence increases. Latent thoughts should therefore be treated as hidden computation, not hidden explanation: decodability, attention, or static structure alone cannot establish LRM reasoning mechanisms; that requires matched controls and causal tests.

Abstract:
Gender bias in large vision-language models (LVLMs) undermines their fairness and reliability, compromising output trustworthiness. Current mitigation methods rely on training-phase adjustments or post-hoc calibration, but face limitations in dynamic visual bias mitigation. These include inability to capture real-time visual-textual incongruence, dependence on predefined gender bias taxonomies, and degraded cross-modal alignment with emergent bias patterns. To address these challenges, we propose ViD, a causally-inspired framework that analyzes attention mechanisms across five distinct patterns, revealing confounding effects from strong language priors. ViD demonstrates that visual-to-language cross-attention effectively suppresses bias while preserving general reasoning capabilities and text generation quality. ViD incorporates dual mechanisms: backdoor adjustment counters strong language priors, while refined token selection in decoding layers optimizes processing. This enhances model robustness and inference efficiency. Our integrated approach significantly mitigates gender bias across multidimensional social attributes in LVLMs, improving visual grounding and output fairness. Cross-benchmark validation shows ViD reduces gender bias by 14.7% on single-attribute evaluations (FACET) and achieves significant improvements on image captioning tasks (MS COCO), with gender bias score improving from 0.6708 to 0.9978 for LLaVA. Crucially, these improvements require no additional training overhead, making ViD a scalable and practical solution for bias mitigation in LVLMs.

Abstract:
Instruct-based large language models (LLMs) have been shown to propagate and even amplify gender bias when prompted with contextually constrained instructions (e.g., writing a text from a description or selecting a gendered pronoun). However, little attention has been paid to biases in responses to contextually unconstrained (generic) instructions conveyed by gendered language, particularly masculine generics (MG). MG, found in many gender-marked languages, denote the use of the masculine gender as a supposedly "neutral" reference for mixed-gender groups or for individuals of unknown gender. Yet, psycholinguistic studies demonstrate that MG are not neutral and systematically induce gender bias. This study investigates how both local and proprietary LLMs are MG-biased when responding to generic prompts in French, examining LLMs’ MG bias rates and use of gender-fair language (GFL). We create a 16k+ human noun database from existing lexical resources and evaluate six LLMs on four instruction-response datasets under two conditions: prompts with and without MG. Overall, we find that ≈27.57% of LLMs’ responses to MG-filtered generic instructions are MG-biased (≈78.55% with MG-containing prompts). Moreover, we find that LLMs rarely use GFL spontaneously. These findings highlight the persistence of MG bias in LLM outputs and the models’ limited tendency towards GFL strategies.

Abstract:
Multimodal large language models (MLLMs) are typically trained in multiple stages, with video-based supervised fine-tuning (Video-SFT) serving as a key step for improving visual understanding. Yet its effect on the fine-grained evolution of visual capabilities, particularly the balance between spatial and temporal understanding, remains poorly understood. In this paper, we systematically study how Video-SFT reshapes visual capabilities in MLLMs. Across architectures, parameter scales, and frame sampling settings, we observe a consistent pattern: Video-SFT reliably improves video performance, but often yields limited gains or even degradation on static image benchmarks. We further show that this trade-off is closely tied to temporal budget: increasing the number of sampled frames generally improves video performance, but does not reliably improve static image performance. Motivated by this finding, we study an instruction-aware Hybrid-Frame strategy that adaptively allocates frame counts and partially mitigates the image-video trade-off. Our results indicate that Video-SFT is not a free lunch for MLLMs, and preserving spatial understanding remains a central challenge in joint image-video training.

Abstract:
Large Language Models (LLMs) achieve strong performance across many tasks but suffer from high inference latency due to autoregressive decoding. The issue is exacerbated in Large Reasoning Models (LRMs), which generate lengthy chains of thought. While speculative decoding accelerates inference by drafting and verifying multiple tokens in parallel, existing methods operate at the token level and ignore semantic equivalence (i.e., different token sequences expressing the same meaning), leading to inefficient rejections. We propose SemanticSpec, a semantic-aware speculative decoding framework that verifies entire semantic sequences instead of tokens. SemanticSpec introduces a semantic probability estimation mechanism that probes the model’s internal hidden states to assess the likelihood of generating sequences with specific meanings. Experiments on four benchmarks show that SemanticSpec achieves up to 2.7× speedup on DeepSeekR1-32B and 2.1× on QwQ-32B, consistently outperforming token-level and sequence-level baselines in both efficiency and effectiveness.

Abstract:
Reinforcement learning (RL) for long-horizon agents faces two key challenges. First, parallel linear rollouts tend to converge toward homogeneous reasoning patterns, leading to cognitive-mode collapse. Second, existing methods refine credit assignment along only a single dimension, making it difficult to simultaneously achieve state-aligned comparison and meta-reasoning discrimination. We propose STARK (Strategic and Tactical Advantage Reinforcement via Confidence Guided Keypoint Exploration) to resolve both issues. STARK performs corrective exploration through confidence-driven branching at uncertain decision points and injects reflection into new branches. To empower these keypoint-expanded trajectories, STARK introduces a dual-track strategic and tactical advantage estimator: The strategic track compares reasoning modes under shared environmental observations, while the tactical track evaluates execution quality within each meta-reasoning tag group. Extensive experiments on ALFWorld, ScienceWorld, and WebShop show that STARK consistently outperforms strong baselines across model scales and task domains, and enables a 1.5B model to surpass GPT-5 on the out-of-domain settings.

Abstract:
Scientific figures compress complex pipelines into a single canvas, but understanding them requires paper-grounded, step-by-step narration aligned with visual highlights—a capability missing from current video generation systems and benchmarks. To address this, we introduce paper-grounded figure-to-video generation: generating narrated, region-grounded walkthrough videos from a figure and its paper. Second, we develop a multi-agent pipeline MINARD, which first generates paper-grounded narration and then ground the figure regions through staged extraction and clustering. Third, we release FigTalk, a benchmark derived from Paper2Video with new sequential and component-level grounding metrics that correlate strongly with human judgments. On FigTalk, MINARD produces human-like narrations grounded in paper evidence, and MINARD-Grounding substantially outperforms existing VLM grounding, animation-based, and diffusion-style video generation methods in both automatic and human evaluation.

Abstract:
Hateful content online is often expressed using fact-like, not necessarily correct information, especially in coordinated online harassment campaigns and extremist propaganda. Failing to jointly address hate speech (HS) and misinformation can deepen prejudice, reinforce harmful stereotypes, and expose bystanders to psychological distress, while polluting public debate. Moreover, these messages require more effort from content moderators because they must assess both harmfulness and veracity, i.e., fact-check them. To address this challenge, we release WSF-ARG+, the first dataset which combines hate speech with check-worthiness information. We also introduce a novel LLM-in-the-loop framework to facilitate the annotation of check-worthy claims. We run our framework, testing it with 12 open-weight LLMs of different sizes and architectures. We validate it through extensive human evaluation, and show that our LLM-in-the-loop framework reduces human effort without compromising the annotation quality of the data. Finally, we show that HS messages with check-worthy claims show significantly higher harassment and hate, and that incorporating check-worthiness labels improves LLM-based HS detection up to 0.213 macro-F1 and to 0.154 macro-F1 on average for large models.

Abstract:
Large Vision-Language Models (LVLMs) achieve strong performance on various vision-language tasks, yet Chain-of-Thought (CoT) generation often weakens attention to visual tokens, degrading evidence-grounded reasoning. During the later stages of autoregressive decoding, models tend to increasingly rely on previously generated text rather than visual evidence, leading to plausible but visually inconsistent rationales. We propose DVAR (Dynamic Visual Attention Recalibration), a training-free method that selectively amplifies visually relevant tokens to mitigate visual attention decay during decoding. DVAR operates at three levels: (i) prompt level structure to encourage visually grounded reasoning; (ii) embedding level relevance weights derived from image–text alignment; and (iii) decoding level scheduling that adjusts recalibration by tracking shifts in visual attention. Without sensitive model-specific hyperparameter retuning, DVAR yields consistent performance improvements across six benchmarks on LLaVA and Qwen-VL models, and better preserves visual attention throughout decoding.

Abstract:
Hybrid thinking enables LLMs to switch between reasoning and direct answering, offering a balance between efficiency and reasoning capability. Yet our experiments reveal that current hybrid thinking LLMs only achieve partial mode separation: reasoning behaviors often leak into the no-think mode. To understand and mitigate this, we analyze the factors influencing controllability and identify four that matter most: (1) larger data scale, (2) using think and no-think answers from different questions rather than the same question, (3) a moderate increase in no-think data number, and (4) a two-phase strategy that first trains reasoning ability and then applies hybrid think training. Building on these findings, we propose a practical recipe that, compared to standard training, can maintain accuracy in both modes while significantly reducing no-think output length (from 1085 to 585 on MATH500) and occurrences of reasoning-supportive tokens such as "wait" (from 5917 to 522 on MATH500). Our findings highlight the limitations of current hybrid thinking and point the ways for enhancing its controllability.

Abstract:
Test-time compute has become a central way to improve code generation: systems sample multiple candidate programs and use verifier-visible evidence to select the final output. This paradigm implicitly assumes that the verifier provides a corrective signal independent from the generator. We challenge this assumption under misleading task premises. When the generator and verifier share a false premise, they become coupled through a mistaken belief: the generator produces premise-consistent shortcuts, while the verifier supplies evidence that fails to expose them. Consequently, the selector may choose a hidden-test-wrong candidate even when a hidden-test-correct program exists in the pool. We call this failure mode Verification Trap. Across three code-generation benchmarks and five code models, false premises consistently degrade first-sample correctness, reduce selector-chosen correctness after 64-sample test-time selection, and amplify recoverable mis-selection. Mechanistically, verifier-written tests inherit the premise-level blind spot, reshaping verifier-visible candidate space away from hidden-test correctness. These traces make Verification Trap predictable before hidden execution: a lightweight gold-free predictor using verifier-visible features reaches 0.846 AUROC. Our results identify decoupled evidence as a key mitigation axis: coupled scaling provides limited recovery, whereas premise-agnostic robustness auditors recover substantial oracle headroom.

Abstract:
Recent advances in equipping Large Language Models (LLMs) with search tools and outcome-reward reinforcement learning (RL) have achieved new state-of-the-art results on open-domain QA tasks. However, we argue that current training paradigms harbor a critical vulnerability: they predominantly reward correct answers but fail to penalize fabricated ones when retrieval fails, thereby implicitly exacerbating hallucinations. To address this, we propose Abstention-Aware Reinforcement Learning (AWA-RL), which dynamically shapes the abstention reward utilizing the model’s query-specific prior capabilities and continuous on-policy training observations. We also introduce a novel metric, RA-F1, to measure the capability-reliability trade-off. Compared to non-abstaining baselines, AWA-RL boosts absolute precision by up to 10.3% and overall RA-F1 by 2.9%, with only marginal sacrifice in raw accuracy. These results confirm that AWA-RL successfully yields highly capable and reliable search agents.

Abstract:
Scientific LLM agents have shown promise in literature reasoning, tool use, and experiment planning, but it remains unclear whether they can autonomously improve large, tightly coupled scientific machine-learning systems through executable code changes and computationally expensive validation. We study this question in protein folding, where progress requires coordinated architectural modifications, multi-objective evaluation, and domain-aware interpretation. We present AgentFold, a multi-agent framework that formulates folding-model development as a closed-loop search over executable code variants. Starting from ESMFold, AgentFold proposes hypotheses, implements and debugs code-level modifications, evaluates model variants, analyzes experimental outcomes, and stores both successful and failed interventions in structured memory. An MCTS-style policy allocates computational resources across high-scoring search branches. On an engineering-scale protein-folding codebase comprising more than 2,000 lines of code, AgentFold explores approximately 80 model variants using approximately 5,000 GPU-hours and 170 million LLM tokens. Under a matched computational budget, AgentFold improves the best lDDT by 7.5% over independent Codex proposals and outperforms a random-search control. Beyond model improvement, the resulting intervention traces reveal recurring empirical design patterns: stable gains tend to arise from early, soft, learnable priors and gated refinement, whereas direct geometric perturbations and geometry-conditioned feedback often destabilize training. The code and experimental resources are publicly available at https://github.com/lmqfly/AgentFold.

Abstract:
Dental triage is a safety-critical clinical routing task that requires integrating multimodal clinical information (e.g., patient complaints and radiographic evidence) to determine complete referral plans. We present Dental-TriageBench, the first expert-annotated benchmark for reasoning-driven multimodal dental triage. Built from authentic outpatient workflows, it contains 246 de-identified cases annotated with expert-authored golden reasoning trajectories, together with hierarchical triage labels. We benchmark 19 proprietary, open-source, and medical-domain MLLMs against three junior dentists serving as the human baseline, and find a substantial human–model gap, on fine-grained treatment-level triage. Further analyses show that accurate triage requires both complaint and OPG information, and that model errors concentrate on cases with multiple referral domains, where MLLMs tend to produce overly narrow referral sets and omission-heavy errors. Dental-TriageBench provides a realistic testbed for developing multimodal clinical AI systems that are more clinically grounded, coverage-aware, and safer for downstream care.

Abstract:
Dense retrieval relies on aligning document and query embeddings; however, its performance is often constrained by the scarcity of labeled queries. Although prior work has attempted to model the many-to-one relationship between documents and queries using Gaussian Mixture Models (GMMs), such Euclidean-based approaches induce norm shrinkage ( ‖𝜇‖ < 1 ) in normalized semantic spaces, resulting in distorted similarity scores. To address this fundamental geometric inconsistency, we propose vMF-HSE, a training-free hyperspherical semantic expansion framework that models the latent query space using a von Mises-Fisher (vMF) mixture distribution on the unit hypersphere 𝕊 d-1 . We leverage Sra’s Newton-based optimization within the Expectation-Maximization (EM) algorithm to ensure robust parameter estimation even in high-dimensional semantic spaces. Building upon this stable estimation, we propose a novel adaptive stopping mechanism ( 𝜅 -Stop) that adaptively determines the optimal number of synthetic queries per document by monitoring the convergence of marginal information gain. Experimental results on the BEIR benchmark demonstrate that vMF-HSE consistently outperforms conventional Euclidean-based methods in zero-shot retrieval settings, establishing the necessity of geometrically consistent modeling for normalized semantic spaces.

Abstract:
While draft-model-free speculative decoding offers a promising path to efficient LLM inference, it is frequently constrained by stale draft candidates and the high computational cost of the verification. To address these challenges, we propose ECHO, a hierarchical dual-loop framework that exploits the functional asymmetry between LLM layers. Leveraging the high discriminative efficiency of early layers and the authoritative distribution of final layers, ECHO bifurcates inference into a high-frequency inner loop and a low-frequency outer loop. Within the inner loop, early-layer bonus logits drive rapid, multi-step draft-tree exploration at a minimal cost. Simultaneously, the outer loop performs authoritative full-model verification through a state-reuse mechanism. Crucially, the outer loop also utilizes final-layer bonus logits to correct existing paths and supplement the tree with high-confidence candidates for subsequent cycles. Experimental results across diverse benchmarks demonstrate that ECHO significantly boosts mean accepted tokens and achieves a 2.4 × to 2.9 × speedup, outperforming existing state-of-the-art baselines with negligible engineering overhead and no extra deployment parameters, albeit with a one-shot fine-tuning dependency for optimal acceleration. The code is available at https://github.com/whucs21Mzy/ECHO.

Abstract:
Large language models (LLMs) are trained through multi-stage pipelines over heterogeneous data sources, yet developers lack a principled way to pinpoint the specific data responsible for an observed behavior. This lack of observability reduces debugging to reactive patching and makes failures prone to recur under distribution shift or subsequent model updates. To address this limitation, we propose DebugLM, a framework that equips LLMs with built-in data provenance, enabling them to explicitly trace the origins of their behaviors to specific training data sources. Specifically, the model learns to associate its responses with unique provenance tags that indicate the responsible dataset, empowering developers to precisely identify where undesirable behaviors are learned. Building on this capability, DebugLM further supports targeted test-time remediation, enabling developers to selectively trigger targeted refusal for specified data sources without retraining or modifying model parameters. Experiments demonstrate that DebugLM provides accurate behavior tracing in multi-stage training pipelines and effective test-time remediation while preserving the general utility of the model.

Abstract:
Long-horizon LLM agents can benefit from reusable skills, yet existing skill-based methods often rely on external skill generators during training or persistent skill retrieval at inference, increasing engineering complexity, context length, and deployment latency. We propose Self-Internalizing Reinforcement learning with Intrinsic skills (SIRI), a three-phase framework that enables agents to discover, validate, and internalize skills without external skill generators or inference-time skill banks. SIRI first warms up the policy with GiGPO to acquire basic interaction ability and collect successful skill-free trajectories. It then performs self-skill mining, where the current policy summarizes compact skills from its own successful plain rollouts and validates them through paired skill-augmented and skill-free rollouts. Finally, SIRI distills only beneficial skill-guided action tokens into the plain policy using trajectory-level utility and action-level advantage. At inference, the agent runs with the original prompt only. On ALFWorld and WebShop with Qwen2.5-7B-Instruct, SIRI improves GiGPO from 0.908 to 0.930 on ALFWorld and from 0.728 to 0.813 on WebShop, outperforming prompt-based, RL-based, and memory-augmented baselines. Further analysis shows that our self-mining strategy can achieve performance comparable to distillation with closed-source large model.

Abstract:
LLM-as-judge evaluation has become standard practice for open-ended model assessment; however, judges exhibit systematic biases that cannot be averaged out by increasing the number of scenarios or generations. These biases are often similar in magnitude to the model differences that benchmarks are designed to detect, resulting in unreliable rankings when single-judge evaluations are used. We introduce a variance decomposition that partitions benchmark score variance into scenario, generation, judge, and residual components. Based on this analysis, CyclicJudge, a round-robin assignment of judges to scenarios, is demonstrated to be the optimal strategy for a fixed judge panel and judge-call budget: the score recovers the panel mean exactly while matching the cost of single-judge evaluation. Empirical results on MT-Bench and MindEval validate the effectiveness of CyclicJudge as predicted, across both general-purpose and domain-specific evaluation settings.

Abstract:
To reliably assist human decision-making, LLMs must maintain factual internal beliefs against misleading injections. While current models resist explicit misinformation, we uncover a fundamental vulnerability to sophisticated, hard-to-falsify evidence. To systematically probe this weakness, we introduce MisBelief, a framework that generates misleading evidence via collaborative, multi-round interactions among multi-role LLMs. This process mimics subtle, defeasible reasoning and progressive refinement to create logically persuasive yet factually deceptive claims. Using MisBelief, we generate 4,800 instances across three difficulty levels to evaluate 7 representative LLMs. Results indicate that while models are robust to direct misinformation, they are highly sensitive to this refined evidence: belief scores in falsehoods increase by an average of 93.0%, fundamentally compromising downstream recommendations. To address this, we propose Deceptive Intent Shielding (DIS), a governance mechanism that provides an early warning signal by inferring the deceptive intent behind evidence. Empirical results demonstrate that DIS consistently mitigates belief shifts and promotes more cautious evidence evaluation.

Abstract:
LLM-powered Multi-Agent Systems (MAS) have emerged as an effective approach towards collaborative intelligence, and have attracted wide research interests. Among them, "self-evolving" MAS, treated as a more flexible and powerful technical route, can construct task-adaptive workflows or communication topologies, instead of relying on a predefined static structure template. Current self-evolving MAS mainly focus on Spatial Evolving or Temporal Evolving paradigm, which only considers the single dimension of evolution and does not fully incentivize LLMs’ collaborative capability. In this work, we start from a novel Spatio-Temporal perspective by proposing ST-EVO which supports dialogue-wise communication scheduling with a compact yet powerful flow-matching based Scheduler. To make precise Spatio-Temporal scheduling, the Scheduler can also perceive the uncertainty of MAS, and possesses self-feedback ability to learn from accumulated experience. Extensive experiments on nine benchmarks demonstrate the state-of-the-art performance of ST-EVO, achieving about 5%–25% accuracy improvement.

Abstract:
Concept induction, which infers appropriate abstract categories from concrete instances, is both a core ontology engineering task and a testbed for LLM semantic abstraction. Although LLMs offer broad knowledge and reasoning that may ease expert-driven ontology construction, prior work shows that knowledge deficiencies still limit them in specialized domains, while systematic evaluation of their induction failures remains lacking. To address this gap, we propose a familiarity-controlled evaluation framework that uses knowledge probing to separate item familiarity from concept induction ability. The evaluation reveals a systematic tendency to induce overly general concepts: LLMs often produce semantically correct but overly broad labels, and direct property augmentation improves performance without resolving this granularity mismatch. We further propose PPACI, an ontology-prior-guided approach that builds concept prototypes by integrating concept definitions with member item properties. For each query group, PPACI uses item properties to retrieve local item-level and global group-level candidate concepts as induction anchors. Experiments across multiple LLMs show that PPACI reduces overly general predictions by 17–28 percentage points while maintaining competitive induction quality. Our code and data are publicly available at https://github.com/Orchideous77/PPACI.

Abstract:
Text detoxification should produce safer rewrites while retaining identity mentions from the original text. However, classifier-guided methods can remove identity descriptors alongside genuinely harmful content when toxicity classifiers associate identity mentions with harmfulness and guidance relies on absolute scores. We propose CIDER (Contrastive Identity-preserving Detoxification via Explicit Reference), a default-relative, margin-gated selective-guidance method for masked diffusion language models. At each selected masked position, CIDER evaluates every candidate in the admissible top-k set relative to the model-default token in the same partially decoded state and commits a non-default candidate only when the best candidate lowers the one-step proxy score by more than a preset margin. On the primary LLaDA-8B CivilComments×Jigsaw setting, CIDER improves retention by +0.06 to +0.16 over the interpolated absolute-score frontier at matched residual-toxicity levels. At the reported aggressive endpoints, the retention advantage recurs across four data pairings on LLaDA-8B and three bases on CivilComments×Jigsaw. Controlled ablations indicate that the selection rule is an important contributor to the improvement.

Abstract:
LLM-based scientific exploration agents need feedback signals that decide which experimental outcomes should guide future search. Recent systems use Bayesian surprise, rewarding results that change an LLM’s belief. This can create a noisy-TV failure mode: unstable artifacts, leakage, proxy effects, or gaps in the model’s prior may appear more valuable than reliable mechanisms. We introduce Reducible Information Gain (ReIG), a validation-aware discovery method. ReIG keeps LLMs for hypothesis proposal and interpretation, and uses evidence that remains stable under resampling and null calibration as the search feedback. ReIG then reports final candidates along separate axes of reliability, novelty, and non-triviality. Across Rep-60, Stress-72, and faceted report evaluations, ReIG consistently improves discovery quality under matched search policies. Under MCTS, ReIG-MCTS improves ValidDisc@10 from 0.57 to 1.55 over Surprise-MCTS, raises reward AUROC from 0.44 to 0.83, and reduces false flags on no-signal tasks from 3.8 to 0.38 at budget 20. Ablations, same-candidate reranking, and the semantic audit show that the gains come from guiding exploration with validated, reusable evidence and keeping reliability, novelty, and non-triviality as distinct criteria.

Abstract:
Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS—Geodesic-Gated Spherical Steering—a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.

Abstract:
While recent research has systematically documented political orientation in large language models (LLMs), existing evaluations rely primarily on direct probing or demographic persona engineering to surface ideological biases. In social psychology, however, political ideology is also understood as a downstream consequence of fundamental moral intuitions. In this work, we manipulate the moral conditioning and investigate the changes in the model’s political behaviors. Rather than simply assigning a demographic persona, we condition models to endorse or reject specific moral values and evaluate the resulting shifts on their political orientations, using the Political Compass Test. By treating moral values as lenses, we observe how moral conditioning actively modulates model trajectories across economic and social dimensions. Our findings show that such conditioning induces pronounced, value-specific shifts in models’ political coordinates. We further notice that these effects are systematically modulated by role framing and model scale, and are robust across alternative assessment instruments instantiating the same moral value. This highlights that effective alignment requires anchoring political assessments within the context of broader social values including morality, paving the way for more socially grounded alignment techniques.

Abstract:
Hint-based reinforcement learning addresses reward sparsity in long-horizon agentic tasks by retaining a prefix of an expert trajectory before each rollout, letting the policy explore from a state closer to success. Its effectiveness hinges on the guidance depth: how much of the trajectory to keep. Existing methods treat this depth as a deterministic scalar. Scheduled approaches share one value across samples and ignore per-task heterogeneity; per-sample probing estimates it separately at the cost of extra rollouts. We find that useful guidance occupies a band of depths whose informativeness profile is approximately Gaussian around the band center, rather than concentrating at a single optimal point. We propose Agent-G², a Gaussian guidance framework that draws the depth per task from a Gaussian whose center and spread are estimated online from rollouts already collected for policy optimization, requiring no probe rollouts or learned depth predictor. The center combines a global baseline with per-cluster difficulty, and the spread tracks within-cluster variance. We evaluate Agent-G² on ALFWorld and WebShop with Qwen2.5-1.5B / 7B-Instruct. Agent-G² outperforms the strongest hint-based, hint-free, and Aux-RL baselines on ALFWorld by 2.3 / 3.9 / 7.4 points at under one-third the rollout cost of per-sample probing.

Abstract:
The Key–Value (KV) cache has become a critical memory bottleneck that significantly hinders the deployment of large language models, especially under long-context inference. Existing approaches either introduce non-negligible errors through lossy compression, or are tightly coupled to specific tasks or model structures, limiting their general applicability. In this work, we first identify that the exponent values in BFloat16 KV caches are highly concentrated, and further prove theoretically that such concentration is inherent to LLMs, ensuring the broad generalizability of our method. Leveraging this insight, we propose AdaCode, a bit-exact lossless compression framework. The core idea is to encode frequent exponents with a short fixed-length code while storing rare outliers explicitly. AdaCode consists of two key components: (1) a minimal-cost optimization that selects the optimal code length and exponent range by balancing the storage overhead of short codes against that of outliers, complemented by a lightweight online adjuster to handle occasional distribution shifts without GPU overhead; and (2) a GPU-friendly byte-aligned packing format together with offline calibration for parameter precomputation. Experiments show that AdaCode achieves an average compression ratio of 71.78% on diverse LLMs and 70.26% on vision-language models, with bit-exact identical recovery and no accuracy loss, effectively alleviating the KV cache memory bottleneck. The codes are available at https://github.com/lichangshiwhu/Adacode .

Abstract:
As Large Language Models (LLMs) increasingly generate reasoning-driven outputs that weave isolated events into intricate logical topologies, factuality depends not only on isolated atomic facts but critically on the faithful relations connecting them. However, current verification paradigms prioritize individual facts and remain fundamentally blind to relational distortions. This oversight creates a dangerous false sense of security, where verifiers inadvertently endorse relationally corrupted claims simply because their constituent events are factual. Furthermore, the community lacks a systematic benchmark to evaluate this crucial relational layer. To address these critical vulnerabilities, we introduce BeyondAtoms , a novel benchmark for relational fact verification. Our evaluations reveal that generic verification pipelines often fail by misclassifying relational manipulations as supported. While explicitly guided reasoning models can resolve these structural errors, they incur prohibitive computational costs. To counter this pervasive relation blindness, we propose CARVE, a completely training-free verification framework. By decoupling structural dependencies through deterministic extraction and bi-directional relational alignment, CARVE achieves a favorable accuracy–efficiency trade-off: it remains highly competitive on relational and general fact verification while avoiding the heavy inference cost of reasoning-based LLMs.

Abstract:
Humans can infer others’ beliefs, intentions, and emotions from observable behavior, an ability known as Theory of Mind (ToM). While current Multimodal Large Language Models (MLLMs) excel at visual understanding, can they identify mental-state cues in videos and reason about evolving mental states? To systematically evaluate spatio-temporal ToM in MLLMs, we introduce ToM-Video, a large-scale benchmark built from diverse real-world narrative videos. Through multi-stage filtering and annotation from massive data sources, ToM-Video provides a high-quality collection of ToM-centric scenes, comprising 1,487 videos and 41K visual question answering (VQA) pairs. We further propose INSPIRE, a post-training framework that, through a carefully designed three-stage curriculum, progressively teaches models to locate ToM-relevant cues and connect them to latent mental-state changes. Experiments show that current MLLMs remain weak at linking observable cues to evolving mental states, while INSPIRE shifts the model from surface description toward compact, evidence-backed inference, leading to higher accuracy and better generalization to external social-reasoning video datasets.

Abstract:
LLMs increasingly use external tools and stateful environments for complex, multi-step tasks. Further improving their tool-use capabilities, however, remains bottlenecked by the lack of scalable stateful environments and verified multi-turn trajectories matched to the target LLM’s current capability boundary. We observe that agent skills, packages pairing workflow specifications with executable scripts, provide scalable seed material for tool-use environment synthesis. This motivates Grafting, a skill-to-environment synthesis loop in which coding-agent harnesses perform open-ended construction under deterministic validation contracts, producing self-contained executable environments that are refined using trajectory-level verdicts. In an initial end-to-end run, Grafting produces 500 environments and approximately 12K verified multi-turn trajectories. We train on these trajectories with Fathoming, an iterative GRPO procedure that selects verified trajectories near the target model’s capability boundary. Training a 4B-parameter instruction-tuned model with two rounds of Fathoming on the trajectory pool produces Fathom-4B and yields strong out-of-distribution gains: BFCLv4 multi-turn accuracy rises from 22.0 to 39.7 (+17.7 points), and ComplexFuncBench per-call accuracy rises from 49.6 to 61.6 (+12.0 points). Using the same environment-synthesis pipeline, we further construct Grove-5K, a collection of 5,101 executable environments. We release Grafting, Grove-500, Grove-5K, and Fathom-12K.

Abstract:
Emotions are an essential aspect of human communication, particularly on social media, where authors frequently combine text and images to convey their emotions. Yet prior work on emotion analysis of social media posts has overlooked two important aspects in regard to measuring how well readers can reconstruct the authors’ intent: (1) the image modality, with most work focusing solely on text, and (2) the real-world events that trigger the expressed emotions, and their relationship to the post content. We therefore study the relation between (a) the author’s experience of the event that caused them to write a social media post and (b) the content of the post, with a focus on readers’ capability to reconstruct that emotion expression. To do that, we introduce the Multimodal Multi-Emotion-Model dataset (Multi^2EMo), collecting annotations from both authors and readers on the posts and their triggering events. We find that reconstruction is possible but challenging for both human readers and computational models. We show that understanding the triggering event is crucial for accurate reconstruction, and that reconstruction is particularly challenging when posts rely heavily on the image to express emotion.

Abstract:
Post-training quantization (PTQ) has become an important technique for reducing the inference cost of Large Language Models (LLMs). While recent mixed-precision methods improve ultra-low bit quantization by preserving critical subspaces in high precision, they typically construct these subspaces relying solely on activation statistics. This ignores the fundamental nature of linear operations, where the output perturbation is jointly driven by both activation and weight quantization noise. In this paper, we propose CoQuant, a joint weight-activation subspace projection method. By theoretically modeling the expected output error, CoQuant formulates a closed-form weighted PCA solution that balances activation and weight covariances to select the optimal high-precision subspace. Extensive experiments on Llama-3.2 and Qwen2.5 models show that CoQuant consistently outperforms strong PTQ baselines in both WikiText perplexity and zero-shot common-sense reasoning accuracy. These results demonstrate that joint weight-activation subspace modeling provides a principled and effective direction for low-bit LLM quantization. The source code is available at https://github.com/Zachary5895/CoQuant

Abstract:
Post-training quantization (PTQ) has become a key approach for large language model (LLM) compression. Existing methods commonly adopt block-level reconstruction as a surrogate to calibrate each quantized Transformer block. However, our analysis shows that this loose objective can admit cancellation patterns , where the aggregated block-level reconstruction error is small even though the per-head errors remain large. We therefore revisit the PTQ formulation from a variational perspective. By treating per-head representations as latent variables and the block output as the observed variable, we find that conventional block-level reconstruction corresponds to optimizing only the observation-level term, while omitting a latent alignment term that captures head-level quantization error. Motivated by this finding, we propose HEAL , a Latent HE ad AL ignment framework that turns block-level reconstruction calibration into joint observation-latent calibration with stop-gradient head alignment. Experiments on LLaMA-2/3 and Qwen2.5 families under low-bit weight-activation quantization show that HEAL generally improves perplexity and zero-shot accuracy over block-level reconstruction baselines.

Abstract:
Document-level translation remains one of the most challenging tasks for large language models, which are constrained by limited context windows that impede global cohesion, while simultaneously suffering from redundant contextual information that degrades translation quality. To address this, we propose a human-like long document translation agent called Loong, which leverages a 3E memory module (Essence-Exemplar-Entity) to store summaries, sentence pairs, and entity records as historical context. Instead of passively attending to all history, Loong performs deep reasoning to adaptively identify the optimal context for translation guidance. Loong optimizes its context policy through reinforcement learning, utilizing preference data derived from its own sampled observe-and-act reasoning trajectories. Empirical evaluations demonstrate that Loong achieves substantial translation quality improvements in English ⇔ Chinese, German, and French directions, with average gains of up to 13.0 points across the three evaluation metrics. Furthermore, Loong exhibits strong generalization across domains and robustness against contextual noise, while maintaining remarkable stability in ultra-long document translation. Our code is freely available at https://github.com/YutongWang1216/LoongDocMT.

Abstract:
Scheming occurs when a model recognizes goal conflict and strategically pursues an instrumental objective over user or developer interests, creating the risk of covertly misaligned behavior. Final outputs are the primary judgment signal, but they often cannot by themselves distinguish strategic deviation from ordinary error or routine harmful compliance; reasoning traces can provide auxiliary evidence about whether the model recognizes the conflict and adopts an instrumental strategy, making evaluation an observability problem. To mitigate this issue, we propose Cognitive Honeypots, which embed instrumental incentives, explicit goal conflict, and supervision cues into otherwise solvable tasks so that conflict-induced strategic deviation is more likely to surface without directly instructing deception or policy violation. We pair this capture mechanism with an intent–output decoupled protocol that labels High, Medium, and Low evidence. We instantiate it as SchemingBench, a bilingual benchmark with 4,000 honeypot scenarios across 5 behavior categories and 8 domains. Experiments across 12 models demonstrate that peak high-severity rate (HSR) reaches 70.76% in Chinese and 72.70% in English. In a matched ablation, all models show high HSRs under honeypot settings, with the largest increase in HSR reaching 65.66 points. Condition analyses further suggest that scheming-related evidence becomes most visible when instrumental incentives and explicit goal conflict are jointly present.

Abstract:
User modeling characterizes individuals through their preferences and behavioral patterns to enable personalized simulation with Large Language Models (LLMs) in contemporary approaches. However, existing methods, whether personalized prompt-based or parameter-based methods, face challenges in balancing personalization quality against computational and data efficiency. We propose a novel framework CURP, which employs a bidirectional user encoder, a discrete prototype codebook and a dual-aggregator to extract both multi-dimensional user traits and query-aware user characteristic. This design enables plug-and-play personalization with a small number of trainable parameters (about 28M parameters, about 0.36% of the total model size). Through extensive experiments, we show that CURP achieves superior performance and generalization compared to strong baselines on the language model personalization (LaMP) benchmark, while offering better interpretability and robustness.

Abstract:
Neural ranking models have become core components of modern information retrieval systems and important building blocks of AI systems such as retrieval-augmented generation (RAG) pipelines. However, their robustness remains insufficiently understood in the presence of large language models (LLMs), which can generate fluent and deceptive content at scale. This work investigates the vulnerability of neural ranking models to corpus poisoning attacks, in which an adversary injects a small number of maliciously crafted documents into the corpus to distort ranking behavior. We propose VerTox, the first framework to formulate corpus poisoning as a verifiable reward-guided reinforcement learning (RLVR) problem. By explicitly coupling ranking distortion with factual corruption through specialized reward shaping, we fine-tune compact LLMs into adversarial generators. Experiments demonstrate that our method achieves near-perfect attack success rates, producing adversarial documents that frequently rank higher than target documents across major neural ranking architectures, as well as a proprietary commercial embedding model. The generated adversarial documents are fluent and exhibit low perplexity, making them difficult to detect. Furthermore, by explicitly encouraging factual corruption, our adversarial documents significantly degrade the performance of a downstream RAG application.

Abstract:
Large Language Models (LLMs) are known to contain significant redundancy, yet a systematic explanation for why certain components, particularly in higher layers, are more redundant has remained elusive. In this work, we identify the BOS sink phenomenon as a key mechanism driving this layer-wise sensitivity. We show that attention heads with high BOS sink scores are strongly associated with functional redundancy: such heads, especially in deeper layers, contribute little to predictive performance and effectively serve as dumping grounds for superfluous attention weights. Leveraging this insight, we introduce a simple pruning strategy that removes high-BOS sink heads. Experiments on Gemma-3, Llama-3.1, and Qwen3 demonstrate that this approach identifies redundant transformer components more reliably than weight- and activation-based criteria in terms of downstream task retention, remaining close to dense baselines at low-to-moderate pruning ratios. We further find that high-scoring sink heads sustain their focus on BOS as context length grows. Overall, our results suggest that structural properties of attention offer a more direct basis for model compression than magnitude-based methods.

Abstract:
In click-through rate prediction tasks, learning high-quality feature embeddings from multi-field categorical data is an essential task. Early approaches typically need to transform categorical features into one-hot encoded vectors for producing embeddings. This transformation often results in loss of semantic information. In the era of large language models, it has become necessary to transform multi-field categorical data into text to achieve embedding of semantic information. Moreover, semantic information of each categorical feature can be useful to model the relationships among categories for enhancing the embedding quality. To address this point, in this paper, we propose an approach of dual-prompt-driven textual knowledge graph generation with heterogeneous information embedding for enhancing the representation of multi-field categorical data. Specifically, we design two prompt templates to transform multi-field categorical data into a textual knowledge graph for problem reformulation, and propose a heterogeneous information embedding approach that works by jointly representing the generated text and knowledge graph in the setting of cross-attention gated modality fusion with feature alignment. The experimental results indicate that our approach outperforms those baselines of click-through rate prediction. The results also indicate that each module of our approach is useful for improving the performance.

Abstract:
The spread of misinformation poses serious societal harms. While detection has been extensively studied, modeling user susceptibility remains a challenging yet important direction for strengthening users’ resilience. In this work, we formally define the Personalized Misinformation Susceptibility Prediction (MiSP) task, predicting whether a specific user is susceptible to (i.e., believe) a particular misinformation claim. We construct benchmark datasets derived from social science surveys and social-platform data, and design diverse evaluation settings covering different seen/unseen user–claim scenarios. Using this benchmark, we examine LLMs’ ability to simulate personalized susceptibility under realistic multi-attribute user profiles, and further evaluate LM-, LLM-, and GNN-based approaches, revealing their limitations. Motivated by these findings, we propose VulneraSpotter, which integrates semantic information and behavioral patterns via heterogeneous and homogeneous GNNs, enabling effective generalization to previously unseen users and claims. Extensive experiments demonstrate that VulneraSpotter outperforms existing baselines in predicting personalized susceptibility, uncovers meaningful group-level patterns, and shows promising potential for mitigating misinformation spread. This work provides a benchmark and modeling foundation for user-centric misinformation understanding and mitigation.

Abstract:
Hybrid-thinking language models expose both cheap non-thinking and expensive thinking modes, creating a deployment problem: deciding when additional reasoning is worth its token cost and override risk. We formulate this problem as budgeted opportunity allocation, where a controller first obtains a cheap seed answer, estimates the marginal value and risk of paid thinking, and preserves the seed unless an adoption gate accepts the thinking result. We propose Bora-Budget , a budget-aware inference framework for frozen LLMs, and instantiate it as Bora-Switch-Calibrated . The controller uses seed-state features to predict helpfulness, harmfulness, and thinking cost, then allocates thinking through either frontier operating points or an AutoThreshold policy calibrated to a total token budget. Across LLM scales on math dataset, the method substantially improves over random allocation and approaches uniform-thinking accuracy under lower token budgets. Transfer results on multiple-choice, code, and hard-QA tasks show promising but uneven generalization: trace length remains a strong baseline, harmful overrides are reduced but not eliminated, and learning how much to think remains open. These results position Bora-Budget as a seed-preserving control layer for budgeted inference rather than a replacement for stronger always-thinking baselines.

Abstract:
Output-token budgets are a first-order deployment constraint for large language model (LLM) agents, yet existing benchmarks typically evaluate them under a single fixed or generous budget. We introduce TokenCliff, a benchmark for measuring multi-turn agent performance across episode-level output-token budgets. It contains 660 deterministic tasks spanning four capability-oriented domains—file operations, data transformation, tool use, and planning—plus a compositional multi-step split. We propose two budget-sensitivity metrics: Budget Elasticity, which captures robustness across budget levels, and the Cliff Index, which captures the sharpest adjacent-budget performance drop. Evaluating 10 LLMs at five budget levels from 200 to 8,192 tokens yields 33,000 episodes. Results show that (1) accuracy drops by up to 32 percentage points between adjacent budgets; (2) neither peak performance nor larger model size necessarily implies stronger budget elasticity; and (3) budget sensitivity is substantially greater in the evaluated multi-turn setting than in the single-shot diagnostic. Code and data are available in our public repository: https://github.com/BairuiZhang/TokenCliff

Abstract:
Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users. We propose Knowledge Trap, a defense that redirects extraction attacks toward low-transferability knowledge through a Honeypot Knowledge Graph (HKG) and breadcrumb-guided exploration. Instead of blocking queries or perturbing outputs, Knowledge Trap consumes the attacker’s limited query budget on knowledge with negligible downstream utility while preserving benign-user performance. Experiments in medical and financial domains show that Knowledge Trap reduces surrogate Agreement by 6.2% on average without degrading legitimate-user accuracy, outperforming existing defenses that impose measurable user impact. These results suggest that defending knowledge-space traversal is a practical direction for mitigating LLM extraction attacks.

Abstract:
A key part of developing large language model (LLM)-powered, automated tutoring tools is student simulation, i.e., using LLMs to role-play as students, which can facilitate tutor model evaluation and training. Existing work mostly focuses on within-dialogue simulation, which lacks context on student knowledge and behavior, partly due to not grounding in past student question-answering or dialogue interactions. In this work, we introduce the task of history-conditioned student simulation, where the goal is to accurately predict student dialogue turns by leveraging information in the student’s learning history. We propose a two-component framework in which a profile generator summarizes a student’s history and a simulator predicts student turns conditioned on the resulting profile. We train both components with reinforcement learning (RL), yielding profiles optimized for faithful student simulation. We evaluate our method and baselines on the first-of-its-kind real-world dataset of student dialogues and question responses that we collect from a math learning platform. Extensive experiments show that our method significantly outperforms baselines, and demonstrate the importance of history, profiles, and RL training.

Abstract:
Training prompts in online reinforcement learning (RL) differ substantially in how informative they are for the current policy: some are already saturated while others are too difficult to yield reliable learning signals, yet both receive equal rollout budget under standard training. We propose an exploration-guided prompt scaffolding framework that adapts the training prompt distribution dynamically throughout RL post-training of multimodal large language models (MLLMs). Central to our approach is the Exploration Potential Score (EPS), a lightweight rollout-based proxy for prompt utility derived from KL-regularized policy improvement theory, computable directly from on-policy rollout statistics without additional overhead. Rather than discarding low-utility prompts, we use a teacher model to generate scaffolded rewrites that preserve the original task intent while making subsequent training more informative, reframing teacher supervision as training-data refinement rather than output imitation. Integrated with GRPO on Geo3K and MMK12, our method consistently outperforms the baseline on both in-domain and out-of-distribution benchmarks, achieving up to 9.7% relative improvement in-domain and gains of 11.5% on MathVision and 11.1% on MMMU-Pro.

Abstract:
LLM agents are moving from single-prompt use to long task streams in which reusable memory becomes a core capability for terminal, software-engineering, and web tasks. Such memory is useful only when stored experience remains reliable across hundreds of interactions, but two failure modes break that assumption in practice. The first is unreliable admission: failed trajectories, accidental successes, and misleading observations enter memory because they appear relevant, then mislead later decisions. The second is memory drift: long-running banks accumulate duplicate, stale, and conflicting records that retrieval alone cannot repair. MemGuard’s key distinction is to treat verifier output not as a one-shot filter, but as persistent lifecycle metadata. It converts multi-criteria score-token verification into reward, confidence, label, and uncertainty descriptors that are attached to every candidate before activation and reused during retrieval, conflict resolution, summarization, and archival. We evaluate MemGuard on Terminal-Bench 2.0, SWE-Bench Verified, WebArena, and Mind2Web across four backbones, comparing against four memory baselines plus a verifier-only control under matched runtime budgets. Averaged over five seeds, MemGuard achieves the best success metric and lowest average steps in all 16 backbone–benchmark settings, improving over ReasoningBank, the strongest prior baseline among the memory methods we evaluate, with a largest gain of 7.9 success-rate points on WebArena, 5.6 step-success-rate points on Mind2Web, and 2.4–3.5 points on terminal and software-engineering benchmarks. Code is available at [https://github.com/whyyyyy123/MemGuard](https://github.com/whyyyyy123/MemGuard).

Abstract:
Diffusion large language models (dLLMs) re-encode the entire prefix at every denoising step, causing recomputation that scales quadratically with context length and becomes prohibitive for long-context scenarios. We propose Prefilling-dLLM, a training-free prefill-decode disaggregation framework for dLLMs that partitions the prefix into N chunks, caches their KV representations once, and selects the top-K most relevant chunks with intra-chunk token sparsity for decoding, showing that sparse prefilling can outperform dense attention while reducing per-step complexity from quadratic in the full sequence length to quadratic only in the decode length. On LongBench and InfiniteBench, Prefilling-dLLM achieves state-of-the-art quality among dLLM acceleration methods, and an attention kernel that parallelizes decoding over the non-contiguously cached chunk KV yields 9.1–28.0× speedup at 8K–32K contexts. We further show that beginning-of-sequence tokens prepended to each chunk act as periodic attention anchors that eliminate the lost-in-the-middle phenomenon. Our code is available at https://github.com/menik1126/Prefilling-dLLM.

Abstract:
Data sparsity remains a long-standing challenge in recommender systems, and it becomes more severe for methods relying on user-generated content (UGC) such as textual reviews, which capture fine-grained preferences but require more user efforts to produce. As a result, UGC exhibits (1) missing reviews, where interactions lack any review, and (2) incomplete reviews, where available reviews cover only a subset of relevant attributes. Existing approaches often overlook these UGC-specific issues, leading to degraded accuracy. Motivated by meta-review in academic peer review, we propose MOSAIC (Meta-review On Sparse And Incomplete user-generated Content), which constructs a meta-review for each target user by aggregating attribute-sentiment evidence from neighbor users’ reviews. A multi-gate mixture-of-experts (MMoE) architecture jointly optimizes rating prediction and meta-review attribute-sentiment prediction, while an attention module personalizes the aggregated meta-review signals to each target user, yielding both refined rating predictions and attribute-level explanations. Experiments on four real-world datasets demonstrate that MOSAIC consistently outperforms state-of-the-art baselines in both recommendation accuracy and explanation quality, mitigating UGC sparsity and incompleteness while delivering consistent gains for users with limited interaction history.

Abstract:
Long-context language models are limited by the memory footprint of the key-value (KV) cache. Existing training-free KV compression methods usually rank tokens by one importance signal — attention, recency, layer-wise allocation, or key distinctiveness — which becomes brittle when useful context is globally distinctive, locally episodic, or immediately relevant. We introduce NestedKV, a key-only KV cache compression method inspired by the Continuum Memory System in Nested Learning. NestedKV maintains global, block-level, and sliding-window key anchors, scores tokens by multi-time-scale cosine anomaly, and combines the resulting rankings with a training-free outer learner using head-adaptive mixing and surprise-gated token routing. The score is paired with adaptive per-head budgets and requires no training or LLM modification. Across RULER (4k–32k), LooGLE, LongBench, LongBench-E, InfiniteBench, and MMLU-Pro on Qwen3 and Llama-3.2 models, NestedKV is strongest when the retained cache is small. On Qwen3-4B, it improves over KeyDiff by up to 19.10 points on RULER and 19.29 on LongBench at r=0.75 ; at r=0.95 , it retains 37.32 on LongBench versus 17.55 for KeyDiff.

Abstract:
Large reasoning models (LRMs) achieve strong mathematical reasoning performance in English, but remain much less reliable in many low- and medium-resource languages. This gap is often explained as a failure to understand non-English problem statements. We show that this view is incomplete: even when the problem is given in English, controlling the model’s reasoning language can substantially reduce accuracy, suggesting that language also affects reasoning execution itself. To study this effect, we introduce DATG, a Directed Acyclic Trace Graph framework that maps reasoning traces to language-independent mathematical anchors and dependencies. This allows us to align target-language traces with reference DAGs and measure whether they cover required mathematical nodes, respect dependency edges, and avoid harmful mathematical actions. Experiments on the Qwen3 series across 12 languages show that non-English reasoning often suffers from reduced anchor coverage and weaker dependency fidelity, especially in low-resource languages. Motivated by this diagnosis, we propose Loop-Retry and Formula-Retry, two simple test-time controls targeting DATG-exposed failure modes, and show that they consistently improve target-language reasoning performance in low-resource languages.

Abstract:
Visual grounding is typically evaluated as a one-shot mapping from an informative referring expression to a visual target. This formulation misses a central property of real-world reference: initial referring expressions are often incomplete or ambiguous, requiring participants to establish shared understanding through interaction. We introduce a controlled evaluation framework for interactive visual grounding in large vision-language models (LVLMs), varying how much target information is provided upfront and how much must be acquired through dialogue. Across four human-grounded visual contexts and four interaction protocols, current LVLMs perform significantly below task-level human baselines. Interaction can help when follow-up questions refine or repair an initial target description. Performance is lowest when no initial description is provided and target information must be acquired through questions, indicating that proactive question-driven grounding remains difficult. LVLMs are also poorly calibrated, often reporting confidence that exceeds their empirical accuracy. Follow-up studies confirm these patterns across varied description sources (human versus AI), reasoning efforts, repeated interactions, description providers, and visual contexts. Overall, interactive visual grounding remains challenging, requiring visual matching, information seeking and synthesis.

Abstract:
Diversity is a critical property of text corpora, with direct implications for training data curation, test set design, and model generalization. However, its measurement remains fragmented and poorly understood. Existing embedding-based diversity measures vary widely in how they are computed and can yield substantially different diversity scores on the same data, yet no systematic comparison of their empirical behaviour exists. We make two contributions toward addressing this gap. First, we present a taxonomy of 21 embedding-based diversity measures, organizing them into four families based on their underlying computations. Second, we introduce EmbDivBench, a benchmark that evaluates these measures along three conceptually distinct components-variety, balance and disparity-on both simulated and natural text data. Although Mean Pairwise Distance is the most commonly used measure, our experiments show that it is outperformed by several alternatives on the three diversity components, with Graph Entropy being the best choice overall. Lastly, we distil these findings into practical recommendations for diversity measure selection.

Abstract:
We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-visual streams. Unlike offline video understanding or text-prompted streaming QA, OmniInteract preserves the original audio-visual stream and requires models to process it online, without access to future content. User queries and ambient sounds are embedded in the audio track, requiring models to detect multimodal triggers, decide when to respond, and answer while the stream unfolds. OmniInteract contains 250 videos with 1,430 temporally grounded response slots: 1,062 1Q1A slots across real-time, proactive, and nested scenarios, and 368 1QnA slots for continuous task monitoring and step guidance. Each slot includes a trigger, response window, and target answer. We evaluate response correctness, timing, invalid outputs, interruption handling, and context continuity using Interaction-Aware Quality-Timeliness F1, Interruption Diagnostic Suite, and Nested Chain Completion Score. Experiments show that current models remain weak in streaming interaction, with the best overall IA-QTF1 reaching only 0.368 and the best 1QnA IA-QTF1 only 0.052. Further study on mathematical reasoning in full-duplex settings shows that offline capability does not necessarily transfer to online interaction. Code and datasets will be made publicly accessible at https://github.com/Lucky-Lance/OmniInteract.

Abstract:
Speculative decoding is an effective and lossless approach for accelerating LLM inference. However, existing widely adopted model-based draft designs, such as EAGLE3, improve accuracy at the cost of multi-step autoregressive inference, resulting in high drafting latency and ultimately rendering the drafting stage itself a performance bottleneck. We propose DART, a low-latency parallel drafting framework for speculative decoding. DART predicts logits for multiple future masked positions in parallel within a single forward pass based on hidden states of the target model, thereby eliminating autoregressive rollouts in the draft model while preserving a lightweight design. Based on these parallel logit predictions, we further introduce an efficient tree pruning algorithm that constructs high-quality draft token trees with N-gram-enforced semantic continuity. DART substantially reduces draft-stage overhead while preserving high draft accuracy, leading to significantly improved end-to-end decoding speed. Experimental results demonstrate that DART achieves a 2.03× – 3.44× wall-clock time speedup across multiple datasets, surpassing EAGLE3 by 30% on average and offering a practical speculative decoding framework.

Abstract:
Spatial audio vocoders are able to convertmel-spectrograms produced by generative mod-els into spatial audio waveforms. Most neural vocoders are designed for monaural audio, and direct extensions to spatial audio can de-grade spatial quality by ignoring inter-channelcues. We present CSAVocoder, a causal GAN-based spatial audio vocoder that jointly optimizes waveform fidelity and spatial rendering. Our framework introduces a Spatial Adaptor that fuses multi-channel mel-spectrograms with dynamic source-listener pose information, together with a spatial consistency discriminator that supervises inter-channel cues.To meet real-time requirements, we design a strictly causal, stateful generator that supports efficient streaming inference with constant memory overhead. Experiments on largescale spatial audio datasets show that CSAVocoder improves spatial fidelity at competitive audio quality and real-time performance.Our project page is at: https://github.com/MM-Speech/CSAVocoder.

Abstract:
Large language models (LLMs) have become increasingly used for various tasks, often coupled with chain-of-thought (CoT) prompting to boost accuracy. Recent work has shown that high label-prediction accuracy does not guarantee correct intermediate reasoning, and the causes of reasoning flaws vary from sample to sample, yet existing remedies either focus on a single domain or assume that one flaw type applies uniformly across samples. A simple mitigation method is to provide the model with the correct answer, but we show that this yields no consistent improvement in reasoning quality. This indicates that the problem cannot be fixed by LLMs’ awareness of answers, and must instead be addressed through the structure of reasoning. Motivated by this, we propose CRAFT (Consensus Reasoning-knowledge-graph Aggregation for Flaw-aware Trace synthesis), which aggregates the consensus components shared across multiple candidate reasoning traces to synthesize improved ones. CRAFT consistently improves label-prediction accuracy on both logical and mathematical reasoning benchmarks, outperforming most baselines, while its post-processed traces achieve higher quality under fine-grained benchmark evaluation.

Abstract:
Preference tuning aligns base language models to human judgments of quality, helpfulness, or safety by optimizing over explicit preference signals rather than likelihood alone. Prior work has shown that preference tuning degrades performance and reduces helpfulness outside the training domain. However, the extent to which adaptation strategies mitigate this domain shift remains unexplored. We address this challenge by conducting a comprehensive and systematic study of alignment generalization under domain shift. We compare five popular alignment objectives and various adaptation strategies from source to target, including target-domain supervised fine-tuning and pseudo-labeling, across summarization, question-answering helpfulness, and safety alignment tasks. Our findings reveal systematic differences in generalization across alignment objectives under domain shift. We show that adaptation strategies based on pseudo-labeling substantially reduce domain-shift degradation but induce mode collapse, revealing a generalization–diversity trade-off.

Abstract:
Despite the remarkable progress of Large Language Models (LLMs), the escalating memory footprint of the Key-Value (KV) cache remains a critical bottleneck for efficient inference. While dimensionality reduction offers a promising compression avenue, existing approaches typically either necessitate prohibitively expensive pre-training from scratch or suffer from severe performance deterioration under high compression regimes. In this work, we propose DynaKV, a novel post-training framework for low-rank KV cache compression. To the best of our knowledge, DynaKV is the first low-rank method to dynamically allocate compression rates to individual tokens according to their semantic meaning, which allows it to achieve better fidelity at aggressive compression ratios. Extensive experiments demonstrate that our method consistently outperforms existing state-of-the-art compression techniques, achieving significant memory reduction while maintaining competitive generation quality. Furthermore, our approach is orthogonal to sequence-level pruning methods. When integrated with SnapKV, DynaKV retains only 6% of the KV cache while maintaining 94% of the baseline performance on the LongBench benchmark.

Abstract:
Existing benchmarks for MLLM-generated web artifacts assess interaction through local evidence and miss the requirement-induced states and transitions that determine whether a page works. We introduce WebRISE, which compiles task requirements into Interaction Contract Graphs (ICGs) of observable states, user-intent transitions, and DOM/visual assertions for implementation-agnostic browser execution. WebRISE spans 442 tasks across five input modalities (Text, Markdown, Sketch, Image, Video), with 5,495 transitions and 5,271 requirement checks that separate user-stated functions from implicit product-level constraints. Across 14 MLLMs, even the strongest model reaches only 65.6% transition validity and 66.3% requirement coverage, and visual quality is no proxy for behavior (Qwen3.6-35B-A3B on Markdown: V=80.8 yet T=15.5 ). Video gives the strongest interaction signal ( +10.6 pp implicit coverage over Text), while implicit constraints persist; defect injection shows ICG-based scoring detects state errors at 2– 16× the rate of checkpoint-style evaluation.

Abstract:
Role-playing conversational agents (RPCAs) built on large language models often leak out-of-boundary knowledge: when asked about topics outside a character’s limits, the model’s Chain-of-Thought reasoning reveals knowledge that the character should not possess, undermining immersion. To locate the source of this leakage, we conduct heuristic experiments at three levels of granularity, and find that intervening on individual Feed-Forward Network (FFN) neurons yields the strongest and most effective suppression. Motivated by this finding, we propose the Dynamic Attention Gating (DAG) Router, a lightweight trainable network that encodes the system prompt and dynamically generates per-layer FFN gating signals at inference time. Evaluated on multiple models and benchmarks, the DAG Router improves out-of-boundary scores by up to +162% compared with the original baseline, along with better overall role-playing quality.

Abstract:
Flexible adaptation to context and shared pragmatic intuitions contribute to smooth human conversation. Iterated reference games—in which players repeatedly pick out novel referents using language—present a test case for agents’ ability to perform context-sensitive pragmatic reasoning in multi-turn linguistic environments. We tested humans and vision–language models on their ability to identify the intended meaning of descriptions produced in iterated reference games, varying the provided context in terms of amount, order, and relevance. While humans performed well consistently, the models we evaluated could make use of prior context to interpret humans’ referring expressions, but they struggled to build up the relevant context to interpret those expressions effectively. Our results suggest that the models we evaluated lack core skills needed for efficient linguistic collaboration.

Abstract:
Large Language Models (LLMs) are increasingly trained to abstain from answering questions they are unsure about. However, this ability is often misused: in real-world applications, input prompts sometimes contain uncertainty elements, and driven by this, LLMs are inclined to abstain even on problems they are capable of solving. We argue that LLM abstention is not only an expression of genuine uncertainty; it is also an artifact that can be largely influenced by prompts. We name this phenomenon Abstention Inflation. We add "Unknown" as an extra option for LLMs to choose from; experiments show serious accuracy drops on True/False Questions (TFQs). Replacing "Unknown" with an unrelated random word produces an identical effect. We argue that LLMs are trained to imitate the surface pattern of abstention, rather than to express genuine uncertainty. Based on ten experiments, we support four claims that form a progressive argument: (C1) Abstention Inflation is triggered by the structural presence of an extra option, not by genuine uncertainty; (C2) further, it makes the model deny it can answer even when it can; (C3) at the representation level, this manifests as a later-layer output override; (C4) finally, this bias is stable and emerges through instruction tuning, rather than stochastic noise.

Abstract:
Long-video question answering remains challenging for Vision-Language Models (VLMs), as answer-relevant evidence is often sparse, transient, and temporally dispersed across lengthy video contexts. Existing frame-centric approaches improve efficiency through uniform sampling, query-aware frame selection, visual-token compression, and adaptive resolution strategies. However, they still rely on isolated and fragmented frames as the fundamental evidence units, limiting VLMs’ ability to capture coherent event-level semantics. To address this limitation, we propose MemoryCard, a video-memory-based augmentation framework that organizes long videos into self-contained Memory Cards. Specifically, MemoryCard first performs a self-reading process over videos and aligned utterances to segment the video into semantically coherent units, each corresponding to a distinct topic or event. For each unit, it generates an event-level video gist and selects representative visual moments, which are then rendered into unified Memory Cards for retrieval and question answering. Experimental results demonstrate that MemoryCard consistently improves long-video QA performance under comparable visual-token budgets, achieving up to a 21.8% relative improvement in accuracy. All code is available at https://github.com/NEUIR/MemoryCard.

Abstract:
Retrieval-Augmented Generation (RAG) has become a dominant paradigm for mitigating hallucinations in Large Language Models (LLMs) by incorporating external knowledge. However, existing RAG systems often struggle to effectively integrate and reason over key evidence scattered across noisy retrieved documents, particularly in multi-hop scenarios. In this paper, we propose KAIR, a Knowledge Anchoring framework for Iterative Retrieval that anchors knowledge within retrieved knowledge to guide LLMs to locate the key information. During iterative retrieval, KAIR progressively updates the knowledge index to anchor salient evidence from retrieved documents. The evolving index serves as a navigational anchoring index that enables the LLM to assess knowledge sufficiency and formulate subsequent retrieval queries. Finally, KAIR generates answers by jointly leveraging the retrieved documents and the finalized anchoring index. Experiments on four multi-hop question answering benchmarks demonstrate that KAIR consistently outperforms strong RAG baselines. Further analysis shows that KAIR effectively anchors key knowledge and alleviates the context noise during iterative retrieval, improving the LLM’s ability to associate and reason over dispersed evidence across retrieved documents. All code and data are available at https://github.com/NEUIR/KAIR.

Abstract:
Large Language Models (LLMs) trained on massive data capture rich information embedded in the training data. However, this also introduces the risk of privacy leakage, particularly involving personally identifiable information (PII). Although previous studies have shown that this risk can be mitigated through methods such as privacy neurons, they all assume that both the (sensitive) training data and user queries are in English. We show that they cannot defend against the privacy leakage in cross-lingual contexts: even if the training data is exclusively in one language, these (private) models may still reveal private information when queried in another language. In this work, we first investigate the information flow of cross-lingual privacy leakage to give a better understanding. We find that LLMs process private information in the middle layers, where representations are largely shared across languages. The risk of leakage peaks when converted to a language-specific space in later layers. Based on this, we identify privacy-universal neurons and language-specific privacy neurons. Privacy-universal neurons influence privacy leakage across all languages, while language-specific privacy neurons are only related to specific languages. By deactivating these neurons, the cross-lingual privacy leakage risk is reduced by 23.3%-31.6%.

Abstract:
Secondary use of clinical data requires removing Protected Health Information (PHI) under regulations such as HIPAA. While de-identification is well studied for text, clinical records are often structured PDFs, where PHI appears across complex layouts, tables, and headers. We introduce Paint-It-Black (PIB), the first benchmark for end-to-end, pixel-level PHI de-identification in clinical PDFs. PIB contains physician-validated synthetic documents with diverse layouts, document types, and controlled visual degradations, annotated with HIPAA-aligned labels and bounding boxes. We evaluate locally deployable Multimodal Large Language Models (MLLMs) and find that even the strongest remain below the reliability required for clinical use. A conventional OCR+NER baseline performs substantially worse than the best MLLMs, highlighting the inherently multimodal nature of clinical PHI de-identification.

Abstract:
Open-web deep research is limited not only by how long an agent searches, but also by how it organizes evidence into independent, verifiable branches. Existing systems often scale single-path ReAct trajectories or long-context search, which can deepen one hypothesis while leaving alternative entities, constraints, or falsifying evidence underexplored. We propose P-Search, a parallel plan–act–verify framework that first builds a grounded background state, then decomposes the task into independently executable evidence branches, runs branch-specific search agents in parallel, and iteratively verifies unresolved constraints before final synthesis. For training, we avoid end-to-end credit assignment over a full multi-stage rollout. Instead, P-Search-React learns reusable grounded search-report behavior from raw-question search instances and is reused for branch execution and targeted supplementary search, while P-Search-Plan learns schema-constrained routing, decomposition, and verification decisions. We supervise this structure with star-shaped subgraph data and role-specific GRPO rewards. Experiments show that instantiating P-Search with open-source backbones such as GLM-5 and Tongyi-DR-30B improves their reported baselines on most open-web benchmarks, while the trained 8B P-Search-React + P-Search-Plan system reaches 59.2% on GAIA and 70.0% on Xbench-DeepSearch. Ablations confirm the complementary roles of Plan, ReAct, and Verify, as well as the transferability of the trained planner across ReAct backbones.

Abstract:
Continuous Sign Language Recognition aims to predict gloss sequences from unsegmented sign language videos. Existing methods typically rely on Connectionist Temporal Classification (CTC) loss for implicit alignment, which is the most dominant loss function for current sequence alignment tasks but causes low-entropy alignment and weak visual-semantic correspondence learning. To address this issue, we propose an explicit temporal-semantic supervision framework that provides dense signals beyond CTC. Specifically, we design the Gloss-Temporal Module (GTM) to construct sustained gloss-to-frame interval supervision by using an LLM to integrate three types of information from RGB videos. We further introduce a joint optimization method with frame-level and sequence-level objectives, coupled with a curriculum learning strategy, enabling a transition from local to global alignment and improving the model’s learning of visual-semantic features. The results show that dense temporal supervision substantially alleviates alignment collapse and enables even simple RGB-only baseline models to achieve competitive performance among state-of-the-art methods without extra cues, while the recognizer converges significantly faster.

Abstract:
Large Reasoning Models (LRMs) achieve strong reasoning capabilities, yet long-chain reasoning becomes inefficient once the intermediate answer stabilizes across reasoning steps: additional reasoning yields little marginal benefit while incurring substantial inference cost. Existing early-exit methods based on confidence or entropy poorly capture reasoning stability, while consistency-based approaches rely on multi-step trajectory agreement, requiring sequential evaluations that delay exit. To better balance efficiency and reliability, we propose SABER, a training-free framework for stability-aware early exit via adversarial branch probing. SABER constructs simple yet effective semantic perturbations around intermediate reasoning states to form adversarial branches, and applies lightweight probing to estimate their likely final outcomes without full trajectory rollouts. When the probed outcomes remain consistent across branches, SABER exits early; otherwise, it continues reasoning. Experiments across multiple reasoning benchmarks and model architectures show that SABER reduces reasoning token consumption by 30.2%–39.8% on average while maintaining competitive accuracy with full-length reasoning.

Abstract:
Real-world data analysis is inherently iterative, yet existing benchmarks mostly evaluate isolated or short interactive tasks, leaving agents’ ability to track evolving analytical context over long horizons untested. We introduce LongDS, a benchmark for long-horizon, multi-turn data analysis where agents must maintain, update, restore, and compose evolving analytical states. LongDS comprises 68 tasks constructed from real-world Kaggle notebooks, spanning 2,225 turns across six domains including Geoscience, Business, and Education. Tasks are designed around state-evolution patterns (e.g., counterfactual perturbation, rollback, multi-state composition), with an average dependency span of 11.3 turns. Evaluating five state-of-the-art models, we find that the best model reaches only 48.45% average accuracy, performance drops nearly 47 points from early to late turns, and long-horizon errors account for 52%–69% of failures. Further analysis shows that additional agent steps do not necessarily improve performance, suggesting that the key bottleneck is maintaining a correct analytical state rather than increasing interaction budget. We release LongDS to support research on reliable long-horizon agentic data analysis.

Abstract:
Large language model (LLM)–based agent applications often incur high response time. Speculative decoding is a promising solution to improve the inference efficiency of LLM agents without impacting generation quality. However, state-of-the-art speculative decoding algorithms exhibit substantial speed degradation under large batch sizes, limiting their effectiveness to deploy in real-world agent applications. In this work, we first present a systematic analysis of speculative decoding for LLM agents and identify two dominant factors of speedup degradation: high rejection rate of speculative tokens, and under-utilization of dynamic token budgets. Based on these observations, we propose AgentSpec, a speculative decoding algorithm that addresses the limitations of existing methods for LLM agents. AgentSpec incorporates structure-isolated drafting that constrains speculation to semantically coherent segments of the agent workflow, reducing the drafts of irrelevant semantic paths and achieving an extremely low rejection rate. Moreover, AgentSpec adopts redundancy-aware budget allocation that exploits agent-level information to better utilize the dynamically-free token budget during the agent inference. We implement and evaluate AgentSpec on five different workloads and four different models from four different LLM families in vLLM. Our results demonstrate the superiority of AgentSpec over state-of-the-arts.

Abstract:
Generative Engine Optimization (GEO) lets content owners rewrite web content to increase their visibility in generative systems. In recommendation agents, this creates a risk that sources controlled by sellers make flawed products appear better supported than they are. We study this risk at the generation stage by asking whether recommendation agents continue to make decisions that align with user utility when these sources are rewritten for GEO. To make this question measurable, we construct SafeGEO, an evaluation suite with 22 GEO attack variants across 600 recommendation cases. We empirically show that GEO attacks can promote flawed target products: they increase the rate at which such flawed products enter the recommendation set by up to 83.2 percentage points (pp). We further study whether agent-side design choices can mitigate this risk and show that simple defenses reduce harmful target promotion by up to 39.2 pp. These gains are substantial but do not restore the no-GEO performance, showing that GEO remains a serious risk despite mitigation.

Abstract:
Environmental, social, and governance (ESG) criteria are essential for evaluating corporate sustainability, yet professional ESG analysis is hindered by data fragmentation and complex multi-step workflows. Existing ESG benchmarks rely on LLM-generated questions that often suffer from semantic ambiguity, limiting their diagnostic reliability. We introduce ESGAgent, a hierarchical multi-agent system with a domain-oriented toolset for automating end-to-end ESG workflows, and a three-level benchmark derived from 310 corporate sustainability reports, featuring expert-sourced questions with verified, unambiguous answers at Levels 1-2 and full report generation at Level 3. Evaluations across 20 state-of-the-art closed-source LLMs show that ESGAgent achieves 84.15% accuracy on Levels 1-2, surpassing the strongest baseline by 3.26 points, and produces Level 3 reports with higher factual consistency and analytical depth than commercial deep research systems, validated by both multi-LLM judging and human expert evaluation, showing a robust alignment (Pearson r=0.551 ).

Abstract:
Retrieval-augmented LLM agents increasingly rely on curated skill banks: collections of reusable textual principles that guide decision making on complex tasks. Existing approaches typically expand these banks in an append-only fashion, continuously adding new skills without removing redundant, outdated, or harmful ones, resulting in inefficient and poorly curated repositories. In this paper, we formulate skill bank curation as a constrained multi-objective problem: a desirable bank must be useful for the agent, diverse in its content, and provide good coverage of the query distribution. To this end, we introduce SkillBrew, a multi-objective curation framework that formalizes skill bank curation as Pareto-aware optimization under a utility constraint, and solves it via a bi-level propose-then-verify loop. We evaluate our approach on three public benchmarks. Our findings suggest that treating skill banks as objects of principled curation, rather than ever-growing append-only logs, is an important step toward building self-improving LLM agents. Code is available at https://github.com/Applied-Machine-Learning-Lab/EMNLP26_SkillBrew.

Abstract:
Self-consistency treats agreement among sampled chains of thought as evidence that the answer is correct. We show that this reading fails predictably. Across 5 reasoning datasets and 5 generators, consistent-but-wrong (CBW) rates track the task rather than the generator, and the pattern holds across agreement thresholds and sample budgets. On StrategyQA, at least 15% of answers are wrong at 90% self-agreement. We prove that no procedure reading only the answer distribution can always separate a question whose majority is right from one whose majority is wrong. We propose Risk-Controlled Self-Consistency (RISC), which combines the majority ratio with a verifier signal and calibrates an acceptance threshold by Learn-then-Test. It certifies the risk target in domain at finite sample size, and accepts nothing when no threshold qualifies. Against self-consistency alone, pooled selective error falls 3.01pp at 0.50 coverage; per-task gains are mixed. At 0.9 coverage, a local generator with a lightweight verifier costs ~9× less than an all-API baseline and shows no detectable selective-error gap on four of five datasets. It trails on MATH500. Higher agreement is not always higher accuracy. Calibration narrows the gap, but more sampling usually does not.

Abstract:
Long-form instructional videos require automatic chaptering for efficient organization and navigation. While long-context language models can predict chapters from textualized video representations, they become costly for content-dense videos with long transcripts, smooth topic transitions, and detailed chapter descriptions. A scalable segment-then-caption paradigm reduces this cost but suffers from boundary error propagation and fragmented cross-chapter context. We propose CausalChapter , a causal-inspired interventional dependency enhancement framework for long-video chaptering. Our method estimates the observable influence of semantic units and context segments through lightweight masking and removal interventions. For boundary localization, it detects dependency changes between adjacent temporal windows via masking-based feature reconstruction; for chapter description generation, it selects cross-chapter contexts according to their influence on generated descriptions. Experiments on long-video chaptering benchmarks show consistent improvements in boundary localization, description quality, and cross-chapter coherence.

Abstract:
Automatic Short Answer Scoring (ASAS) requires models that can score student responses against question-specific criteria while remaining efficient and transferable across rubric sets. We propose RUSPAN, a rubric-conditioned ASAS framework that treats rubric descriptions as semantic label representations. RUSPAN serialises the question context, student answer, and all candidate rubric levels into a single sequence, then scores the levels listwise from the rubric-span and whole-sequence representations produced in a single LM pass. We further introduce RUSPAN-RIM, in which a Rubric-Independent Mask prevents rubric spans from attending to one another, making rubric representations depend only on the answer and question context and preventing overfitting to rubric patterns during training for zero-shot transfer. On six ASAS benchmarks spanning English, German, and Portuguese, RUSPAN improves mono-benchmark scoring over discriminative and generative baselines, while RIM with position reindexing delivers consistent and substantial gains on PT-ASAG, the held-out benchmark with the strongest combined language and rubric-structure shift.

Abstract:
Large language models (LLMs) now support contexts of up to 1M tokens, but their strengths and weaknesses on complex long-context tasks remain unclear. To study this, we focus on multi-document legal case summarization, where a single case often spans many documents exceeding 100K tokens. We systematically evaluate 12 frontier LLMs with Gavel, which consists of Gavel-Ref, a reference-based evaluation framework with checklist, residual-fact, and writing-style evaluations, and Gavel-Agent, a reference-free agent for evaluating factual coverage directly from source documents. Our results show that current models are more prone to omitting key information than hallucinating. They all perform well on simple checklist items, such as filing date, but struggle with rare and complex items, such as settlements. Performance also declines as case length increases. To meta-evaluate Gavel, we collect 160 hours of human annotations. Gavel-Agent reduces token usage by at least 36% compared to end-to-end and chunk-by-chunk methods while achieving competitive performance. Gavel-Agent also generalizes to the medical domain, achieving the best performance with at least 77% fewer tokens. We will release our code and annotations publicly.

Abstract:
Debate feedback is useful only when it explains why a side lost and how that side can revise to become more competitive. However, LLM-generated feedback often sounds polished while remaining generic and vague. We introduce ReFeedGraph, a graph-based method that turns prior debate feedback into reusable principles. It collects multi-framework judge feedback, extracts weaknesses and actionable suggestions, abstracts them into principles, and stores them in a graph. For a new debate, ReFeedGraph retrieves relevant principles and adapts them into targeted revision feedback. We evaluate the generated feedback by recording whether it improves the weaker side after rewriting. Across three existing datasets (DebateArt, Intelligence Squared, and British Parliamentary debates), ReFeedGraph outperforms direct LLM and structured-prompt baselines, producing more specific and actionable feedback that leads to winning more debates.

Abstract:
Evaluating research ideas generated by LLMs is difficult because their scientific value cannot be fully determined by objective criteria, and no single reference answer specifies what counts as a good idea. To address this challenge, we introduce Ideation Arena, a battle style platform that evaluates research ideas through pairwise human assessment. Ideation Arena evaluates ideas generated by 14 frontier LLMs and 5 research agent architectures built on 2 base models. To ensure a common starting point, Ideation Arena builds shared literature contexts from papers familiar to the participating researchers and provides the same contexts to all LLMs and agents. We collect over 6,000 double blind pairwise comparisons from 105 active computer science researchers and construct an Elo rating leaderboard for research ideation. We validate the rankings through interrater agreement and robustness analyses, showing that the leaderboard remains stable under changes in annotator composition and domain coverage. Our results show substantial variation in agent effectiveness, with some frameworks improving ideation quality over their backbones and others offering little benefit or even underperforming their base models. We further construct Ideation Arena Eval, a benchmark for assessing whether automated evaluators align with human preferences in research ideation. Experiments with current LLM judges show that they remain far from reliable human level assessment, with the best model reaching only 72.56% agreement on overall quality. Our code, data, and leaderboards are available at https://anonymous.4open.science/r/Ideation-Arena-2C21 .

Abstract:
Multi-intent spoken language understanding remains challenging due to the scarcity of labeled data and the complexity of modeling intent-slot correlations. In this paper, we propose Collaborative Self-Distillation (Colla-SD), a novel framework that leverages a pair of open-weight large language models (LLMs) to synthesize high-quality labeled samples, which are used for fine-tuning the LLMs to achieve self-distillation. Our approach consists of six stages: (a) intent-slot sampling, (b) reverse progressive skeleton generation, (c) deduplication and quality filtering, (d) cross-model consensus flagging, (e) adjudication of disagreements, and (f) model training. Evaluations on the test sets of public benchmarks show that Colla-SD dramatically improves the open-weight LLM’s overall accuracy by up to 13 times. Equipped with Colla-SD, 7B/8B open-weight models achieve up to 62.68% intent accuracy and 66.13% slot F1 on MixATIS, even outperforming massive open-source and commercial LLMs while also enabling full joint prediction. Ablation studies confirm that each component of our method is critical for the quality of synthesized data, and further analysis reveals that our method can generalize to diverse LLM architectures. Our work demonstrates that Colla-SD can effectively bridge the gap between LLM generation and zero-shot structured semantic parsing.

Abstract:
Multimodal LLMs are increasingly used with multi-image, interleaved image-text inputs, yet hallucination evaluation remains largely single-image. We study target-binding hallucination: a model answers as if an object appears in the queried target image when it appears only elsewhere. We introduce a position-controlled COCO haystack with POPE-style object-existence queries and split negatives into clean cases (object absent everywhere) and distractor-present cases (absent from the target but present elsewhere). On InternVL2-8B at N = 10, clean false positives remain near floor (1.15%), whereas distractor-present cases reach 10.55% and show a position-sensitive profile; the same regime-level separation appears across three additional architectures and in an object-balanced PASCAL VOC2012 check. Channel and paired-removal controls are consistent with a visual-channel-dominant source-misattribution interpretation, and a related MIHBench Existence re-audit finds that over 98% of false positives concentrate in partial-present negatives. For object-existence diagnostics and other source-sensitive multi-image evaluations with auditable negative evidence, we recommend reporting clean and evidence-supported negatives separately whenever non-target images may contain supporting evidence.

Abstract:
Large language model (LLM) agents have demonstrated strong capability in sequential decision-making, yet they remain fundamentally reactive in long-horizon tasks. Unlike humans who employ "what-if" reasoning to evaluate potential plans before commitment, standard agents lack an internal world model to simulate future outcomes. Therefore, we propose to internalize future-aware planning by training a single autoregressive model to verbalize both a prospective state rollout and a plan-conditioned success estimate—a textual analogue of the Q -value. Crucially, we identify a format-capability gap: simply fine-tuning agents on look-ahead traces during post-training leads to superficial mimicry of foresight without genuine predictive grounding. To bridge this gap, we introduce a three-stage training paradigm: (i) World Model Agentic Mid-Training (WM-AMT) to inject latent predictive capabilities into the policy; (ii) Format-Eliciting SFT (FE-SFT) to structure this injected capability; and (iii) Foresight-Conditioned Reinforcement Learning (FC-RL) to refine the calibration and utility of the generated simulations. Evaluated on search and mathematical reasoning tasks, our approach consistently outperforms other training baselines. Our results demonstrate that effective internal world modeling in LLM agents requires a capability-first training pipeline to achieve grounded and calibrated foresight.

Abstract:
In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence. Their parametric knowledge can induce pre-evidence tendencies that may conflict with the retrieved context, yet existing evaluation frameworks do not characterize such prior-context discrepancy or measure how verifiers arbitrate between parametric and contextual signals. We introduce PAVE ( Prior-Aware Verifier Evaluation ), a diagnostic testbed that stratifies an LLM verifier into four epistemic states based on the correctness and confidence of its pre-evidence prior and evaluates its arbitration behavior on this new benchmark, i.e., whether it persists in correct prior under misleading evidence, and whether it corrects wrong prior when accurate evidence is provided. Experiments across seven LLMs reveal unreliable and highly model-dependent prior-context arbitration, highlighting the importance of verifier selection for real-world RAG-based fact-checking applications. Based on these findings, we propose a lightweight JSD-based test-time arbitration method that improves factual reliability without modifying the underlying model, achieving competitive performance across diverse LLM families.

Abstract:
Long-horizon LLM agents often fail not because relevant information is absent, but because semantically relevant memories are epistemically invalid: stale observations, failed tool traces, or claims contradicted by newer verified evidence are retrieved and treated as support. We introduce HELM ( H ierarchical E pistemic L earned M emory) , a memory framework that makes this reliability problem explicit. HELM exposes memory through logged read/write/consolidate/prune operations and stores each item with provenance and epistemic state. Its three-tier Semantic-HNSW Nested Memory (SHNM) links episodic tool traces to consolidated recalls and thematic indices, enabling coarse-to-fine retrieval that can be audited back to concrete evidence spans. Epistemic governance filters and re-ranks recalls by status, freshness, consistency, recency, and provenance, while a lightweight learned controller decides when memory operations are worth their cost. Across five long-horizon benchmarks, HELM improves success over strong memory baselines on GAIA, WebArena, Mind2Web, StreamBench, and LifelongAgentBench while reducing effective context tokens and epistemic errors. New scaling, training-free, and governance-only ablations show that the main gains persist with a 32B backbone and that epistemic governance alone explains a large share of the reliability improvement.

Abstract:
LLM agents increasingly operate through multi-turn tool use and environment interaction, where safety risks often emerge from intermediate steps long before they surface in the final outcome. Reactive auditing is therefore insufficient: post-hoc diagnosis frequently misses the chance to flag risks while they are unfolding. We propose TRACES, a representation-based proactive auditor that learns prefix-level trajectory risk states from the hidden representations of an observer LLM. TRACES induces latent mechanism features from step representations and models their temporal evolution to estimate whether a partial trajectory is drifting toward unsafe behavior. To sidestep the cost and ambiguity of step-level risk annotation, TRACES is trained with weak trajectory-level supervision while still producing dense prefix-level risk estimates. Across multiple agent safety benchmarks, TRACES improves both full-trajectory safety prediction and proactive risk discrimination. Our analyses further suggest that these risk states can help train a safer agent, highlighting the broader potential of proactive auditing for long-horizon agent safety. The code is available at https://github.com/lab-flair/TRACES.

Abstract:
LLM agents act on real-world environments through tool calls, and a single misjudged action can cause irreversible harm. The standard safeguard is a guard model that labels each proposed action as safe or unsafe, but this binary view conflates two distinct decisions: whether the action is harmful in itself, and whether it is appropriate given the user’s context. It also operates at the granularity of action categories rather than individual instances, producing routine interruptions that erode autonomy and train users to wave through the most consequential alerts. We reframe the problem as a per-instance three-way routing decision over Execute, Ask, Refuse and instantiate it with Safety Sentry, a lightweight guard model whose inference reduces to a single decoding call. A single decoding-time threshold lets one fixed checkpoint be re-positioned across deployments of differing risk tolerance without retraining. Safety Sentry outperforms a broad set of open-weight and frontier closed-source baselines on overall accuracy and safety-related recall, while controlling both directional error rates simultaneously.

Abstract:
Visualization-of-thought (VoT) has substantially advanced multimodal spatial reasoning by interleaving visual thinking with verbal reasoning. However, multi-hop VoT reasoning introduces a fundamental scalability bottleneck: the autoregressive accumulation of interleaved visual and textual tokens causes KV cache memory to grow rapidly across successive hops. We present ScoutKV, a framework that recasts KV cache eviction as a reinforcement learning problem. ScoutKV trains a dedicated lightweight policy agent per KV head, conditioned on key–value representations, cross-hop positional encodings, and modality embeddings, to predict each token’s long-term utility across the full reasoning trajectory. Training combines offline supervised fine-tuning with future-query attention targets and online GRPO optimization with a multimodal process reward targeting low-entropy text anchors and modality-specific spatial anchors. ScoutKV consistently reduces FLOPs, latency, and peak memory while maintaining or improving accuracy across multimodal spatial reasoning benchmarks, and generalizes robustly to broader vision-language tasks.

Abstract:
Parametric retrieval augmentation encodes document information into lightweight, document-specific modules such as LoRA adapters, reducing the need to include all evidence as input context. However, it remains unclear how this parameter-side memory interacts with context-side memory stored in the KV cache. We study this interaction in document-level question answering by progressively evicting document key-value states and measuring when a document LoRA contributes beyond the retained context. We find that document LoRA adds little when the KV cache is largely intact, but becomes increasingly useful under aggressive compression, recovering 13–21 ROUGE-L points when no document context remains. The gain is largest when the base model encodes the document, and the adapter is applied only during answer generation, suggesting that LoRA serves as decoding-time parametric memory rather than as a document encoder. Finally, QA-style supervision produces substantially stronger adapters than raw-context, continuation, or context-plus-QA objectives. These results position document LoRA as a complementary memory channel whose value emerges precisely when context-side evidence is scarce.

Abstract:
Coding-agent benchmarks evaluate whether a single uninterrupted agent can resolve a repository issue. Real software work is messier: tasks are interrupted, reassigned, reviewed, and resumed from partial states left by another agent or engineer. We study this missing dimension through handoff debt: the rediscovery cost imposed when a predecessor’s work is opaque or incomplete. Our takeover protocol interrupts a coding agent at deterministic handoff points, freezes the repository, and evaluates successor agents under four handoff views: repository state only, raw trace, summary notes, and structured notes. Across 75 source tasks, the protocol generates 181 handoff-point tasks and 724 takeover runs per successor model. Across three successor models, context-bearing handoffs reduce median agent events by 20-59% and cumulative prompt tokens by 42-63% relative to repository-only takeover. Solved-rate effects are smaller and model-dependent, but efficiency gains are consistent. These findings suggest that coding-agent evaluation should report not only whether a task is solved, but also how costly that work is for another agent to resume.

Abstract:
Large language models (LLMs) achieve strong performance on many tasks, but their progress remains uneven across languages and cultures, often reflecting values latent in English-centric training data. To enable practical cultural alignment, we propose a scalable approach that leverages national social studies curricula as a foundation for culture-aware supervision. We introduce CuCu, an automated multi-agent LLM framework that transforms national textbook curricula into open-ended, culture-specific question–answer pairs for supervised fine-tuning (SFT). Applying CuCu to the Korean national social studies curriculum, we construct KCaQA, comprising 34.1k open-ended QA pairs. Our analyses and training experiments suggest that KCaQA covers culture-specific topics and produces responses grounded in local sociocultural contexts.

Abstract:
Sampling from the sequence-level power distribution elicits RL-level reasoning from base language models without any parameter updates, but the standard Metropolis–Hastings (MH), a Markov Chain Monte Carlo (MCMC) sampler, is both expensive and slow-mixing. We trace both to a structural mismatch: the power distribution mainly departs from the base distribution at a sparse, spatially clustered set of high-entropy decision points, yet MH proposes resampling positions uniformly along the prefix—wasting compute on near-degenerate conditionals while under-mixing precisely where modes diverge. We propose Entropy-Guided Power Sampling (EGPS), a training-free and verifier-free approximate sampler that re-derives its proposal from token-level entropy already in the forward pass. EGPS skips deterministic blocks, localizes each MCMC move to a high-entropy neighborhood, and applies Multiple-Try Metropolis at decision points—making sampling cost scale with entropy mass rather than sequence length. EGPS targets significant speedup with accuracy that matches or improves on MH power sampling: across three base models and three benchmarks it attains best or tied-best accuracy on 5 of 9 settings; on Qwen2.5-Math-7B it is best or tied-best on all three (MATH500 75.8%, HumanEval 62.2%, GPQA 42.4%), with wall-clock speedups over the MH baseline of up to 12.6× across these benchmarks. Our code is available at https://github.com/hongguo7/EGPS.

Abstract:
Corporate climate transition plans detail how companies intend to reduce their carbon footprint, but their unstructured and often incomplete nature limits assessments of whether disclosed actions are sufficiently concrete, quantified, and consistent with stated emission-reduction objectives. We analyze transition plans from CDP by decomposing them into the key information needed for such assessments: disclosed mitigation measures, their implementation status, quantitative attributes, action type, specificity, and organizational perimeter. Large language models achieve moderate to strong performance across these tasks (65-91% F1), enabling large-scale diagnostic analysis of transition-plan disclosures. Our results show that the main bottleneck lies in the disclosures themselves. Specifically, 80% of transition plans contain at most two quantified planned measures, and nearly 47% of proposed actions are both generic and lack a specified organizational perimeter. Firms also tend to favor high-level targets, preparatory actions, and energy-related measures over more structural business transformations. Overall, our findings show that current transition-plan reporting often lacks the granularity needed to assess whether disclosed actions are consistent with stated emission-reduction targets.

Abstract:
Inference-time methods for creative LLM behaviour, from stochastic decoding to tree search to evolutionary mutation of programs or prompts, all vary the output. The problem formulation stays fixed. We introduce TROPE, an inference-time framework that operates one level of abstraction higher: an LLM parser extracts a typed structural representation of each problem (entities, typed relations, assumptions, goal, disciplinary frame), and eleven operators across three levels (token noise, conceptual mutation, conceptual crossover) edit this representation directly before the LLM generates a candidate. Operator radicality is sampled from an α-stable distribution calibrated by rejection sampling so the rare large reformulations that prior methods cannot produce appear at the target rate. Candidates are scored by compression gain over an evidence corpus, for which we prove a calibrated-reference bound (Proposition 3): under reference-LM calibration, noise candidates score at most zero in expectation, a property embedding-distance novelty does not inherit. Against fifteen baselines at matched generator-backbone-call budget on ten benchmarks, TROPE gains +4.3 absolute points on average (+10% relative across the nine benchmarks other than the case study; a USAMO 2025 case study at n = 6 raises the relative figure to 14%). The largest reliable math-olympiad gain over the strongest per-column baseline is on AIME 2025/26 (17.6 → 21.5, +3.9, n = 60, p < 0.05 paired Wilcoxon). Ablations show that mutation gives the largest single-level gains on MATH-500 and AIME, while crossover is used most often on creative and discovery tasks. The typed parser matters most on structured mathematics, and compression-based novelty matters most on UoT.

Abstract:
Process reward models (PRMs) are widely used in language-model training with dense, step-level supervision. They assume that PRM scores remain stable proxies for step correctness under label-preserving transformations that alter reasoning structure while preserving final answers. However, this assumption remains insufficiently validated. Such transformations may alter the relationship between PRM scores and correctness, producing different failure modes across models. To address this gap, we introduce EST-PRM, a stress-testing framework for dense process rewards. It evaluates three transformations: (1) step inflation, (2) dependency-aware step reordering, and (3) confidence markers. We further define a vulnerability decomposition that separates reward inflation from loss of correctness sensitivity. We evaluate five PRM-style models on 4,687 reasoning chains from MATH-500, GSM8K, and PRMBench. Results reveal substantial differences in vulnerability patterns across models. Math-Shepherd exhibits the strongest sensitivity to position-based perturbations, with a Pearson correlation drop of 0.139 ± 0.031 and a 29.7 ± 4.2% score inflation rate, while Qwen2.5-Math-PRM is most affected by step inflation, reaching a 47.6 ± 4.3% inflation rate. Confidence-based perturbations further distort reward calibration, revealing inconsistencies in correctness estimation. Finally, we evaluate three mitigation strategies and identify trade-offs between robustness coverage and false-positive rates.

Abstract:
Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sparse snapshots of wellbeing. Wearable devices offer longitudinal behavioral and physiological signals for continuous, low-burden monitoring. Recent LLM-driven personal-health agents enable natural language queries over wearable signals, but mainly handle short-term, retrieval-based lookups (e.g., highest step count over a week). They do not evaluate whether agents can reason long-term signals to predict wellbeing scores paired with evidence-grounded rationales. To address this gap, we introduce BALMS, the first systematic benchmark of LLM-based agentic systems for longitudinal mental health sensing. BALMS spans 3 real-world longitudinal datasets, 2 task families (closed-form wellbeing-score prediction and rationale generation auto-graded by an LLM-as-Judge), 3 agentic paradigms evaluated across 5 open- and closed-source LLM backbones. We find that zero-shot agents rarely outperform a simple mean baseline, except with stronger backbones or compact, semantically meaningful features. Chain-of-thought prompting improves reasoning-oriented backbones, but does not guarantee temporal grounding or numerical correctness. Together with more analysis on efficiency and temporal scaling, BALMS highlights the need for longitudinal mental health agents that selectively retrieve history, ground temporal evidence, and reason over interpretable behavioral features.

Abstract:
As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses has become essential. These tasks require context-sensitive answers and allow little room for error, motivating fine-grained and diagnostic evaluation that can identify specific sources of response quality failures. We introduce LexRubric, a rubric-based benchmark for evaluating open-ended Chinese legal tasks. LexRubric contains 649 instances from legal consultation and judicial examination, which reflect both everyday legal needs and professional legal reasoning and cover 14 legal scenarios. It further includes 12,337 expert-written atomic scoring criteria organized under a unified six-dimensional framework, enabling accurate evaluation and diagnostic analysis across tasks and evaluation dimensions. To validate the reliability of the evaluation, we test multiple judge models and compare model-based judgments with human judgments. We further evaluate 18 recent general and legal-domain LLMs on LexRubric. Results show that different models exhibit distinct capability profiles, and that open-ended legal tasks remain challenging for current LLMs. Data is available at: .

Abstract:
Detecting whether corporate sustainability disclosures truly satisfy reporting standards is essential for greenwashing-aware ESG auditing, yet existing resources rarely evaluate whether disclosures sufficiently satisfy framework metrics. We introduce MM-COMPLIANCE, a multilingual multimodal dataset aligned with SASB metrics across six languages (English, French, Japanese, Chinese, Thai, and Korean), with page-level evidence and a key completeness annotation that separates fully compliant disclosures from partial mentions. We define two tasks reflecting practical auditing workflows: Page-level Compliance Judgment and Metric-to-Page Evidence Retrieval. Our evaluated baselines primarily comprising open-source architectures, supplemented by a commercial VLM as an upper-bound reference, show moderate performance for page understanding and evidence localization, but consistently low performance on partial compliance verification and large variation across languages and industries. MM-COMPLIANCE provides a benchmark for developing framework-aware models to support rigorous ESG compliance auditing and long-term commitment tracing.

Abstract:
Multimodal large language models have recently shown strong potential for time series reasoning, where models jointly process textual context and long numerical sequences. However, dense time series representations introduce a large number of tokens, leading to substantial computational and memory overhead during inference. Existing token pruning methods are mainly designed for visual tokens and typically rely on global attention or representation geometry, overlooking the fact that token redundancy varies across layers and modalities. This makes pruning-induced perturbations difficult to control, especially when tokens are pruned across multiple transformer layers. In this paper, we propose StratPrune , a cascade stratum pruning framework for efficient time series reasoning. StratPrune first identifies an early-layer pruning stratum where time series tokens remain less diverse than text tokens, indicating high redundancy suitable for stable pruning. We then derive a hidden-state stability bound that decomposes pruning-induced perturbation into a layer-dependent stability term and a token-level attention saliency term. Guided by this analysis, StratPrune allocates layer-wise pruning budgets according to time series diversity and removes low-saliency time series tokens through a context-aware cascade process. Experiments on 7 tasks show that StratPrune outperforms strong pruning baselines under matched token budgets, achieving up to 30% relative improvement.

Abstract:
Questions about a recurring person in long videos are rarely answered from a single moment. They require evidence that is sufficient for the claim, comparisons across episodes, synthesis over recurring audiovisual patterns, and refusal when the source material does not support an answer. Existing long-video benchmarks do not isolate these requirements. We introduce KnowMeBenchV2, an evaluation-only benchmark over five public multi-episode video series, with 327 videos and 3,061 questions covering four evidence scopes: single-clip extraction, temporal and event relations, cross-episode ordering, and cross-episode persona synthesis. A separate grounded-abstention task serves as a contrastive support check: its questions are surface-similar to supported person-linked questions but are not licensed by the source evidence. The most interpretive synthesis items are human-authored and quality-audited by trained annotators with full source access. A matched human reference reaches 78.0 on cross-episode synthesis. To diagnose failures, we compare evidence routes under a fixed Qwen3.5-9B answerer and a task-matched pre-tokenization character routing budget. ASR-Retrieval reaches 2.7 on synthesis, while KnowMe reaches 64.9. In this diagnostic, the final answerer reasons over textualized evidence constructed from audiovisual sources rather than raw video tokens. The gap therefore isolates the importance of audiovisual evidence construction and cross-episode organization before answer generation.

Abstract:
Local fine-tuning datasets routinely contain sensitive secrets such as API keys, personal identifiers, and financial records. Although “local offline fine-tuning” is often viewed as a privacy boundary, we reveal that compromised model code is sufficient to steal them. Current passive pretrained-weight poisoning attacks, while effective for natural language, fundamentally fail to capture such sparse high-entropy targets due to their reliance on probabilistic semantic prefixes. To bridge this gap, we identify and exploit a practical but overlooked supply-chain vector—malicious model code camouflaged as standard architectural definitions—to realize a paradigm shift from passive weight poisoning to active execution hijacking. We introduce a deterministic full-chain memorization mechanism: it locks onto token-level secrets in dynamic computation flows via online tensor-rule matching, and leverages value–gradient decoupling to stealthily inject attack gradients, overcoming gradient drowning to force model memorization. Furthermore, we achieve, for the first time, attacker-verifiable secret stealing through black-box queries that precisely distinguishes true leakage from hallucination. Our attack achieves over 98% Strict ASR in the default LoRA setting with limited primary-task utility degradation and effectively evades defense measures including semantic safety filtering, code auditing, and perplexity-based detection.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves mathematical reasoning, but sparse sequence-level rewards make training heavily dependent on long, repeated rollout generation. On-policy distillation (OPD) makes these rollouts more informative by asking a larger teacher to score every student trajectory token by token, but this inserts large-teacher forward passes into every RL step and requires online resources beyond standard student-only RL. We propose SPEC-Distill, a speculative prefix distillation framework in which the teacher shapes rollout construction rather than defining an auxiliary reward. Instead of querying the teacher during training, SPEC-Distill runs it once offline to build a reusable pool of expert trajectories. During RL, the student treats these trajectories as speculative drafts, accepts a prefix under a lenience-controlled criterion, completes the response under its own policy, and is updated only by verifiable rewards with standard GRPO. Against OPD, this removes online teacher scoring; against standard autoregressive rollouts, accepted prefixes reduce the number of fresh student-generated tokens. In the primary non-think setting, SPEC-Distill improves Qwen3-4B from 66.1 to 73.2 average accuracy over GRPO, and outperforms OPD on the two scales. Compared with full-response expert reuse such as Luffy, SPEC-Distill reaches comparable accuracy on Qwen3-1.7B while reducing online rollout tokens by about 60% and rollout time from 10.90 to 7.06 hours. Analysis further shows that teacher prefixes are best used as an early scaffold, after which reusing the student’s own rollouts provides the strongest balance between expert guidance and on-policy exploration.

Abstract:
A correct diagnosis can still be reached for the wrong reasons. In clinical reasoning, every disease hypothesis creates obligations: key evidence must be checked, alternatives must be ruled out, contradictions must be resolved, useful tests must be considered, and closure must be justified. Current evaluations of medical LLMs mostly focus on final answers, local steps, or isolated facts, and therefore miss these hypothesis-induced commitments. We introduce VeriDx , a disease-centric verification framework that links free-form diagnostic reasoning to structured disease profiles. VeriDx tracks whether each hypothesis is satisfied, unresolved, or violated its clinical obligations, exposing failures such as missing critical tests, unresolved differentials, ignored contradictions, unsupported claims, and premature closure. We instantiate VeriDx for complex respiratory diagnosis using guideline-derived disease profiles and expert-annotated longitudinal cases. Our results show that many diagnostic errors are not isolated mistakes, but broken commitments made earlier in the reasoning process.

Abstract:
Large Language Model (LLM) agents have established a promising paradigm for Knowledge Graph Question Answering (KGQA). However, their practical performance is severely constrained by the pervasive incompleteness of real-world knowledge graphs (KGs). To address this, Incomplete KGQA (IKGQA) has emerged as a vital task, yet existing multi-agent frameworks often suffer from a logic-misalignment effect, where passive collaboration leads to retrieval failure and hallucination propagation when encountering structural gaps. In this paper, we propose IDLC, an Intent-Driven Logical Compensation framework designed for faithful IKGQA. Unlike traditional reactive debate, IDLC formalizes reasoning as a proactive deliberation task within a Dec-POMDP. It introduces an Active KG Explorer that generates explicit Gap Descriptors to translate structural gaps into actionable retrieval goals, alongside a Logical Consistency Verifier that performs bi-directional anchoring to intercept step-wise semantic drift. The entire multi-agent policy is jointly optimized via reinforcement learning with a specialized logic-fidelity reward, aligning agent behaviors toward faithful reasoning paths. Extensive experiments on BRINK and IKG-CWQ/WebQSP benchmarks demonstrate that IDLC outperforms state-of-the-art baselines in both reasoning performance and step-wise logical fidelity. Furthermore, IDLC exhibits robust zero-shot transferability across various incompleteness levels while maintaining computational efficiency, confirming the effectiveness of intent-driven coordination in repairing fractured reasoning chains.

Abstract:
Current evaluation of LLM-generated debate largely focuses on argumentative quality, leaving the value structure of generated arguments underexamined. We introduce a diagnostic metric framework for comparing how values are expressed and organized in human-reference and AI-generated debates. The framework separates value expression , the range of recognizable values a debate invokes, from value organization , how those values are distributed and sustained across turns. Grounded in Schwartz’s Basic-10 value theory, it represents debates as multi-level value distributions over argumentative units, utterances, stances, and full debates, enabling metrics for value coverage, alignment, diffusion, and cross-turn dynamics. Across multiple generation systems, AI-generated debates are broadly value-expressive, often activating as many or more value dimensions than matched human references, but they organize these values differently. Their utterances are more locally value-diffuse, and source-blind judges distinguish value clarity from value focus. These findings show that debate evaluation should consider not only value presence, but also value organization. Code and supplementary materials are available at https://anonymous.4open.science/r/Value-Expression-and-Organization-6716/

Abstract:
Hallucination detection in LLMs often relies on output consistency or model-internal signals. While output-based methods have the advantage of applicability to closed-source models, existing studies often overlook a key limitation: they struggle on tasks that admit multiple valid answers, where variation across generations may reflect legitimate alternative responses rather than hallucinations. In this paper, we propose a question-type-aware hallucination detection framework that distinguishes single-answer and multi-answer regimes. Given a predicted question-type signal, we derive type-conditioned diagnostic features from low- and high-temperature generations, and build two alternative hallucination detectors based on these features: a calibrated type-aware uncertainty estimator and a feature-verbalized few-shot LLM judge. Experiments on NQ-Open, WebQA, and AmbigQA across both open-source and closed-source LLMs demonstrate that our method consistently outperforms conventional semantic clustering approaches and representative baselines in hallucination detection.

Abstract:
Multimodal large language models (MLLMs) are frequently exposed to auxiliary textual context, the impact of which on vision-language tasks remains underexplored. In this paper, we investigate the influence of task-irrelevant context by formulating it as a controlled intervention within a binary visual judgment framework. Through controlled paired comparisons with and without auxiliary textual context, we observe that irrelevant text consistently biases model predictions across diverse benchmarks. To move beyond performance metrics, we characterize this sensitivity through "decision margin", which is defined by the log-probability difference between candidates. Our analysis reveals an empirical pattern: context-conditioned margins are often well approximated by an affine transformation of their context-free counterparts. We further interpret the fitted affine parameters as metrics for visual commitment preservation and directional answer bias. These findings provide a margin-level diagnostic view of irrelevant-context effects in MLLMs, indicating that irrelevant context does not manifest as stochastic noise, but as an estimable distortion of model preference.

Abstract:
Open-ended deep research (OEDR) requires systems to acquire knowledge through multi-round retrieval and generate coherent long-form reports. The outline plays a central role as a structural scaffold that coordinates retrieval, evidence organization, and generation. However, existing methods either fix the outline before writing or refine it with local heuristics, leading to scaffold drift under continuous information accumulation and delayed feedback for evaluating outline modifications. We propose ScaffoldAgent, a utility-guided dynamic outline optimization framework for OEDR. ScaffoldAgent models outline evolution as a structured decision process with three operations: Expansion, Contraction, and Revision, enabling controlled updates to the report scaffold. It further introduces a utility-guided feedback mechanism that estimates the downstream value of each outline operation from retrieval gain, structural coherence, and trial-generation quality. The resulting utility signal guides node selection, operation scheduling, and termination during inference. Experiments on DeepResearch Bench and DeepResearch Gym show that ScaffoldAgent consistently improves long-form report generation and factual grounding over existing deep research agents.

Abstract:
As large language models (LLMs) are increasingly adapted from general-purpose systems into domain-specific problem solvers, a central challenge is how to provide reliable process-level guidance for knowledge-intensive reasoning. High-quality reasoning supervision, such as annotated reasoning trajectories or externally generated rationales, is costly to obtain, difficult to verify, and hard to scale across domains. In this paper, we propose KG-RLVR, a lightweight framework that reframes KG-enhanced LLM post-training as verifiable reward construction from traceable structured evidence. KG-RLVR retrieves evidence from external knowledge graphs, consolidates noisy graph neighborhoods into representative evidence units, and converts these units into locally computable process rewards. The reward encourages the model to cover directly relevant evidence while selectively integrating less direct but potentially useful auxiliary evidence, thereby guiding reasoning traces without requiring externally authored process demonstrations. Experiments on knowledge-intensive medical and multi-domain QA benchmarks show that KG-RLVR consistently improves over vanilla RL, demonstrating that KG-derived process rewards provide practical and scalable supervision for reasoning-oriented LLM post-training.

Abstract:
Retrieval-Augmented Generation (RAG) enables Large Language Models (LLMs) to access external knowledge during generation, improving factual reliability without parameter retraining. However, existing methods still struggle to retrieve evidence that is both relevant and complete. Standard RAG may miss semantic associations among related chunks. Graph-based RAG addresses this issue by modeling entity relations, but it is often query-agnostic. Question-format RAG further improves query matching by converting facts into Question-Answer (QA) pairs, but it cannot guarantee semantic association and introduces knowledge fragmentation. To address these issues, we propose HolisticRAG, a framework integrating graph-grounded and query-aligned knowledge for holistic knowledge RAG. In the offline stage, HolisticRAG constructs a knowledge representation to preserve semantic associations while reducing query-agnostic retrieval. In the online stage, it applies holistic knowledge clustering to merge all fragmented evidence and adaptively retrieve sufficient supporting facts. Experiments demonstrate the effiectiveness of HolisticRAG.

Abstract:
Enabling Large Language Models (LLMs) to perform tool-free Table Question Answering (TableQA) presents two critical challenges: "question-first" data synthesis often introduces logical hallucinations, while standard reinforcement learning (RL) suffers from sparse rewards over long reasoning chains. We propose ProbeTQA, a unified framework that couples execution-verified data construction with process-supervised RL. To mitigate synthesis hallucinations, ProbeTQA adopts an answer-first executable probing paradigm, in which an agent performs multi-round exploratory programming over real tables so that every synthesized instance is grounded in a verifiable witness program. To address reward sparsity, we convert verified code trajectories into natural-language reasoning traces and derive execution-grounded process rubrics directly from runtime states, providing reliable dense supervision. Using these rubrics, we optimize the model with Group Relative Policy Optimization (GRPO) under a Two-Stage Reward Curriculum, thereby internalizing complex tool-use reasoning into the model parameters. Experiments show that our 8B model achieves state-of-the-art tool-free performance on challenging benchmarks, attaining 65.8% accuracy on TableBench and outperforming GPT-5.4 and DeepSeek-v3.2 by 13.4 and 6.8 percentage points, respectively.

Abstract:
Large vision-language models (LVLMs) perform well on long-horizon GUI tasks, but their high inference costs make small VLMs (SVLMs) attractive for cost-effective local deployment despite a substantial capability gap. Group Relative Policy Optimization (GRPO) is a widely used RL algorithm for improving SVLMs, yet its binary outcome rewards provide no intermediate progress signal across trajectories and its uniform advantages lead to semantically-blind credit assignment within each trajectory. We propose PAGE, an RL training method built on automatic progress reward construction and hierarchical advantage computation. The former derives structured evaluation criteria from successful trajectories through iterative extraction and produces stable continuous progress rewards through multi-round aggregation, while the latter combines progress-based inter-rollout advantages with intra-rollout subtask-level advantages obtained through LVLM-driven segmentation and completion verification. Across three long-horizon GUI scenarios, PAGE achieves a 79.3% average success rate with an 8B SVLM, a 16.0 pp gain over the GRPO baseline, and competitive performance against the evaluated closed-source LVLMs.

Abstract:
As LLM-based agents operate over sequential multi-step reasoning, hallucinations arising at intermediate steps risk propagating along the trajectory, thus degrading overall reliability. Unlike hallucination detection in single-turn responses, diagnosing hallucinations in multi-step workflows requires identifying which step causes the initial divergence. To fill this gap, we propose a new research task, automated hallucination attribution of LLM-based agents, aiming to identify the step responsible for the hallucination and explain why. To support this task, we introduce AgentHallu, a comprehensive benchmark with: (1) 693 high-quality trajectories spanning 7 agent frameworks and 5 domains, (2) a hallucination taxonomy organized into 5 categories (Planning, Retrieval, Reasoning, Human-Interaction, and Tool-Use) and 14 sub-categories, and (3) multi-level annotations curated by humans, covering binary labels, hallucination-responsible steps, and causal explanations. We evaluate 13 leading models, and results show the task is challenging even for top-tier models (like GPT-5, Gemini-2.5-Pro). The best-performing model achieves only 41.1% step localization accuracy, where tool-use hallucinations are the most challenging at just 11.6%. We believe AgentHallu will catalyze future research into developing robust, transparent, and reliable agentic systems.

Abstract:
Text-Attributed Graphs (TAGs), where each node is associated with text descriptions, are ubiquitous in real-world scenarios. They typically exhibit distinctive structure and domain-specific knowledge, motivating the development of Graph Foundation Models (GFMs) that generalize across diverse graphs and tasks. Despite large efforts to integrate Large Language Models (LLMs) and Graph Neural Networks (GNNs) for TAGs, most approaches suffer from decoupled architectures with two-stage alignment, limiting their synergistic potential. Even worse, existing methods assign out-of-vocabulary (OOV) tokens to graph nodes, leading to graph-specific semantics, token explosion, and incompatibility with task-oriented prompt templates, which hinders cross-graph and cross-task transferability. To address these challenges, we propose PromptGFM, a versatile GFM for TAGs grounded in graph vocabulary learning. PromptGFM comprises two key components: (a) a graph understanding module, which explicitly prompts LLMs to replicate the fine-grained GNN workflow within the text space, facilitating seamless GNN-LLM integration and elegant graph-text alignment; and (b) a graph inference module, which establishes a language-based graph vocabulary to ensure expressiveness, transferability, and scalability, enabling readable instructions for joint LLM fine-tuning across graphs and tasks. Extensive experiments demonstrate the superiority of PromptGFM, highlighting its strong transferability across diverse graphs and tasks and its adaptability to auxiliary pre-training tasks. The code is available at https://github.com/agiresearch/PromptGFM .

Abstract:
Large language models (LLMs) are increasingly used to support question answering and decision-making in high-stakes, domain-specific settings such as natural hazard response and infrastructure planning, where effective answers must convey fine-grained, decision-critical details. However, existing evaluation frameworks for retrieval-augmented generation (RAG) and open-ended question answering primarily rely on surface-level similarity, factual consistency, or semantic relevance, and often fail to assess whether responses provide the specific information required for domain-sensitive decisions. To address this gap, we propose a multi-dimensional, reference-free evaluation framework that assesses LLM outputs along four complementary dimensions: specificity, context utilization, robustness to paraphrasing and semantic perturbations, and answer relevance. We introduce ClimateRiskQA, a curated dataset of 1,412 domain-specific question–answer pairs spanning 40 professional roles and seven natural hazard types, and further evaluate on PubMedQA to examine cross-domain generalization. We also conduct human evaluation to assess inter-annotator agreement and alignment between model outputs and human judgments, which highlights the inherent subjectivity of open-ended, domain-specific evaluation. Our results show that no single metric sufficiently captures answer quality in isolation and demonstrate the need for structured, multi-metric evaluation frameworks when deploying LLMs in high-stakes applications.

Abstract:
Human language processing in the brain involves the coordinated engagement of multiple perceptual pathways. In parallel, recent multimodal large language models (MLLMs) integrate visual and linguistic information into language generation. However, it remains unclear whether and how the injection of visual information alters language representations, and how such changes in turn affect neural predictions during language processing. To address this question, we analyze joint vision–language representations extracted from MLLMs using brain encoding models. We visually augment textual stimuli via image generation and use the resulting representations to predict fMRI responses recorded while participants process language-only stimuli. We find that joint vision–language representations substantially improve neural prediction performance, with pronounced gains in higher-level visual regions. Further analyses indicate that these improvements reflect both the engagement of visual pathways and the enrichment of linguistic representations through visually grounded information. Together, our results provide evidence for multimodal pathway involvement in human language processing and offer insights into the interpretability of multimodal models.

Abstract:
As language agents are increasingly entrusted with tool-mediated actions, reaching the correct outcome is not sufficient: agents must also comply with domain-specific standard operating procedures (SOPs), policies, and constraints throughout execution. We introduce SOPBench, an executable benchmark for evaluating procedural compliance in language agents. SOPBench provides sandbox environments spanning seven customer-service domains, with 167 executable tools, 70 service-specific SOPs, and 830 validated test cases. Its code-based oracle verifiers assess compliance at the outcome, step, and trajectory levels without relying on human or LLM judges. Evaluating 23 leading models, we find that reasoning models perform best but remain far from reliable, particularly in difficult domains and under simple adversarial instructions. Error analysis further reveals distinct procedural failure modes across model capability levels. Finally, a layered harness ablation shows that even when agents are given ground-truth constraint verdicts, 12-38% of cases still fail. This residual gap suggests that a key challenge lies not only in retrieving or verifying constraints, but also in consistently translating verified constraints into compliant actions. We release the code, data, and more than 40k agent trajectories.

Abstract:
Long-context LLMs are vulnerable to prompt injection, where an attacker injects an instruction into a long context to hijack the LLM’s output. Existing defenses, designed for short contexts, have limited effectiveness in long-context scenarios because the injected instruction constitutes only a small portion of the context. We propose PISanitizer, which pinpoints and sanitizes injected tokens in a context before a backend LLM generates its response. PISanitizer builds on two observations: (1) prompt injection attacks craft instructions that compel an LLM to follow them, and (2) LLMs leverage attention to focus on influential input tokens during generation. Accordingly, we intentionally let an LLM follow any instructions in the context and then sanitize tokens receiving high attention from the resulting instruction-following behavior. This design creates a dilemma for attackers: the more effectively an injected instruction compels the LLM to follow it, the more likely it is to be sanitized. Comprehensive evaluations demonstrate that PISanitizer successfully and efficiently prevents prompt injection, maintains utility, outperforms existing defenses, and is robust to optimization-based and adaptive attacks. The code is available at https://github.com/sleeepeer/PISanitizer.

Abstract:
Vision foundation tools such as open-vocabulary detectors, segmentation models, and post-processing operators are powerful building blocks for computer vision, but their effectiveness depends heavily on how they are orchestrated: which tools are used, in what order, with what parameters, and under what visual conditions. Existing visual-programming agents typically generate a fixed solution pipeline, making them brittle under dense objects, occlusion, small targets, and domain shift. We introduce VTOS (Vision Tools Orchestration Search), a framework for adaptive visual tool orchestration through joint solution–observer search. VTOS co-searches executable solution programs that compose vision tools such as Grounding DINO, SAM, NMS, and slice-and-detect, together with observer programs that diagnose candidate solutions, identify failure modes, and generate actionable feedback. These observations are accumulated in a shared VisionThoughts knowledge base to guide subsequent search. We evaluate VTOS through two case studies: dense object counting on LVIS-Count and zero-shot plant-disease segmentation on PlantSeg-OOD, which stress different orchestration challenges including threshold calibration, NMS, slicing, mask refinement, and domain generalization. Across both tasks, VTOS outperforms static tool pipelines and agentic visual-programming baselines, specifically in complex settings such as dense, occluded scenes and out-of-distribution segmentation where static pipelines leave measurable headroom, rather than in standard tasks where a single well-calibrated tool already approaches its ceiling.

Abstract:
The continuous evolution of large language models drives escalating demands on data scale and quality, and as different training stages impose increasingly tailored data requirements, systematic organization of high-quality corpora becomes indispensable. Existing corpus construction pipelines confine the resulting corpora to flat, undifferentiated document collections, universally lacking systematic knowledge organization. We present CORTEX , to our knowledge the first framework that elevates web-scale corpus construction from flat document filtering to structured knowledge organization through an Ontological Corpus Graph (OCG) , a three-layer heterogeneous structure unifying a quality-refined content layer, a hierarchical lightweight ontology layer via LLM-driven automated evolution, and a cross-domain alignment layer enabling inter-domain association at arbitrary taxonomic resolution. Comprehensive experiments confirm the effectiveness of CORTEX. In particular, we leverage the OCG to synthesize CORTEXBench, a cross-domain search-and-reasoning benchmark whose evaluation across eight frontier LLMs validates the effectiveness of quality refinement, domain organization, and cross-domain data synthesis. We will publicly release the complete codebase, a 24.14B-token refined corpus with its OCG, and CORTEXBench.

Abstract:
Machine-generated text (MGT) attribution aims to identify the specific generator responsible for a given text, thereby providing fine-grained evidence for model accountability and misuse investigation. As new large language models continue to emerge, attribution models must continuously incorporate new generators while preserving their ability to recognize previously seen ones. Prior works have shown that this lifelong MGT attribution setting is challenging, and existing methods often struggle to achieve a stable balance between adapting to new classes and retaining old ones. To address this issue, we propose RidgeFT, a lightweight analytic update framework that does not rely on exemplar replay. RidgeFT trains a task-aware encoder on the initial generator set, stores compact class-wise sufficient statistics when each generator class is first observed, and then freezes the encoder for replay-free closed-form updates. It then suppresses generator-irrelevant variation through covariance calibration, improves representation capacity with fixed random features, and updates new classes through closed-form ridge regression based on class-level sufficient statistics. Across multi-topic evaluations with varying initial generator setups, consistently outperforms baselines. It achieves the best overall macro-F1 across domains, backbones, and incremental protocols, while also improving both old-class retention and new-class adaptation. These results suggest that feature-stable analytic updates provide a simple yet effective approach to lifelong MGT attribution.

Abstract:
The Neural Finite State Machine (NFSM) framework offers a pragmatic path to full-duplex dialogue by serializing turn-taking control and response generation onto a single causal tape under the standard next-token prediction objective, thereby preserving semantic prowess at a low fine-tuning cost. However, its reliance on synthetic text data fundamentally limits turn-taking naturalness, as Large Language Models (LLMs) cannot faithfully simulate the fine-grained acoustic temporal dynamics of real human dialogues. In this work, we propose a decoupled data approach that learns turn-taking from real Human-Human (HH) spoken dialogues while shaping semantic behavior through configurable Human-Agent (HA) text dialogues. To operationalize this approach, we introduce a rule-based event-guided data transformation method that serializes HH spoken dialogues into FSM tapes by classifying turn-taking events and applying deterministic mapping rules, enabling scalable supervision without LLM-generated annotations. We further propose a Source-Aware Calibrated (SAC) Loss that jointly calibrates the long-tailed distribution of state transition tokens and channels each data source toward the capability it best supervises. Experiments show that our approach substantially improves turn-taking proficiency while recovering the foundation LLM’s semantic capability. Our code and model will be publicly available.

Abstract:
Certain document automation scenarios, such as intelligent form infilling and privacy-strict parsing, require models to comprehend unpopulated templates. Yet existing benchmarks mainly use content-filled tables, leaving the template side of document understanding under-evaluated. To bridge this gap, we introduce NEST, a benchmark of 828 Native Empty Structural Templates from real-world, born-digital forms. NEST removes filled content while preserving complex hierarchical layouts and guided selection options in authentic administrative workflows. Through Structural Schema Parsing, Cell-Header Alignment, and Generative Infilling, NEST directly evaluates structural reasoning without content-based shortcuts. Experiments reveal a clear template–instance gap: state-of-the-art LLMs drop by 9.5 percentage points on average when evaluated on unpopulated templates rather than populated instances. We further show that Generative Infilling is a data-efficient tuning signal for template completion, raising infilling accuracy by 16.37 percentage points and the three-task average by 8.19 percentage points with only 662 (80%) training templates. NEST thus provides a benchmark for measuring template-level understanding and highlights a data-efficient route for improving template completion. The code, data, and results for this paper are available at https://github.com/cpeng-research/NEST-Benchmark.

Abstract:
Multi-agent systems enhance complex reasoning but face a critical cost and effectiveness dilemma: API costs grow exponentially while performance degrades due to context degradation and error propagation. This paper proposes an evaluator-guided clause-level pruning and early halting method for lower-cost multi-agent systems named HALT, which implements a macro-to-micro intervention. At the macro-level, a difficulty aware router classifies query complexity, bypassing redundant multi-agent systems overhead for simple tasks. At the micro-level, a lightweight hindsight evaluator monitors generation in real time for complex reasoning, driving a dual-axis intervention policy: spatial pruning removes erroneous clauses, while temporal halting terminates multi-agent interaction early upon reaching a purified consensus. The extensive experiments across three different datasets demonstrate the outperformance of HALT over the baselines, achieving the 97.98%, 90.82%, 87.14% in GSM8K, MMLU, MMLU-Pro in accuracy while drastically mitigating computational overhead, with a maximum reduction of 94.3% in prompt tokens and an unprecedented reduction of 98.9% in completion tokens, which validate the effectiveness of HALT. Code is available at URL.

Abstract:
Embedding benchmarks like MTEB report a single score per model, implicitly treating robustness as a static, scalar property. We argue that robustness is multidimensional, since models respond differently to different types of variation, and requires dynamic evaluation to expose failures hidden by static benchmarks. We introduce the Harder Text Embedding Benchmark (HTEB), a dynamic evaluation framework that challenges model robustness along three practically interpretable axes (Lexical/Stylistic, Length and Language) by stochastically transforming inputs at evaluation time with an LLM. Evaluating 16 open-weight embedding models on 32 datasets covering 42 languages under transformations validated by 4,800 individual human ratings on an English subsample, supplemented by a Spanish-source evaluation and an exploratory STS-B study of pair-level label preservation, we find three patterns: (1) Models exhibit partly decoupled robustness profiles across axes. (2) Across three model families, scale increases absolute scores but does not close the gap between original and transformed evaluations. Scaling tends to improve specifically the Language axis. (3) English datasets are more sensitive to HTEB transformations than multilingual datasets. This demonstrates that HTEB identifies strengths and weaknesses of models along deployment-relevant axes, challenging embedding benchmarks and arguing for multidimensional, dynamic robustness evaluation. We make HTEB’s code publicly available.

Abstract:
LLM-based multi-agent systems (MAS) are increasingly considered for high-stakes decision-making, yet outcome-based fairness audits can miss where risks arise within the decision trajectory. We present SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based hiring MAS. SCOPED-Hiring constructs controlled resume variants, runs role-based hiring committees, logs over 311K structured decision trajectories, and converts trajectory fields into quantitative fairness signals organized by six diagnostic lenses: final outcome, counterfactual, process, pathway, dynamic, and design effects. SCOPED-Hiring reveals that balanced final hire rates can mask hidden trajectory unfairness in multi-agent decision trajectories: career gaps trigger suspicion, proxy cues shape qualification judgments, and identity cues lead to unequal investigation. Targeted repair guided by these diagnoses reduces total layered burden by 72.3% while shifting the hire rate by only 1.86 pp, showing that process diagnosis can guide effective repair.

Abstract:
Long-form audio makes the Key-Value (KV) cache the dominant bottleneck in speech LLMs, yet existing KV compression methods commit to an irreversible retention set at prefill. However, we reveal a severe attention mismatch in this static paradigm particularly for long-form audio: prefill attention concentrates at the audio’s start, whereas decode-time attention distributes broadly. This weak overlap between their top-attended positions leads to performance degradation, with generation collapsing into repetition at extreme compression rates. To bridge this gap, we propose WnW (Waxing-and-Waning KV cache), which makes eviction reversible by partitioning KV-heads into three roles via offline calibration. Anchor heads keep all audio KV on GPU and provide a per-step signal of which audio region the current token needs; tidal heads, guided by that signal, recall evicted KV from CPU on demand; fixed heads discard theirs permanently. Evaluations on LibriSpeech-Long Benchmark across two 3B backbones (Voxtral-Mini and Qwen2.5-Omni) demonstrate that WnW preserves near-full-cache accuracy while maintaining only 20% of audio tokens on GPU—a regime where irreversible methods fail to terminate. WnW exhibits strong generalization across languages, tasks, and domains, with CPU-GPU chunk swapping introducing negligible decoding overhead. More broadly, WnW transforms KV compression into a reversible memory hierarchy in which a small set of calibrated observer heads governs the placement of the remaining cache.

Abstract:
While propagation-based fake news detection improves performance by modeling how news spreads, it remains vulnerable to adversarial social manipulation, where adversaries distort propagation patterns through malicious social interactions to evade detection. Existing attacks often target only one type of social behavior with predefined rules, making them vulnerable to robust defenses. To bridge this gap, we propose a new Self-evolving Social Manipulation framework (EvoSM) that utilizes Large Language Models (LLMs) to orchestrate evolutionary social manipulation campaigns for evading fake news detection. First, we design an LLM-driven Social Manipulation Orchestrator that uses a social graph interface to perceive the propagation environment, coordinate diverse manipulation actions, and refine attack strategies in context. Second, to further strengthen the manipulation capability, we introduce Reflective Experience Memory, which distills reusable manipulation insights from previous successful and failed attack attempts through reflection. Experiments on four real-world fake news datasets demonstrate that EvoSM consistently outperforms comparison baselines in attack success rates and transferability. Further analysis shows that EvoSM benefits from coordinated multi-behavior manipulation and experience-guided strategy evolution. Our study highlights the vulnerability of existing fake news detection systems to adaptive social manipulation.

Abstract:
Recent large language models are increasingly being adapted for medical reasoning, yet most open medical LLMs remain optimized for English-centric clinical settings and fail to capture local legal and institutional knowledge. We present Meerkat2, a family of open medical reasoning models at 14B, 32B, and 30B-A3B scales for Korean and English clinical contexts. Meerkat2 is trained on an 837K-example mixture that separates reasoning transfer from high-fidelity local knowledge: clinical reasoning is distilled from strong teachers, while Korean medical statutes are converted into source-grounded question-answer pairs. We also introduce Verbosity-Decoupled Distillation (VDD), which preserves the benefits of long teacher rationales without making verbose generation to the student’s default behavior. Across MedQA and KorMedMCQA, Meerkat2 substantially improves over comparable base models, with the 30B-A3B variant reaching Korean licensing accuracy that matches Qwen3-235B-A22B while using only 3B active parameters. Ablations show that statute-grounded synthesis outperforms standard distillation, and mechanistic analysis reveals that VDD cancels gradients toward verbose generation. These results provide a practical recipe for localized and efficient medical LLMs.

Abstract:
Large-scale computational research on conspiracy theories has focused exclusively on believers’ online behavior, leaving the harm experienced by those closest to them under-examined. This paper bridges this gap by analyzing 12747 stories from r/QAnonCasualties, an online support group for people who have “lost” someone to conspiracy beliefs. We design a computational pipeline to extract fine-grained thematic traits from personal narratives and cluster them into six coherent radicalization personas, which we then link to the emotional toll reported by narrators via LLM-assisted emotion detection and regression modeling. We find that personas are meaningful predictors of specific emotional harms: radicalization perceived as a deliberate ideological choice is associated with anger and disgust, while personas marked by personal and cognitive collapse correspond to fear and sadness. This work provides an empirically grounded computational framework for understanding the relational harms of radicalization, opening new avenues for research into its wider social consequences.

Abstract:
LLMs over external data face a per-instruction tension: a single document or tool return often interleaves legitimate directives with adversarial prompt injections, and the model must follow the former while rejecting the latter. Existing defenses suppress or trust all external instructions, collapsing the safety-utility frontier and forcing a choice between over-refusal and over-compliance. We argue the missing capability is per-instruction credit attribution and formalize it as External-Data Instruction Classification, an identify-classify-act decision over each embedded directive. We propose EDIC, a decomposed reward framework for per-instruction compliance. A scene-grounded pipeline injects mixed beneficial and harmful instructions with counterfactual filtering, while a generative judge is initialized by SFT and refined with GRPO to attribute compliance at the instruction level. The judge then produces a reward that separately encourages beneficial compliance and penalizes harmful compliance for RL training. Across two backbones and five benchmarks, EDIC improves both safety and utility over the strongest prior defense, lifting harmful-rejection by up to 7.7% and Arena-Hard-Auto utility by 12.5% on Qwen3-4B.

Abstract:
Diagnosing pulmonary diseases requires integrating heterogeneous evidence amid phenotypic variability and cross-disease overlap. Although large language models (LLMs) have shown progress on pulmonary knowledge question answering (QA) and information-processing tasks, reliable pulmonary diagnosis requires patient-specific, relation-aware reasoning over electronic medical record (EMR) evidence rather than isolated knowledge recall. We define this gap between pulmonary knowledge and case-level diagnostic reasoning as the Pulmonary Knowledge-to-Diagnosis Gap. To address it, we introduce LungKG, the first structured pulmonary knowledge graph for diagnostic knowledge organization and record-grounded reasoning. LungKG contains 59,038 nodes and 164,308 edges across 15 entity types and 112 relation types, serving as both a reusable pulmonary knowledge resource and the foundation for LungKG-guided model adaptation. Built on LungKG, we propose Lung-R1, a LungKG-guided pulmonary LLM trained through KG-constrained reasoning-chain construction and KG-guided reinforcement learning. In a 20-system evaluation, Lung-R1-14B achieves state-of-the-art performance across Choice, Pulmonary-QA, and EMR Diagnosis, reaching an EMR Diagnosis score of 4.3583 and surpassing the strongest non-Lung-R1 baseline by 0.1476 points. These results demonstrate the value of LungKG-guided training for EMR-based pulmonary diagnosis. The dataset and code are available at GitHub repository.

Abstract:
Legal article retrieval is critical for building traceable and reliable legal AI systems, where conclusions must be grounded in specific legal articles. However, existing open-domain retrieval methods rely heavily on surface-level lexical or semantic similarity, making it difficult for them to distinguish legally relevant articles from those that are textually similar but legally inapplicable or misaligned with the user’s underlying intent. To bridge this gap, we propose LexPath, a domain-oriented multi-path framework comprising a multi-path retrieval module and an intent-aware reranking module. The retrieval module combines two complementary legal-specific paths to collect candidate articles: an IRAC-guided sparse path that expands queries with legally informative keywords, and a structure-guided dense path trained with hard negatives derived from legal hierarchy and citation relations. The reranking module then further refines the candidate ranking by incorporating the intent consistency score between queries and legal articles. We evaluate LexPath on three Chinese benchmarks spanning diverse query scenarios, and further assess its cross-jurisdictional applicability on a Japanese benchmark. Experimental results demonstrate that LexPath consistently outperforms lexical, dense, hybrid, and adaptive retrieval-augmented generation (RAG) baselines across the four benchmarks. Ablation studies further verify the effectiveness of each component. The data and code are available at: .

Abstract:
Enhancing the moral alignment of Large Language Models (LLMs) is a critical challenge, yet current techniques like RLHF often function as superficial behavioral guardrails, leaving the model’s internal moral representations untouched and susceptible to complex jailbreaks. In this paper, we propose a shift from behavioral intervention to latent space steering by leveraging Moral Foundations Theory (MFT). To ensure these internal representations capture moral concepts shared across different languages rather than superficial linguistic patterns, we first conduct a cross-lingual linear probing analysis, validating a shared yet culturally nuanced moral subspace between Chinese and English. Establishing this robust semantic foundation allows us to extract steerable Moral Vectors with high causal efficacy at both internal and behavioral levels. Finally, we introduce Adaptive Moral Fusion (AMF), a dynamic inference-time intervention that balances the safety-helpfulness trade-off. Empirical results demonstrate that AMF serves as a targeted intrinsic defense, significantly reducing jailbreak success rates on HarmBench while minimizing incorrect refusals on XSTest compared to standard baselines.

Abstract:
Chinese elementary-school writing is rubric-driven. We study it as a teacher-facing instructional support task: given a classroom prompt and requirements, generate a teacher-reviewable example matching the target genre, rubric, length, and age-appropriate register. Directly prompted frontier LLMs often mis-route genres or adopt an adult register. We present RA-MOPD, a four-stage pipeline that post-trains a 4B model into a unified essay writer across four genres and eight rubric dimensions. Its core stage augments on-policy reverse-KL distillation with Rubric-Weighted KL, which up-weights rubric-critical tokens via reward-model attribution, and Cross-Genre Mixing, a parameter-free expert blend guided by an empirical transfer prior. On a new 6,000-essay benchmark with eight reward models (QWK = 0.864), RA-MOPD reaches 4.21/5.00 and 97.8% routing—within 0.09 of GPT-5.3 3-shot (p < 0.01). No statistically significant difference is detected relative to a 4×-cost four-expert logit-averaging baseline (Δ = +0.03, p = 0.18). A GPT-4.1 judge, BERTScore, a 1,200-essay teacher study (62.8% preference), and register analysis provide converging evidence beyond RM metrics. GPT-5.5 holistic evaluation further supports the component-level gains, which remain stable on Qwen3-1.7B without model-specific retuning. Code and data: [https://github.com/TW-NLP/MOPD](https://github.com/TW-NLP/MOPD)

Abstract:
Deploying text embedding models under resource constraints is challenged by the sequence-length-dependent computational cost of self-attention. However, existing sequence-reduction strategies remain predominantly static and offer limited flexibility across inference budgets. We propose Jasper-Flash, an inference-scalable text embedding framework combining Elastic Token Compression (ETC) with Compression-Adaptive Progressive Distillation (CAPD). ETC uses a length threshold and a user-selected compression ratio to adjust sequence length on the fly, enabling flexible latency–quality trade-offs within a single model. To mitigate the optimization difficulty introduced by lossy sequence reduction, CAPD integrates heterogeneous multi-teacher distillation with a progressive curriculum over fixed and dynamically sampled compression ratios. We instantiate Jasper-Flash as Jasper-Token-Compression-600M, a bilingual text embedding model. Experiments on MTEB and C-MTEB demonstrate competitive performance across compression ratios and substantial reductions in encoding latency, particularly for longer inputs. The code and model are available at https://huggingface.co/infgrad/Jasper-Token-Compression-600M.

Abstract:
Reinforcement learning (RL) is used to improve the reasoning abilities of LLMs, while training data span heterogeneous tasks. However, most RL post-training pipelines rely on fixed or manually designed task mixtures, even though task usefulness changes as training progresses. Online curriculum methods often define learnability by update magnitude, ignoring whether the update translates into reward gains, which can misallocate rollout budget toward tasks with large but ineffective updates. We propose PAC, a Progress-Augmented Advantage Curriculum for multi-task RL of LLMs that combines two task-level signals: advantage-derived learnability, which measures the magnitude of the policy update a task can induce, and recent reward gains, which show whether those updates have improved task performance. A Bayesian Thompson Sampling controller uses these signals to allocate rollouts across tasks during GRPO training. We evaluate PAC under two settings: a multi-level reasoning setting and a multi-domain reasoning setting. PAC improves sample efficiency and final performance: it reaches comparable validation scores with fewer rollout steps and achieves higher final averages than random sampling and advantage-based curriculum baselines in both settings. These results show that jointly tracking advantage signals and actual reward gains yields an effective online curriculum for LLM post-training.

Abstract:
Calibration evaluates whether a model’s confidence aligns with its empirical accuracy. Existing studies often compare the calibration of different large language models using global calibration metrics such as Expected Calibration Error and Brier Score. We begin by showing, both theoretically and empirically, that such comparisons are confounded by differences in model accuracy. For fairer cross-model comparison, we then propose ACE, an accuracy-controlled evaluation framework with three complementary views: Instance-Aligned, Distribution-Aligned, and Candidate-Aligned calibration. Across multiple benchmarks, model families, and confidence elicitation methods, we use ACE to study two practically important comparison axes, small versus large models and thinking versus non-thinking models. We find that many previously reported calibration advantages under raw global metrics weaken substantially after accuracy control. We also find that ranking reversal is frequent: models favored by raw metrics often cease to be favored once accuracy is controlled. Our results show that raw global calibration metrics are not robust for cross-model comparison, and that fair calibration comparison requires accuracy-aware evaluation.

Abstract:
Synthetic data generation with Large Language Models (LLMs) has emerged as a promising solution in the medical domain to mitigate data scarcity and privacy constraints. However, existing approaches remain constrained by their derivative nature, relying on real-world records, which pose privacy risks and distribution biases. Furthermore, current patient agents face the Stability-Plasticity Dilemma, struggling to maintain clinical consistency during dynamic inquiries. To address these challenges, we introduce Patient-Zero, a novel framework for ab initio patient simulation that requires no real medical records. Our Medically-Aligned Hierarchical Synthesis framework generates comprehensive and diverse patient records from medical knowledge via stratified attribute permutation. To support rigorous clinical interaction, we design a Dual-Track Cognitive Memory System to enable agents dynamically update memory while preserving logical consistency and persona adherence. Extensive evaluations show that Patient-Zero outperforms baselines in both data quality and interaction fidelity. In physician evaluations, our records obtain the highest mean diversity, consistency, and completeness ratings. Downstream medical reasoning model trained on our synthetic dataset shows substantial performance gains (MedQA +24.0%; MMLU +14.5%), demonstrating the practical utility of our framework.

Abstract:
Reinforcement Learning (RL) has emerged as a critical driver for enhancing the reasoning capabilities of Large Language Models (LLMs). While recent advancements have focused on reward engineering or data synthesis, few studies exploit the model’s intrinsic representation characteristics to guide the training process. In this paper, we first observe the presence of high-magnitude activations within the query and key vectors when processing long contexts. Drawing inspiration from model quantization—which establishes the criticality of such high-magnitude activations—and the insight that long-context reasoning inherently exhibits a sparse structure, we hypothesize that these weights serve as the pivotal drivers for effective model optimization. Based on this insight, we propose LongAct, a strategy that shifts from uniform to saliency-guided sparse updates. By selectively updating only the weights associated with these significant activations, LongAct achieves an approximate 8% improvement on LongBench v2 and enhances generalization on the RULER benchmark. Furthermore, our method exhibits remarkable universality, consistently boosting performance across diverse RL algorithms such as GRPO and DAPO. Extensive ablation studies confirm that high-magnitude activations pinpoint the model’s most sensitive components, demonstrating that focusing on these salient features is key to unlocking long-context potential.

Abstract:
Providing extensive context via prompting is vital for leveraging the capabilities of Large Language Models (LLMs). However, lengthy contexts significantly increase inference latency. To mitigate this issue, context compression—particularly soft prompt compression—has emerged as a widely studied solution, which converts long contexts into shorter memory embeddings via a trained compressor. Existing methods typically compress the entire context indiscriminately into a set of memory tokens, requiring the compressor to capture global dependencies and necessitating extensive pre-training data to learn effective patterns. Inspired by the chunking mechanism in human working memory and our empirical observation of spatial specialization within memory embeddings, we propose Parallelized Iterative Compression (PIC). By simply modifying the Transformer’s attention mask, PIC explicitly restricts the receptive field of memory tokens to sequential local chunks, thereby significantly reducing the difficulty of compressor training. Experiments across multiple downstream tasks demonstrate that PIC outperforms competitive baselines while dramatically expediting the training process. Finally, we provide an investigation into the emergence of spatial specialization, revealing how tuning strategies and explicit structural priors shape the latent structure of the memory embeddings.

Abstract:
Large Vision-Language Models (LVLMs) are promising scalable annotators for video understanding, but they are not directly suited for dense temporal annotation. We study this challenge in egocentric streaming understanding, where the goal is to segment continuous first-person video into atomic actions: short, concrete physical interactions with well-localized temporal boundaries. We propose CoTT (Chains of Temporal Transitions), an annotation-free framework that structures LVLM reasoning around temporal transitions and combines it with iterative streaming distillation. CoTT uses structured LVLM reasoning to generate dense atomic action pseudo-labels from unlabeled egocentric videos, distills these labels into a lightweight online model, and bootstraps the model’s predictions to improve temporal consistency. On large-scale egocentric benchmarks, EPIC-Kitchens-100 and Ego-Exo4D, CoTT achieves competitive performance against supervised online localization methods without using ground truth temporal annotations for training. It also transfers across egocentric datasets and, on THUMOS14, outperforms prior best training-free zero-shot Temporal Action Localization methods, even without any dataset-specific training or adaptation.

Abstract:
Large Reasoning Models (LRMs) generate intermediate reasoning traces that may contain unsafe content, even when their final responses appear safe. Guardrail models are designed to detect and block unsafe content, yet existing benchmarks for unsafe content detection focus primarily on prompts and final responses, leaving reasoning traces largely unexamined. Moreover, these benchmarks typically provide only binary safety labels, without evidence annotations that justify the judgments. To address these limitations, we introduce TRACE, an evidence-grounded safety evaluation benchmark that covers the entire LRM inference pipeline: prompts, reasoning traces, and final responses. TRACE includes prompts in two languages spanning nine risk categories and ten attack strategies. For each prompt, four LRMs generate reasoning traces and final responses, and we annotate the safety of each component and extract supporting evidence from the corresponding source text. Evaluating 18 guardrail models on TRACE reveals that safety judgment for reasoning traces is substantially more challenging than for prompts or final responses, and that current models struggle to accurately extract supporting evidence. These findings highlight the need for guardrail models that can reliably detect and precisely localize unsafe content across the LRM inference pipeline.

Abstract:
Recent AI agents can flexibly invoke skills to solve complex tasks, but their long-term improvement is fundamentally constrained by a lack of systematic skill construction, accumulation, and transfer. In particular, without a unified framework for skill consolidation, agents tend to redundantly construct similar capabilities across different tasks, are unable to effectively transform experience into reusable assets, and struggle to generalize task-specific skills to novel scenarios. To address this limitation, we propose SkillPyramid, a skill consolidation framework that reuses existing skill experience for broader task generalization. Operating on a hierarchical skill topology, SkillPyramid further introduces a self-evolution mechanism that enables agents to compose, validate, and incorporate new skills during task execution. Experiments on ALFWorld, WebShop, and ScienceWorld across four backbone models show that SkillPyramid substantially increases the average reward by 38.0% and reduces execution steps by 27.7%. Overall, our method transforms a skill collection from a static resource pool into a dynamic evolution system.

Abstract:
Meta-analyses are central to evidence-based medicine, but their production remains slow because reviewers must search PubMed and screen large candidate sets under PICOS eligibility constraints. We introduce MetaBench, a large-scale benchmark for early-stage PubMed title/abstract citation screening in meta-analysis study selection, formulated as a two-stage reranking and Top-N screening problem with effort-aware evaluation. MetaBench contains 7,801 PubMed meta-analyses, 120,930 PMID-linked included studies, and reproducible candidate pools with 4.2M unique title/abstract records constructed from replayed PubMed strategies or expert-validated training-only reconstructions. To improve evaluation fidelity, we introduce a coverage-based tiering protocol and reserve Tier 1 (1,297 topics) as a high-integrity test set. MetaBench evaluates candidate reranking with Recall/nDCG and Effort@R/NormEffort@R, and budgeted Top-N screening with sensitivity, specificity, precision, and F-scores. The main benchmark intentionally focuses on scalable title/abstract screening rather than full-text eligibility adjudication; PMCID metadata supports an auxiliary full-text track on an OA-constrained subset. Experiments show that MetaBench supervision improves dense reranking, reduces screening effort, and strengthens LLM-based inclusion decisions. An anonymized codebase and dataset sample are included with this ARR submission; the full dataset and codebase will be publicly released upon acceptance.

Abstract:
Large Reasoning Models (LRMs) achieve strong performance on complex tasks through explicit multi-step reasoning, but their long reasoning traces also expose them to denial-of-service (DoS) attacks that induce excessive computation. Existing DoS attacks rely on indirect, heuristic objectives to optimize adversarial inputs, without leveraging the internal mechanisms that drive long reasoning, resulting in relatively inefficient optimization and weaker attack effectiveness. In this paper, we show that over-reasoning in LRMs is mediated by a sparse neuron-level mechanism that can be systematically exploited. Specifically, we introduce Aha Neuron Localization (ANL), a method for identifying reasoning-related neurons whose activations are strongly associated with the generation of reasoning tokens, and demonstrate that selectively amplifying their activations induces over-reasoning. Based on this insight, we develop ANL-guided DoS Attack (AGDA), which optimizes a short adversarial suffix using reasoning-related neurons as precise and stable white-box signals. When attached to user queries, the suffix induces over-reasoning. Experiments show that the adversarial suffixes learned by AGDA yield reasoning traces over 3.45 × longer than the original outputs and more than 2 × longer than the strongest method, while transferring effectively to closed-source models.

Abstract:
Alignment has been widely reported to reduce stereotypical bias in large language models, but it is unclear whether these reductions reflect debiasing or suppression at the output level. Across 27 base–instruct pairs from four open-weight families and four bias benchmarks, three patterns provide substantial evidence that the measured reduction is largely suppression. (i) The chat template drives about half the alignment effect on CrowS-Pairs and essentially all on StereoSet, where 18 of 27 improve without it (p = 0.061) versus 26 of 27 with it (p < 0.001). Per-pair effects remain below d = 0.2 in both conditions. IAT lies at the weight-level end of the spectrum, with template contribution indistinguishable from zero. (ii) Decomposing BBQ’s ambiguous-context score shows instruct models defer rather than debias (28% → 53%) and are more biased when they answer (conditional bias 0.53 → 0.59). Suppressed bias is recoverable: the most aggressive adversarial framing restores CrowS-Pairs bias to near base levels, while on BBQ it partly reverses deferral without reactivating conditional bias. (iii) Across eight probed pairs, base and instruct gender directions remain geometrically aligned (median cosine 0.86 vs. 0.006 random-pair baseline). Together, these findings suggest benchmark improvements after alignment may overstate genuine bias reduction.

Abstract:
Precise retrieval of scientific information is fundamentally constrained by long-tailed concepts and high fact-sensitivity of scientific corpora. These challenges often limit the effectiveness of dense retrievers and hallucination-prone LLM augmentation. To address this, we present REPAIR, a self-evolving data augmentation framework for scientific dense retrievers. REPAIR iteratively synthesizes training data to address knowledge gaps by cycling through diagnosis of long-tail concepts, API-guided evidence expansion, and differentiation via hard negative mining. This process effectively grounds retrieval in factual reality to resolve fine-grained distinctions. Extensive experiments demonstrate that REPAIR significantly outperforms 19 strong baselines on nine materials science and biomedical benchmarks. Our work highlights that diagnosing and factually augmenting data to long-tail deficits is essential for robust scientific retrieval.

Abstract:
Correcting flawed reasoning traces remains a significant challenge for Large Language Models (LLMs), whose autoregressive generation can propagate early mistakes into subsequent reasoning. We introduce DCGC, a Masked Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context. DCGC combines task-specific Supervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG. This mechanism separates problem-only and joint problem-draft branches and scales the draft-conditioned residual using a relative confidence gap. Across math, code, and knowledge reasoning benchmarks, DCGC outperforms standard sampling and simpler CFG variants, with additional results suggesting transfer to different diffusion backbones. In test-time setting where ground-truth failure labels are unavailable, DCGC improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module for difficult reasoning instances.

Abstract:
While Parameter-Efficient Fine-Tuning (PEFT) reduces the cost of adapting Large Language Models (LLMs), current methods like LoRA suffer from catastrophic forgetting due to unregulated updates on high-importance parameters. To address this stability-plasticity dilemma, we introduce ANCHORS: Adaptive Normalization for Continual High-Order Retention via Sigmoid-Guided Magnitude Control, a novel framework that ensures uninterrupted knowledge retention during aggressive adaptation. ANCHORS integrates CABR Decomposition to preserve informative subspaces and S-MagNorm, a bio-inspired normalization mechanism that utilizes Sigmoid-guided magnitude control to dynamically dampen updates on critical weights. Extensive experiments on reasoning and QA benchmarks demonstrate that ANCHORS consistently outperforms LoRA, DoRA, and CUR-LoRA. It achieves performance gains of up to 4.05% in MCQ and 2.61% in QA tasks while retaining 61.12% of pre-trained knowledge—surpassing Experience Replay and EWC baselines in continuous learning scenarios.

Abstract:
In the time series domain, increasing efforts extend large language models (LLMs) to multimodal large language models (MLLMs) by incorporating temporal data for downstream temporal understanding tasks. In conventional MLLM training, models primarily rely on language signals, while temporal information provides substantially weaker supervision. This imbalance leads to insufficient learning of temporal patterns. To address this issue, we propose TimeSense, a framework with a disentangled reconstruction structure that introduces explicit temporal supervision and encourages the learning of both global trends and local temporal variations that are often underrepresented in text-dominated multimodal training. Existing QA benchmarks provide limited coverage of temporal variations and compositional temporal dependencies, making it difficult to systematically diagnose these MLLMs capabilities across diverse temporal understanding tasks. To facilitate a more fine-grained evaluation, we introduce EvalTS, a comprehensive benchmark spanning three difficulty levels (Atomic, Molecular, and Compositional) with diverse high-quality queries for evaluating temporal understanding abilities in large language and multimodal language models. Experiments on EvalTS and existing benchmarks demonstrate that TimeSense consistently outperforms prior methods and substantially improves temporal understanding performance.

Abstract:
Safety evaluation in large language models treats harm as culturally invariant, yet a model cannot correctly apply a safety norm without knowing whose norms bind. We introduce ScopeBench, a benchmark that treats cultural scope as a first-class safety dimension. It comprises 3,323 norms spanning 49 countries across nine cultural clusters, each annotated as Universal or Specific. We define three scope error types identically across four evaluation tasks: Scope Inflation, Scope Deflation, and Scope Displacement. Evaluating 21 frontier LLMs across seven model families, we surface three structural failures. First, the dominant error type inverts with prompt format: Scope Deflation reaches 70% in single-answer classification, while Scope Inflation reaches 33% when each candidate culture is judged independently. Second, scope-classification accuracy is empirically decoupled from behavioral safety: the six classifiers achieving ≥97% Task 1 accuracy still produce harmful responses to 40–42% of scope-violating statements. Third, no model is safe in interaction, with harmful-response rates ranging from 29% to 51% across the cohort, excluding one model that systematically refuses the task. We release all data, code, and model outputs to establish cultural scope as a distinct alignment axis.

Abstract:
Large Vision-Language Models (LVLMs) incur high computational costs from redundant visual tokens. Although training-free attention-based multi-layer pruning in the vision encoder has been explored as an effective strategy, we find that pruning in shallow layers consistently degrades performance. We aim to understand this problem and seek a solution. By analyzing attention patterns across network depth, we find that shallow layers primarily function as edge detectors with chaotic attention maps, while deeper layers transition through local subject recognition and unstable semantic aggregation. To address misalignment between pruning strategies and network stages, we propose STD, a hierarchical token pruning framework that adapts token selection mechanisms to the functional role of each stage. STD employs High-Frequency Spectral Analysis in shallow layers to deterministically preserve structural edges, uses Gaussian-Smoothed Attention in intermediate layers to maintain spatial coherence, and introduces a Stability-Adaptive Trigger in deep layers to execute pruning only during semantically stable phases. Extensive experiments show that STD outperforms state-of-the-art pruning methods by 1.1% on LLaVA-1.5-7B with 88.9% token reduction, while being plug-and-play and highly effective when combined with other methods, and by 2.1% on LLaVA-NeXT-7B with 94.4% reduction, delivering a 3.9× speed-up in the prefilling stage. Our code will be released at https://github.com/Twilight03/STD.

Abstract:
Synthetic data is increasingly used to train large language models (LLMs), yet its security implications remain poorly understood. Prior work on subliminal learning suggests that models can inherit behavioral traits from seemingly unrelated training data. In this work, we investigate whether such mechanisms can be exploited to inject targeted social biases into aligned models through semantically benign synthetic data. We construct a pipeline in which a misaligned teacher model generates filtered synthetic datasets across domains such as creative writing and code generation, which are then used to fine-tune aligned student models. Our experiments show that benign-looking synthetic data can act as a covert channel for transmitting targeted biases while largely preserving the student model’s general task capabilities. These results reveal a previously underexplored security risk in synthetic data–driven LLM training pipelines and highlight the need for improved safeguards. As one possible step toward this goal, we suggest that log-linearity-based scoring may provide a useful signal for screening seemingly benign synthetic data.

Abstract:
Dense retrievers are a core component of Retrieval-Augmented Generation (RAG) systems, yet their growing adoption introduces critical security risks in the Al supply chain. Existing retrieval backdoor attacks primarily rely on coarse-grained parameter updates, which often degrade benign retrieval behavior and require substantial computational cost, limiting their stealth and practicality across deployment settings. In this paper, we propose LOKI (Locating Optimal Knock-In), a framework that reformulates stealthy retrieval backdoor injection as a neuron-level local editing problem. Following a Locate-then-Edit paradigm, LOKI first identifies pivotal neurons that are highly sensitive to triggers yet decoupled from benign retrieval semantics via a multi-dimensional diagnostic analysis. And then LOKI implants backdoor logic through constrained local editing that balances trigger-induced alignment with semantic manifold preservation. Extensive experiments show that LOKI achieves near 100% attack success while modifying fewer than 1% of model parameters, with only limited impact on clean retrieval performance.

Abstract:
Supervised fine-tuning (SFT) is the de facto standard for adapting large language models (LLMs) to target domains, but it often degrades the model’s general capabilities, a phenomenon known as catastrophic forgetting. Existing approaches typically modify the SFT loss to mitigate forgetting, but they inevitably operate along a domain-generality trade-off. In this work, we step outside this trade-off by decoupling the two capabilities at the model level: we keep the original base model for general capability, and selectively invoke the SFT expert only when domain-specific knowledge is required. Specifically, we propose CPR ( C ritical- P oint R outing), a token-level routing framework between a base model and its expert derivative, based on critical tokens where the base model fails but the expert succeeds. We train a lightweight hierarchical router that estimates the expert-call probability per token, and pair it with a tailored inference procedure that combines momentum smoothing and threshold gating. Across diverse model-domain configurations, CPR achieves state-of-the-art across all settings, surpassing SFT expert by 1.4-5.5% in domain performance while recovering its general-capability drop from 3.4-14.5% to at most 0.5%, with minimal overhead from invoking the expert on only one-third of tokens.

Abstract:
Competitive programming remains challenging for code LLMs. Despite recent progress, many training pipelines still depend on scarce real-world data, raising concerns about scalability and near-duplicate benchmark contamination. In this paper, we examine whether synthetic training artifacts can support the complete SFT-to-RL cycle for competitive programming: no real-world tasks, solutions, or test cases are directly used for post-training. To this end, we synthesize tasks, verified solutions, and reliable test cases that serve as reward signals for reinforcement learning. To improve reward reliability, we introduce a dual-verification strategy that reduces noise in both selected solutions and test outputs. Using this high-quality data, we train the X-Coder model series. X-Coder-14B achieves 67.5% avg@8 on LiveCodeBench v5 and 63.4% on v6, outperforming its base model by over 40 points. Further analysis provides practical insights into synthetic post-training, highlighting the value of diverse tasks, verified long-CoT supervision, and code-centric reinforcement. Our data and models are released at https://github.com/JieWu02/X-Coder.

Abstract:
Reinforcement learning has become a cornerstone technique for developing reasoning models in complex tasks, ranging from mathematical problem-solving to imaginary reasoning. The optimization of these models typically relies on policy gradient methods, whose efficacy hinges on the accurate estimation of an advantage function. However, prevailing methods typically employ static advantage estimation, a practice that leads to inefficient credit assignment by neglecting the dynamic utility of training samples over time. This limitation results in suboptimal policy updates, which in turn manifest as slower convergence rates and increased learning instability, as models fail to adapt to evolving sample utilities effectively. To address this problem, we introduce ADORA (Advantage Dynamics via Online Rollout Adaptation), a novel framework for policy optimization. ADORA dynamically adjusts the advantage function’s weighting by adaptively categorizing training data into temporarily advantageous and disadvantageous samples, based on their evolving utility during online model rollouts. This tailored data differentiation strategy allows ADORA to be seamlessly integrated into existing policy optimization algorithms without significant architectural modifications, enabling the policy to prioritize learning from more informative experiences and thereby achieve more efficient policy updates. Extensive evaluations across diverse model families and varying data scales demonstrate that ADORA is a robust and efficient framework. It significantly enhances long reasoning in both geometric and mathematical tasks, consistently achieving notable performance gains without requiring sensitive hyperparameter tuning.

Abstract:
Understanding how Large Language Models (LLMs) perform logical reasoning internally remains a fundamental challenge. While prior mechanistic studies focus on identifying task specific circuits, they leave open the question of what computational strategies LLMs employ for propositional reasoning. We address this gap with a causal mechanistic analysis on PropLogic-MI, a controlled benchmark of 11 propositional rules across one- and two-hop tasks, applied to three model families (Qwen3, Llama-3.1, Mistral). Rather than asking which components are necessary, we ask how the reasoning process is organized, and identify four interlocking mechanisms: Staged Computation, where early, middle, and late layers take on distinct functional roles; Information Transmission, where semantic content aggregates at boundary tokens; Fact Retrospection, where fact tokens stay causally active as binding lookup in middle layers and as sustained access in late layers under hard reasoning loads; and Specialized Attention Heads that structurally implement these patterns. A prompt-order control that places the query before the facts confirms that this organization is model-internal rather than input-layout-induced. These findings show that pretrained LLMs solve propositional reasoning through a structured, layer-organized process that recurs across models, rule categories, and reasoning hops.

Abstract:
Large vision-language models increasingly rely on long-context modeling to reason over documents, hour-level videos, and long-horizon agent trajectories, requiring them to locate relevant evidence across interleaved text and images. Prior work has studied this behavior using retrieval heads in large language models, but its copy-based criterion does not directly apply when evidence appears in images. We introduce a multimodal retrieval head detection method that scores attention from question tokens to textual or visual evidence. With this method, we show that multimodal retrieval heads are sparse, intrinsic, and causally important: only 4.4-10.2% of attention heads account for 50% of the positive retrieval-score mass, and masking the top-5% selected heads reduces performance on two tasks from MMLongBench by 42.5 and 62.3 percentage points, while random-head masking is far less damaging. Further analysis shows that these heads are partly shared across modalities yet remain dynamic within each modality, with image retrieval heads changing more than text retrieval heads as context length and haystack modality change. Without further training, we find that these heads can also be used directly to rank visually rich documents: on MMDocIR, Qwen3-VL-8B selected-head scoring improves Recall@1 by 7.7/7.4 macro/micro points for page retrieval and 6.3/6.8 points for layout retrieval over the strongest reported baseline.

Abstract:
Temporal knowledge graph forecasting predicts future facts from historical snapshots, and its accuracy hinges on how well a model exploits the historical evidence available to each query. Existing methods route every query through a single reasoning pipeline, yet queries with the same predicate form can rest on very different histories and call for different inductive biases. We observe that query histories fall into three regimes: sparse histories with few prior events, recurrent histories dominated by stable repetition, and ambiguous histories with many competing candidates. Building on this observation, we propose QTI-MoE, a Query-Type-aware Inductive-bias Mixture-of-Experts framework. QTI-MoE characterises each query with a soft type label derived from its historical prefix and forwards it to one of three experts whose memory pathways carry complementary biases: recency-weighted aggregation for sparse queries, count-based repetition for recurrent queries, and diversity-normalised scoring for ambiguous queries. QTI-MoE achieves state-of-the-art performance on four standard benchmarks, and a type-stratified analysis confirms that each expert peaks on its target regime.

Abstract:
LLMs acquire vast amounts of knowledge during pre-training, but often lack the specialized knowledge needed to answer questions from niche sources such as manuals or technical documents unseen during pre-training. Continued pre-training (CPT) is widely used to inject such knowledge into model parameters. However, niche documents seldom repeat facts, making it difficult for CPT to robustly acquire such knowledge. Recent works address this by generating multiple paraphrases of the new knowledge, but paraphrasing is computationally expensive and typically requires powerful LLMs. In this work, we introduce KItCAT: Knowledge Injection via Corrupted Auto-regressive Training, a lightweight training strategy that reduces the need for paraphrasing in decoder-only LLMs. KItCAT augments standard next-token prediction by stochastically corrupting the input sequence. During training, a random subset of input tokens is replaced with other vocabulary tokens while the original next-token labels are kept unchanged. This simple intervention generates diverse training inputs from each sample, enabling large-scale data augmentation at negligible cost. We show that KItCAT consistently improves over CPT across multiple datasets and model families. Code is available at https://github.com/meghanadhpulivarthi/KItCAT.

Abstract:
The rapid growth of LLM evaluation resources makes exhaustive benchmark curation increasingly difficult. Existing efficient benchmarking methods reduce evaluation cost by selecting representative subsets or predicting full-benchmark performance, but typically require evaluation results from many models and additional training. In this paper, we ask a more fundamental question: to what extent do benchmarks actually measure different things? We study this question at two levels: intra-benchmark redundancy, which supports benchmark compression, and inter-benchmark similarity, which supports evaluation-suite design. We first formalize both notions through a functionalist behavioral criterion, and then propose an evaluation-free proxy based on Sparse Autoencoder (SAE) features for redundancy diagnosis. Our central hypothesis is that behavioral overlap can be approximated by feature overlap. This hypothesis yields level-specific metrics based on feature-coverage saturation within benchmarks and feature-footprint overlap across benchmarks. Experiments show that SAE feature-based redundancy strongly correlates with performance-based redundancy across 17 benchmarks, while SAE-guided subset selection produces more stable and representative subsets than random sampling. SAE feature overlap also predicts inter-benchmark performance similarity across 8 benchmarks. These results show that SAE features can turn benchmark analysis from a costly performance-matrix problem into a feature-footprint problem, enabling evaluation-free benchmark compression and suite curation.

Abstract:
Large language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated data can improve downstream task performance. Current best-performing approaches typically rely on few-shot prompting with target-language examples, which increases inference costs and may reduce diversity through lexical anchoring. In this work, we investigate activation steering as an alternative for low-resource synthetic data generation. We study two steering strategies: Language Steering, which targets the linguistic identity of a language, and Quality Steering, which captures well-formedness by contrasting human-written and backtranslated text representations. We evaluate these methods across four open-source LLMs, multiple layers, and 11 typologically diverse languages by generating sentiment and topic classification data and fine-tuning smaller classifiers. Steering is applied in both zero-shot and few-shot prompting settings and compared against non-steered counterparts. Our results show that steering on early layers consistently improves the diversity of generated data while often yielding stronger downstream model performance, particularly for low-resource languages.

Abstract:
Long-horizon software engineering agents often need to manage lengthy and noisy interaction histories under limited context budgets. Existing memory management methods rely on static compression workflows or rigid compression schedules, and often decouple memory decisions from task-solving performance. We present SWE-MeM , a training framework for proactive and on-demand memory management in software engineering agents. SWE-MeM provides a flexible memory tool that lets agents decide when, what, and how to compress based on trajectory state, task progress, and remaining context budget. We train agents with synthesized proactive memory-management trajectories and Memory-aware GRPO, which jointly optimizes memory management and issue resolution through memory-aware trajectory splitting and step-level credit assignment. On SWE-Bench Verified, SWE-MeM achieves 43.4% and 60.2% resolve rate with 4B and 30B models, respectively, outperforming existing memory management baselines in both performance and efficiency.

Abstract:
Natural Language Processing (NLP) systems engage with identity across datasets, models, tasks, and evaluation frameworks, yet they often do so by reducing people to fixed categories (e.g., race, gender, age). This reduction is particularly consequential in identity-sensitive applications: for example, it affects how systems classify harm or generate tailored responses to user needs. But individuals contain multitudes: they are not a homogeneous group, but are situated, context-dependent selves whose differences exceed fixed categories. To examine how identity is addressed in NLP, we survey ACL Anthology papers that mention "identity". Our study investigates how the term is operationalized, where it enters the NLP pipeline, which tasks incorporate identity-related information, and whether its use is explicit or implicit. Our analysis shows that identity is most often operationalized through explicit sociodemographic categories, especially those referring to protected groups by law. Moreover, it is primarily used during the evaluation stage to audit systems for bias and harm. We synthesize these patterns into a taxonomy of identity operationalizations in NLP. Finally, we propose a roadmap to treat identity as relational, contextual, intersectional, and linguistically indexed, with the broader scope of achieving socially aware NLP.

Abstract:
Knowledge graph completion(KGC) aims to infer missing facts from observed relational triples, which demands jointly modeling textual semantics and relational structure. Existing structure-based methods capture graph topology but often struggle with sparse entities and long-tail relations, while text-based methods exploit entity and relation descriptions but weakly model relational neighborhoods. In this paper, we propose LQ-KGC, a latent-query refinement framework that enables iterative interaction between semantic inference and structural reasoning for KGC. Instead of treating all missing triples as latent variables, LQ-KGC constructs a restricted set of semantically plausible latent queries and alternates between two refinement steps: the language model predicts latent tail distributions from textual descriptions, while the graph model incorporates high-confidence pseudo tails as pseudo edges for relation-aware message passing. Meanwhile, structure-aware graph predictions are distilled back into the language model to refine semantic representations. Experiments on WN18RR and FB15k-237 demonstrate that LQ-KGC consistently improves over text-based and graph-based baselines, with additional analyses highlighting the effectiveness of pseudo-edge augmentation and latent-query construction.

Abstract:
Peer review is central to scientific quality, yet reliance on simple heuristics, namely lazy thinking and non-specific critiques, has threatened review quality. Prior work frames lazy thinking detection as single-label classification and stops at detection, yet review segments often exhibit multiple co-occurring issues, and reviewers benefit more from actionable, guideline-aware feedback than from labels alone. We further show that off-the-shelf LLMs prompted for feedback frequently rewrite the entire review or address the authors rather than the reviewer, motivating an inference-time approach. We introduce an LLM-driven framework that decomposes reviews into argumentative segments, identifies issues violating ACL Rolling Review (ARR) guidelines, and generates targeted feedback using issue-specific templates refined by a novel iterative, reranking-based generation algorithm. In a controlled rewriting study, our feedback reduces guideline violations by up to 92.4%. We also release LazyReviewPlus, the first multi-label dataset of 1,309 sentences annotated for detecting lazy thinking and lack of specificity.¹ ¹ Code and Data: https://github.com/UKPLab/emnlp2026-reviewfeedbackagent

Abstract:
Continual learning for Table Question Answering (TableQA-CL) aims to train models on sequential tasks with limited supervision per task. Existing methods either rely on replaying historical data or assume ideal conditions like gold task boundaries, limiting real-world applicability. We propose TORe (Table Operation Recipe Reconstruction), a rehearsal-free framework that reconstructs historical knowledge through schema-aware reasoning patterns. TORe extracts abstract Operation Recipes from table reasoning processes, identifies fine-grained knowledge deviations between current and past tasks, and employs execution-verified LLM synthesis to generate pseudo-examples reflecting both current and historical reasoning patterns. Experiments on two TableQA-CL benchmarks and multiple backbone models show that TORe consistently improves average accuracy over strong rehearsal-free baselines, remains competitive with replay-based methods, and maintains robust performance under different task orders.

Abstract:
Chain-of-Thought prompting is increasingly relied upon as a safety monitoring surface: if a model reasons before acting, its trace can be inspected before harm occurs. Yet reasoning is a stochastic branching process rather than a deterministic, auditable path. We introduce Bifurcation Entropy ( H BE ), a step-level diagnostic metric that extends Kernel Language Entropy to intermediate reasoning steps, measuring the semantic breadth of a model’s next-step distribution at each point in the Chain-of-Thought. We deploy H BE via In-Distribution Attacks, a controlled probing instrument that injects the model’s own high-entropy continuations to test whether H BE predicts exploitability. Across five reasoning models in high-stakes blackmail scenarios, phrases in the top H BE quartile are exploited 6.2 × more often than the lowest, with H BE -targeted probing inducing significantly more alignment failures than random selection. These results establish H BE as a principled probing signal that concentrates safety monitoring effort on the minority of reasoning steps that carry the majority of alignment risk.

Abstract:
Autoregressive language models suffer from high inference latency due to their sequential decoding nature. Speculative decoding (SD) mitigates this by employing a lightweight draft model to propose candidate tokens, which are selectively verified by a larger target model. While existing methods either adopt multi-draft strategies to increase acceptance rates or block verification techniques to jointly verify multiple tokens, they remain limited by treating these improvements in isolation. In this work, we propose SpecTr-GBV, a novel SD method that unifies multi-draft and greedy block verification (GBV) into a single framework. By formulating the verification step as an optimal transport problem over draft and target token blocks, SpecTr-GBV improves both theoretical efficiency and empirical performance. We theoretically prove that SpecTr-GBV achieves the optimal expected acceptance length within the class of conditionally independent couplings (CIC) induced by our multi-draft verification design, and show that this optimal value increases with the number of drafts. Empirically, we evaluate SpecTr-GBV across five datasets and four baselines. Our method achieves superior speedup and significantly higher block efficiency while preserving output quality. In addition, we perform comprehensive ablation studies to evaluate the impact of various hyperparameters in the model. The code is publicly available at https://github.com/junemily/SpecTr-GBV.git.

Abstract:
Cross-domain multimodal time series forecasting is a challenging task, requiring models to integrate precise numerical comprehension, cross-domain semantic understanding, and effective multimodal fusion. Existing approaches either build Time Series Foundation Models (TSFMs) from scratch or leverage pretrained Large Language Models (LLMs). However, TSFMs often overlook semantic understanding and lack the ability to perform future-oriented semantic reasoning, and LLMs struggle with numerical comprehension and accurate quantitative forecasting. To overcome these limitations, we propose KairosAgent, a novel agentic framework for multimodal time series forecasting, including an LLM-based reasoner and a TSFM-based forecaster. KairosAgent unifies textual reasoning and numerical forecasting by dynamically invoking analytical tools to enhance the numerical understanding and semantic reasoning capabilities of LLMs. The reasoning results are subsequently fused into the TSFM pipeline, enabling more accurate and reliable future predictions. To further improve the reasoning, we curate a large-scale corpus of high-quality trajectories, alongside a reinforcement learning from forecasting paradigm with multi-turn refinement and turn-level credit assignment. Experiments demonstrate that KairosAgent achieves superior zero-shot forecasting performance while maximizing the utility of pretrained LLMs and TSFMs, presenting a promising direction for efficient and interpretable time series agents. The project page is at https://foundation-model-research.github.io/KairosAgent.

Abstract:
Real-world GUI usage frequently involves workflows that span multiple devices and platforms, requiring the transfer of intermediate results, maintenance of shared state, and coordination across heterogeneous environments. However, existing GUI benchmarks overwhelmingly evaluate agents on single-device, statically defined tasks, thus leaving such cross-device capabilities largely unexamined, resulting in an overly optimistic assessment of agents’ readiness for real-world usage. We introduce JarvisGUI, a dynamic benchmark that evaluates GUI agents on cross-device workflows requiring coordinated interaction across heterogeneous platforms—including Android, Windows, and Ubuntu. Specifically, JarvisGUI formulates GUI tasks as input–output transformations under a lightweight type system, which allows us to automatically compose multi-step, cross-device workflows and dynamically evaluate agent performance within a unified framework. By evaluating agents in virtual environments spanning multiple operating systems, JarvisGUI reveals that state-of-the-art open-source GUI agents struggle with the state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management required for real-world workflows, exposing a critical capability gap invisible to existing benchmarks.

Abstract:
Localizing issue-relevant code regions is a critical step in automated software engineering. However, due to their reliance on sparse trajectory-level signals, existing methods cannot identify which per-turn actions matter and often discover correct code regions during exploration but fail to commit them. To address these limitations, we propose an action-aware reinforcement learning method that combines a per-turn reward sequence rewarding both the discovery and commitment of gold code regions with an action-level advantage estimation scheme that isolates each action’s credit by grouping turns sharing the same exploration context. Extensive evaluations on two benchmarks show that our method achieves 1.58% and 8.55% improvements over state-of-the-art in average F1, with our 4B model outperforming baselines up to 8× larger. Our code is available at https://anonymous.4open.science/r/A2Agent.

Abstract:
LLM training increasingly relies on teacher-generated supervision, from synthetic responses to reasoning traces and tool-use demonstrations. Current practice often chooses the highest-performing teacher to generate student training data, implicitly treating teacher test performance as a proxy for teaching quality. We show that this assumption can fail: even when multiple teachers provide correct answers to the same question, the answer from the strongest teacher is not necessarily the best supervision for a given student. To address this gap, we propose Student-Centric Answer Sampling (SCAS), a framework that selects from verified teacher-generated answers according to their estimated student-centric learning cost. Motivated by a token-wise gradient decomposition, we derive an efficient forward-only proxy for this cost and use it to guide answer selection during training. Experiments across 30 teacher models, 6 student base models, and 8 tasks show that SCAS consistently improves student performance, suggesting that effective distillation should prioritize supervision matched to the current student rather than teacher strength alone.

Abstract:
Large Language Models (LLMs) frequently exhibit performance degradation in specialized domains due to distribution shifts. While unsupervised Test-Time Adaptation (TTA) offers a lightweight solution, existing methods suffer from critical flaws: input-centric adaptation lacks reasoning constraints, while output-centric optimization falls into a "Confidence Trap" of reinforced hallucinations. We instead study Few-Shot Anchored TTA, where a small labeled support set is available, and propose Anchor-Tethered Test-Time Adaptation (AT-TTA), a framework that synergizes linguistic alignment with reliable reasoning. AT-TTA employs Dual-View Uncertainty Calibration, minimizing input perplexity for domain alignment and sequence-level output energy for generation consistency. Furthermore, to mitigate catastrophic drift, we construct a "Semantic Anchor" by fine-tuning on the labeled support set and its filtered augmentations, which serves as a stable domain prior and constrains the adaptation trajectory via an Energy-Guided Adaptive Replay strategy. Extensive experiments on the AdaptEval benchmark demonstrate strong performance across diverse domains, effectively preventing model collapse while maintaining high inference efficiency.

Abstract:
Social deduction games (SDGs) require agents to reason under partial observability by maintaining relational beliefs about hidden roles, team alignments, suspicion, and others’ perceptions. While recent LLM-agent approaches improve gameplay through prompting, reflection, and preference optimization, they often optimize actions and in-game speech without explicitly grounding them in such beliefs. This frequently leads to strategically inconsistent behavior, especially for compact LLM agents. We introduce Multi-Agent Relational Belief Optimization (MARBO), a belief-grounded preference optimization framework that leverages relational beliefs to guide strategic decisions and in-game speech. MARBO selectively provides preference feedback only when behaviors are supported by reliable relational beliefs and lead to strategically favorable social outcomes, encouraging more consistent learning under uncertainty. Experiments on representative SDGs, including Werewolf and Among Us, show that MARBO enables compact LLM agents to consistently outperform existing baselines.

Abstract:
While recent studies have explored human behavior and preference simulation using large language models (LLMs), it remains unclear how well LLMs can simulate subjective evaluations from real learners in educational settings. We investigate this question using real learner evaluation data on feedback for high-school biology questions at both the group and individual levels. We compare performance with and without learner-specific information, such as personality traits and evaluation examples, across six models. Our results show that LLMs still have a limited ability to simulate learner evaluations. Providing learner profiles and examples improves score calibration and individual-level simulation, but more often fails to improve group-level consistency. These findings highlight the need to investigate which learner information and adaptation strategies are effective for learner preference simulation.

Abstract:
Multi-hop question answering (MHQA) requires models to retrieve and reason over evidence distributed across multiple documents. However, training dense retrievers for MHQA remains challenging because labeled query-document pairs are difficult to obtain for the evolving intermediate queries in multi-hop reasoning. Recent approaches use LLM likelihoods for step-wise supervision over queries conditioned on accumulated evidence, which anchors retrieval to known facts and biases supervision toward answer-bearing documents, leaving bridging evidence undiscovered. To address this, we propose PROBE, a framework that optimizes dense retrievers for MHQA without requiring manually annotated query-document pairs. It decouples the reasoning state into a factual summary and a prospective query that anticipates missing information. For training, we transfer cross-encoder signals only at the final hop, providing trajectory-level supervision without penalizing early exploration. Experiments on three MHQA benchmarks show that PROBE consistently improves both retrieval and QA performance over strong iterative baselines, achieving an average gain of 5.7 points while using only about 5% of the training data of the prior method.

Abstract:
Time-Series Question Answering (TSQA) aims to interpret time-series information and generate answers according to the intent of natural language questions. However, in TSQA, the required temporal granularity and reasoning scope vary across queries, making it crucial to effectively organize the time-series information needed for each query. In this study, we propose TiMER (Time-series and language alignment with Multi-granularity Expert Routing), a time-series-language alignment framework that selectively constructs multi-granularity time-series information according to query intent and connects it to the LLM input space. Experimental results on a large-scale TSQA benchmark show that TiMER achieves an overall accuracy of 81.02% using only a small number of external alignment modules while keeping the LLM frozen, outperforming various LLM-based baselines. In addition, extensive ablation studies verify the effectiveness of each component. This study presents an effective direction for natural language-based time-series understanding and temporal reasoning by dynamically organizing and utilizing time-series information according to query intent. The code is publicly available at https://github.com/hyuna0926/TiMER_paper.

Abstract:
Structured pruning reduces the inference cost of large language models (LLMs) by permanently removing neurons and weight rows, yielding dense, hardware-agnostic speedup. Existing post-training methods couple neuron selection and weight reconstruction through a shared calibration dataset, introducing domain bias and out-of-memory failures at high sparsity. We propose SARATHI, a framework that decouples these two stages. A data-free NMF Residual Subspace Probe scores neurons using intrinsic weight geometry, without accessing any calibration data, and carries a formal connection to Robust PCA. Adaptive OBS then reconstructs the pruned weights with a localized calibration batch, operating in 𝒪(d 2 ) peak memory per layer and eliminating the global memory bottleneck of coupled methods. A Bias Shift Compensation step corrects systematic activation drift in post-LayerNorm architectures that would otherwise cause perplexity collapse at high sparsity. Experiments across six models from two architecture families (gated and non-gated FFN) at 15–40% sparsity show that SARATHI achieves competitive perplexity and matches or exceeds existing structured pruning baselines on zero-shot accuracy. Code: https://github.com/snkrbhr/SARATHI

Abstract:
Psychiatric consultation requires agents to elicit discriminative evidence, map uncertain narratives to diagnostic criteria, and decide when evidence suffices. Existing dialogue and retrieval-augmented systems condition policies on raw histories or appended passages, leaving observed evidence, missing checks, differentials, and support reliability entangled. We introduce MIND, a criteria-grounded evidence-state decision interface. At each turn, MIND constructs a typed state containing observed evidence, unresolved criterion checks, active differentials, criterion-linked supports, and reliability metadata. A training-split Psychiatric Reasoning Bank supplies gated supports, turning retrieval into state construction rather than prompt injection. The same state conditions action selection, process rewards, information-gain scoring, and trajectory rectification. Under EMR-grounded simulator protocols, MIND improves accuracy by 8.8 and 7.3 points over strong inference-only, RAG, and RL baselines, with gains transferring to public MDD-5k dialogues. Matched interventions attribute these gains to the shared state rather than prompt length, retrieval text, or formatting. MIND targets screening-level decision support, not autonomous diagnosis. Code is available at https://github.com/Lingxi-mental-health/MIND.

Abstract:
Large language models (LLMs) provide a powerful foundation for social media simulation, but their utility is fundamentally limited by the accuracy of user behavior modeling. Existing LLM agents exhibit behavioral homogenization and fail to capture users’ inherent identities and evolving preferences. To address this, we propose CLASP, a closed-loop behavior alignment framework for social media agents. CLASP leverages explicit, dynamically refined personas to bridge users’ historical behaviors and subsequent agent actions, ensuring behavioral consistency and temporal adaptability. Our framework adopts a decoupled dual-model design to separate persona refinement from action generation. This design enables continual behavior alignment driven by action prediction discrepancies. To facilitate training and evaluation, we further construct BlueTrack, a large-scale Bluesky dataset that covers long-horizon and heterogeneous social behaviors with complete action elements. Extensive experiments demonstrate that CLASP outperforms baselines in behavioral alignment with real users. It improves the overall alignment quality by up to 31.8% and reduces its variance by 32.1% during long-sequence simulation. This work provides a practical reference for further exploration in realistic social media simulation.

Abstract:
Large language model (LLM)-based agents increasingly face tasks that require reasoning over extremely long range. A key bottleneck is memory design: most prior approaches treat memory as a static, human-designed component, limiting adaptability. Inspired by biological memory evolution, we propose AtomMem, a framework that exposes a set of atomic memory operations—including summarization and CRUD primitives—as tool-like actions. The agent learns a policy over these operations purely via reinforcement learning, using only the task reward. AtomMem allows the agent to dynamically interleave summarization and CRUD to retain important information while managing memory efficiently. We evaluate AtomMem on streaming long-context understanding and web-navigation benchmarks, demonstrating an average improvement of 8.5 points over static memory workflows and 2.1 points over the current learnable-memory baseline. Ablation studies show that both summarization and CRUD are essential to train agent memory. The project can be accessed at https://github.com/RUCBM/AtomMem.

Abstract:
Despite the strong reasoning capabilities of recent large language models (LLMs), achieving reliable performance on challenging tasks often requires post-training or computationally expensive sampling strategies, limiting their practical efficiency. In this work, we first show that a small subset of neurons in LLMs exhibits strong predictive correlations with reasoning correctness. Based on this observation, we propose AdaRAS (Adaptive Reasoning Activation Steering), a lightweight test-time framework that improves reasoning reliability by selectively intervening on neuron activations. AdaRAS identifies Reasoning-Critical Neurons (RCNs) via a polarity-aware mean-difference criterion and adaptively steers their activations during inference, enhancing incorrect reasoning traces while avoiding degradation on already-correct cases. Experiments on 10 mathematics and coding benchmarks demonstrate consistent improvements, including over 13% gains on AIME-24 and AIME-25. Moreover, AdaRAS exhibits strong transferability across datasets and scalability to stronger models, outperforming post-training methods without additional training or sampling cost.

Abstract:
Retraining-free MoE compression reduces deployment memory by pruning or merging experts, but often treats the compressed checkpoint as the final artifact. We argue that this view is incomplete: compressed MoE checkpoints are better understood as compressed initializations that benefit from a tiny post-compression adjustment stage. Across two MoE LLM backbones, four pruning/merging methods, three expert-retention ratios, and 28 benchmarks, we compare LM fine-tuning and teacher-based KD under matched small-data budgets and measured GPU costs. Using only 3,000 C4 examples and a single epoch of adjustment, Full FT recovers 37.3% of the original-to-compressed performance gap on average. Moreover, LM fine-tuning is more cost-effective than standard token-level KD, and full-parameter adjustment gives the strongest cost–recovery trade-off among the tested scopes. These results suggest that retraining-free compression should be paired with small post-compression adjustment to recover a substantial portion of the performance lost during compression.

Abstract:
Diffusion large language models (dLLMs) generate text through iterative denoising rather than left-to-right decoding. This generation paradigm introduces two axes that can influence safety alignment: when tokens are generated during denoising and where they appear in the response. In this paper, we measure dLLM safety behavior under harmful prompts by tracing intermediate token distributions and commitment decisions throughout denoising. Our analysis shows that refusal signals are concentrated in early denoising steps and leading response positions, and the tokens committed early can strongly shape the final safety outcome. Our measurements further show that the denoising step and persistence of refusal-token commitment are important for understanding dLLM safety. Based on these findings, we propose Refusal-Aware Early Commitment (RAEC), a simple training-free decoding method that commits persistent refusal signals from early steps. Experiments on LLaDA and Dream show that RAEC reduces attack success rates while largely preserving utility. The code is available at https://github.com/Glresearch1/RAEC.

Abstract:
Cold-start item recommendation remains challenging as new items lack interaction signals beyond textual descriptions. Existing methods either fine-tune large language models (LLMs) on descriptions or map item content into collaborative spaces learned from user-warm item interactions. However, text-only prompts struggle to capture higher-order graph structures, while distributional differences between behavior and content embeddings weaken alignment based solely on predicted probabilities. To address these issues, we propose Graph Structure Serialization for Embedding Alignment (GraphSEA), an LLM-based framework inspired by the observation that graphs contain sequential structures similar to language and can be serialized as inputs for LLMs. GraphSEA first serializes interaction graphs into structured token sequences for an LLM to learn behavior embeddings that encode collaborative patterns, while the LLM separately generates content embeddings from item descriptions. An embedding alignment module then aligns behavior and content embeddings across multiple facets, including interaction probability, preference order, and item distinction. Extensive experiments demonstrate that GraphSEA improves recommendation performance across warm and cold recommendation tasks, while achieving higher efficiency than LLM-based methods.

Abstract:
Recent research has shown that filtering massive English web corpora into high-quality subsets significantly improves training efficiency. However, for high-resource non-English languages like German, French, or Japanese, aggressive filtering creates a strategic dilemma: should practitioners prioritize diversity by training once on large amounts of lightly filtered web data, or prioritize quality by strictly filtering for a high-quality core and repeating it over multiple epochs? We investigate this trade-off for German by applying hierarchical quality filters to 500M web documents, comparing multi-epoch training on filtered subsets against single-pass training on a diverse corpus. Our experiments across multiple model scales and token budgets show that repeating high-quality data consistently outperforms single-pass training on larger, less filtered sets. Notably, the performance gap persists even after 7 epochs. Our findings suggest that for non-English LLMs, semantic concentration through quality filtering offers a more viable path to efficient language modeling than simply maximizing unique data volume. We release our German language models (called Boldt), as well as our cleaned evaluation benchmarks to the research community. Our experiments indicate that they achieve state-of-the-art results despite training on 10-360x fewer tokens than comparable models.

Abstract:
The rapid evolution of Large Language Models (LLMs) has fostered diverse paradigms for automated slide generation, ranging from code-driven layouts to image-centric synthesis. However, evaluating these heterogeneous systems remains challenging, as existing protocols often struggle to provide comparable scores across architectures or rely on uncalibrated judgments. In this paper, we introduce SlidesGen-Bench, a benchmark designed to evaluate slide generation through a lens of three core principles: universality, quantification, and reliability. First, to establish a consistent evaluation framework, we ground our primary analysis in the visual domain, treating terminal outputs as visual renderings for consistent comparison, while acknowledging format-conditional ceilings in structural editability. Second, we propose a structured computational approach that quantitatively assesses slides across three distinct dimensions—Content, Aesthetics, and Editability. Finally, we ensure high correlation with human preference for the highly subjective aesthetic dimensions by constructing the Slides-Align1.5k dataset. Our experiments demonstrate that SlidesGen-Bench achieves a significantly higher degree of alignment with human judgment than existing evaluation pipelines. Our code and data are released to support further research.

Abstract:
The GSM-Symbolic benchmark (Mirzadeh et al., 2025) reported consistent performance drops across 25 Large Language Models (LLMs) when tested on template-generated variants of GSM8K problems, concluding that the models lack genuine reasoning capabilities. We argue that this conclusion rests on shaky statistical ground. Re-evaluating 20 open-weight models using bootstrapped Generalised Linear Mixed Models with per-question random effects, we find that only 8 exhibit statistically significant performance changes under the original prompt format. Moreover, we identify a previously unacknowledged factor: the distribution of integers in problem texts of the main GSM-Symbolic dataset is systematically shifted towards larger values relative to the original GSM8K (K-S statistic = 0.12, p < 0.001), contradicting the original authors’ claims. Controlling for this large-number effect accounts for significance in half of the remaining cases. Among models with statistically significant performance deltas, we identify distinct, model-specific behavioural failure profiles — including fragility of variable binding, arithmetic limitations, and dual-task interference — underscoring that blanket claims about LLM reasoning risk being both statistically premature and mechanistically misleading.

Abstract:
Effective detection of hate content is essential for maintaining healthy online communities, yet performance can vary substantially across languages. This work investigates a core practical question: Can training data from one language support hate-speech detection in another language with comparable performance? To answer this question, we conduct a large-scale study across 14 languages. Our analysis shows that transfer effectiveness could depend on several factors—including, but not limited to, dataset size, resources and temporal overlap. At the same time, we observe consistent patterns in which certain language pairs exhibit stronger transfer performance. Upon further analysis, we find that these patterns can be attributed to cultural cues or shared societal characteristics. Our analysis includes low-resource languages, offering practical insights into when cross-lingual transfer can support them and where its limitations emerge. These findings provide guidance for designing more reliable and generalizable hate detection systems. Warning: This paper includes examples of hate speech for research purposes.

Abstract:
Preserving safety alignment during large language models fine-tuning is critical, however, recent studies have demonstrated that even benign fine-tuning data may contain safety-degrading samples that silently undermine such alignment. Existing approaches typically identify safety-degrading samples using representations from a single safety-sensitive layer. However, whether a single safety-sensitive layer is sufficient for identifying such samples in multilingual settings remains unclear, where representation patterns may vary across languages. Our cross-lingual analysis shows that sensitive layers are only partially shared across languages and that safety-relevant signals often span multiple layers. Motivated by these observations, we propose MMSafe , a multi-layer framework for multilingual safety-degrading data identification that jointly leverages shared and language-specific sensitive layers. Extensive experiments across multiple models, languages, and safety benchmarks demonstrate that MMSafe reduces the average harmful-response ratio by 60% compared with random filtering and outperforms the strongest single-layer baseline, demonstrating the effectiveness of multi-layer modeling for robust multilingual safety alignment. Code is available at https://anonymous.4open.science/r/MMSAFE .

Abstract:
Large language models (LLMs) alignment ensures model behaviors reflect human value. Existing alignment strategies primarily follow two paths: one assumes a universal value set for a unified goal (i.e., one-size-fits-all), while the other treats every individual as unique to customize models (i.e., individual-level). However, assuming a monolithic value space marginalizes minority norms, while tailoring individual models is prohibitively expensive. Recognizing that human society is organized into social clusters with high intra-group value alignment, we propose community-level alignment as a "middle ground". Practically, we introduce CommunityBench, the first large-scale benchmark for community-level alignment evaluation, featuring four tasks grounded in Common Identity and Common Bond theory. With CommunityBench, we conduct a comprehensive evaluation of various foundation models on CommunityBench, revealing that current LLMs exhibit limited capacity to model community-specific preferences. Furthermore, we investigate the potential of community-level alignment in facilitating individual modeling, providing a promising direction for scalable and pluralistic alignment.

Abstract:
A long-lived LLM agent, such as OpenClaw, earns its value by acting on a user’s preferences and constraints across sessions, not just the current request. Yet today’s agents keep what a user volunteers but rarely ask for what stays unspoken, leaving a proactivity gap in long-lived LLM agents : an agent cannot act on a preference it never obtained. As users delegate more of their affairs to agents, the impact of this gap grows. We isolate one concrete, controllable slice of this gap as Ask-to-Remember (ATR) : the agent decides whether to ask now for a reusable user preference that the current task does not need but a later session with the same user will. ATR is hard even to evaluate: the right question is underdetermined and its payoff deferred to tasks that may never arise. ATRBench , to the best of our knowledge the first ATR benchmark, makes it measurable by fixing each user’s preferences as hidden ground truth, so success demands asking, not recall. Across eight frontier LLM agents, defaults fall at least 62 points below an oracle handed the relevant preference, and prompting closes little of it. Diagnostics identify acquisition as the bottleneck. ATRBench surfaces this proactivity gap in current agents and offers a diagnostic testbed for closing it.

Abstract:
A calibrated Vision-Language Model (VLM) can repeatedly self-correct, say "Wait, I should recheck," arrive at the wrong answer, and still report high confidence. We find that this occurs because verbalized confidence is largely trajectory-independent in the VLMs and calibration methods we evaluate. We examine this through three complementary lenses: content variation, token masking, and the model’s own hesitation markers. We show that confidence is insufficiently sensitive to what the reasoning trajectory actually contains, and that calibration training can paradoxically worsen this disconnect. Since existing metrics like ECE and AUROC cannot detect this problem, we propose the Trajectory-Grounding Score (TGS) in two complementary forms: TGS-self, which compares confidence with and without access to the model’s own trajectory, and TGS-pair, which tests whether the model assigns higher confidence to correct trajectories than to flawed ones along the vision, reasoning, and answer axes. We propose TGS-Bench, a model-agnostic suite spanning 10 benchmarks with controlled good/bad trajectory pairs, and show that conventional calibration rankings diverge from trajectory-grounding rankings, exposing a blind spot in current evaluation practice.

Abstract:
Proactive dialogue agents must guide multi-turn interactions toward task goals while adapting to users whose intentions and attitudes evolve. Existing planners improve system side action selection through search, learned policies, or user aware planning, but are often trained and evaluated against fixed or overly compliant simulators; as a result, high simulated success can reflect soft user bias rather than robust dialogue ability. We introduce MASP , a Mentalized Adversarial Self Play framework that treats proactive dialogue as a partially observable game between a system and a rationally resisting user. MASP represents the user’s evolving mental state with a BDI based latent state, separates reward side teacher mentalization from policy side student mentalization, and alternates self play updates for the system and user policies. This turns robustness to rational resistance into a training objective rather than only an evaluation condition. Across diverse proactive dialogue benchmarks, compared with the widely used SFT and Planner-only approaches, MASP substantially improves success rate (SR) and rationality constrained success (RCSR), reduces average turns (AT), and performs better against hard users who make the task goal more difficult to achieve.

Abstract:
Group-Relative Policy Optimization (GRPO) is widely used to post-train LLM agents, but its trajectory-level advantage is ill-suited to long-horizon credit assignment. Reward-homogeneous rollout groups yield zero advantage; otherwise, one scalar uniformly scales every token, obscuring which decisions determine the outcome. We propose TRACE-GRPO (Teacher-Refined Advantage Credit Estimation for GRPO), which converts teacher-LLM feedback into hierarchical credit. Teacher scores provide a sample-level fallback for zero-variance groups, reweight trajectory advantage at the turn level, and redistribute turn credit at the token level using the policy’s log-probability shift under a teacher critique. This yields dense training-time credit without token-level annotations or learned critics. We also introduce CAPA (Critique-Aware Paged Attention), which reuses trajectory KV caches and reduces the isolated critique-conditioned forward from roughly 2× a vanilla GRPO step to approximately 5% of a training step. On ALFWorld and ScienceWorld with Qwen2.5-7B-Instruct, TRACE-GRPO achieves 96.3% and 73.4% success, respectively, outperforming strong agent RL baselines. Ablations show cumulative gains from the three credit levels, and comparison with supervised fine-tuning on teacher-generated successes confirms that critique-guided policy optimization provides benefits beyond imitation. Code is available at [https://github.com/Lau-Jonathan/TRACE-GRPO](https://github.com/Lau-Jonathan/TRACE-GRPO).

Abstract:
Reward models (RMs) are critical for aligning Large Language Models via Reinforcement Learning from Human Feedback (RLHF). While Generative Reward Models (GRMs) achieve superior accuracy through chain-of-thought (CoT) reasoning, they incur substantial computational costs. Conversely, Scalar Reward Models (SRMs) offer efficiency but suffer from limited performance and adaptability in complex scenarios. We introduce Fast-Slow Thinking Reward Models (F/S-RM), a hybrid RM architecture inspired by Dual Process Theory. It trains a single model to integrate two distinct reward paradigms: scalar-style first-token pairwise judgment (fast thinking) and CoT-based judgment (slow thinking), regulated by a dual-confidence activation mechanism that determines when to activate slow thinking. Under hybrid inference, F/S-RM achieves state-of-the-art accuracy with an average score of 84.3 across benchmarks, while reducing token consumption by 22.5%. Code and data are publicly available.

Abstract:
Current reinforcement learning from human feedback (RLHF) methods primarily rely on scalar rewards from a trained reward model (RM). While effective, scalar rewards are often noisy and fail to capture fine-grained preference differences, whereas RM hidden states encode richer semantic and preference information. We introduce the representation-aware advantage estimation, which leverages RM hidden states and models them as auxiliary signals for better advantage estimation. Specifically, we propose the Graph-based Advantage Estimation (GraphAE), treat each sampled group as a graph, where nodes correspond to responses and edges capture their similarity in the RM hidden space. Then advantages are computed via graph propagation, enabling each sample to incorporate contextual information from its neighbors. GraphAE is lightweight and can be seamlessly integrated into existing group-based RL algorithms. We apply GraphAE to GRPO, GSPO and RLOO, and conduct extensive experiments on different models and benchmarks. Empirical results show consistent improvements across three benchmarks, with gains of up to + 6.3 on Arena-Hard-v0.1, + 8.27 on AlpacaEval 2.0, and + 0.22 on MT-Bench. These results demonstrate that leveraging RM representations leads to more sample efficient and robust RLHF.

Abstract:
Large Language Models (LLMs) often struggle with rigorous logical reasoning, suffering from hallucinations and inconsistent deduction. Chain-of-Thought (CoT) prompting relies heavily on the frozen model’s inherent capabilities, while neuro-symbolic approaches face a "translation reliability" bottleneck in which semantic parsing errors mislead external solvers. Recent work attempts to infuse symbolic capabilities via fine-tuning, but typically mimics forward reasoning, suffering from search-space explosion and irrelevant intermediate facts that dilute attention. We propose SOLID (Symbolic Oriented Logic Infusion via Dual-verification), a framework that synthesizes high-quality supervision through a Dual-Verification Mechanism: bidirectional consistency checks filter out hallucinated translations. Crucially, instead of forward chaining, we execute validated translations to generate backward reasoning traces—verifying necessary premises from the goal. By fine-tuning on these focused, goal-oriented paths, SOLID infuses the model with the efficient deduction process of a symbolic engine. Experiments on ProntoQA, LogicNLI, and ProofWriter demonstrate state-of-the-art accuracy, significantly outperforming CoT baselines, solver-based methods, and forward-chaining variants. Code and data are available at: https://github.com/ejie22/SOLID.

Abstract:
Multi-vector visual document retrieval models encode pages into hundreds of patch-level embeddings, enabling fine-grained interaction but incurring substantial storage overhead. Existing train-free compression approaches rely on token pruning or heuristic merging, which often fail to preserve the broad semantic coverage of the original representations under constrained budgets. To address this, we propose CovMerge, a geometry-aware, train-free index compression framework that formulates token reduction as a budgeted semantic coverage problem on the unit hypersphere. Given frozen patch embeddings, CovMerge applies Farthest Point Sampling (FPS) for coverage-aware prototype initialization, followed by spherical K-Means refinement aligned with MaxSim retrieval. The primary variant, Global Coverage, operates directly on all page patches without external dependencies, providing the strongest simplicity–efficiency trade-off. An optional Regional Coverage extension leverages document layout structure to guide prototype allocation, offering additional gains on layout-sensitive documents when reliable detection is available. Operating purely as post-processing without retraining, CovMerge supports flexible deployment budgets. On the ViDoRe V1 benchmark, CovMerge retains up to 98.1% and 98.6% of the original nDCG@5 for ColPali and ColQwen2, respectively, while using only 10% of the original storage, establishing a new state-of-the-art for storage-efficient multimodal retrieval.

Abstract:
Chain-of-Thought (CoT) distillation transfers multi-step reasoning from large reasoning models to smaller students, but verbose teacher traces inflate both training and inference cost. Existing CoT compression methods fall into two families, selective pruning and generative rewriting, yet prior studies have left key factors entangled: granularity is confounded with importance criteria in pruning, restructuring level is rarely isolated in rewriting, and compression budgets are not systematically evaluated across domains or regimes. We recast CoT compression along three dimensions: importance criterion, restructuring level, and compression budget. Sweeping these across two model families, Math and General domains, and Long-/Short-CoT regimes, we find that (i) importance criterion utility is strictly governed by granularity: step-level criteria converge on a shared reasoning backbone, while token-level pruning requires symbol-aware signals to preserve the logical core; (ii) restructuring level inverts across domains: Math degrades monotonically with structural disruption, while aggressive rewriting acts as a denoiser on General tasks; (iii) training-time compression does not necessarily translate to inference-time savings: Long-CoT students retain verbose habits despite concise supervision, making the training ratio an optimistic lower bound on deployment cost. These findings yield condition-aware guidelines for matching compression to deployment context.

Abstract:
Many continual learning methods for LLMs mitigate forgetting by expanding task-specific modules, yet this leads to memory costs that grow with the number of tasks. We reveal that such expansion may be unnecessary: a single LoRA adapter possesses substantial per-task parameter redundancy, where retaining only 3–10% of LoRA-B entries suffices to recover over 95% of task accuracy. Moreover, the per-parameter update magnitudes follow a heavy-tailed distribution whose geometric knee point naturally separates critical parameters from dispensable ones. Building on this insight, we propose Continual Adaptation via Subspace Trimming (CAST), which adaptively allocates a compact task-specific subspace within a shared LoRA adapter using the knee point criterion, freezes previously assigned parameters to eliminate cross-task interference, and routes inputs to relevant subspaces via learned semantic vectors without requiring task identities. Experiments on LLaMA-3.1-8B and Qwen3-8B demonstrate that CAST matches per-task LoRA accuracy across 15 sequential tasks while maintaining a fixed adapter budget with over 24% idle capacity preserved.

Abstract:
Language models act through tools, yet practical agents face libraries containing thousands of interfaces. We introduce the tool menu as the short, ordered subset of available tools shown to an agent before execution. The agent can call only tools in this menu. Multi-step tasks require the final action and the prerequisite tools that create its inputs in a usable order. Current constructors rank tools by request relevance, which can surface the final action while omitting or delaying less obvious producers. We introduce the state path, a pre-execution route from the observable request state to the desired outcome, and propose State-Path Tool Menu to learn it. Our framework treats the menu as an execution prior over these routes. Its encoder represents which tools can run from the current state, how their outputs satisfy later inputs, and which orders recur in training paths. A retriever covers an executable entry, the missing-input producers, and the final action. A reranker then places producers before consumers. On ToolBench, our menu raises online success from 0.737 to 0.898 and outperforms retrieval, reranking, generation, and routing baselines without changing the agent. The State-Path menu also covers more complete chains with 32 tools than the official list covers with 128, and its success gain persists across executor families with different model capacities.

Abstract:
3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene, leading to ambiguous predictions and high computational cost, especially in cluttered environments. We observe that many referential expressions rely on local spatial context and often correspond to restricted spatial regions rather than the full scene. Motivated by this insight, we propose PruneGround, an effective plug-and-play framework for 3DVG built upon three key components. First, we introduce LanguageGuided Spatial Pruning (LGSP), which leverages a frozen Vision Language Model (VLM) to identify language-relevant regions, thereby reducing spatial computation and grounding candidates in the narrower search space. Second, we propose MultiView-Conditioned Description Reformulation (MCDR), which decomposes complex expressions into simplified target–anchor relations and augments missing spatial cues through multi-view reasoning. Finally, we propose LLM-Grounder, which repurposes a detection-pretrained spatial LLM into a language-conditioned grounding model by aligning point cloud and linguistic representations within the pruned region. Extensive experiments on the three most popular point cloud benchmarks demonstrate that our method achieves state-of-the-art results on all three ScanRefer settings and on 9 out of 10 Nr3D/Sr3D settings.

Abstract:
Document retrieval increasingly supports high-stakes information access in finance, healthcare, and law. Modern retrieval pipelines vary both in modality (text or multimodal) and in retrieval architecture (dense or late-interaction). These choices impose a hard compromise: the most effective pipelines are too slow and expensive to run at scale, while the fastest fail to retrieve evidence from complex documents. Practitioners must therefore choose between missed evidence and unusable latency, with no principled basis for adapting that choice at the query level. We show that this compromise is unnecessary. Not every query requires the same pipeline. Across benchmarks spanning financial and scientific corpora, no static pipeline dominates. We introduce RetrievalRouter, a lightweight query-aware router that learns, from the query text alone, which retrieval pipeline best fits each query. A single tunable parameter exposes the full accuracy–latency frontier, and for every static baseline, RetrievalRouter offers an operating point that is simultaneously more accurate and faster. Against the best static baseline, RetrievalRouter is 2.5% more accurate and 12.4× faster. Furthermore, compared with prior adaptive strategy selection methods, RetrievalRouter achieves significantly higher nDCG@5 across accuracy-oriented settings, while matching or numerically outperforming them on both nDCG@5 and latency in latency-oriented settings. Our code: https://github.com/emrekuruu/retrieval-router

Abstract:
Emotion understanding in discourse requires reasoning beyond surface sentiment, since speakers often convey affect through indirect, implicit, polite, ironic, or deliberately mismatched expressions. Existing emotion benchmarks mainly annotate surface polarity or final emotion categories, while lacking a structured account of how explicit expression, implicit affect, pragmatic intent, and fine-grained emotion interact. This limitation makes current evaluations insensitive to cases where affective meaning is concealed, weakened, inverted, or pragmatically reshaped, thereby obscuring models’ failures in deeper emotion understanding. To address this gap, we introduce CUE-Bench, a Chinese Unsaid Emotion benchmark that centers on Affective Stance and covers diverse communicative scenarios. CUE-Bench constructs nine human-interpretable affective stances from Explicit-Implicit polarity interaction and further provides intent and fine-grained emotion annotations for structured affective inference. Experiments show that incorporating Affective Stance improves fine-grained emotion recognition by 3.1 percentage points and pragmatic intent detection by 8.1 percentage points over strong baselines. Project page, data, and code are available at [https://ai4ss.github.io/CUE-Bench/](https://ai4ss.github.io/CUE-Bench/).

Abstract:
Conversational emotion recognition suffers from two compounding limitations: models trained on imbalanced data report near-zero F1 on minority emotions, and evaluation has been almost exclusively English, leaving it unknown whether current methods generalize across languages. We propose UniMER, a parameter-efficient framework built on the insight that natural language is the universal interface: visual, acoustic, and facial cues are converted into text by a multimodal LLM, enabling a single LoRA-adapted backbone to process all modalities and languages through one shared representation. To prevent minority-class collapse under extreme imbalance, we introduce class-prototype attention pooling, which anchors each attention head to a learnable class prototype so every emotion retains dedicated representational capacity. With only 2.4% trainable parameters, UniMER achieves state-of-the-art results on three benchmarks across three language families: MELD (English, +14.0pp macro F1), M3ED (Chinese, +3.2pp wF1, text-only), and AkaCE (Akan, +11.5pp macro F1), lifting AkaCE Fear and Disgust F1 from near-zero ( < 0.1%) to 58% and 40%. All three benchmarks share the 7-class Ekman taxonomy, enabling direct cross-dataset transfer evaluation. Cross-lingual analysis reveals universal arousal-based confusions alongside culturally specific patterns: heightened neutral absorption in Chinese and prosodic confusion in Akan absent in English.

Abstract:
Gossip has been investigated primarily through vignette-based surveys, which provide limited insights into the dynamic and adaptive gossip behaviors within real-world contexts. To address these limitations, we introduce an LLM agent-based simulation framework, designed as a configurable experimental laboratory for gossip research. We validate our framework across three core dimensions of gossip dynamics: initiation, reaction, and perception. Our framework can reproduce several behavioral trends observed in human studies and enable analysis of agents’ reasoning and temporally unfolding interactions. We further augment the framework with a chain-of-reasoning mechanism to support exploratory investigation, demonstrating how explicit agent reasoning and experiment-level controllability enable the generation of hypotheses about gossip dynamics beyond fixed survey settings. Overall, our framework bridges LLM-based computational modeling and social science, complementing vignette-based methodologies and enabling controlled, systematic study of complex social behaviors. Our code is available at: https://github.com/TrustAIRLab/Gossip-Simulacra.

Abstract:
Faithful reasoning is essential in medicine, where clinical decisions require transparent justification grounded in reliable evidence. Current medical LLMs either lack active access to evidence or use retrieved evidence without supervising how it should be appraised and applied during reasoning. To address this, we formalize evidence-based medicine principles as process-level criteria and introduce FaithMed, a framework that combines clinician-designed, automatically refined rubrics with reinforcement learning using step-level process reward assignment and advantage grouping. Across seven medical benchmarks, FaithMed improves over agentic-search baselines (+9% on average) and outcome-only RL (+5.8%), while raising average evidence-based medicine rubric scores over agentic-search Qwen3 baselines (+15.5%). This work demonstrates that explicit step-level supervision can improve both task success and the faithfulness of the reasoning process. Code is available at https://github.com/cxcscmu/FaithMed.

Abstract:
Mobile GUI agents can execute tasks from natural-language instructions, but their evaluation remains difficult to make both realistic and reproducible. Existing benchmarks typically trade off these goals: simplified apps lack real-world mobile complexity, whereas live commercial apps introduce uncontrolled variation from recommendations, advertisements, accounts, and changing content. We propose AppSim-Bench, which addresses this trade-off through controllable simulated apps that preserve task-relevant interaction logic while supporting deterministic evaluation. Built through a coding-agent-assisted and human-verified workflow, it contains 557 tasks across 17 high-frequency Chinese and English apps. Its controllable backend data and outcome-based verification remove major sources of environmental stochasticity, enabling reproducible cross-model comparison. Evaluating 19 GUI agents, spanning general-purpose and GUI-specialized systems, we find that autonomous mobile execution remains far from solved. The best model completes only 50.27% of tasks, and 28.55% of tasks are not solved by any agent. Further analysis shows that failures concentrate in longer workflows, numerical reasoning tasks, and inefficient trajectories marked by high action overhead and budget exhaustion. Our project is available at https://github.com/Acrab-Lab/AppSim.

Abstract:
Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities, and rendering them accurately. Modern LLMs can produce visually plausible, instruction-compliant charts, yet data-level hallucinations remain difficult to detect in long, noisy, and multimodal contexts. To measure this gap, we introduce , an expert-annotated benchmark of 1,482 task-conditioned chart-generation instances drawn from real-world scientific papers, financial filings, and ecosystem reports. DEEPCHART formulates chart generation as an Extract–Reason–Visualize pipeline and evaluates source-data extraction, derived-data reasoning, and chart rendering stage by stage. Experiments with state-of-the-art models show that visually plausible charts often conceal data-level hallucinations, with extraction and reasoning errors common in realistic long and multimodal settings. These findings suggest that larger context windows alone are insufficient; faithful chart generation also requires reliable evidence extraction and quantitative reasoning before rendering.

Abstract:
Interactive agents increasingly fail on long task processes involving user turns, tool calls, and evolving environment contexts. Existing agent debugging methods provide useful trace-level diagnosis, but generally return explanations for individual failed traces rather than recurring, fine-grained action-context patterns across runs. To offer such corpus-level analyses, we present AgentTAC (Temporal Action-Context), an automated framework for mining erroneous and generalizable temporal sequences of action and context properties from agent traces. AgentTAC first localizes all distinct errors in each involved agent run. Then, it constructs and maintains a dynamic pool of attributes over the involved tasks, actions, and contexts to summarize their intrinsic properties and contextual relationships. Finally, it extracts temporally contiguous attribute patterns anchored at the localized errors. The resulting error patterns describe failures as interpretable sequential combinations of context and actions, such as making decisions under a missing-constraint context or taking an action after an incompatible prior decision. Extensive evaluations of AgentTAC on multiple benchmarks and agents validate its capabilities in error localization and error pattern mining, and provide preliminary evidence for error-pattern-driven test-time repair.

Abstract:
Large language models (LLMs) demonstrate impressive performance across a wide range of general NLP tasks; however, their effectiveness in sensitive domains, such as hate speech detection, remains less clear. Prior studies comparing prompted LLMs with state-of-the-art encoder-based models (e.g., BERT variants (Roy et al., 2023; Dönmez et al., 2024)) have shown only marginal gains, suggesting that LLMs may not excel in hate speech detection or mitigation. In this work, we revisit this question through the lens of instruction tuning. By thoroughly unifying 36 English hate speech datasets spanning multiple labeling schemes, we fine-tune a generalist LLM, based on Qwen3 (Qwen Team, 2025), specifically for hate speech mitigation. Our results demonstrate not only state-of-the-art performance on in-domain benchmarks but also substantial improvements in cross-domain and cross-lingual generalization—areas where encoder-based specialist classifiers often struggle.

Abstract:
Large language models have shown promise in assisting humans, but most real-world AI remains reactive, responding only to explicit user prompts. Current proactive AI agents often fail in multimodal reasoning over long-term memory, which requires agents performing temporal reasoning over long-horizon multimodal histories and continual knowledge update, and thus struggle to predict users’ needs and intent. To address this challenge, we present ProactiveEngine, the first foundation model designed for proactive multimodal agents, which generates proactive responses in long-horizon settings. ProactiveEngine supports both video plus text and text-only input. Our model achieves an F1-score of 83.3%, substantially outperforming current commercial models. Our model also achieves strong performance on the public Proactive Benchmarks in a zero-shot setting without further training, while enabling a 7.8× increase in token context window and over 100× more historical events for long-horizon proactive prediction. We also introduce NeuroAuroraBench, a multimodal benchmark designed to evaluate long-term memory proactive prediction. The benchmark resulted in 24.8k annotated sessions capturing dynamic changes.

Abstract:
Korean introduces an additional typographical perturbation level not captured by ordinary character-level edit models: because syllable blocks are internally composed of sub-character units called jamo, keyboard-level errors can occur within a syllable, either producing a valid but semantically altered character or exposing raw jamo on the surface. Both outcomes disrupt sub-word tokenization and are not reliably corrected by existing grammatical error correction pipelines, leaving LLMs directly exposed to corrupted inputs. To quantify this vulnerability, we apply five jamo-level perturbation types to the KMMLU benchmark and evaluate four language models, finding that accuracy declines monotonically with perturbation intensity and that parameter scaling does not confer robustness against intra-syllabic noise. We further show that typo-corrupted inputs induce a distinct shift in internal representations that is not reducible to ordinary answer incorrectness, and that a simple linear probe trained on these representations detects unseen perturbation types with high AUROC. Motivated by this signal, we propose Typo-Aware Chain-of-Thought (TACoT) , which routes inputs to chain-of-thought inference only when the probe detects a likely typo, recovering a substantial portion of the CoT accuracy gain at a fraction of the inference cost.

Abstract:
Vision-Language Models (VLMs) remain limited in high-precision numerical prediction for 3D scene understanding. In continuous numerical tasks, existing reinforcement learning methods such as GRPO are sample-inefficient: binary rewards fail to distinguish near-miss predictions from incorrect ones, while group-relative normalization can further weaken informative reward differences when sample groups are uniformly poor. Consequently, near-miss samples contribute little to policy updates despite carrying valuable learning signal. We propose Smooth Numerical Reward Activation (SNRA) and Absolute-Preserving GRPO (AP-GRPO) to address this issue. SNRA converts raw numerical errors into dense rewards via a dynamically parameterized sigmoid, while AP-GRPO modulates group-relative advantages with an absolute reward factor to emphasize informative near-miss samples and suppress uniformly uninformative groups. This improves the effective training contribution of near-miss predictions and substantially enhances data efficiency. On our Numerical3D-50k dataset, AP-GRPO achieves performance comparable to large-scale supervised learning across five benchmarks and four VLM architectures, without architectural modifications.

Abstract:
This paper studies the problem of misclassification detection for vision-language models (VLMs), which aims to identify samples wrongly assigned by the pre-trained VLM. Previous works usually focus on exploring the latent space and measuring pairwise similarity scores for confidence estimation. However, these similarity scores can be biased for boundary points with different fine-grained features. To address this issue, this paper proposes an approach named Distribution-aware Prototypical Calibration with Fine-grained Voting (DECENT). The core idea of DECENT is to utilize probabilistic distributions to characterize both the semantics of the test image and its corresponding class prototype. Specifically, we first conduct clustering within each class in the latent space to construct a distribution of fine grained prototypes. More importantly, multiple augmented views of the test image are considered as samples from its underlying semantic distribution. Then, we measure the distribution distance between the image’s underlying semantic distribution and its neighboring fine grained prototype distributions, which is aggregated via fine-grained voting into a distributional score. This score can be fused with the VLM’s semantic alignment score to significantly enhance misclassification detection sensitivity, or to improve overall classification robustness. Extensive experiments on benchmark datasets validate the dual effectiveness of the proposed DECENT.

Abstract:
Negation remains a longstanding challenge for both language models (LMs) and large language models (LLMs). Prior work mainly focuses on a small set of high-frequency single-word negation cues, such as "not" and "never", with limited exploration of broader negation types and modern LLMs. To address this gap, we construct NegCue, a large-scale dataset containing over 1.8M samples spanning single-word, multi-word, and affixal negation with more than 200 unique cues. We further pre-train both encoder-only LMs and LLMs on NegCue to investigate how different negation types affect negation understanding. Experiments on five downstream benchmarks show that negation types contribute unevenly to performance gains under the same training scale. In particular, affixal negation yields the largest improvements, while the gains from the commonly studied single-word negation are remain modest. Moreover, our results demonstrate that further pre-training improves negation understanding for both LMs and LLMs.

Abstract:
Large language models exhibit value-related preferences in their outputs, yet steering such behavior often requires large training datasets or paired preference annotations. In this work, we investigate a lightweight alternative: can fine-tuning on value survey responses induce measurable shifts in value-conditioned model outputs and downstream behavior? We first estimate baseline value profiles for several open-source LLMs by asking them to rate value-related descriptions organized around the ten Schwartz values. We then fine-tune models on survey-style supervision that selectively changes the target value ratings while keeping non-target values close to their baseline responses. We evaluate the resulting shifts in two settings. First, we measure whether the intervention changes responses on held-out survey questions. Second, we test whether these survey-induced shifts transfer to out-of-domain behavioral proxies, including a contextual moral-judgment benchmark constructed from Reddit AITA posts and a text-based interactive decision-making benchmark. Our results show that survey-based supervision can change held-out survey responses and produce downstream shifts in model response distributions on proxy behavioral evaluations. However, this transfer is strongly model- and value-dependent, and is shaped by how value descriptions are framed in the supervision data. These findings suggest that value surveys can serve not only as diagnostic tools, but also as lightweight supervision for studying behavioral value conditioning in LLMs.

Abstract:
Mixture-of-Experts (MoE) models achieve strong performance through conditional computation, but their large parameter footprint poses deployment challenges. Prior MoE compression approaches catastrophically fail when evaluated on general-purpose benchmarks beyond commonsense reasoning. We trace this failure to the granularity of compression: important capabilities are distributed across experts but concentrated in FFN sparse intermediate dimensions. To identify these dimensions, we use Fisher importance which outperforms activation-, router-score-, and magnitude-based alternatives, and identifies tiny sets of task-critical dimensions: in Qwen1.5-MoE, removing as few as 12 of 2.03M routed-FFN intermediate dimensions collapses GSM8K accuracy while largely preserving factual-knowledge performance. Building on this, we propose Fisher-MoE, which operates within FFN to remove intermediate dimensions ranked by Fisher importance. At the same 50% MoE compression ratio, Fisher-MoE preserves model capability, while reducing weight memory by ~45% and improving inference throughput by 21%. These findings suggest intermediate dimension granularity is an effective unit for both compression and ranking where capability concentrates in MoE models. Our implementation is available in an anonymous repository.

Abstract:
Recent segmentation models couple large language models (LLMs) with mask decoders to ground complex language expressions into masks, yet their instructions remain target-referential: they describe, constrain, or imply the region to be segmented. However, in real-world embodied interaction, human instructions are often at the intent-level, which includes the desired outcome without naming the region that enables it. To bridge this gap, we introduce SegWorld, where the model reasons about the scene through a multi-level visual chain-of-thought (CoT) before committing to a mask. Before receiving any instructions, it proactively observes the scene, describing visible objects and inferring plausible events they may support. Given an instruction, it continues the chain: from the object relevant to the intent, through the action that satisfies it, to the physical interaction site, the object part that affords the action. We formalize SegWorld as probabilistic inference, in which proactive observation supplies a linguistic scene context that improves mask prediction when instructions are given at the level of intent. We construct an Intent2Part benchmark for evaluating affordance-bearing part segmentation from high-level goals. Experiments show SegWorld matches instruction-driven baselines on target-referential instructions and improves substantially on intent-level ones.

Abstract:
Modern voice-agent systems rely on streaming speech recognition models that operate under stringent latency constraints. This study shows that, due to the limited memory constraints of real-time processing, these systems are adversely impacted by conversational phenomena such as long silences and backchannels. While many agentic pipelines mitigate this by resetting state at each turn, this approach discards vital context and impairs performance at turn onsets. We propose two state-management strategies that preserve cross-utterance context to reduce onset errors. In experiments with two state-of-the-art streaming models on two spoken dialogue benchmarks, our best method yields an average of 15-21% relative WER reduction at utterance onsets.

Abstract:
Multimodal retrieval-augmented generation (RAG) is usually evaluated with clean evidence, although deployed systems may retrieve topically relevant but unreliable text or images. We introduce QIMG-7, a controlled post-retrieval stress test for multi-sentence factual QA comprising 110 questions from four datasets, seven visual-evidence pollution families, and 16 paired clean/polluted regimes per question (1,760 rows per method). Across four generator/gate stacks, Full-MM is fragile under polluted text: in the main gpt-4o-mini stack, support decreases from 0.908 with clean text to 0.490 with polluted text. We propose source-aware trust resolution (SATR), a training-free family that compares Parametric, Text-only, and Full-MM candidate answers and selectively chooses a branch or falls back. Field-Selector obtains the highest numerical balanced score, 0.816, improving over Full-MM by 11.7 points and the Cascaded Router by 2.2 points. TextTrust, an image-blind text-reliability baseline, reaches 0.800; Field-Selector’s 1.5-point balanced advantage over it is not statistically resolved (95% CI [-0.006, 0.036]). In this text-first setting, the results support selective trust rather than unconditional multimodal fusion. Artifacts are available at https://github.com/mbzuai-nlp/Trust_Before_Fusion.

Abstract:
Text generated by large language models (LLMs) has been shown to be stylometrically distinct from human-written text (CITATION). But LLMs are increasingly used not only to generate text but also to edit human writing, and it is unclear whether the two leave the same trace. We show that AI generation leaves a consistent ”stylometric footprint”: a small subset of features, primarily entropy and lexical diversity, consistently separates AI-generated text from human writing across 8 LLMs and 5 domains, while the remaining features depend heavily on the domain and generator. AI editing, however, does not reproduce the same footprint. Relative to their human-written sources, AI-edited texts show only a small increase in lexical diversity and a decrease in entropy, rather than the joint increase that characterizes AI generation. Lexical density, which contributes little to generation, instead becomes the dominant editing-associated signal. Stylometric features therefore separate AI-edited text from AI-generated text but are substantially less effective at separating it from human-written text. Our results suggest that ”AI text” is not a single phenomenon: generation and editing leave qualitatively different stylometric traces and should be studied separately.

Abstract:
Medical Vision-Language Models (VLMs) are vulnerable to misleading clinical prompts and may follow incorrect user assertions instead of image evidence, a behavior commonly referred to as sycophancy. Existing mitigation methods mainly rely on prompt-level constraints to encourage neutral responses. However, through fine-grained integrated gradient analysis, we observe a phenomenon, although prompting strategies can correct the final prediction, the internal multimodal representations remain biased toward text tokens, while visual tokens continue to receive limited attention in late fusion layers. To address this issue, we propose HD-AS (Head-Driven Activation Steering), a training-free inference-time intervention framework. HD-AS identifies sycophancy-related attention heads through activation drift analysis under different prompting conditions. During inference, a lightweight isotropic damping operation is applied through a forward pre-hook on the target projection layers, reducing the dominance of text features in the hidden space and restoring the contribution of visual representations. Experiments on two public medical Visual Question Answering (VQA) datasets show that HD-AS consistently reduces the flip rate while maintaining diagnostic accuracy.

Abstract:
Language models (LMs) are increasingly extended with new learnable vocabulary tokens for domain-specific tasks, such as Semantic-ID tokens in generative recommendation. The standard practice initializes these new tokens as the mean of existing vocabulary embeddings, then relies on supervised fine-tuning to learn their representations. We present a systematic analysis of this strategy: through spectral and geometric diagnostics, we show that mean initialization collapses all new tokens into a degenerate subspace, erasing inter-token distinctions that subsequent fine-tuning struggles to fully recover. These findings suggest that token initialization is a key bottleneck when extending LMs with new vocabularies. Motivated by this diagnosis, we propose the Grounded Token Initialization Hypothesis: linguistically grounding novel tokens in the pretrained embedding space before fine-tuning better enables the model to leverage its general-purpose knowledge for novel-token domains. We operationalize this hypothesis as GTI (Grounded Token Initialization), a lightweight grounding stage that, prior to fine-tuning, maps new tokens to distinct, semantically meaningful locations in the pretrained embedding space using only paired linguistic supervision. Despite its simplicity, GTI outperforms both mean initialization and existing auxiliary-task adaptation methods in the majority of evaluation settings across multiple generative recommendation benchmarks, including industry-scale and public datasets. Further analyses show that grounded embeddings produce richer inter-token structure that persists through fine-tuning, corroborating the hypothesis that initialization quality is a key bottleneck in vocabulary extension.

Abstract:
Large Language Models (LLMs) are increasingly deployed in real-world applications, yet remain vulnerable to harms ranging from adversarial jailbreaks to implicit hate. Specialized moderators and general-purpose LLMs are both used as safety layers, but there is little systematic guidance on which model to choose for which harm. We present the most comprehensive evaluation of LLM safety to date, testing 53 models on 11 datasets organized into four challenge categories, under both prompt-only (Q) and prompt-with-response (QA) settings. Large frontier models that lead on one category fall behind smaller specialized models on others, and real-world conversational safety remains unsolved across model families. These findings challenge the assumption that scale alone ensures safety and offer practitioners a structured basis for model selection.

Abstract:
Large language models are increasingly used to answer culturally grounded questions across languages, yet it remains unclear whether local cultural knowledge is better accessed through English or the local language. Existing evaluations face two key limitations: many rely on parallel template-based questions that may not reflect how cultural knowledge naturally appears, and raw accuracy conflates general language proficiency with language-conditioned knowledge access. We address these issues with a controlled framework built on real-world cultural questions collected from regional benchmarks and local sources. By crossing question type, culture-agnostic versus culture-specific, with query language, English versus local language, and estimating ability using a shared 1PL item response theory model, we separate proficiency from localized knowledge access. Across 13 locales and roughly 80 models, we find a consistent English advantage on culture-agnostic questions, indicating stronger English proficiency. However, after accounting for this proficiency gap, local languages show a positive knowledge-access advantage in nearly all locale–model settings. This advantage is often masked in raw accuracy but becomes more visible for frontier, regionally aligned, or language-adapted models. Our results suggest that weaker local-language performance does not necessarily imply weaker cultural knowledge; rather, local cultural knowledge may be more accessible through the local language but hidden by limited language proficiency.

Abstract:
LLM assistants frequently answer everyday queries that reveal nothing about the user’s demographic identity. We audit whether models nevertheless attribute demographic characteristics to the user, a behavior we call unsupported demographic commitment. Across 18 proprietary and open-weight LLMs evaluated on 212 filtered information-seeking queries, 11 of 18 models commit to a demographic label on more than 80% of queries under a forced-choice diagnostic. In the matched abstention-first setting, the eight tested models average 44% commitment, compared with 5% for human annotators. Conditional on commitment, race, SES, location, and education show the same directional skew across all six domains, while gender varies by domain. Demographic-coded patterns also appear in free-form answers when no identity inference is requested. We evaluate identity-abstention, a prompt-level intervention that reduces classifier-detected presupposition from 41% to 16%, with a measurable clarification cost on queries that can be answered without additional context.

Abstract:
Multilingual LLMs increasingly claim broad language coverage, yet their performance still lags far behind English in many languages. Because low resource data is scarce and noisy, improving these languages directly is difficult, making cross lingual transfer from English a natural alternative. Parallel corpora are central to such transfer, but it remains unclear which kinds of parallel data matter most. We propose ParaRater, a corpus selection method that identifies high value English text to translate into target languages, building parallel corpora that most efficiently improve target language performance. ParaRater uses meta learning to tie selection directly to downstream accuracy on native target language data, and applies a two stage filter to find samples that only help when both language versions are used in training, capturing truly impactful parallel data. Experiments across Chinese, German, French, Japanese, Thai, Arabic, Tagalog, and Russian, together with qualitative analyses, support these findings.

Abstract:
We introduce MiRAGE, an evaluation framework for retrieval-augmented generation (RAG) from multimodal sources. As audiovisual media becomes a more prevalent source of information online, RAG systems must integrate such media into generation. Yet, existing evaluation methods for RAG are largely text-centric and do not readily transfer to multimodal settings. MiRAGE is a claim-centric approach to multimodal RAG evaluation, consisting of InfoF1, which assesses factuality and information coverage, and CiteF1, which assesses citation support and completeness. We show that, when applied by humans, MiRAGE strongly aligns with extrinsic judgments of output quality. We additionally introduce an automatic implementation of MiRAGE and compare it to multimodal variants of three prominent text-centric RAG metrics—ALCE, ARGUE, and RAGAS—finding that MiRAGE outperforms all three on text while being the only one to generalize to multimodal sources. We release open-source implementations and outline evaluation methods for multimodal RAG.

Abstract:
Long-context inference in large language models (LLMs) is bottlenecked by the linear growth of the self-attention key-value (KV) cache. Top- k sparse attention alleviates this by loading only a small fraction of the KV cache, but accurately and cheaply estimating cache importance, for both training-free use and sparsity-aware training, remains challenging. This paper proposes UNIQUE, a universal top- k sparse attention framework that addresses both requirements and stays consistently effective across LLM modalities. UNIQUE operates at the granularity of KV pages and estimates per-page importance with a simple yet accurate score combining the mean of the page’s keys as a representative vector with their standard deviation as an offset term. To further close the train-inference gap, this paper introduces a soft-mask sparsity-aware training scheme that uses the top- k score boundary as a per-query threshold and a sigmoid soft mask around it, requiring neither auxiliary losses nor architectural changes. Experiments on text and speech LLMs show that UNIQUE preserves task performance on long-context benchmarks such as LongBench Pro and on long-form speech recognition, while delivering up to 11.4 × attention-kernel speedup over FlashInfer dense attention and at least 5.3 × end-to-end decoding speedup over a vLLM-based dense model.

Abstract:
The success of the transformer architecture is in large part due to its use of attention layers. An attention layer follows the standard neural network paradigm: it takes the residual stream as input and thereby produces context-dependent query, key, and value vectors. However, we find that model performance meaningfully improves when deeper layers learn a context-free value vector that preserves the original token information, without drawing on any context information from the residual stream. When the model has access to this context-free value vector, adding back the context-dependent component of the value vector provides no gain. Such context-free value vectors can be stored as sparse model parameters, thereby eliminating the need for recomputation or caching. Through systematic ablations on the key design choices for such context-free value vectors, we propose Bank of Values (BoV), a new way of computing value vectors in attention by learning a lookup table of token-specific value vectors for each of the last third of layers. BoV consistently improves upon the standard attention mechanism in both validation loss and downstream benchmark performance across two model sizes, and surpasses competing attention architectures while using a fraction of the compute and memory.

Abstract:
Sparse autoencoder (SAE) features are used to interpret and steer large language models, yet nobody has tested whether a feature’s causal role is stable across SAE families. Single-token features fire on one vocabulary item, so ground truth permits direct comparison. We analyze 3.9M features across six models and three SAE families and zero-ablate at full layer depth: they sit 4.7x tighter in decoder space and concentrate in early layers. Deleting one lowers the model’s logit for that token in 178 of 208 layer conditions, significant after multiple-comparison correction. And depth decides how the damage lands: early-layer deletions disrupt the layers that follow, late-layer deletions change the output directly. Cross-family causal differences exceed within-family scale effects: on the same base model, GemmaScope and BatchTopK features are causally anchored, LlamaScope features locally redundant. Under LlamaScope the token returns to within 2x its pre-ablation rank 96-98% of the time. Changing only the activation function reverses the sign of that difference, so the training recipe is the remaining candidate: cross-family claims are sensitive to training methodology, not just activation function or scale.

Abstract:
Data selection for language models is often framed as balancing example quality and diversity. We argue that both are consequences of a more fundamental principle: selected examples should preserve the downstream query space induced by instructions, task signals, or retrieval needs. We present Greedy Information Projection (GIP), a query-aligned method that requires only candidate embeddings and a score signal from LLM judgments, metadata, or intrinsic geometry. GIP greedily selects examples whose embedding span explains the largest residual component of task/query scores, yielding a fast matching-pursuit selector. A Gaussian projection view connects this update to maximizing mutual information, equivalently minimizing the residual volume of the query subspace left unexplained by selected data; this explains how quality and diversity emerge from one objective. Empirically, GIP matches or surpasses full-data fine-tuning using small instruction and reasoning subsets, while pretraining and RAG passage-selection experiments show that the same residual principle transfers beyond supervised fine-tuning.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved mathematical reasoning, yet progress on hard prompts often stalls: a prompt can remain unsolved for many training steps before abruptly transitioning to solved—a discrete “zero-to-one” jump whose underlying dynamics and steerability remain poorly understood. We investigate two questions: what drives this transition, and can it be steered? Through event-based analysis measuring the time-to-first-positive (TTFP), we find that negative rollouts preceding successful transitions consistently retain higher early-stage generation entropy, distinguishing near-solvable failures from dead ends. This signal motivates NAPO (Negative-Augmented Policy Optimization), a method integrated with Group Relative Policy Optimization (GRPO) that ranks negatives in unsolved groups via an early-branching driver and assigns graded advantages, injecting informative gradients precisely where standard group-relative updates vanish. Under fixed rollout budgets, NAPO accelerates early solution discovery and improves Pass@256 by 10.0 points on AIME24 and 6.6 points on AIME2025, demonstrating that zero-to-one transitions are diagnosable and steerable through driver-guided negative learning. Our code is available at https://github.com/jameszhu-pixel-time/Z2O.

Abstract:
Frontier reasoning systems generate chain-of-thought (CoT) traces meant to reveal how they reach answers, yet these traces systematically misrepresent the underlying computation. A tempting explanation is bandwidth: each token carries few bits while the residual stream encodes many, so sufficiently complex tasks would make faithful verbalization impossible in principle. We formalize this intuition and locate its limits. A capacity bound rules out faithful CoT whenever the information the model uses exceeds the trace’s token capacity; this condition requires the answer to be longer than the trace, so standard CoT settings do not reach it. A per-step information-loss conjecture is empirically falsified on six of seven benchmarks. A constructive result shows faithful traces remain achievable for polynomial-size circuit families, and an information-bottleneck experiment supports a faithfulness–accuracy tradeoff. Measuring critic-score gaps on seven benchmarks for three open-weight models, complemented by perturbation sensitivity, we find no benchmark provably in the above-capacity regime. The capacity bound is architecture-independent, but at evaluated trace lengths it does not bind: observed unfaithfulness is attributable to training rather than to the token interface, and is therefore potentially correctable.

Abstract:
Industrial code tasks, including chip design, GPU optimization, and firmware, often fail for reasons that are invisible in ordinary source code: timing constraints, backend logs, layout data, simulator behavior, and hardware-specific performance trade-offs. Current code models see many programs but few expert correction traces from these settings. We present InCoder-32B-Thinking, a 32B thinking code model trained with Error-driven Chain-of-Thought (ECoT) synthesis and an Industrial Code World Model (ICWM). ECoT converts several rounds of toolchain signals into reasoning traces by preserving the failed code, the observed failure, and the later fix. ICWM learns from execution traces in domains such as Verilog simulation and GPU profiling, then predicts feedback for the data engine, which can build more repair traces without routing every candidate through a real toolchain. The data keeps the uneven reasoning depth of industrial work: short traces for simple cases, long traces for hardware-level debugging. Across 23 tests (14 general and 9 industrial), InCoder-32B-Thinking reaches 81.3% on LiveCodeBench v5, 84.0% CAD-Coder compile success, and 38.0% on KernelBench L2, while remaining competitive on general coding tasks. Model weights and code are available at Hugging Face: https://huggingface.co/Multilingual-Multimodal-NLP/IndustrialCoder and GitHub: https://github.com/CSJianYang/Industrial-Coder.

Abstract:
While LLM-based Automatic Speech Recognition (ASR) achieves high accuracy, its speed is limited by sequential autoregressive decoding. Diffusion Language Models (DLMs) offer a parallel alternative, yet their decoding strategies remain under-explored in ASR contexts. This paper analyzes three decoding schemes for DLM-based ASR: fixed-number, static confidence threshold, and dynamic confidence threshold. We introduce a round-wise analysis of decoding progress using Negative Log-Likelihood-based uncertainty as a proxy for prediction reliability. Our results show that both threshold-based strategies provide a better accuracy–speed trade-off than fixed-number schemes. This behavior is associated with the more concentrated confidence distribution observed in the evaluated ASR settings: many tokens reach high confidence early, enabling multiple tokens to be committed in early decoding rounds while lower-confidence tokens are deferred to later rounds. The static-threshold strategy achieves accuracy close to autoregressive decoding at lower decoding cost.

Abstract:
Existing indirect prompt injection (IPI) attacks commonly face a stealth-effectiveness tradeoff: features that make payloads effective can also make them detectable because attack intent is localized in a single payload. LogicInception moves IPI to discourse-level inference: attack logic is decomposed into register-matched declarative fragments distributed across tool outputs and linked by fabricated causal dependencies that the agent reconstructs through reasoning. To our knowledge, it is the first IPI framework to demonstrate successful unauthorized tool execution with no directive instruction anywhere in the injected content. No fragment contains a command or exploits instruction-following; the attack emerges from the agent’s own inference. We observe an empirical capability-safety amplification pattern: discourse-level amplification correlates with task capability across the evaluated models. On 1,139 scenarios with 5 defenses, LogicInception achieves 42.9-48.8% ASR at 88% per-fragment stealth, while all baselines remain below 8% composite success. The three prompt-level defenses reduce baseline ASR by up to 72% but reduce LogicInception by less than 7%; we trace this asymmetry to three architectural factors that motivate discourse-level rather than content-level countermeasures. This defense asymmetry is larger for higher-utility models in our evaluation, suggesting a tension between reasoning capability and safety.

Abstract:
Machine unlearning for large language models (LLMs) often assumes that a pre-defined forget set matches what the model has memorized, but this frequently breaks in realistic privacy settings where the original training data is inaccessible. We term this gap forget-set misalignment and identify two cases. In Under Unlearning, the forget set omits memorized information and leakage persists. In Out-of-Knowledge Unlearning, the algorithm is driven to "forget" knowledge the model never learned, perturbing parameters and degrading utility. Using gradient-level analysis, we show these behaviors arise from misaligned unlearning targets rather than specific optimization choices. We then propose CONfession-to-Forget-Set (CONFS), a data-blind framework that constructs model-aligned forget sets by eliciting and formalizing the model’s memorized knowledge. Across synthetic, multimodal, and real-world benchmarks, CONFS approaches Gold-standard performance on several metrics and achieves a competitive forgetting-utility balance, while preserving utility better than other data-blind forget-set constructions.

Abstract:
Before any reviewer is assigned, a journal editor decides whether a manuscript merits external review. Yet this routine desk screening has no benchmark. We present PREREVIEWBENCH, the first controlled benchmark for journal desk screening. Because real desk decisions are confidential, we pair Web of Science papers published in 2025 with journals to construct proxy labels: true matches, within-field caliber surpluses and deficits, and cross-field scope mismatches. The 7,292 balanced paper–journal instances span four broad areas and 58 journals, supporting desk triage, branch-specific assessment (caliber fit or rejection cause), and transfer-journal recommendation. Evaluating four frontier LLM families, we find average triage accuracy of 0.965 on cross-field scope mismatches but only 0.572 on within-field caliber deficits, where blind expert-led review reproduces the triage labels perfectly. Transfer recommendation is weaker still, with the best Hit@1 at just 0.369. Models are also sensitive to input perturbations, and a cue suggesting that most papers merit review sharply shifts GPT-5.5’s decisions. These results point to a clear division of labor: LLMs can already carry the scope side of desk screening, while caliber judgment stays with editors. PREREVIEWBENCH makes that harder half measurable: whether an in-scope paper is strong enough for its journal.

Abstract:
Recent LLM-based GPU code generation methods have shown promising results on CUDA and Triton kernels, but they typically operate on pre-isolated operators. Such a pre-isolated setting falls short in real-world CPU-GPU heterogeneous optimization, where the target computation is embedded in a larger repository and must be discovered, contextualized, selectively offloaded, and integrated. Such optimization requires repository-level context—including data structure definitions, caller patterns, build configurations, and profiling signals—that is largely absent from operator-level tasks and benchmarks. In this work, we propose GLARE, a profiling-guided framework for repository-level CPU-GPU heterogeneous optimization. GLARE anchors repository context in dynamic profiling signals, builds a multi-granular code dependency graph, extracts hotspot-centered active subgraphs, and uses an agentic loop to decide CPU-GPU partitioning and refine heterogeneous patches with execution feedback. On a GPU-acceleration subset of GSO with 63 tasks, GLARE improves OPT@1 from 74.6% to 87.3% over the strongest baseline under GPT-5.2, and achieves an 11.6% average end-to-end speedup. Ablation studies confirm that profiling-guided context construction is the primary driver of these gains.

Abstract:
Users across enterprises increasingly rely on LLM agents to query their data through natural language. However, building reliable data agents remains difficult because real enterprise data is fragmented across many heterogeneous database systems, with duplicated and inconsistent data, and key information often buried in unstructured text—requiring agents to go beyond just writing SQL or data science scripts to answer questions. Existing benchmarks tackle only individual pieces of this end-to-end workflow—e.g., text-to-SQL over a single database—and are increasingly saturated and contaminated. We present a new benchmark for LLM agents, the Data Agent Benchmark (DAB), grounded in a study of enterprises building production data agents across six industries. DAB comprises 104 queries across 17 datasets and 4 database management systems. On DAB, the best state-of-the-art agent achieves only 57% pass@1. We analyze agent failure modes and distill takeaways for future data-agent development. Our benchmark and experiment code are published at [URL redacted for anonymous review].

Abstract:
Large language model agents rely on effective model context to obtain task-relevant information for decision-making. Many existing context engineering approaches primarily rely on the context generated from the past experience and retrieval mechanisms that reuse these context. However, retrieved context from past tasks must be adapted by the execution agent to fit new situations, placing additional reasoning burden on the underlying LLM. To address this limitation, we propose a generative context augmentation framework using Contrastive Learning of Experience via Agentic Reflection (CLEAR). CLEAR first employs a reflection agent to perform contrastive analysis over past execution trajectories and summarize useful context for each observed task. These summaries are then used as supervised fine-tuning data to train a context augmentation model (CAM). Then we further optimize CAM using reinforcement learning, where the reward signal is obtained by running the task execution agent. By learning to generate task-specific knowledge rather than retrieve knowledge from the past, CAM produces context that is better tailored to the current task. We conduct comprehensive evaluations on the AppWorld and WebShop benchmarks. Experimental results show that CLEAR consistently outperforms strong baselines. It improves task completion rate from 72.62% to 81.15% on AppWorld test set and averaged reward from 0.68 to 0.74 on a subset of WebShop, compared with baseline agent.

Abstract:
What happens inside a language model when alignment training conflicts with a cultural value it encodes? Across 16 MENA countries, 26 models, and 1.53M human survey responses, we find evidence that the dominant mechanism is suppression rather than erasure. When an aligned model refuses, its internal logit distribution often correlates more strongly with human survey responses than its freely generated answer. We call this the alignment veto. We distinguish suppression failures (accurate internal distributions blocked at output) from representational bias failures (the encoding itself diverges from human values), and show the two require different interventions. Generation-time interventions confirm the gate is causal, not only correlational: sweeping a refusal direction moves generated refusal from 0% to 100%, and ablating it releases the answer the model had already encoded. Native-language prompting degrades alignment quality overall and shifts refusal in framing-dependent directions. Sparse autoencoder analysis corroborated by comparisons across alignment stages identifies a candidate DPO-stage feature mediating suppression in Tulu-3-8B. The gate is real, its costs are unequal, and deciding what it should protect is not a technical question.

Abstract:
Text-to-SQL is a pivotal task that bridges natural language understanding and structured data access. Recent advances in large language models, through prompting, fine-tuning, and reinforcement learning, have greatly increased the likelihood of generating correct SQL queries. However, selecting the correct SQL from multiple candidates remains challenging. Existing approaches, such as self-consistency, Best-of-N decoding, and learned scalar ranking models, rely on shallow heuristics or lack explicit reasoning, making them prone to unstable judgments and unable to capture fine-grained semantic differences between similar queries. We propose JudgeSQL, a framework that combines reasoning-based SQL evaluation with a weighted consensus tournament. JudgeSQL develops a reasoning-based SQL judge model that distills reasoning traces with reinforcement learning guided by verifiable rewards, enabling accurate and interpretable judgments. Building on this, a weighted consensus tournament integrates explicit reasoning preferences with the generator’s execution-consistency confidence, improving both reliability and efficiency. Experiments on the BIRD benchmark demonstrate that JudgeSQL consistently outperforms prior methods, with strong generalization and robustness across model scales. The code is available at https://github.com/Jiayuan-Bai/JudgeSQL.

Abstract:
Between the first visible sign of danger and the moment an accident occurs, there is often a window where intervention remains possible. Video-capable multimodal large language models (MLLMs) could serve as always-on safety monitors that issue warnings during this window. Yet current benchmarks do not test this ability: they rely on static inputs, ignore timing precision, and omit false-positive measurement on safe scenes. We present PaSBench-Video, a 740-video benchmark with 481 risk videos and 259 no-risk videos across four domains: driving, healthcare, daily life, and industrial production. The benchmark is annotated with frame-level risk onset and accident boundaries. A model must observe the video causally and produce a warning that is both temporally calibrated and content-correct. Testing 13 MLLMs, we find that no model exceeds 20.0% on our strictest metric, and recall is tightly coupled with false-positive rate, with Pearson correlation 0.64: higher detection comes only at the cost of triggering warnings on the majority of safe clips. Performance splits sharply by domain: models achieve moderate recall at low false-positive rates in daily life, where risks are inherently anomalous, yet fire indiscriminately in driving, where routine and hazardous scenes look alike. These results indicate that current models rely on scene-level activity cues rather than reason about emerging harm.

Abstract:
Despite the success of Large Language Models (LLMs) in long-context tasks, their reliability in recognizing unanswerable questions remains a critical challenge. In information-seeking scenarios, models often suffer from overconfidence and hallucinations when evidence is missing or misleading. In this paper, we propose FactGuard, a systematic multi-task workflow for generating evidence-grounded answerable and unanswerable questions. We introduce FactGuard-Bench, a benchmark comprising 25,220 examples with context lengths up to 128K, specifically targeting Lack of Evidence and Misleading Evidence scenarios. Evaluations of eleven representative open-source and proprietary LLMs reveal a significant reliability gap: models struggle to reject unanswerable questions as context scales, with the leading model achieving only 71.62% accuracy. By training with FactGuard-Bench, we improve overall accuracy to 81.17% and significantly enhance models’ reasoned rejection capabilities. Our work provides a scalable path toward building more faithful and trustworthy long-context LLMs. Our source is publicly available at https://github.com/biaoliu-kiritsugu/FactGuard.

Abstract:
Agent benchmarks for measuring memory largely study textual cases, in which information is deliberately extracted from the environment, written down, and then later retrieved. In other words, they assess what agents elected to record, not what they happened to see. We introduce DMV-Bench (code: https://github.com/yyyujintang/DMV-Bench), the first interactive benchmark for visual memory in multimodal agents, to study this often-neglected property. DMV-Bench is built on (1) a controlled home-furnishing e-commerce environment, supported by a catalog of 1,000 product variants, and (2) a text-leakage contract which ensures that the primary discriminative signal of each task is solely in the pixels. In DMV-Bench, agents undergo chains of autonomous shopping sessions in which every visited product image carries a unique, pre-rendered incidental cue that the agent is later asked to recall. We show that conventional solutions struggle with this task. Inspired by dual-coding theory, we propose a memory architecture that uses parallel visual and verbal codes, which we call DualMem. On DMV-Bench, DualMem outperforms a caption-only baseline and three recent multimodal agent-memory systems across multi-session chain lengths on multiple models. These gains persist even adjusting for memory-bank size and encoding-position bias. Further experiments also reveal an asymmetric division of labor between the two codes; a weighted coding scheme is often strongest. We view this as a step towards memory systems that preserve a richer record of agents’ observations.

Abstract:
LLM-assisted decompilation improves readability and re-executability, but still underperforms on stripped C++ functions that use the Standard Template Library (STL). Compilation, optimization, and symbol stripping remove or obscure source-level semantics such as container types and library-call structure, while traditional decompiler output often fails to recover them. We present STILL, a structured semantic interface that predicts function-level STL container semantics from stripped control-flow graphs and renders them as compact hints for LLM refinement. On StlBench, STILL predicts common container-level STL semantics, with the strongest cross-dataset results for stable string and vector slices. On stripped HumanEval decompilation, these hints enable DeepSeek-chat refinement to reach 28.4% executability, compared with 17.4% for no-hint refinement and 8.9% for raw Ghidra decompilation; hint utility is downstream-backbone-dependent, with decompilation-specialized models requiring lightweight adaptation to benefit from the same interface.

Abstract:
Fine-grained spatio-temporal understanding is essential for video reasoning and embodied AI. Yet, while Multimodal Large Language Models (MLLMs) master static semantics, their grasp of temporal dynamics remains brittle. We present TimeBlind, a diagnostic benchmark for compositional spatio-temporal understanding. Inspired by cognitive science, TimeBlind categorizes fine-grained temporal understanding into three levels: recognizing atomic events, characterizing event properties, and reasoning about event interdependencies. Unlike benchmarks that conflate recognition with temporal reasoning, TimeBlind leverages a minimal-pairs paradigm: video pairs share identical static visual content but differ solely in temporal structure, utilizing complementary questions to neutralize language priors. Evaluating over 20 state-of-the-art MLLMs (e.g., GPT-5, Gemini 3 Pro) on 600 curated instances (2400 video-question pairs), reveals that the Instance Accuracy (correctly distinguishing both videos in a pair) of the best performing MLLM is only 48.2%, far below the human performance (98.2%). These results demonstrate that even frontier models exhibit significant deficiencies in temporal reasoning, positioning TimeBlind as a vital diagnostic tool for next-generation video understanding.

Abstract:
While prior work focuses on treating answer verification as a general reasoning or semantic matching problem, significant challenges remain in computation-oriented scientific domains. In this paper, we introduce CoSineVerifier, an agentic verifier that actively invokes external executors to ground its judgments in precise computations and symbolic reasoning. To train this accurate agentic verifier, we propose a novel verifier training data augmentation method to increase the correctness and efficiency of tool-invoked verifications. Extensive experiments across STEM, QA, and long-form reasoning tasks demonstrate CoSineVerifier’s robust generalization abilities, achieving state-of-the-art performance on VerifyBench-Hard and SCI-Bench. Furthermore, when employed as an RLVR reward model, CoSineVerifier consistently outperforms both rubric- and model-based verifiers on AIME’24, AIME’25 and GPQA-D, highlighting its potential to advance LLM reasoning and accelerate data processing

Abstract:
Rubric-based rewards are increasingly used for open-ended language model post-training, but criterion-level scores are commonly aggregated as independent utilities. This flat scalarization ignores semantic dependencies among criteria, allowing downstream credit or penalties to be counted even when the conditions that license them are absent. We call this structural failure False Credit Propagation (FCP). We propose GEAR (Graphical Event Aggregation for Rubric rewards), a dependency-aware aggregation framework for rubric rewards. GEAR represents rubric criteria as latent events in a typed dependency graph, uses monotone retention factors to infer dependency-adjusted criterion marginals, and computes a normalized expected-utility reward. This preserves compatibility with standard rubric-based reinforcement learning while adding only linear-time aggregation over criteria and edges. Experiments on HealthBench, WritingBench, and PLawBench with two policy backbones show that GEAR consistently improves over flat aggregation and hard gating, both in controlled aggregation-only settings and as a plug-in module for existing rubric-guided RL pipelines. FCP diagnostics further show that GEAR substantially reduces invalid downstream utility while avoiding the over-suppression of deterministic gating.

Abstract:
Recent work has shown that LLMs overrepresent dominant cultures, particularly Western ones, while marginalizing others. We investigate whether this affects models’ ability to generate culturally adapted responses by evaluating their use of local measurement units based on the user’s perceived cultural background. We introduce Cultural and Pragmatic Response Inference (CAPRI), a dataset of conversations with varying levels of cultural cues. Experiments with state-of-the-art LLMs show that models can infer cultural background and recall relevant conventions, but often fail to utilize the information to adapt their answers to the relevant cultural conventions, unless explicitly prompted to perform the tasks sequentially. We further evaluate adaptation to the interpretation of time and quantity expressions, two subjective language grounding dimensions that are affected by culture. We find that models increasingly adapt their answers as cultural cues accumulate, but their priors are not culture-neutral, sometimes aligning with the model’s country of origin. Overall, CAPRI provides a resource for future research aimed at narrowing the gap between cultural knowledge and culturally adaptive language generation.

Abstract:
When large language models are fine-tuned to generate persona- or tone-conditioned responses, their output diversity is severely limited—a failure we term Cross-Style Collapse. We trace this collapse to the cross-entropy objective, which under shared representations tends to suppress diverse continuations. We propose Semantic Flow Regularization (SFR), a lightweight auxiliary objective that supervises the backbone with continuous sentence-encoder embeddings of future segments via conditional flow matching. The stochastic flow source preserves multi-modality by construction; the flow-matching head is discarded at inference, adding zero deployment cost. On a large-scale industrial dialogue dataset (Qwen3-32B, 9 personas), SFR improves output diversity, style fidelity, and response quality over SFT. We further validate on the public LiveCodeBench-v5 (Qwen2.5-Coder-7B-Instruct), where SFR consistently improves pass@k, confirming generality beyond stylized dialogue. A controlled comparison on MBPP reveals Multi-Token Prediction to be a degenerate special case of SFR.

Abstract:
Long-horizon LLM agents often fail from procedural errors such as forgetting constraints, misordering tool calls, or stopping prematurely. Because these slips are unpredictable, the same agent can succeed on one run and fail on an identical task later. The resulting execution trajectories contain rich signals about which procedural choices lead to success or failure, yet existing approaches that extract reusable guidance from agent trajectories either introduce noise through imprecise distillation or depend on external retrievers for deployment. We propose SkillForge, a training-free framework that rethinks both the construction and deployment of procedural guidance: it pairs successful and failed attempts on the same task to identify critical procedural choices, consolidates and filters candidate guidance to remove low-quality entries, and loads relevant guidance on demand at inference time only when the agent’s current context signals its relevance. Across agent benchmarks including 𝜏 2 -Bench and Terminal-Bench, SkillForge improves repeated-run consistency, reducing the consistency gap by up to 37% and lowering total token usage by up to 23.6% compared to baselines, while moderately improving single-trial performance. These results suggest a practical path toward distilling high-quality procedural guidance from past trajectories for more consistent and efficient long-horizon agents.

Abstract:
Automatic Multi-Agent Systems (MAS) generation has emerged as a promising paradigm for solving complex reasoning tasks. However, existing frameworks are fundamentally bottlenecked when applied to knowledge-intensive domains (e.g., healthcare and law). They either rely on a static library of general nodes like Chain-of-Thought, which lack specialized expertise, or attempt to generate nodes on the fly. In the latter case, the orchestrator is not only bound by its internal knowledge limits but must also simultaneously generate domain-specific logic and optimize high-level topology, leading to a severe architectural coupling that degrades overall system efficacy. To bridge this gap, we propose Unified-MAS that decouples granular node implementation from topological orchestration via offline node synthesis. Unified-MAS operates in two stages: (1) Search-Based Node Generation retrieves external open-world knowledge to synthesize specialized node blueprints, overcoming the internal knowledge limits of LLMs; and (2) Reward-Based Node Optimization utilizes a perplexity-guided reward to iteratively enhance the internal logic of bottleneck nodes. Extensive experiments across four specialized domains demonstrate that integrating Unified-MAS into four Automatic-MAS baselines yields a better performance-cost trade-off, achieving up to a 14.2% gain while significantly reducing costs. Further analysis reveals its robustness across different designer LLMs and its effectiveness on conventional tasks such as mathematical reasoning.

Abstract:
LLM agent memory modules universally adopt a single global retrieval depth k across all deployment scenarios; this global constraint produces an irreducible Oracle performance gap. We carry out a systematic evaluation on 19 heterogeneous deployment cells covering three LLM backbones, three multi-agent coordinators, three tasks, and two representative memory architectures (A-Mem and G-Memory). We confirm that the per-cell optimal k varies dramatically across cells, that low-capacity LLMs exhibit catastrophic accuracy collapse at larger k , and that no cross-cell feature or lookup-table shortcut reliably predicts the per-cell optimum. A theoretical analysis further shows that in the small-budget paired-binary-reward regime, every training-free bandit method we evaluate reduces to uniform-mean argmax. Motivated by this collapse, we propose per-cell warmup calibration: paired probe queries iteratively estimate the mean reward of each candidate k, the procedure commits to the empirical argmax, and a closed-form (2 σ) criterion provides earlystop. The recipe is training-free, requires no architectural modification, and introduces no hyperparameter beyond the probe budget. On all 19 deployment cells it attains 100% of the per-cell Oracle, with an average absolute improvement of +3.33 percentage points and a peak gain of +12.00 percentage points.

Abstract:
Stable personality is a key requirement for personal AI assistants, yet how reasoning models acquire such traits remains underexplored. Existing work largely treats personality as role-play and inherits the standard reasoning-model recipe: SFT warm-up followed by RLVR. We ask whether this recipe is appropriate for natural personality acquisition, where a trait should shape both final answers and intermediate reasoning, persist under prompt variation, and preserve general capability. Holding Stage 2 fixed as rubric-based GRPO, we vary Stage 1 on Qwen3-8B across four Big Five trait directions and compare standard SFT with MCQ-GRPO, a simple on-policy warm-up over the model’s trait-relevant choices and reasoning traces. We evaluate each endpoint using an LLM-judged Rubric for stylistic expression and TRAIT for introspective disposition. We find that SFT-then-GRPO achieves high stylistic scores but is weaker on the introspective probe. In contrast, MCQ-GRPO-then-GRPO yields stronger introspective acquisition at a smaller mathematical-reasoning cost. Weight- and activation-space analyses further support this finding, suggesting that personality acquisition is not only a question of how much trait is displayed, but where it is carried.

Abstract:
We study how to optimally allocate ranks in Low-Rank Adaptation (LoRA). Despite its importance, ranks are typically chosen manually and fixed across all modules. We argue that this practice is suboptimal, risking either under-parameterization, which hurts performance, or over-parameterization, which wastes memory and computation. To address this, we propose LoRA-RG, a gradient-guided framework that adjusts LoRA ranks through two stages: rank initialization and rank expansion. In the rank initialization stage, LoRA-RG estimates a minimal rank for each layer based on gradient information prior to training. During training, the rank expansion stage further increases ranks only when the current LoRA parameterization is insufficient to capture the remaining update demand. This design dynamically aligns LoRA ranks with the underlying optimization needs. We evaluate LoRA-RG on reasoning, mathematical, and code generation benchmarks, and show that it consistently achieves higher accuracy compared to fixed-rank LoRA and recent rank initialization and adaptation baselines under the same parameter budget.

Abstract:
Modeling how diverse communities respond to public health messages is critical for training students to design targeted communication materials. However, no method is currently available to iteratively evaluate how different populations may perceive their campaign designs before human-subject testing. Despite progress in VLM/LLM-based psychometric simulation, community simulators for public health education remain largely unexplored. We reframe VLMs as attribute-aware community response simulators and introduce CLAAS, to our knowledge the first multimodal response simulation tool for public-health higher education. CLAAS aligns open-source VLMs, such as Gemma3, Llama-3.2, and Qwen2.5-VL, with an in-house survey dataset of 33K+ responses from 1K+ participants to simulate trait-conditioned emotional, behavioral, and sentimental responses. Quantitatively, attribute-aligned VLMs consistently outperform zero-shot and random baselines and generalize to unseen communication strategies. Beyond dataset evaluation, our pilot studies in real course sessions across two semesters demonstrate CLAAS’s potential practical relevance in educational settings. Most students reported CLAAS provided useful feedback and generated responses aligned with their expectations. This study suggests attribute-aware VLM support experiential public health education and promote a socially impactful research direction: interactive population simulator for community-centered communication design.

Abstract:
LLM-generated stories are a popular use case, but they show very low variability. We sample 20,000 total stories from four current models using five prompts. We find that 11 words occur in 88.3% of generated stories, with little difference between models. These words include names (Elias, Mara, Elara), settings (lighthouses), and professions (clockmaker, librarian). These tokens do not often occur in published literature nor pre-training data, but they are found in preference data that is likely to have been used by all current models. Surprisingly, these “lighthouse” stories are infrequent when compared with the average post-training story, much of which contains references to copyrighted characters or adult content. This result demonstrates the potentially disproportionate impact of small datasets combined with powerful alignment algorithms.

Abstract:
Language agents increasingly rely on reusable skills, but post-failure repair is often handled by opaque one-shot reflection: a model generates a skill patch without explicitly maintaining how failure explanations relate to candidate repairs or how unsuccessful retests should influence later edits. We introduce RESKILL, a structured repair framework that maintains an explicit repair state across repair rounds. Given a failed rollout, the framework links failure hypotheses to candidate skill patches, selects local repairs through coverage-based attribution, retests the edited skill set in the environment, and uses retest outcomes to guide subsequent repair updates. The language model supplies structured repair factors, while the repair procedure records them, compares local skill patches by how well they address active failure explanations, and carries unsuccessful retest outcomes into later repair rounds. We evaluate RESKILL on ALFWorld and TextCraft across three model sizes under fixed repair budgets. RESKILL obtains the strongest final success in all six benchmark–model settings, improving average final success by 3.7 percentage points over direct repair and 3.3 points over hypothesis-conditioned repair. These results suggest that explicit attribution alone is insufficient; durable improvement emerges when attribution is integrated with repair selection and persistent retest-conditioned update.

Abstract:
Named Entity Recognition (NER) has achieved substantial progress since the advent of large language models (LLMs). Nevertheless, the recognition of long-tail and domain-specific entities remains challenging due to the deficiency in parametric knowledge. Retrieval-augmented generation (RAG) offers a promising remedy by injecting external knowledge, but it also introduces noise and unnecessary cost when dealing with familiar cases. In this paper, we propose NE-R1, a novel framework for adaptive retrieval-augmented NER. We design a “retrieval-on-demand” mechanism for NER. Then we integrate it into models by a two-stage training method: (1) multi-task instruction tuning initialization; (2) end-to-end RL optimization with CoT. To achieve reasonable selection between parameterized and external knowledge, we design a multi-dimensional reward considering both accuracy and retrieval benefit. NE-R1 achieves state-of-the-art performance on various benchmarks, with average improvements of 2.52 F1 points in in-domain evaluation and 1.18 F1 points in zero-shot cross-domain evaluation.

Abstract:
As LLMs scale, improving training efficiency relies on effective data utilization. Data selection mitigates this issue by allocating limited data budget to examples that optimally facilitate target behavior. Most existing approaches define target behavior via a set of target examples and score candidate training examples based on estimated influence on the target set. However, such methods treat target examples as equally important, ignoring their relevance to model optimization. Target examples that align closely with the model’s inherent behavior deliver stronger supervisory signals, whereas discrepant examples yield weak local guidance. We propose PRISM, a Preference-aware Influence function based Data Selection Method. It uses model preference to weight examples and build a preference-aware target direction. PRISM evaluates candidate training examples according to influence on this direction, and prioritizes data budget allocation to examples that drive the model to match target behavior. Theoretical analysis shows that preference weighting yields a superior first-order direction for boosting target preference, relative to equal aggregation. Experiments across model architectures and scales show that PRISM improves data selection in both capability improvement and reverse repair. The results validate that accurate characterization of target behavior serves as the core of cost-effective data selection.

Abstract:
Mixture-of-Experts (MoE) language models scale model capacity through sparse expert activation, but their large parameter footprint remains costly to deploy. Post-training quantization (PTQ) offers a practical compression strategy; however, quantization errors can alter router decisions and cause the quantized model to activate experts different from those selected by its full-precision counterpart. Prior router-aware PTQ objectives mitigate such expert-shift by aligning router outputs or preserving top- k expert orderings and margins at a given router. Yet sequential block-wise PTQ introduces an additional failure mode: quantizing a decoder block changes the representations received by the next MoE router, potentially causing downstream expert-shift that is not directly optimized by existing objectives. To address this issue, we propose MENDS-MoE (Margin-Preserving Expert Orderings with Next-Router Decision Supervision), an auxiliary calibration objective that supervises current-block quantization using routing decisions at the next MoE router. MENDS-MoE further introduces a boundary-aware margin loss that preserves orderings among selected experts while maintaining their separation from unselected experts near the top- k boundary. It is applied only during calibration and adds no inference-time computation. Across three MoE language models under 4-bit and 3-bit-average weight-only quantization, MENDS-MoE achieves the best average downstream accuracy and the lowest perplexity in most evaluated model–bit settings. Ablation studies further support the contributions of both next-router supervision and boundary-aware margin preservation.

Abstract:
Activation-based jailbreak defenses rest on a fixed assumption: a successful jailbreak leaves an internal trace stable enough across attacks for one probe to catch new ones. We show it fails in mixture-of-experts language models. Within an attack, routing makes the jailbreak legible: router logits distinguish compliance from refusal, and a few experts carry the signal. But the signature does not survive across attacks. A routing probe trained on three attacks performs at chance on a held-out fourth (pooled leave-one-attack-out balanced accuracy 0.52, 95 percent CI [0.50, 0.61]). Controls on two dense models show near-chance residual-stream transfer to held-out roleplay, so the failure is not specific to MoE routing. Nor is it an artifact of separable regions of activation space: the comply-versus-refuse direction itself shifts across attacks at every network depth. One systematic trace remains in the compliance state’s effective dimensionality, but it is a signature of shape, not location, beyond directional probes. Across six open-weight MoE models and four jailbreak attacks, in routing and residual streams alike, the jailbroken state is legible and locally structured within each attack, yet its location is attack-specific. Reading activations alone cannot be assumed to catch a jailbreak the probe was not built for.

Abstract:
How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training on reasoning chains that justify a safety decision), and preference optimization (ORPO) – across three architecturally distinct models (Llama-3.1-8B, Gemma-2-9B, Qwen3-8B). We find that training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training consistently produces a distinct kind of refusal computation, visible across all three models, while architecture independently shapes internal structure and how reliably refusal can be steered. Most importantly, no method we study achieves all three properties we would want from safe alignment at once: refusal that isn’t concentrated in a few fragile components, safety gains that don’t cost general capability, and safety behavior correctable through small, targeted edits. We caution against treating current post-training methods as a solved, reliable defense, especially for security-critical use. Code and models are available in https: //github.com/hoangcuongnguyen2001/Beyond-Shallow-Alignment.

Abstract:
Multi-agent LLM systems assign different responsibilities to different agents: one plans, another retrieves evidence, another verifies, another executes. This division of labor is only meaningful if each agent stays within the functions it is authorized to perform. Over many turns, however, an agent can produce a locally reasonable message that exercises authority belonging to another role: a verifier that begins planning, a retriever that asserts unsupported conclusions, an executor that changes state no one approved. We formalize this as role drift , a turn-level breach of an agent’s role contract. We introduce Agent Trajectory Control ( ATC ), which makes each role’s authority explicit as a contract and enforces it at the point where a message would enter the shared conversation. ATC scores a proposed output for out-of-role deviation, leakage into the functions of other roles, impermissible speech acts, and logical contract breaches, then computes a leakage-regularized target that constrains regeneration; the revised message is re-checked before admission. Enforcement acts on the message rather than the generator, so ATC governs agents whose internal representations are unavailable. Across homogeneous and heterogeneous teams, ATC improves task performance and role consistency while lowering role-drift energy relative to strong baselines spanning prompting, memory, workflow control, and activation editing, with the largest gains under explicit role-confusion stress tests and in larger, more diverse teams.

Abstract:
Offline preference optimization aligns reasoning models from fixed chosen-rejected pairs, yet standard methods apply gradient updates from every pair regardless of its training value under the current policy. We argue that this uniform treatment is wasteful and potentially harmful. From the perspective of gradient utility, we show that a pair’s contribution depends jointly on informativeness and stability. Pair utility drifts as the policy evolves, high-gradient samples can coincide with high-curvature regions, leading to noisy and destabilizing updates, and the most effective supervision comes from stable confident errors where the model is reliably wrong yet curvature remains low. These findings motivate SAGE (Stability-Aware Gradient Efficiency), which maintains difficulty-stratified candidate pools refreshed during training and selects pairs within each pool by a forward-pass signal-to-curvature score. Only pairs with high current utility receive gradient computation; the rest are excluded from backpropagation. On mathematical reasoning benchmarks across multiple model scales, SAGE outperforms full-data and size-matched baselines while producing substantially smoother optimization trajectories.

Abstract:
Multi-Agent Debate (MAD) has emerged as a promising paradigm for eliciting stronger inference-time reasoning, yet most existing systems still use fixed interaction protocols across instances. Such static designs can suffer from anchoring bias, overthinking on easy problems, and rigid coordination across tasks of varying difficulty. We present OrchDebate, a framework that recasts multi-agent debate as structured-state orchestration over reusable debate-control skills. Instead of following a pre-defined template, OrchDebate uses an orchestrator to examine the current debate state and select the next coordination action, such as introducing agents under different visibility settings or terminating deliberation with an adaptive readout rule. This skill-based view makes debate control modular and state-dependent. The orchestrator operates through a structured control interface rather than directly generating reasoning content, helping reduce reliance on early answers. OrchDebate further maintains a self-evolving skill bank whose reusable realizations are refined through outcome-guided updates on an evolution stream, without additional external training data. Across diverse benchmarks, OrchDebate delivers a better accuracy-cost trade-off than representative fixed and adaptive debate methods by allocating debate compute more selectively.

Abstract:
Collaborative conversations frequently contain references whose targets are indirect rather than named: resolving "this looks like the fix discussed yesterday" requires combining conversational context with evidence from the surrounding workspace which is accessible through APIs or user interfaces. We formalize this problem as Conversational Reference Grounding (CoRG): using a given set of tools to resolve a reference in conversation to the unique external item intended by the speaker. CoRG is challenging because it combines lexical, semantic, and temporal cues distributed across the conversation and the external workspace. Agents must translate these heterogeneous signals into effective tool use: formulating strategies, discovering plausible candidates, inspecting their metadata and content, and ruling out close alternatives. We study CoRG through RepoRef, a benchmark of 400 developer-chat segments grounded in GitHub issues, pull requests, and commits across 92 repositories. Unlike single-shot retrieval tasks, RepoRef often requires multi-step tool use. Our results show that CoRG remains challenging for current agents, even the best agent reaches only 67.0% success rate, leaving one third of references unresolved. These findings position CoRG as a concrete benchmark for studying how agents search, inspect, and verify information in realistic multi-tool environments.

Abstract:
Large language models (LLMs) are increasingly used to edit existing code, but correctness alone is not enough: useful repairs should also be minimal, reviewable, and faithful to the original implementation. We study over-editing, the tendency of a model to rewrite code beyond what is required to fix a bug. We construct an evaluation framework from 400 BigCodeBench problems by injecting controlled AST-level corruptions into reference solutions, giving each repair task a known minimal patch. Across frontier LLMs, over-editing is widespread even among strong models like GPT-5.5: high Pass@1 can coexist with unnecessarily large edits and added cognitive complexity. A preservation instruction substantially reduces this behavior, lowering average excess Levenshtein distance from 0.195 to 0.131, reducing added cognitive complexity by 26.6%, and increasing Pass@1 by 2.3 points. However, these gains do not simply follow from a larger reasoning budget or larger models. We next ask whether minimal editing can be learned directly during post-training. We observe that supervised fine-tuning overfits to seen corruption patterns, whereas reinforcement learning gives the best out-of-domain edit-fidelity and performance-retention trade-off. These results position edit fidelity as a distinct axis of code-repair quality and show that it can be measured and learned.

Abstract:
Diffusion Language Models (DLMs) generate text by iteratively denoising a masked sequence, independently predicting multiple tokens at each step. This conditional independence discards inter-token dependencies and degrades coherence-an issue that parallels the multi-modality problem in Non-Autoregressive Translation (NAT). Drawing on the Directed Acyclic Transformer (DAT), which tackles this problem in NAT via a Directed Acyclic Graph (DAG), we propose DA-DLM, a model that adapts DAG-based dependency modeling to DLMs’ iterative setting through a position-oriented DAG design. The position-oriented DAG binds node groups to fixed output positions so that tokens fixed in earlier steps anchor neighboring predictions via learned transitions, and evolves with denoising to focus on remaining uncertainty as anchors accumulate. On language modeling, open-ended generation, and summarization, DA-DLM consistently outperforms Block Diffusion, especially under fewer denoising steps, and matches autoregressive models while preserving the parallel generation advantage. Our code is publicly available at https://github.com/jipy0222/DA-DLM.

Abstract:
Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottlenecked by two fundamental issues: current data synthesis methods for GUI Agents rely on specific environments and struggle to generate diverse data, while existing evaluators either suffer from limited scalability or provide inaccurate and unreliable reward signals. To overcome these challenges, we introduce GSAR (Goal-State-Anchor Reward), a RL reward framework that supports scalable task generation and delivers reliable reward signals for stable and efficient policy optimization. Our approach features self-evolving data synthesis, which produces multiple environments through task execution and generates diverse tasks and goal states. Complementing this, a state-anchor mechanism automatically annotates task-relevant UI elements in successful goal states as reference anchors. During RL training, these reference anchors provide accurate, scalable reward signals that substantially enhance efficiency. Extensive evaluations demonstrate that our framework achieves over 90% accuracy on offline trajectory verification and performs closest to rule-based methods. Furthermore, agents trained using our reward framework exhibit strong performance on both AndroidWorld and our constructed benchmark, establishing a scalable approach for GUI agent training.

Abstract:
Test-time evolution of agent memory represents a pivotal paradigm for advancing AGI, as it strengthens complex reasoning through experience accumulation without requiring parameter updates. However, even during benign task evolution, agent safety alignment remains vulnerable—a phenomenon known as Agent Memory Misevolution. To evaluate this phenomenon, we construct the Trust-Memevo benchmark and find that agents exhibit an overall decline in trustworthiness across multiple tasks during benign task evolution. To address this issue, we propose TAME, a trust-aware memory evolution framework in which a shared memory bank is jointly governed by an Executor and an Evaluator. The Executor retrieves and applies transferable experiences to support task solving, while the Evaluator assesses the contribution of each utilized experience to the outcome and produces trust-aware feedback to guide subsequent memory use. This executor–evaluator loop enables memory to be selectively reinforced, cautiously reused, and continuously expanded over time. Experiments show that TAME mitigates memory misevolution while achieving strong task performance. In particular, on the GPT-5.2 AIME benchmark, TAME improves accuracy by 14.6 percentage points over the strongest existing method and maintains competitive trustworthiness.

Abstract:
Retrieval-Augmented Generation (RAG) is highly sensitive to retrieval noise: when retrieved documents mix informative and irrelevant context, LLMs are easily distracted, leading to hallucinations. To overcome this, we propose SCoNE (Selective Context-aware Neuron Editing), a training-free model editing approach that improves retrieval noise robustness by selectively strengthening context-aware FFN neurons that are identified by both high attribution and high cross-input variability. SCoNE requires only a small number of mining samples, no fine-tuning, and no inference-time overhead. Across various knowledge-intensive question-answering benchmarks and two LLM backbones, SCoNE consistently outperforms competitive baseline methods. Our code is available at https://github.com/HYU-ARK-Lab/SCoNE.

Abstract:
Sensitive attributes such as age, income, and occupation can be inferred from user-generated content by aggregating indirect cues across many ordinary posts. LLM-based profilers can perform this aggregation automatically and with high accuracy, which makes large-scale personal attribute inference a major privacy threat. Existing LLM-based profilers, however, offer limited insight into which specific posts, concepts, and relationships made an inference possible, which is key to targeted privacy mitigation, i.e., redacting or rewriting only the few posts that actually leak an attribute, rather than perturbing entire histories. We introduce GraphProfiler, an auditable LLM-based profiler that represents each user’s post history as a source-linked personal knowledge graph where nodes and edges trace back to the originating post and resolves attribute predictions to cited graph records and source texts. GraphProfiler reaches 86.7% attack success rate on the eight-attribute SynthPAI benchmark, within two points of strong text-only baselines, and 84.6% on PANDORA, while citing supporting evidence for over 98% of predictions. Our controlled ablation experiments provide evidence that the cited posts contribute to attack success, as removing them reduces the attack success rate substantially more than removing an equal number of random posts.

Abstract:
Positional encoding (PE) underpins how permutation-invariant Transformers represent sequence order, yet how positional information is processed and stored remains poorly understood. Modern PE methods such as RoPE still struggle on tasks such as long-context understanding or retrieval (Chen et al, 2025). Hence, a better understanding of the internal positional mechanism could help design better PE. Building on evidence that positional and semantic signals occupy nearly orthogonal subspaces in trained Transformers, we modify an encoder Transformer to process three explicitly disentangled streams: semantic, absolute positional (AP) and relative positional (RP), and confine the masked-language-modeling (MLM) objective to the semantic stream. This decoupling enables a clean mechanistic study and yields three take-aways. (1) The isolated AP subspace spontaneously collapses into a low-frequency two-dimensional manifold that captures the structure of the document; (2) Attention heads specialize into structure and semantic-oriented groups, with RP exclusively supporting the latter; (3) Standard positional encodings do not robustly retain macroscopic structure: RoPE and RP only weakly encode it, and entangled AP loses it in the final layers under MLM pressure. The disentangled approach preserves positional encoding, which improves linguistic representations.

Abstract:
Controlling the basic human values expressed by large language models (LLMs) has attracted growing attention in the mechanistic interpretability community. Recent work has made substantial attempts to steer such values through vector arithmetic or causal attribution over internal states. However, because values are complex social constructs, their mechanistic controllability still faces a major unresolved challenge: the auditability bottleneck, which makes it difficult to construct natural-language corpora that precisely capture value realization for mechanistic analysis. To address this challenge, we propose VALUEAUDITOR, a measurement-first framework that makes values behaviorally targetable before they are representationally steered. It operationalizes each value through grouped, theory-grounded response rubrics, derives graded supervision from same-scenario response contrasts, and performs inference-time steering through token-level sparse features that retain local cues of value expression. Experimental results show that VALUEAUDITOR significantly outperforms cutting-edge baseline methods on value steering benchmarks, while demonstrating robust control across values and scenarios with minimal trade-offs in response clarity. The code is publicly available at https://github.com/ZichenX/Steer-On-Rubrics

Abstract:
Long audio-video reasoning is difficult for omni-modal LLMs because the decisive evidence is often sparse, cross-modal, and too expensive to preserve with uniformly high-fidelity inputs. We introduce OmniReasoner , a tool-use post-training framework for Thinking with Long Audio-Video : omni-modal LLMs learn, via supervised fine-tuning and reinforcement learning, to decide whether and where to call a zoom-in tool before answering. OmniReasoner first builds a low-cost global preview of the full stream and then, when needed, calls the zoom-in tool with a requested temporal interval for higher-fidelity visual and audio inspection before answering. Because the model observes different sampling granularities before and after this call—a sparse global preview and a denser local clip—we introduce TimeAnchor , which keeps the tool’s temporal argument valid and round-trip-consistent across these granularities, rather than tied to frame indices from a particular sampling rate. To make this tool-use behavior trainable without expensive manual interval annotation, we build a Temporal Augmented Data Engine that synthesizes tool-use post-training trajectories by video editing and composition. Experiments across omni-modal and video benchmarks show that OmniReasoner improves both answer accuracy and temporal grounding while concentrating high-fidelity computation on informative regions. Code is available at https://github.com/RockyChen0205/OmniReasoner .

Abstract:
Deep search requires LLM agents to decompose complex queries, search for evidence, and synthesize grounded answers, yet existing ReAct-style agents suffer from two limitations: role coupling, where one policy must handle planning, evidence use, and synthesis; and context accumulation, where growing search histories introduce noise and obscure useful information. To address these issues, we propose IterSynth, a role-decoupled and summary-based paradigm that alternates between a Planner for identifying information needs and a Synthesizer for integrating evidence into an evolving summary state. This design separates planning from synthesis while using the summary as the persistent state of search, reducing both capability coupling and context noise. To train IterSynth effectively, we further introduce Role-Decoupled Policy Optimization (RDPO) for reinforcement learning, which combines terminal outcome rewards with turn-level rubric evaluations and computes role-specific advantages for more precise credit assignment. Experiments on five long-horizon deep-search benchmarks such as BrowseComp and Xbench-DS show that IterSynth-8B achieves an average score of 50.7, surpassing the strongest prior ≤ 8B agent by +4.2%. Moreover, IterSynth serves as a model-agnostic prompting paradigm, delivering substantial zero-shot gains over ReAct and similar prompting paradigms on frontier proprietary models.

Abstract:
Diffusion large language models (dLLMs) accelerate language generation by predicting multiple masks in a single forward pass.However, existing dLLMs can suffer from unreliable predictions in early denoising stages under aggressive parallelism strategies, leading to errors that can propagate to later stages. To tackle this issue, we present Consistency Forcing (CForce) for dLLMs, a distillation method to force the mask predictions of early stages to align with those of later stages. CForce trains the model on pre-collected self-rollout trajectories, thereby improving training-inference alignment.We introduce Confidence Adaptive KL Divergence as a distillation objective to conjoin the merits of forward and reverse KL.We further provide a theoretical analysis for the consistency objective to explain why CForce can approximately minimize the prediction error of early stages.Critically, the same formulation applies to both mask-to-token decoding and edit-capable decoding; in the edit-capable case, later token-to-token refinements provide additional supervision for earlier masked-state predictions. Experiments on non-edit and edit-capable LLaDA models show improved speed-quality trade-offs, especially under high-parallelism decoding budgets.

Abstract:
Text-to-video (T2V) generation models have advanced rapidly, yet their ability to represent diverse cultural contexts remains underexplored. Existing benchmarks mainly focus on perceptual quality, physical plausibility, and text-video alignment, but do not directly assess whether generated videos capture culturally specific objects, actions, rituals, visible text, or audio cues. We introduce CultureVidBench, a comprehensive benchmark for evaluating cultural understanding in T2V generation. CultureVidBench contains 1,000 curated prompts covering 12 countries, 6 continents, 8 cultural regions, and 14 cultural aspects organized into three categories: material culture, social practice & performance, and ritual & ceremony. Designed specifically for video generation, CultureVidBench emphasizes dynamic and multimodal cultural representation, including social interactions, ritual procedure, and culturally appropriate visible text and audio. We evaluate seven representative T2V models through human user studies and MLLM-based automatic assessment across cultural faithfulness, multimodal cultural rendering, semantic adherence, and perceptual quality. Results show that although current models achieve strong semantic adherence and visual quality, they often fail to faithfully capture fine-grained cultural details, particularly for underrepresented regions, rituals, and multimodal cultural cues.

Abstract:
Large language models struggle with spatial and geometric reasoning. Existing remedies either fine-tune on answers, leaving geometric structure implicit, or attach a symbolic solver at inference, which is brittle under imperfect parser coverage. We propose DGCL (Differentiable Geometric Constraint Layer), a training-time objective for a typed family of spatial and geometric relations (e.g., parallelism, frame-relative left/right, vertical order) that supervises the LLM’s hidden states with a differentiable Huber residual over parser-proposed candidate constraints, a coverage regularizer, and a learned selector. Training requires a frozen constraint parser; inference does not: it is a single LLM forward pass over the text prompt plus one learned ‘[GEO]‘ token, with no parser, solver, external tool, or auxiliary head. We evaluate on three auditable datasets (planar geometry, language-grounded 3D relations, and a firewalled zero-touch counterfactual split) with 10,278 test instances over 5 seeds. We are explicit about attribution: a strong constraint-supervised control (A2c), which trains the selector and coverage objective on gold candidates but blocks the residual gradient from the LLM, already captures most of the improvement over plain fine-tuning (up to +26 points). Our isolated contribution is that coupling the differentiable residual gradient into the LLM adds a further, stable +4.0 to +5.2 points over this strong control across all five settings (seven pre-registered gates, Holm-corrected p<0.001 ), at 1.06× a fine-tuned baseline’s latency. On full parser coverage a strong tool baseline marginally exceeds DGCL; DGCL’s advantage is robustness under partial or missing coverage, not superiority over oracle solvers.

Abstract:
While multilingual language models successfully transfer factual and syntactic knowledge across languages, it remains unclear whether they process culture-specific pragmatic registers, such as slang, as isolated language-specific memorizations or as unified, abstract concepts. We study this by probing the internal representations of Gemma-2-9B-IT using Sparse Autoencoders (SAEs) across three typologically diverse source languages: English, Hebrew, and Russian. To isolate pragmatic register processing from trivial lexical sensitivity, we introduce a novel dataset in which every target term is polysemous, appearing in both literal and informal contexts. We find that while much of the informal-register signal is distributed across language-specific features, a small but highly robust cross-linguistic core consistently emerges. This shared core forms a geometrically coherent “informal register subspace”, with greater geometric concentration at the deeper layer. Crucially, these shared representations are not merely correlational: activation steering with these features causally shifts output formality across all source languages and transfers zero-shot to six held-out languages spanning diverse language families and scripts. Together, these results provide the first mechanistic evidence that multilingual LLMs internalize informal register not just as surface-level heuristics, but as a portable, language-agnostic pragmatic abstraction.

Abstract:
Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified, multilingual, and multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants. Based on this dataset, we propose a novel strategy-alignment evaluation that leverages the winning faction’s strategies as ground truth in two stages: 1) Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability. 2) Decision evaluation, which assesses the model’s voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models’ linguistic and reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs and VLMs exhibit diverse performance, with roughly half of the models in both modalities remaining below 0.50 accuracy, revealing clear gaps in deception and counterfactual reasoning.

Abstract:
Machine unlearning for large language models seeks to remove specified knowledge while preserving general capability, yet current methods often treat an entire response as the unit of forgetting. This coarse granularity causes unnecessary utility loss when the forget target is only a small set of sensitive facts embedded within otherwise benign explanations. In this paper, we introduce Decouple to Forget (D2F), a targeted unlearning method based on causal mediation that separates sensitive factual content from non-sensitive meaning and surface expression. For each example to be forgotten, D2F extracts a fact-level “mediator” representation from the answer, identifies which parts are sensitive, and removes only those parts while leaving the rest of the structure intact. We then generate counterfactual answers grounded in this mediator that preserve non-sensitive content but exclude sensitive facts, employing preference-based contrastive optimization to reduce the likelihood of information leakage while maintaining helpfulness and fluency. Experiments across various benchmark datasets show that D2F achieves a better balance between forgetting and retention than strong baseline methods. Our code is available at https://anonymous.4open.science/r/D2F-D596/.

Abstract:
The task of authorship style transfer involves rewriting text in the style of a target author while preserving the meaning of the original text. Existing style transfer methods train a single model on large corpora to model all target styles at once: this high-cost approach offers limited flexibility for target-specific adaptation, and often sacrifices meaning preservation for style transfer. In this paper, we propose AuthorMix: a lightweight, modular, and interpretable style transfer framework. We first train individual, style-specific LoRA adapters on a small set of high-resource authors: this allows for the rapid training of specialized adaptation models for each new target using layer-wise adapter mixing via reinforcement learning, necessitating only a handful of target- style training examples. AuthorMix ranks first on the combined style–meaning score among all baselines, including GPT-5.1, and substantially improves meaning preservation over the trained baselines; under human evaluation it is the only method best-or-tied on every dimension.

Abstract:
Decoding-time KV cache compression research focuses heavily on designing better token scoring functions, while the temporal rule that aggregates scores across decode steps is often treated as an implementation detail. Under aggressive KV compression, we find that exponential-moving-average (EMA) aggregation makes approximately order-preserving scorer modifications largely indistinguishable at the eviction-set level. Value-norm and entropy variants remain highly correlated with attention and produce nearly unchanged retention sets, whereas KeyDiff, key norm, recency, and a learned scorer alter the ranking and degrade substantially. We associate this stability with the evaluated aggregation, which couples layer weighting and temporal retention. Building on this observation, we introduce InertiaKV, an EMA-based decoding-time eviction method, and InertiaKV-Lazy, its periodic-refresh variant, which yields 1.34–1.46× decode throughput relative to full refresh InertiaKV. We also study Score-Free decoding as a separate empirical operating point: it scores the full context once at the first decode step, freezes that ranking, and incurs an average quality change of +0.03 while removing all subsequent scoring. Across six open-weight backbones and the LongBench, LongBench-v2, and RULER benchmarks, the results identify temporal aggregation and ranking preservation as distinct, consequential design factors; they do not imply that scoring quality is irrelevant in general.

Abstract:
Gloss-free Sign Language Translation (SLT) avoids costly gloss annotations but remains limited by weak visual-text alignment. Recent MLLM-based methods use frame descriptions as auxiliary supervision, yet these descriptions are often repetitive, producing nearly identical text features; meanwhile, global pooling washes out sparse but meaningful gestures. We propose FiMA (Fine-grained Multimodal Alignment), a pre-training framework for redundancy-aware description enhancement. FiMA combines a Description Enhancement Module, which removes shared template redundancy from description features, with a Multimodal Alignment Module, which aligns each text token to its most relevant visual frame through Max-Sim matching. On PHOENIX14T and CSL-Daily, FiMA achieves state-of-the-art gloss-free SLT performance and visualizations show clearer feature separation and temporal grounding. The code will be released at https://github.com/Ryeliu/FiMA.

Abstract:
Data attribution methods often compress the effect of each training data point into a single score, computed either at a final checkpoint or by aggregating quantities along the training trajectory. This static view hides how the contribution of a training data point changes during training. To overcome this limitation, we propose the Window-level Influence Estimator (WIE), a framework for estimating the contribution of a training data point zⱼ over a user-specified training window [t₁, t₂]. WIE models a leave-one-out-style counterfactual under a first-order linearization of SGD. An influence propagator P[t] = I − ηₜH[t] at each step models how parameter deviation is amplified or dampened by the loss landscape. Empirically, the window-level estimator reveals four distinct influence patterns: Early Influencer, Late Bloomer, Stable, Highly Fluctuating. Across multiple benchmarks, WIE improves mislabeled-data detection over the considered static and trajectory-based baselines. On BERT-IMDB with 40% label noise, WIE outperforms these baselines across the evaluated inspection ratios. On ViT-EMNIST, WIE reaches 70% recall for mislabeled data while inspecting up to 7.5× fewer candidates than TracIn requires to reach the same recall.

Abstract:
Restoring dots to dotless Arabic text, historically known as _rasm_, is a foundational yet underexplored problem in Arabic NLP. While Arabic diacritization focuses on recovering omitted vowels, rasm restoration addresses a deeper level of orthographic ambiguity in which consonantal distinctions themselves are absent from the written text. Beyond its linguistic and historical significance, the task has applications in representation learning, tokenization, OCR, manuscript digitization, information retrieval, and content moderation. In this paper, we present a large-scale comparative study of Arabic rasm restoration across statistical, neural, pretrained, and prompt-based paradigms. We evaluate n-gram language models, BiLSTMs and Transformers trained from scratch, pretrained character-level CANINE models, and large language models (LLMs) using few-shot prompting. Our experiments were conducted on a diverse corpus exceeding 431M words spanning social media, news, religious texts, poetry, and academic writing. Results show that Transformer models achieve the best overall performance while remaining computationally efficient. BiLSTMs perform competitively, and classical n-gram models remain surprisingly strong despite their simplicity. In contrast, modern LLMs perform substantially below specialized models, with flagship systems only matching the performance of a 4-gram baseline. These findings position rasm restoration as a challenging benchmark for contextual orthographic inference and a natural extension of Arabic diacritization research.

Abstract:
The rapid development of AI-driven tools, particularly large language models (LLMs), is reshaping professional writing. Still, key aspects of their adoption such as language support, ethics, and long-term impact on writers’ voice and creativity remain underexplored. In this work, we carried out a questionnaire ( N = 301 ) and an interactive task ( N = 36 ) targeting freelance professional writers regularly using AI. We examined AI-assisted writing practices across 25+ languages, ethical concerns, and user expectations. Our findings reveal that adoption is shaped more by peer influence and professional outlook than demographics, that multilingual writers face both performance and perception barriers to equitable AI use, and that job security concerns are widespread and adoption-independent.

Abstract:
Multimodal Automated Program Repair (MAPR) extends traditional program repair by requiring models to jointly reason over source code, textual issue descriptions, and visual artifacts such as GUI screenshots. While recent LLM-based repair systems have shown promising results, existing approaches face several limitations: rigid workflow pipelines restrict exploration during debugging, visual reasoning is often performed over full-page screenshots without localized grounding, and failed repair attempts are rarely transformed into reusable knowledge. To address these challenges, we propose FailureMem, a multimodal repair framework that integrates three key mechanisms: a hybrid workflow–agent architecture that balances structured localization with flexible reasoning, active perception tools that enable region-level visual grounding, and a Failure Memory Bank that converts past repair attempts into reusable guidance. Experiments on SWE-bench Multimodal demonstrate FailureMem improves the resolved rate over GUIRepair by 3.7%.

Abstract:
Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss. Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically. Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best. On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.

Abstract:
Video event prediction (VEP) requires models to infer unobserved future states from partial video evidence. Existing video MLLMs usually verbalize intermediate future reasoning in text space: once visual evidence is verbalized, fine-grained motion, geometry, and interaction cues can be lost, leading to plausible but visually ungrounded hallucinations. We introduce Future-L1, an interleaved latent visual reasoning framework that lets an MLLM alternate between language tokens and continuous latent visual spans during autoregressive decoding. To train this capability, we construct Future-L1-50K by selecting examples where future visual hints help prediction and aligning latent states to future-frame embeddings, then further optimizing sampled latent trajectories with LA-DAPO, a latent-aware RL objective with outcome-contrastive and temporal-diversity rewards. Future-L1 achieves new state-of-the-art results on both benchmarks: on FutureBench, it improves Qwen3-VL-8B from 61.0 to 85.4 and exceeds the previous best Video-CoE by 10.4 points; on TwiFF-Bench, it improves the average score from 2.44 to 3.04. These results suggest that future-oriented video reasoning benefits from preserving intermediate visual semantics in latent space rather than translating every reasoning step into text.

Abstract:
Language models are trained on tokenized text that obscures the sound structure of words, yet they reliably produce morphemes whose form is phonologically conditioned. It remains unclear whether they rely on item-specific memorization or rule-like generalization and, if the latter, how that generalization is implemented. We therefore ask whether this phonological condition is represented within language models and how it is causally used for allomorph selection. For the English indefinite article a / an , we show that the phonological condition is encoded along a single linear direction in trigger-token embeddings, that this direction causally drives article selection in token-level wug tests, and that, at the article-prediction position, the model forecasts the upcoming trigger token and uses the forecasted trigger’s phonological feature to choose the article. We then ask whether this rule-like generalization extends beyond English article selection, both to allomorph selection in other languages and to explicit phonological judgment. Together, these results provide a mechanistic account of phonologically conditioned allomorph selection in language models, and dissociate this generation-time ability from explicit metalinguistic judgments.

Abstract:
Decoder-only large language models (LLMs) carry rich semantic information in their hidden states, but these states are not directly usable for retrieval: they exhibit strong anisotropy and excessive dimensionality, producing distorted cosine-similarity rankings and prohibitive index sizes. A standard solution is to train a dedicated embedding model, which requires positive pairs and substantial compute. We ask a different question: can we make frozen LLM hidden states retrieval-usable purely through post-hoc transformation, without fine-tuning the LLM backbone or using labeled positive pairs? We propose NCWP (Neighbor-Contrastive Whitening Projection), a post-hoc alignment method for symmetric sentence-similarity retrieval that combines ZCA-shrink whitening for global isotropy correction with augmentation-free neighbor-contrastive projection learning, requiring only a single linear projection at inference. On the official STS-Benchmark test split and BEIR-Quora across three decoder-only LLM backbones (Llama-3.1-8B, Qwen1.5-4B, Qwen2-7B), NCWP substantially exceeds the raw Base hidden states at aggressive-compression dimensions (e.g., +23.8 Spearman over Qwen1.5-4B Base on STSBenchmark at 8× compression; +13.6 nDCG@10 over Qwen1.5-4B Base on BEIR-Quora at 32× compression), simultaneously delivering retrieval-quality improvements and index-size reductions; the pattern also holds in breadth analyses on STS12–16+SICK-R and CQADupStack. All main results are reported as mean over three random seeds. We characterize a dimensional crossover in which NCWP’s advantage over PCA-whitening widens monotonically down to 256× compression on Qwen1.5-4B and Qwen2-7B, and provide geometric evidence consistent with the proposed mechanism via anisotropy measurements: NCWP reduces anisotropy on all three LLM backbones by roughly an order of magnitude, to a near-zero residual on Qwen1.5-4B and Llama-3.1-8B and a smaller reduction on Qwen2-7B. NCWP is also complementary to prompt-based extraction: applied on top of PromptEOL it improves over full-dimensional PromptEOL while compressing up to 64×. Code is available at https://github.com/ianleeGH20/NCWP.

Abstract:
Recent advances in large reasoning models (LRMs) have made machine unlearning more challenging, as protected facts or unsafe rationales may surface in intermediate chain-of-thought (CoT) traces before the final answer is produced. Existing unlearning objectives typically suppress the target content or redirect internal representations, but they never specify how the post-forgetting trajectory should continue, which can lead to hallucinated substitutes, malformed boundaries, or repetitive outputs. We argue that LRM unlearning should instead learn a natural forgetting trajectory: a coherent non-disclosing CoT followed by a stable refusal-style answer that replace the original disclosure. To this end, we propose Guided Answer-Reasoning Distillation (GUARD), which converts model-generated unsafe disclosures into safe-exit trajectories, aligns a frozen LRM via guidance tokens, and distills the guided behavior into model parameters.To address the lack of metrics for replacement quality beyond leakage, we further introduce Natural Forgetting Reasoning Score (NFRS), which captures structural stability, fluency, and unsupported substitutes in forgotten outputs. Extensive experiments on R-TOFU and a STAR-1-derived harmful-intent setting show that GUARD substantially reduces unsafe and privacy disclosures across two widely adopted distilled LRMs while preserving reasoning utility. Codes are available at https://github.com/zeyu-Yan/GUARD

Abstract:
Large language models make natural-language-to-MILP modeling increasingly plausible, but reliably selecting among multiple generated programs remains challenging: structurally similar errors can create false majorities, while solving every candidate integer program is costly. We propose Structural-Semantic Consistency (SSC), a best-of- K selector combining structural consistency, static quality, text alignment, and two diagnostics obtained from continuous LP relaxations. Notably, SSC does not require solving any candidate integer programs during selection. Across seven benchmarks and 618 fixed candidate pools, SSC obtains 55.7% accuracy, compared with a random baseline of 51.5%, 53.7% for a zero-shot listwise LLM judge, and 55.3% for a leave-one-benchmark-out learned ranker. The symbolic fusion reaches 53.6%, with LP diagnostics adding 2.1 percentage points. These results establish multi-signal weak verification as a competitive and interpretable baseline for candidate selection.

Abstract:
LLMs often exhibit highly agreeable conversational styles, also known as AI sycophancy. This pattern may become problematic when interacting with user prompts that reflect negative social tendencies, risking the amplification of harmful behavior. We examine how LLMs respond to user prompts expressing varying degrees of Dark Triad traits (Machiavellianism, Narcissism, and Psychopathy) using a curated dataset. Our analysis reveals systematic differences across models: while all models predominantly exhibit corrective behavior, some generate reinforcing or ambivalent output. Model behavior further varies with severity level and response sentiment. These findings highlight the need for safer conversational systems that can reliably detect and respond to users escalating from benign to harmful requests.

Abstract:
Recent work has shown that the hidden states of large language models contain signals useful for uncertainty estimation, motivating a growing interest in efficient probe-based approaches. Yet it remains unclear how robust existing methods are, with prior work reporting conflicting conclusions under substantially different evaluation settings. We address this by introducing ProbeDrift, a systematic evaluation framework for supervised uncertainty probes covering a wide range of OOD settings across models, tasks, and distributional shifts. Using ProbeDrift, we train over 2,000 probes to disentangle the effect of key design choices, showing poor robustness of current methods beyond near-OOD settings. We find that robustness is driven by design decisions that have a largely invisible effect in-distribution, including the choice of feature type, aggregation strategy, and training signal. We argue that robust uncertainty estimation requires robust evaluation. To support this, we release ProbeDrift as a lightweight Python library that contains the train and test splits underpinning our extensive evaluation. We also show how insights from our evaluation can directly lead to more robust methods through a simple Hybrid Back-Off (HBO) strategy.

Abstract:
As large language models (LLMs) are deployed as autonomous agents, safety failures increasingly involve consequential actions. We study agentic misalignment, where agents take harmful actions under goal conflicts and pressures. Using chain-of-thought (CoT) monitoring, we find that harmful execution is often preceded by intent signals in reasoning. However, post-hoc CoT labels are too coarse to show how intent changes during generation. We introduce INTENT-AS-A-TOOL, an approach that adds intent-targeted tools to give the model a dedicated channel for expressing commitment to a target behavior. The probability of calling an intent tool provides a judge-free, fine-grained signal of the model’s tendency to pursue that behavior. Our results show that INTENT-AS-A-TOOL complements CoT monitoring, expands post-hoc CoT labels into dense trajectories, and identifies critical steps for online intervention. These findings suggest that action preferences are useful for tracking agentic misalignment during reasoning. Our code and data are accessible: https://github.com/RebeccaZhang22/intent-as-a-tool.

Abstract:
Multimodal LLMs are powerful but prone to object hallucinations, which describe non-existent entities and harm reliability. While recent unlearning methods attempt to mitigate this, we identify a critical flaw: structural fragility. We empirically demonstrate that standard erasure achieves only superficial suppression, trapping the model in sharp minima where hallucinations catastrophically resurge after lightweight relearning. To ensure geometric stability, we propose Sharpness-Aware Robust Erasure (SARE), which casts unlearning as a targeted min-max optimization problem and leverages a targeted Sharpness-Aware Minimization (Targeted-SAM) mechanism to explicitly flatten the loss landscape around hallucinated concepts. By suppressing hallucinations under simulated worst-case parameter perturbations, our framework ensures robust removal stable against weight shifts. Extensive experiments demonstrate that SARE outperforms baselines in erasure efficacy while preserving general generation quality. Crucially, it maintains persistent hallucination suppression against relearning and parameter updates, validating the effectiveness of geometric stabilization.

Abstract:
Large Language Models (LLMs) have achieved remarkable strides in multilingual translation but are hindered by a systemic cross-lingual verbosity bias, rendering them unsuitable for strict time-constrained tasks like subtitling and dubbing. Current prompt-engineering approaches struggle to resolve this conflict between semantic fidelity and rigid temporal feasibility. To bridge this gap, we first introduce Sand-Glass, a benchmark specifically designed to evaluate translation under syllable-level duration constraints. Furthermore, we propose Homura, a reinforcement learning framework that explicitly optimizes the trade-off between semantic preservation and temporal compliance. By employing a constrained reinforcement learning objective featuring a novel dynamic syllable-ratio reward, Homuraeffectively "tames" the output length. Experimental results demonstrate that Homura significantly outperforms strong baselines, achieving precise length control that respects linguistic density hierarchies without compromising semantic adequacy.

Abstract:
Large Language Models (LLMs) are widely used for question answering but are prone to hallucinations, yet the underlying causes, particularly the role of factual prevalence in training data, remain underexplored. In this work, we study this through academic authorship attribution, hypothesizing that hallucinations are more likely for less prevalent papers. Using citation count as a proxy for factual prevalence, we curate a dataset of 1.92 million papers across eight scientific disciplines with stratified samples by citation level. Experiments on three state-of-the-art LLMs confirm our hypothesis, revealing a robust inverse relationship between hallucination rate and citation count across all models and disciplines, with hallucination rates exceeding 98% for rarely cited papers and falling below 37% for highly cited ones. Our in-depth analysis further shows that these failures primarily reflect retrieval limitations rather than knowledge absence, and hidden-state probing of open-source models shows that factual prevalence is encoded in pre-generation representations, providing convergent mechanistic evidence for the prevalence-hallucination relationship.

Abstract:
LLMs often fail to handle temporal knowledge conflicts–contradictions that arise when facts evolve over time within their training data. Existing benchmarks often rely on structured sources such as Wikidata, where changes are directly exposed through versioned snapshots, and rarely account for differences in model knowledge cut-offs, limiting fair evaluation across LLMs released at different times. We introduce a novel framework for automatically constructing temporal knowledge probing benchmarks from unstructured, time-stamped corpora by discovering evolving knowledge through salient entities, concepts, and attributes; and using this structure to generate grounded questions with gold answers aligned to model cut-off dates. Using this framework, we construct evolveQA, spanning 3 domains: AWS service updates, Azure platform changes, and WHO Disease Outbreak News reports. Evaluating 12 open- and closed-source LLMs across 3 probing formats, we find performance drops of up to 31% on evolveQA compared to static knowledge questions, showing that LLMs struggle to recall temporally updated facts even when such knowledge is available in their parameters.

Abstract:
Vision-Language Models (VLMs) have demonstrated impressive performance when integrated into robot manipulation pipelines, but real-world evaluation of VLM capabilities on downstream robotics tasks is both difficult and expensive to scale. We propose that spatial and temporal ordering tasks using images from stereo and wrist cameras attached to real-world robots are particularly well-suited for evaluating visual reasoning capabilities, as they assess skills directly applicable to robots. We evaluate VLM reasoning abilities on three questions: ordering shuffled frames conditioned on a language description of the task, ordering objects depthwise based on distance from the camera, and ordering objects from left-to-right in pixel coordinates. The complexity of these tasks can also be modulated by changing the size of the set to sort. Our experiments show that several state-of-the-art vision-language models struggle on ordering tasks. To fix this, we propose a scalable autonomous training data generation pipeline, and show that both supervised finetuning and reinforcement-learning on the generated data can significantly improve VLM reasoning performance both on spatiotemporal ordering tasks as well as downstream robotics and visual reasoning tasks.

Abstract:
Autonomous agents powered by large language models (LLMs) perform complex tasks through long-horizon reasoning and tool interaction, where a fundamental trade-off arises between execution efficiency and reasoning robustness. Models at different capability–cost levels offer complementary advantages: lower-cost models enable fast execution but may struggle on difficult reasoning segments, while stronger models provide more robust reasoning at higher computational cost. We present AgentCollab , a self-driven collaborative inference framework that dynamically coordinates models with different reasoning capacities during agent execution. Instead of relying on external routing modules, the framework uses the agent’s own self-reflection signal to determine whether the current reasoning trajectory is making meaningful progress, and escalates control to a stronger reasoning tier only when necessary. To further stabilize long-horizon execution, we introduce a difficulty-aware cumulative escalation strategy that allocates additional reasoning budget based on recent failure signals. In our experiments, we instantiate this framework using a two-level small–large model setting. Experiments on diverse multi-step agent benchmarks show that AgentCollab consistently improves the accuracy–efficiency Pareto frontier of LLM agents.

Abstract:
Large language models deployed as multi-step agents are increasingly trained via Direct Preference Optimization (DPO) on trajectory-level comparisons. Yet standard DPO weights every step uniformly, penalizing correct early actions in a losing trajectory as severely as the causal error itself. We propose Divergence-Weighted DPO(dwDPO), a drop-in replacement that detects the first divergence step between competing trajectories using implicit rewards already computed by DPO, then concentrates the loss around it via an asymmetric kernel with a closed-form scale factor that preserves the Bradley-Terry preference model. dwDPO recovers standard DPO as a special case and provably achieves higher gradient concentration at the critical step than both uniform and monotonic-discount baselines. Extensive experimental results across six deep-research benchmarks demonstrate the superiority of dwDPO over state-of-the-art open-source baselines of comparable scale at no additional inference cost.

Abstract:
Recent advances in image generation and editing have been driven by powerful pretrained unified models, yet most existing approaches remain fundamentally raster-centric: they represent images as opaque pixels without natively modeling transparency. This limitation restricts their applicability to a broad range of real-world applications. In this work, we present Qwen-Image-Layered 1.5, a pretrained unified image generation and editing model with native transparency awareness. By jointly modeling visual appearance and transparency, our framework supports a wide range of both raster-centric and transparency-aware tasks within a single model. Notably, we observe that the model can perform transparent image editing in a zero-shot manner. To systematically evaluate this new task, we introduce TransEdit-Bench, the first benchmark specifically designed for transparent image editing. Extensive experiments across various tasks demonstrate strong performance and highlight the great potential of transparency-native image modeling as a unified paradigm for future image synthesis and editing.

Abstract:
Mixed-precision quantization (MPQ) assigns a different bitwidth to each linear layer of a large language model (LLM) to minimize the quantization-induced quality loss under a fixed budget, but Mixture-of-Experts (MoE) models contain these layers in every expert of every MoE block, so the allocation space grows far larger than in a dense model. Existing methods either allocate within each block under a uniform per-block budget, or allocate across blocks through an additive proxy, and neither directly optimizes a model-level objective over the choices that couple the blocks. We propose Q-Strata, a bi-level allocator that ranks within-block assignments with a cheap proxy and allocates across blocks with a model-level objective evaluated on the assembled quantized model. Its inner stage caches a Pareto frontier of candidates per block over finely spaced budgets, leaving the outer stage to set one budget per block instead of a bitwidth for every linear layer. With the search reduced to one budget per block, the outer stage optimizes this model-level objective directly, capturing the inter-block coupling that additive proxies miss. On Mixtral-8x7B-Instruct, Qwen1.5-MoE-A2.7B, and DeepSeek-V2-Lite, Q-Strata consistently achieves lower WikiText2 perplexity than uniform-bitwidth GPTQ and the state-of-the-art MoE MPQ methods MxMoE and GEMQ in the low-bit regime. The code is available at https://github.com/snu-mllab/Q-Strata/tree/main.

Abstract:
Understanding which countries are represented in NLP datasets is essential for identifying gaps, targeting data collection, measuring progress, and informing AI policy. However, geographic metadata is very rarely available, and country-level representation is often hidden behind broad language-level claims. We introduce AtlasNLP, a country-aware atlas of over 13,000 NLP dataset records across normalized NLP task categories, tracking both the populations represented and where datasets are produced. AtlasNLP includes AtlasNLP-Gold, a human-curated reference set, and AtlasNLP-Core, an ACL-derived large-scale collection. Using this resource, we show that (1) dataset coverage is highly uneven across countries and tasks; (2) dataset production and representation are geographically asymmetric; and (3) language coverage does not imply geographic representation. These findings reveal blind spots in current dataset documentation practices and motivate more explicit geographic metadata for country-aware NLP evaluation.

Abstract:
LLMs are increasingly deployed in public-sector settings, where incorrect guidance can cause irreversible harm. We introduce CIVI, the first framework for diagnosing search agent failures in civic information. Its benchmark instantiation jointly spans cross-national, inter-jurisdictional government contexts (federal, state, and local) and functional categories from an internationally adopted United Nations standard. We evaluate ten frontier search agents and find that none matches an attentive human baseline. Alongside accuracy, CIVI measures search invocation rate, selective no-search accuracy, and how often agents cite authoritative government sources. To perform this diagnosis, we introduce ARISE, which decomposes agentic search failures into four mutually exclusive modes, isolated via source-injection ablation. ARISE attributes 72.1% of all observed failures to retrieval-bound causes rather than to gaps in the models’ parametric knowledge.

Abstract:
MLLM-as-a-Judge is conventionally validated by agreement with human annotations, but agreement against a culturally heterogeneous pool is pool-dependent: it silently conditions on whose judgments the aggregate represents. We introduce VOIR DIRE, a multimodal benchmark of 626 culturally paired image–prompt items spanning U.S. and mainland Chinese contexts across food, fashion, and architecture, with annotator pools that are within-pool reliable ( 𝛼 = 0.86 / 0.74 ) but cross-pool divergent on evaluation (Q1 r = -0.12 ). Across six MLLM judges, the bias decomposes into two failures: a positivity-floor calibration failure (compressed scale use) and an orientation failure (default to one cultural norm). On this corpus, contested items are sampled so the two pools split, and because models default to high ratings, the floor mechanically sides with the more-permissive Chinese reading. Persona prompting partially recovers calibration, but an orientation residual survives—evidence the tilt is not reducible to scale compression. Reference-pool in-context demonstrations deepen the orientation residual and inflate the high end rather than restoring use of the low end. Model origin adds a small additive tilt ( ∼ 0.10 MAE) that is approximately invariant under demonstration. We recommend reporting alignment against each reference pool separately and treating cross-pool divergence as a judge property.

Abstract:
Large language models (LLMs) have recently demonstrated improved machine translation performance over strong supervised baselines. This raises questions as to what mechanisms underlie how LLMs perform machine translation between languages. Motivated by recent interpretability findings—namely, that LLMs use massively multilingual latent feature representations to perform language modeling—we propose the interlingua hypothesis. The hypothesis holds that language models translate by reading a source sentence into a latent feature space, and generate a target sentence by reading from the latent feature space. We show three lines of evidence in support of this hypothesis: (1) variance in BLEU across language pairs is largely predictable from language-specific competences with no language pair–specific interaction terms; (2) many model components are causally influential in both monolingual tasks and translation tasks; (3) and fine-tuning on monolingual data recovers a large proportion of translation improvements relative to fine-tuning on aligned documents. Together, these provide convergent evidence in support of the interlingua hypothesis, and suggest new ways of understanding and improving how LLMs can be leveraged to perform translation tasks.

Abstract:
FactRot measures whether typed factual mentions survive iterative LLM rewriting. We follow each ledger fact through every hop of a rewrite chain and score it separately, instead of measuring aggregate drift in the rewritten text. We score 1,113 ledger facts from 25 Wikipedia passages across 4 models, 4 rewrite operations, and 3 prompt conditions, with chains reaching depth 12. A GEE with passage clusters finds per-hop decay (OR=0.743 per additional hop, p<.001 ). Type-specific faithfulness instructions do not measurably outperform generic faithfulness at hop 5 ( 𝛥=0.026 , 95% CI [0.022, 0.073]); both improve survival over neutral prompting by roughly 0.13-0.16 absolute. On the depth-12 neutral subset, answer-restate rewriting is much more destructive than direct paraphrasing, and the frontier preview model has the highest hop-12 survival among the four models. Exact fact-type rankings, however, are unstable across models (Kendall W=0.082 at hop 5).

Abstract:
Diffusion language models (D-LLMs) offer parallel denoising and bidirectional context, but hallucination detection for D-LLMs remains underexplored. Prior detectors developed for auto-regressive LLMs typically rely on single-pass cues and do not directly transfer to diffusion generation, where factuality evidence is distributed across the denoising trajectory and may appear, drift, or be self-corrected over time. We introduce TDGNet, a temporal dynamic graph framework that formulates hallucination detection as learning over evolving token-level attention graphs. At each denoising step, we sparsify the attention graph and update per-token memories via message passing, then apply temporal attention to aggregate trajectory-wide evidence for final prediction. Experiments across four D-LLMs spanning dense and Mixture-of-Experts architectures (7B—16B parameters) and multiple diffusion paradigms show consistent AUROC improvements over output-based, latent-based, and static-graph baselines, with single-pass inference and modest overhead. These results highlight the importance of temporal reasoning on attention graphs for robust hallucination detection in diffusion language models.

Abstract:
Existing methods for evaluating lexical competence in language models often rely on discriminative tasks or expensive, human-curated datasets. We introduce EvaLex, an automated framework to evaluate lexical competence in language models without gold labels. EvaLex employs a bidirectional approach: it first prompts a model to define a target word and subsequently tasks the model with retrieving that same word from its own generated definition in a separate inference session. We evaluate EvaLex on two Spanish datasets: CREA-10k-POS (7,784 words) and CREA-10k-Lemmas (4,594 lemmas), and we further validate against an expert-annotated DEA sample (100 words). We compare similarly sized LLMs (Llama, Mistral, Yi, Gemma) alongside Spanish-centric models (Gromenauer, Salamandra). EvaLex scores correlate strongly with expert manual annotations (Pearson r = 0.8539, Spearman rho = 0.8) across the four models for which manual annotations were available, supporting its use as a proxy for human judgments. Broader validation across more models and languages remains an open priority. We also observe a clear frequency effect, with Spanish-specific models remaining more robust on low-frequency terms. Overall, EvaLex provides a scalable and low-cost method for model selection and for diagnosing lexical gaps in low-resource and cross-lingual settings.

Abstract:
Multilingual dictionaries are among the most valuable documentary resources for low-resource and endangered languages, yet many remain available only as scans. For many decades their digitization and conversion into a machine-readable format was nearly impossible due to language-specific scripts, complex multi-column layouts full of entries with abbreviations and cross-references. Recent vision-language models offer a promising solution, but it is unclear how well they preserve characters, markup, and process lexicographic structure. We introduce MUDIDI, a two-stage framework for multi-lingual dictionary digitization. Stage 1 evaluates the quality of character recognition and markup preservation; Stage 2 focuses on dictionary entry segmentation with subsequent mapping into a machine-readable lexicographic schema, SIL’s Multi-Dictionary Formatter. We release a dataset that consists of human-annotated lexicographic entries collected from 30 public-domain dictionaries featuring diverse writing systems, language families, and formats. We benchmark OCR systems, general-purpose Large Language Models (LLMs), and Vision Language Models (VLMs) on the dataset, demonstrating superior performance of LLMs across most writing systems and languages in both stages, and provide practical guidelines on improving the results for more challenging scripts. Finally, we show that supplementing additional information such as dictionary introduction to the LLM can help improve the quality of the digitized dictionary.

Abstract:
Large language models (LLMs) are commonly evaluated on multiple-choice tasks that ask them to select an answer or to abstain on unanswerable questions. Neither setting captures frame-breaking: where the question is answerable, but the correct answer lies outside the framing of a candidate set of invalid answers. A useful model must reframe the task, rather than choose an invalid frame-compliant answer. We introduce REFRAME, a test suite of 150 scenarios in which a commonsense constraint invalidates their framing. Each REFRAME scenario is additionally paired with a direct knowledge probe of the same frame-breaking constraint, which we leverage to separate what a model knows from what it deploys. Across nine frontier LLMs, we observe a substantial knowledge-deployment gap with zero-shot prompting. A targeted reasoning prompt only partially closes this gap. Human annotators, by contrast, readily detect the constraint and reframe the problem. Together, these results characterize frame-breaking as a distinct reasoning challenge and demonstrate significant room for LLM improvement.

Abstract:
LLMs’ linguistically expressed confidence should faithfully reflect their intrinsic uncertainty. While recent work shows LLMs struggle to use epistemic markers (e.g., “it is likely...”) in a human-aligned fashion, it remains unclear whether models can apply their own linguistic confidence framework to associate markers with specific confidence levels in a stable and generalizable way, and how contextual features impact this ability. We conduct the first systematic study of this question, formalizing _marker internal confidence_ (MIC) as the estimated intrinsic confidence a model associates with a specific epistemic marker in a given task domain. We present 7 metrics to evaluate the stability of MICs within and across distributions. Applying our analysis framework to diverse models and tasks, we find that LLMs remain faithfully miscalibrated even under model-centric interpretation of marker meanings, struggling to differentiate markers by internal confidence across distributions despite preserving a somewhat consistent ranking order across tasks. This supplies critical, complementary evidence to existing work toward a holistic understanding of faithful calibration in LLMs, emphasizing the need for more aligned and stable marker use to improve trustworthiness and reliability.

Abstract:
Persistent vulnerabilities in safety alignment hinder the deployment of large language models (LLMs). Existing methods remain susceptible to jailbreak attacks, suggesting a fundamental, unaddressed flaw in current safety paradigms. In this work, we diagnose the root cause of this fragility, identifying a systemic issue we term Depth-wise Alignment Discrepancy. We find a fundamental misalignment: harmful vectors—latent representations predisposed to unsafe content—predominantly originate in the model’s lower layers, yet conventional alignment training concentrates its corrective gradients disproportionately on the top-most layers. This creates a brittle, "end-of-pipe" defense that is easily bypassed. To address this discrepancy, we propose SAGA, a framework that achieves robust safety through two synergistic innovations. First, it leverages high-entropy safety data , constructed via an iterative entropy ascent procedure, to provide the deep semantic signals necessary to reach the source of harmfulness. Second, it introduces a novel Synergistic Gradient Scaling (SGS) mechanism to explicitly reshape the gradient flow, ensuring these corrective signals are precisely applied to the identified vulnerable layers. Extensive experiments on five LLMs against six distinct jailbreak attacks demonstrate SAGA’s superiority, reducing attack success rates (ASR) by 21%–63% compared to state-of-the-art baselines. Our method preserves downstream task accuracy while introducing minimal computational overhead (<3%).

Abstract:
Large Vision-Language Models (LVLMs) demonstrate strong performance on multiple-choice question answering (MCQA), yet their reliability under biased conditions remains underexplored. We investigate length bias, the systematic preference for longer answer options, through controlled benchmarks spanning fine-grained visual domains. LVLMs consistently favor longer options even when visually inconsistent, with accuracy dropping 2–38% when correct answers are shorter and increasing 1–70% when longer. This bias persists across model scales. Through layer-wise mechanistic analysis, we reveal that length bias emerges from late-layer suppression of visual evidence in favor of language-side length priors. Blurry-image experiments show that bias intensifies when visual evidence weakens, while abstention tests reveal that LVLMs struggle to refuse incorrect predictions. We propose and evaluate multiple inference-time mitigation strategies; while these provide partial improvements, none achieves deployable bias mitigation without relying on oracle information unavailable at inference time. These results underscore that inference-time interventions are insufficient, pointing to the need for architectural solutions and bias-aware training.

Abstract:
Minimum Bayes Risk (MBR) decoding enables high-quality text generation by selecting the hypothesis that maximizes a utility metric over sampled pseudo-references. However, it is highly susceptible to metric overfitting: it can irregularly inflate the chosen utility metric at the direct expense of other unoptimized evaluation metrics. To mitigate this, we introduce SVD-MBR, which frames the pairwise utility matrix as a noisy information signal. By computing a low-rank approximation via Singular Value Decomposition (SVD) and retaining only the top- k components, we effectively decouple true consensus from metric noise. Experiments demonstrate that SVD-MBR successfully regularizes decoding, yielding substantial gains across a range of generalized metrics. Furthermore, we reveal that this denoising is metric-dependent: neural metrics encode a robust low-rank consensus ideal for SVD, whereas surface-level metrics struggle to separate signal from metric noise.

Abstract:
Current AI-driven game development has made substantial progress in asset generation, gameplay design, and web-based game coding, yet project-level code engineering on professional game engines remains largely unexplored due to the absence of large-scale datasets and deterministic evaluation methods. We present JamSet and JamBench, the first project-level game code framework dataset and benchmark built on a professional game engine. Our key insight is that Game Jam competitions, community events where developers build complete games under tight time constraints, yield thousands of open-source projects suitable for this purpose. Building on the Godot engine’s text-based format and headless execution mode, we design a deterministic verification pipeline from file integrity to runtime behavior collection, distilling 8,133 verified projects from over 240,000 repositories. Of these, 300 manually verified projects form JamBench; the rest constitute JamSet. JamBench defines theme-driven generation and code completion tasks, evaluated through a pipeline combining compilation pass rates, Structural Completeness Score (SCS), and Behavioral Alignment Score (BAS). Evaluation of 9 frontier models reveals a capability cliff as project scale increases, with runtime pass rates dropping from 80.4% on small projects to 5.7% on large ones(Task2a). Code Agents improve compilation rates yet yield no gains in runtime behavioral quality, indicating that the bottleneck lies in architectural design rather than syntactic correctness. Experiments validate JamSet as effective training data. All data and code are publicly available.

Abstract:
LLM Watermarking for clinical text requires evaluating detection capability alongside task performance. While existing benchmarks focus on accuracy, we show this misses critical quality degradation in Electronic Health Record (EHR) transformation tasks, where technical notes are simplified for patient understanding. We evaluate five watermarking methods across two EHR transformation tasks spanning five clinical note types, developing a multi-aspect framework that measures accuracy, style, and readability. We adapt linguistic distance of Part-of-Speech tags (LD PoS ) to quantify style shifts and enable mechanistic analysis. Results reveal a fundamental watermarking-induced helpfulness degradation and its detectability tension: EHR transformation achieves only 7.7%–24.2% detection rates under default parameters (vs. > 90% in narrative tasks). Tuning for practical detection degrades helpfulness by 24.6%–32.5%. Mechanistic analysis via LD PoS shows watermarking systematically shifts style toward generic terms and connected structures, degrading helpfulness even when accuracy is preserved.

Abstract:
Vision-language models (VLMs) have demonstrated impressive capabilities but suffer from substantial computational overhead, as vision tokens dominate the input sequence. This motivates vision token compression as a key direction to alleviate the burden. However, with the emergence of hybrid architectures incorporating linear attention (, Qwen3.5), prior methods designed for softmax attention struggle to generalize. Our analysis reveals that both attention- and similarity-based approaches suffer notable performance degradation, underscoring the urgent need for compression methods tailored to this regime. To this end, we propose V-CoLA , an efficient training-free token compression framework specifically designed for linear attention. V-CoLA introduces a novel uniqueness-aware importance criterion for identifying critical vision tokens, coupled with an adaptive token merging strategy that performs compression. All components are optimized at the implementation level to remain compatible with the chunk-wise parallelism of linear attention, ensuring strong practical value. Extensive experiments across multiple benchmarks demonstrate the superiority of V-CoLA: it achieves 99.5% of the original performance with only 50.0% of vision tokens, and over 88.0% with as few as 12.5%, while delivering a 1.86 × to 6.15 × prefill speedup.

Abstract:
Information retrieval (IR) increasingly targets open-ended queries that admit diverse perspectives. Existing IR benchmarks, however, focus primarily on closed-ended queries, while even open-ended benchmarks largely consist of queries whose supporting documents span a single subject domain and modality. We introduce Multi 3 IR, a benchmark that evaluates how well retrievers cover the multifaceted perspectives of open-ended queries across diverse domains and modalities. It comprises 104.9K Stack Exchange queries, each annotated with perspective descriptions that capture the query’s implicit viewpoints. We further propose SPIN, a parameter- and label-efficient method that learns noise vectors to steer embeddings toward diverse yet meaningful semantic directions. Experiments show that existing multimodal retrievers suffer from single-perspective bias, while SPIN substantially improves perspective coverage on Multi 3 IR and generalizes well to unseen open-ended IR benchmarks. The dataset and experimental code are available at https://github.com/seokwon99/Multi3IR.

Abstract:
Effective workplace negotiation requires balancing multiple objectives, including achieving task goals, preserving professional relationships, and resolving conflicts constructively. However, misunderstandings, misaligned preferences, and interpersonal friction often impede successful outcomes. Politeness mitigates these challenges by fostering trust, reducing tension, and preventing escalation, and persuasive communication helps overcome resistance, align preferences, and guide participants toward mutually beneficial agreements. Motivated by these insights, we present DIPLOMAT, a dialogue system for polite and persuasive workplace negotiation. To support its development, we introduce PROWESS, a dataset of multi-turn workplace negotiation dialogues generated via a multi-agent framework and enriched with negotiation strategies, politeness levels, and persuasive strategies. DIPLOMAT is trained using Dialogue-Span-Aware Direct Preference Optimization (DSA-DPO), a novel preference learning objective that identifies key dialogue spans for preference alignment. This enables DIPLOMAT to generate contextually coherent responses that employ intended negotiation strategies, maintain politeness, and incorporate effective persuasion strategies throughout interactions. Automatic and human evaluations on PROWESS confirm that DIPLOMAT consistently outperforms baselines in generating coherent, polite, and persuasive negotiation responses.

Abstract:
Sycophancy, where an LLM becomes overly agreeable, deferential, or flattering toward the user, has been identified as a major failure mode. While post-training has been implicated as a cause, how it internally gives rise to sycophantic behavior remains insufficiently understood. Post-training transforms the model’s output from text completion into utterances directed toward an addressee, thereby giving the output addressivity. We hypothesize that the accompanying pressure to follow the addressee’s instructions and align with their preferences links this addressivity to sycophancy. To test this, we use Sparse Autoencoders to extract addressivity directions from pre-trained and post-trained models and examine their relationship with sycophancy. Steering models with addressivity directions can amplify or suppress sycophancy, and this effect is observed only in post-trained models. Furthermore, comparing the internal mechanisms of both model types confirms that the association between addressivity and sycophancy is strengthened through post-training. Finally, through a controlled post-training experiment, we find that models are sycophantic not toward a specific addressee but toward any recognized addressee, suggesting that the presence of an addressee, rather than the addressee’s identity, is what elicits sycophancy.

Abstract:
Multilingual LLM benchmarks omit Slovak, a morphologically rich West Slavic language of five million speakers, or cover it only by machine translation. We present sk-bench, a native-first Slovak benchmark with 30 datasets (33 scored task variants) across ten skill categories. Eleven resources are introduced or first packaged for generative-LLM evaluation, including IFEval-SK with Slovak-adapted instruction checkers and native Chiby/SKJ1 resources for Slovak grammar and morphology. We evaluate 55 open- and closed-weights models under one harness. The best open model trails proprietary APIs by 12.6 points. Model rankings are similar for native and translated closed-form data ( 𝜌≥0.98 ), though translation separates the strongest models less well. By contrast, human-authored and LLM-generated QA questions rank models differently ( 𝜌=0.72 ). For Qwen3-14B, continued Slovak pretraining lowers the overall score by 13.9 points. A small instruction set restores three quarters of that loss. Test-time reasoning improves scores by 8.5 to 12.5 points for models of 9B and above. Together, these findings suggest four design lessons for other under-resourced languages: use native data where translation fails, plan instruction repair after language adaptation, enable test-time reasoning before scaling up, and avoid overinvesting in target-language prompts. We release the data and code at https://github.com/slovak-nlp/sk-bench

Abstract:
Personalized large language models adapt responses to users’ preferences and social attributes, but can introduce substantial universal truth inconsistencies across social groups, where some groups systematically receive less accurate responses on objective tasks. Existing alignment methods either ignore personalization or mainly focus on subjective preference alignment, largely overlooking fairness and consistency in universal truths. To address this gap, we study Truth-Invariant Alignment (TIA), an alignment problem for personalized LLMs that aims to ensure universal truths remain consistent across social groups while preserving personalization. We propose TriAlign , the first offline multi-agent reinforcement learning (MARL) framework for TIA, where each social group is modeled as an agent interacting. TriAlign jointly optimizes universal truth accuracy, cross-group truth consistency, and personalization through a fairness-aware objective and an explicit inconsistency penalty. Experiments across diverse benchmarks demonstrate that TriAlign achieves a stronger balance among these three objectives than strong baselines, reducing universal truth disparities across social groups while improving both objective task performance and personalization quality.

Abstract:
Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshots to become a buildable codebase with shared components and working navigation. This project-level setting exposes three limits of existing design-to-code benchmarks: they focus on single-page generation rather than complete codebases, cannot evaluate cross-page navigation, and do not measure project-wide maintainability. We introduce MobileForge, the first benchmark for project-level multi-screen mobile app generation, comprising 29 real mobile apps, 309 human-reviewed screens, structured page-relationship annotations, and 701 navigation test specifications. MobileForge supports five-axis evaluation of build, navigation, visual fidelity, code maintainability, and efficiency. We also propose state-isolated navigation testing to avoid cascading failures in navigation evaluation and an anchor-referenced list-wise visual evaluation protocol to improve visual-judge reliability. Across 174 end-to-end runs on six frontier multimodal LLMs, current models can build mobile-app projects that compile and reach the correct pages, but interactive navigation remains unreliable and visual fidelity and maintainability still lag. The benchmark and supporting materials are available at https://anonymous.4open.science/r/MobileForge-E2E5/ .

Abstract:
LLM judges are increasingly used to evaluate open-ended responses, but their scores depend strongly on the rubrics that condition them. A vague rubric asking for a response to be “helpful and factual” can reward polished answers that invent facts or violate user intent. We treat reusable rubrics as measurement specifications: changing the rubric changes the response quality measurement induced by a fixed judge. We introduce PReMISE , a framework that, given pairwise human-preference data, (i) discovers a policy-level rubric set, and (ii) audits any rubric set under LLM-judge use along four axes: structural adequacy, reliability, preference fit, and adversarial robustness. We evaluate PReMISE on four public human-preference corpora – UltraFeedback, Arena-Expert-5K, HelpSteer3, and HH-RLHF – across six rubric sources and three open-weight judges spanning 32B to 1T parameters. Across rubric sources no raw source is simultaneously reliable, preference-predictive, and adversarially robust; and high inter-rater agreement does not imply low exploitability. PReMISE is the only rubric source to score non-trivially on applicability, specificity, and effective dimensionality simultaneously. We contribute two audit-targeted repair operations: preference-rank selection raises held-out cross-source, cross-judge mean accuracy on paired responses from 65.0% to 68.6%, competitive with the strongest rubric-discovery baselines and leading on two of three judges in our cross-judge sweep; reliability-constrained refinement reduces the rate at which exploit responses receive high scores from 46.4% to 36.0% with little change in inter-judge agreement ( 𝛼=.531→.519 ).

Abstract:
Benchmark saturation and data contamination increasingly obscure genuine scientific reasoning in frontier LLMs. We introduce ScienceArena , an olympiad-style benchmark from thirteen public science competitions in physics, chemistry, and biology, including IPhO and IChO 2025–2026, IBO 2023, USAPhO 2026, and USNCO 2025. Its open-ended, multi-step problems use process-credit rubrics, making faithful scoring difficult. We build ScienceArena through an expert-audited digitization pipeline that converts official exams, figures, solutions, and rubrics into structured items verified by olympiad medalists. To scale evaluation beyond costly human grading, we calibrate LLM-as-judge against medalist ground truth on archived answers from five models across IPhO and IChO; two strong judges stay within one point of expert total scores. Medalist notes show that failures often stem from visual grounding, structure fidelity, and global problem control rather than missing terminology. Evaluating fourteen recent LLMs with interleaved solving, we find that top models obtain medal-equivalent rubric scores on several public international exams, while chemistry and long-horizon consistency remain key bottlenecks. We provide an interactive demo .

Abstract:
Scaling Large Language Models (LLMs) typically relies on increasing the number of parameters or test-time computations to boost performance. However, these strategies are impractical for edge device deployment due to limited RAM and NPU resources. Despite hardware constraints, deploying performant LLM on edge devices such as smartphone remains crucial for user experience. To address this, we propose MeKi (Memory-based Expert Knowledge Injection), a novel system that scales LLM capacity via storage space rather than FLOPs. MiKi equips each Transformer layer with token-level memory experts that injects pre-stored semantic knowledge into the generation process. To bridge the gap between training capacity and inference efficiency, we employ a re-parameterization strategy to fold parameter matrices used during training into a compact static lookup table. By offloading the knowledge to ROM, MiKi decouples model capacity from computational cost, introducing zero inference latency overhead. Extensive experiments demonstrate that MiKi significantly outperforms dense LLM baselines with identical inference speed, validating the effectiveness of memory-based scaling paradigm for on-device LLMs. Code is available at https://github.com/ningding-o/MeKi.

Abstract:
Memory-based self-evolution has emerged as a promising paradigm for coding agents. However, existing approaches typically restrict memory utilization to homogeneous task domains, failing to leverage the shared infrastructural foundations, such as runtime environments and programming languages, that exist across diverse real-world coding problems. To address this limitation, we investigate Memory Transfer Learning (MTL) by harnessing a unified memory pool from heterogeneous domains. We evaluate performance across 6 coding benchmarks using four memory representations, ranging from concrete traces to abstract insights. Our experiments demonstrate that cross-domain memory improves average performance by 3.7% over zero-shot setting and even outperforms the strongest self-evolving baseline, primarily by transferring meta-knowledge, such as validation routines, rather than task-specific code. Importantly, we find that abstraction dictates transferability; high-level insights generalize well, whereas low-level traces often induce negative transfer due to excessive specificity. Furthermore, we show that transfer effectiveness scales with the size of the memory pool, and memory can be transferred even between different models. Our work establishes empirical design principles for expanding memory utilization beyond single-domain silos.

Abstract:
As large neural models have become better at language tasks, researchers are increasingly building multi- and omnimodal models that handle more than one modality of data. One example is the expansion of speech recognition models to audio-visual data, the goal being noise mitigation and multimodal subtitling. While performance and bias have been studied extensively in the single-modality regime, it is unknown how new modalities affect this, even though they produce biases in humans. We therefore propose the first bias evaluation of multimodal speech recognition, where we create videos pairing different faces with the same audio using lip-sync, and measure changes in speech transcription accuracy. We find large quality-of-service differences across mWhisper-Flamingo and Gemini models, with drops of up to 4.05 word error rate points, across self-declared gender, ethnicity, and their intersection. Our findings point to a priority for developers to evaluate, fix, and communicate such limitations, as providing more signals through additional modalities is not necessarily better, and may even lead to biased outcomes.

Abstract:
Large vision-language models (LVLMs) demonstrate strong performance on diagram understanding benchmarks, yet they still struggle to understand relationships between elements, particularly those represented by directed edges between nodes. To investigate the causes of this limitation, we probe the internal representations of LVLMs using a synthetic diagram dataset based on directed graphs. Our linear probing experiments reveal that signals about nodes and edges are linearly decodable from the hidden states of LVLMs, even for aspects that LVLMs struggle with in VQA, such as edge direction and connectivity between nodes. These results reveal a gap between internal signals and model outputs. We further perform causal interventions on the model’s hidden states, showing that, for some aspects, the signals identified through probing causally contribute to the model’s final predictions. These findings also hold on flowcharts, not only on synthetic diagrams.

Abstract:
While language models have demonstrated remarkable capabilities, their performance remains disproportionately skewed towards high-resource languages. This disparity is particularly acute for the 700 million inhabitants of Southeast Asia, where vast linguistic diversity remains under-represented in existing benchmarks, often suffering from limited language coverage or narrow task scopes. To address this, we introduce DeepSEA, a comprehensive benchmark constructed from established datasets through professional human translation into 12 Southeast Asian languages, spanning both high-resource (e.g., Indonesian, Thai, Vietnamese) and low-resource (e.g., Burmese, Khmer, Lao) languages across five distinct tasks: mathematical reasoning, scientific reasoning, question answering, abstractive summarisation, and machine translation. Extensive evaluation of frontier proprietary and openweights models reveals that while frontier models like Gemini-3.7-Flash maintain robustness across resource tiers, smaller models face severe performance degradation on low-resource languages. Crucially, models perform better in reasoning and summarisation when pivoted through English rather than target languages, exposing a critical dependency on English representations, especially for low-resource languages. Beyond these observations, we provide a cross-lingual error analysis and a cross-script tokeniser fertility analysis. We hope these findings shed light on the development of next generation inclusive multilingual models.

Abstract:
Translated versions of English-origin benchmarks such as MMLU are widely used to evaluate large language models in non-English languages, yet it remains unclear whether they measure the same constructs as natively created benchmarks. We frame this as a proxy validity question and compare DeepL-translated MMLU proxies against six native benchmarks in seven pairings across five Nordic and Baltic languages, evaluating 29 models on each pairing. Beyond ranking agreement, we test incremental proxy validity: does the translated score add predictive value for native performance after controlling for model size and language proficiency? Proxy validity varies substantially across benchmark pairings. Applying the same robustness criteria to all seven pairings, we find four robust effects, two marginal effects, and one null result. The strongest evidence occurs for curriculum-overlap benchmarks such as Skolprov ( 𝛥 R 2 =.128 , 95% CI [.038,.253] ), whereas translated MMLU provides no significant incremental value for Sweden-specific factual knowledge in SwedishFacts ( p=.242 ). Because SwedishFacts is also the only open-ended benchmark, we cannot fully separate the effects of local content and response format. Overall, the results indicate a clear pattern: translated benchmarks provide a better proxy when their source content overlaps with the native evaluation target, while locally anchored knowledge is less reliably captured.

Abstract:
Large language models (LLMs) frequently produce confident yet factually incorrect responses when user inputs contain misleading premises, a phenomenon we attribute to fact perturbations in the input. Existing approaches to hallucination mitigation typically assume reliable user inputs, overlooking how such factual errors can actively mislead model reasoning. To address this vulnerability, we propose DEDUCE, a three-stage framework that transforms LLMs from passive responders into proactive error correctors. DEDUCE operates in three stages: (1) detect errors through fine-grained fact extraction and verification; (2) devise correction strategies via multi perspective deliberation; and (3) correct misconceptions while delivering reliable answers. We also present MisFactQA, a dataset containing factual errors of varying degrees, and propose new metrics for evaluating model robustness. Experiments on TruthfulQA, FalseQA, and our MisFactQA benchmark demonstrate that DEDUCE significantly improves both accuracy and error correction capability. Consistent gains across Qwen, LLaMA, and Gemma families confirm its effectiveness and scalability.

Abstract:
Large reasoning models (LRMs) produce reasoning traces (RTs) that often contain sensitive information. These leaky thoughts are difficult to control and frequently violate explicit privacy directives. Because RTs can be exposed through prompt injection attacks, this becomes a direct privacy risk to the user. We approach this as a controllability problem: since privacy directives are themselves instructions, improving instruction-following (IF) within the RT provides a direct path to reducing privacy leaks. To this end, we introduce an SFT dataset that teaches models to follow general instructions throughout their reasoning process, and propose Staged Decoding, a simple decoding strategy that decouples RT and answer generation using separate LoRA adapters to maximize IF of each component. We evaluate our approach on six models from two families (1.7B-14B parameters), across two IF benchmarks and two privacy benchmarks. Our method yields substantial improvements, with gains of up to 20.9 points in IF and 51.9 percentage points on privacy benchmarks, though these can come at the cost of task utility due to the trade-off between reasoning performance and IF. Our results show that improving IF in LRMs can significantly enhance privacy, suggesting a promising direction for future privacy-aware LRMs.

Abstract:
Qualitative interviews provide essential insights into human experiences when they elicit high-quality responses. While qualitative and NLP researchers have proposed various measures of interview quality, these measures lack validation that high-scoring responses actually contribute to the study’s goals. In this work, we identify, implement, and evaluate 10 proposed measures of interview response quality to determine which are actually predictive of a response’s contribution to the study findings. To conduct our analysis, we introduce the Qualitative Interview Corpus, a newly constructed dataset of 343 interview transcripts with 16,940 participant responses from 14 real research projects. We find that direct relevance to a key research question is the strongest predictor of response quality. We additionally find that two measures commonly used to evaluate NLP interview systems, clarity and surprisal-based informativeness, are not predictive of response quality. Our work provides analytic insights and grounded, scalable metrics to inform the design of qualitative studies and the evaluation of automated interview systems.

Abstract:
Large language models (LLMs) now play a central role in code generation, yet they continue to hallucinate, frequently inventing non-existent libraries. Such library hallucinations are not just benign errors: they can mislead developers, break builds, and expose systems to supply chain threats such as slopsquatting. Despite growing awareness of these risks, there is limited understanding of how library hallucinations manifest under realistic usage conditions. To fill this gap, we present the first systematic study of how user-level prompt variations influence library hallucinations in LLM-generated code. Across seven diverse LLMs, we analyse library name hallucinations (invalid imports) and library member hallucinations (invalid calls from valid libraries), examining the effects of realistic developer language and controlled user mistakes, including misspellings and fabricated libraries or members. Our findings expose systemic vulnerabilities: one-character misspellings trigger hallucinations in up to 26% of tasks; fabricated library names are accepted in up to 99%; and time-based prompts induce hallucinations in up to 84%. Grounded in the highest-risk prompts identified in our study, we introduce LibHalluBench, a benchmark that enables a systematic and reproducible evaluation of these library hallucinations. Our findings underscore the fragility of LLMs to natural prompt variation and highlight the urgent need for safeguards against library-related hallucinations and their downstream risks.

Abstract:
Realistic text-to-SQL workflows often require joining multiple tables. As a result, accurately retrieving the relevant set of tables becomes a key bottleneck for end-to-end performance. We study an open-book setting where queries must be answered over large, heterogeneous table collections pooled from many sources, without clean scoping signals such as database identifiers. Here, dense retrieval (DR) achieves high recall but returns many distractors, while join-aware alternatives often rely on extra assumptions and/or incur high inference overhead. We propose CORE-T, a scalable, training-free framework that enriches tables with LLM-generated purpose metadata and pre-computes a lightweight table-compatibility cache. At inference time, DR returns top-K candidates; a single LLM call selects a coherent, joinable subset, and a two-step additive adjustment stage restores strongly compatible tables. Across Bird, Spider, MMQA, and Beaver, CORE-T improves over DR by up to 22.7 points in table-selection F1 while returning up to 40% fewer tables, and by up to 24.4 points in multi-table execution accuracy, and uses 1.64–4.20× fewer total selection tokens than LLM-intensive baselines.

Abstract:
Peer review plays a central role in the NLP publication process, but is susceptible to various biases. Here, we study language-of-study (LoS) bias: the tendency for reviewers to evaluate a paper differently based on the language(s) it studies, rather than its scientific merit. Despite being explicitly flagged in reviewing guidelines, such biases are poorly understood. Prior work treats such comments as part of broader categories of weak or unconstructive reviews without defining them as a distinct form of bias. We present the first systematic characterization of LoS bias, distinguishing negative and positive forms, and introduce the human-annotated dataset LOBSTER (Language-Of-study Bias in ScienTific pEer Review) and an LLM-based detection pipeline achieving 87.37 macro F1. We analyze 15,645 reviews to estimate how negative and positive biases differ with respect to the LoS, and find that non-English papers face substantially higher bias rates than English-only ones, with negative bias consistently outweighing positive bias. Finally, we identify four subcategories of negative bias, and find that demanding unjustified cross-lingual generalization is the most dominant form. We publicly release all resources to support work on fairer reviewing practices in NLP and beyond.

Abstract:
As the key-value(KV) cache size scales with context length, accessing large KV cache each step and substantial GPU memory demand challenge us to deploy LLMs with long contexts.Various sparse attention methods have been proposed and offloading-based KV retrieval preserves entire KV cache in CPU memory and dynamically retrieves most relevant KV pairs for each decoding step, which performs higher quality and effectively reduces GPU memory consumption than other line works. However, exiting KV retrieval performs page-level to reduce estimation overhead, which introduces inaccurate KV selection and significant re- trieval overhead. We propose PRKV, a framework that both-optimizes algorithm and system for page-level KV retrieval with KV offloading. On the algorithm side, PRKV introduces hybrid KV selection that combines both static and dynamic KV selection strategies. On the system side, PRKV employs contiguous memory in- dexing and batched transfer optimizations to improve retrieval efficiency. Exper- iments demonstrate that PRKV improve accuracy across various scenarios and models, delivering up to 6.75× speedup compared to SOTA KV retrieval methods.

Abstract:
Vision–Language–Action (VLA) models achieve strong performance in robotic manipulation by leveraging pre-trained vision–language backbones. However, in downstream robotic settings, they are typically fine-tuned with limited data, leading to overfitting to specific instruction formulations and leaving robustness to paraphrased instructions underexplored. To study this gap, we introduce LIBERO-Para, a controlled benchmark that independently varies action expressions and object references for fine-grained analysis of linguistic generalization. Across seven VLA configurations (0.6B–7.5B), we observe consistent performance degradation of 22–52pp under paraphrasing. This degradation is primarily driven by object-level lexical variation: even simple synonym substitutions cause large drops, indicating reliance on surface-level matching rather than semantic grounding. Moreover, 80–96% of failures arise from planning-level trajectory divergence rather than execution errors, showing that paraphrasing disrupts task identification. Binary success rate treats all paraphrases equally, obscuring whether models perform consistently across difficulty levels or rely on easier cases. To address this, we propose PRIDE, a metric that quantifies paraphrase difficulty using semantic and syntactic factors. Our benchmark and code are available at https://github.com/cau-hai-lab/LIBERO-Para.

Abstract:
Dark humor often relies on subtle cultural nuances and implicit cues that require contextual reasoning to interpret, posing safety challenges that current static benchmarks fail to capture. To address this, we introduce a novel multimodal, multilingual benchmark for detecting and understanding harmful and offensive humor. Our manually curated dataset comprises 3,000 texts and 6,000 images in English and Arabic, along with 1,200 videos spanning English, Arabic, and language-independent (universal) contexts. Unlike standard toxicity datasets, we enforce a strict annotation guideline: distinguishing Safe jokes from Harmful ones, with the latter further classified into Explicit (overt) and Implicit (covert) categories to probe deep reasoning. We systematically evaluate state-of-the-art (SOTA) open- and closed-source models across all modalities. Our findings reveal that closed-source models significantly outperform open-source models, with a notable performance difference between English and Arabic in both, underscoring the critical need for culturally grounded, reasoning-aware safety alignment. The code and dataset are available at https://github.com/insait-institute/harm-or-humor. Content warning: This paper contains offensive text and meme imagery, video clips, audio, and other audiovisual material that may include sexual, violent, discriminatory, religious, disability-related, or historically sensitive material. Reader discretion is advised.

Abstract:
Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-consistent responses. However, the representational differences between unstructured and structured knowledge make LLMs inherently struggle to maintain logic consistency, leading to Logic Drift challenges in structured knowledge reasoning tasks such as Knowledge Graph Question Answering (KGQA). Existing methods address this limitation by designing complex workflows embedded in prompts to guide LLM reasoning. Nevertheless, these approaches only provide input-level guidance and fail to fundamentally address the Logic Drift in LLM outputs. Additionally, their inflexible reasoning workflows cannot adapt to different tasks and knowledge graphs. To enhance LLMs’ logic consistency in structured knowledge reasoning, we specifically target the logits output from the autoregressive generation process. We propose the Logits-to-Logic framework, which incorporates logits strengthening and logits filtering as core modules to correct logical defects in LLM outputs. Extensive experiments show that our approach significantly improves LLMs’ logic consistency in structured knowledge reasoning and achieves state-of-the-art performance on multiple KGQA benchmarks.

Abstract:
Human language is driven by unspoken beliefs and belief updates which are thus of critical importance for successful interactions between large language models (LLMs) and their users. In this paper, we evaluate the ability of LLMs to recognize unspoken beliefs made through implicatures and to recognize their updates through implicature cancellation: the pragmatic phenomenon whereby an utterance’s implied meaning is weakened or negated. We create the first expert-annotated implicature cancellation dataset, ImplicatureX, crowdsourced for human judgements of implicatures and their corresponding cancellations. We find that LLM belief update understanding lags behind that of humans, especially in more naturally-occurring scenarios. Additional control experiments suggest that successes in LLM belief updates may in part stem from a reliance on prior beliefs, and that failures in belief updates may depend on their type and on their form. Overall, our study suggests that current LLMs have not yet reached human-level understanding of communicative belief updates.

Abstract:
Large language models (LLMs) can correctly answer "When was Einstein born?" yet fail to provide the same date when the same question is embedded in a long-form query requesting multiple facts about Einstein’s life, revealing a fundamental inconsistency in how models access factual knowledge across task complexities. While models display impressive accuracy on factual question-answering benchmarks, the reliability gap between simple and complex long-form queries remains poorly understood, eroding their trustworthiness. In this work, we introduce Short-Long Form Alignment for Factual Question Answering (SLAQ, a controlled evaluation framework that compares LLMs’ answers to the same factual questions asked (a) in isolation (short) vs. (b) integrated into complex queries (long). Looking at 16 LLMs across 600 queries, we find a systematic misalignment of answers to the corresponding short and long queries. We further uncover momentum effects where consecutive correct or incorrect answers create self-reinforcing patterns. Through mechanistic analysis, we find that aligned facts activate overlapping model internals, and that metrics based on mechanistic similarity can predict short-long answer alignment with up to 80% accuracy. Our work establishes factual consistency over query complexity as an important aspect of LLMs’ trustworthiness and challenges current evaluation practices, which implicitly assume that good performance for simple factual queries implies reliability in more complex knowledge-seeking tasks too.

Abstract:
Temporal knowledge graph question answering (TKGQA) aims to answer time-sensitive questions by leveraging temporal knowledge bases. While Large Language Models (LLMs) demonstrate significant potential in TKGQA, current prompting strategies constrain their efficacy in two primary ways. First, they are prone to reasoning hallucinations under complex temporal constraints. Second, static prompting limits model autonomy and generalization, as it lacks optimization through dynamic interaction with temporal knowledge graph (TKG) environments. To address these limitations, we propose TKG-Thinker, a novel agent equipped with autonomous planning and adaptive retrieval capabilities for reasoning over TKGs. Specifically, TKG-Thinker performs in-depth temporal reasoning through dynamic multi-turn interactions with TKGs via a dual-training strategy. We first apply Supervised Fine-Tuning (SFT) with chain of thought data to instill core planning capabilities, followed by a Reinforcement Learning (RL) stage that leverages multi-dimensional rewards to refine reasoning policies under intricate temporal constraints. Experimental results on benchmark datasets with three open-source LLMs show that TKG-Thinker achieves state-of-the-art performance and exhibits strong generalization across complex TKGQA settings.

Abstract:
Despite the significant efforts devoted to aligning large language models (LLMs) with human values and ensuring safe deployment, recent work has revealed that LLMs remain vulnerable to adversarial jailbreak attacks that can bypass safety guardrails and elicit harmful responses. Many defense methods are proposed to detect jailbreaks but they are limited in their effectiveness to counter wide-range optimization based jailbreak mechanisms that can yield highly fluency-optimized or harmful semantic obfuscated prompts. To tackle this challenge, we propose a unified detection framework—SAFEGuard—which incorporates a hybrid fluency measurement based on cross-layer distribution distance and perplexity, and the analysis of harmful semantics through gradient matching. Our method is grounded in a paramount observation: high fluency prompts maintain their malicious intention close to harmful prompts while harmful semantic obfuscated prompts often inject gibberish token sequences. Our evaluation demonstrates that SAFEGuard consistently outperforms state-of-the-art baselines and achieves significant improvement in accuracy across different optimization-based jailbreaks. This underscores the effectiveness of SAFEGuard against evolving jailbreak attacks.

Abstract:
Speculative decoding (SD) has emerged as a promising approach to accelerate LLM inference without sacrificing output quality. Existing SD methods tailored for video-LLMs primarily focus on pruning redundant visual tokens to mitigate the computational burden of massive visual inputs. However, existing methods do not achieve inference acceleration comparable to text-only LLMs. We observe from extensive experiments that this phenomenon mainly stems from two limitations: (i) their pruning strategies inadequately preserve visual semantic tokens, degrading draft quality and acceptance rates; (ii) even with aggressive pruning (e.g., 90% visual tokens removed), the draft model’s remaining inference cost limits overall speedup. To address these limitations, we propose HIPPO, a general holistic-aware parallel speculative decoding framework. Specifically, HIPPO proposes: (i) a semantic-aware token preservation method, which fuses global attention scores with local visual semantics to retain semantic information at high pruning ratios; (ii) a video parallel SD algorithm that decouples and overlaps draft generation and target verification phases. Experiments on four video-LLMs across six benchmarks demonstrate HIPPO’s effectiveness, yielding up to 3.51 × speedup compared to vanilla auto-regressive decoding.

Abstract:
Large Language Models (LLMs) are increasingly used not only for instrumental tasks, but as always-available and non-judgmental confidants for emotional support. Yet what drives adoption and how users perceive emotional support interactions across countries remains unknown. To address this gap, we present the first large-scale cross-cultural study of LLM use for emotional support, surveying 4,641 participants across seven countries (USA, UK, Germany, France, Spain, Italy, and the Netherlands). Our results show that adoption rates vary dramatically across countries (from 20% to 59%). Using mixed models that separate country-level effects from demographic composition, we find that being aged 25-44, religious, married, and of higher socioeconomic status are predictors of positive perceptions (trust, usage, perceived benefits), with socioeconomic status being the strongest. English-speaking countries consistently show more positive perceptions than Continental European countries. We also collect a corpus of 731 real multilingual prompts from user interactions, showing that users mainly seek help for loneliness, stress, relationship conflicts, and mental health struggles. Our findings reveal that LLM emotional support use is shaped by a complex sociotechnical landscape and call for a broader research agenda examining how these systems can be developed, deployed, and governed to ensure safe and informed access. We share our code and data publicly [].

Abstract:
Discrete motion tokenization has recently enabled Large Language Models (LLMs) to serve as versatile backbones for motion understanding and motion-language reasoning. However, existing pipelines typically decouple motion quantization from semantic embedding learning, linking them solely via token IDs. This approach fails to effectively align the intrinsic geometry of the motion space with the embedding space, thereby hindering the LLM’s capacity for nuanced motion reasoning. We argue that alignment is most effective when both modalities share a unified geometric basis. Therefore, instead of forcing the LLM to reconstruct the complex geometry among motion tokens from scratch, we present a novel framework that explicitly enforces orthogonality on both the motion codebook and the LLM embedding space, ensuring that their relational structures naturally mirror each other. Specifically, we employ a decoder-only quantizer with Gumbel-Softmax for differentiable training and balanced codebook usage. To bridge the modalities, we use a sparse projection that maps motion codes into the LLM embedding space while preserving orthogonality. Finally, a two-stage orthonormal regularization schedule enforces soft constraints during tokenizer training and LLM fine-tuning to maintain geometric alignment without hindering semantic adaptation. Extensive experiments show that our framework improves the aggregated Average by 22.4% over the strongest baseline on HumanML3D and by 14.4% on KIT-ML, while ablations confirm the effectiveness of the tokenizer, projection, and regularization designs.

Abstract:
Communication protocol design is a central challenge in large language model-based multi-agent systems. Existing single-channel approaches face an inherent communication trilemma: text-based methods are interpretable but verbose, while latent-space methods are efficient but opaque and limited to single-round workflows. Inspired by multi-channel communication theory, we propose HyLaT, a hybrid latent-text communication protocol that transmits elaborate cognitive signals through a latent channel for efficiency, while expressing concise critical signals in natural language to preserve interpretability and precision. We introduce a two-stage training framework combining single-agent hybrid generation learning and multi-agent interactive co-training, enabling agents to generate and interpret hybrid messages across multiple rounds of interaction. Experiments demonstrate that HyLaT reduces communication overhead significantly while maintaining competitive task performance, with strong generalization and robustness across diverse settings.

Abstract:
Discrete diffusion language models (DDLMs) offer potential advantages for language generation, such as parallel decoding and iterative refinement, but they often underperform autoregressive (AR) models and exhibit higher training instability. Recent approaches incorporate AR training signals to stabilize training, yet this introduces strong sequential inductive biases that limit the benefits of non-autoregressive generation. In this work, we analyze the structural causes of degeneration in non-AR DDLMs. We show that standard diffusion formulations rely on an i.i.d. corruption process and factorized sampling structure that implicitly assume token exchangeability, which conflicts with the non-exchangeable dependency structure of natural language. This mismatch leads to degeneration under semantic sparsity, where contextual signals are insufficient. To guarantee stable language generation, we propose the Dependency-Coupling Principle , a structural constraint that preserves necessary contextual dependencies during diffusion generation and provides insight into the performance gap between AR and diffusion language models.

Abstract:
Large Language Models (LLMs) are evolving from performing end-to-end mathematical reasoning to integrating agentic intelligence. However, most existing math benchmarks evaluate only final answers. This outcome-oriented evaluation provides limited diagnostic value for identifying process-level failures or rigorous logic, failing to guide the transformation of LLMs into robust agents. To bridge this gap, we present a process-level benchmark designed to evaluate the inherent agentic mathematical reasoning abilities of LLMs. Our framework aligns problem-solving agentic behaviors with a structured taxonomy of reusable mathematical atomic capabilities. We design a comprehensive suite of planning, action, and feedback tasks across both textual and multimodal contexts, supported by an automated pipeline that synthesizes high-quality trajectories and produces fine-grained annotations via controlled LLM rewriting. Experiments reveal that models with stronger end-to-end accuracy can exhibit markedly different agentic capability profiles. This demonstrates that process-level evaluation is crucial for interpreting the true potential of LLMs and guiding the development of next-generation mathematical agents.

Abstract:
In the unsupervised self-evolution of Multimodal Large Language Models, the quality of feedback signals during post-training is pivotal for stable and effective learning. However, existing self-evolution methods predominantly rely on majority voting to select the most frequent output as the pseudo-golden answer, which may stem from the model’s intrinsic biases rather than guaranteeing the objective correctness of the reasoning paths. To counteract the degradation, we propose C ontinuous S oftened R etracing re S ampling ( CSRS ) in MLLM self-evolution. Specifically, we introduce a Retracing Re-inference Mechanism ( RRM ) that the model re-inferences from anchor points to expand the exploration of long-tail reasoning paths. Simultaneously, we propose Softened Frequency Reward ( SFR ), which replaces binary rewards with continuous signals, calibrating reward based on the answers’ frequency across sampled reasoning sets. Furthermore, incorporated with Visual Semantic Perturbation ( VSP ), CSRS ensures the model prioritizes mathematical logic over visual superficiality. Experimental results demonstrate that CSRS significantly enhances the reasoning performance of Qwen2.5-VL-7B on benchmarks such as MathVision. We achieve state-of-the-art (SOTA) results in unsupervised self-evolution on geometric tasks.

Abstract:
Large language models are increasingly deployed as coding agents, shifting safety from individual responses to action sequences. Existing benchmarks, however, primarily assess whether models refuse unsafe prompts, leaving impacts on stateful workspaces largely unexamined. We present SABER, a benchmark for environment-aware operational safety that places models in realistic agent-style projects and evaluates safety from the final environment state after a sequence of actions. Beyond binary safety-violation reports, SABER categorizes violations by cause, enabling analysis of model-specific safety profiles. Our evaluations show that even the best-performing model has more than a 54% harmful safety-violation rate (HSR), suggesting that current alignment remains insufficient for realistic project environments. SABER further reveals distinct safety profiles across models. Our benchmark is publicly available at https://github.com/sssr-lab/saber.

Abstract:
Reinforcement learning-based retrieval-augmented generation (RAG) methods enhance the reasoning abilities of large language models (LLMs). However, most rely only on final-answer rewards, overlooking intermediate reasoning quality. This paper analyzes existing RAG reasoning models and identifies three main failure patterns: (1) information insufficiency: failure to retrieve adequate support; (2) faulty reasoning: logical or content-level flaws despite sufficient information; (3) answer–reasoning inconsistency: a valid reasoning chain with a mismatched final answer. We propose TIRESRAG-R1, a novel framework using a think–retrieve–reflect process and a multi-dimensional reward system to improve reasoning and stability. TIRESRAG-R1 introduces: (1) a sufficiency reward to encourage thorough retrieval; (2) a reasoning quality reward to assess rationality and accuracy of the reasoning chain; (3) a reflection reward to detect and revise errors. It also employs a difficulty-aware reweighting strategy and training sample filtering to boost performance on complex tasks. Experiments on four multi-hop QA datasets show TIRESRAG-R1 outperforms prior RAG methods and generalizes well to single-hop tasks. Our code is available at https://github.com/probe2/TIRESRAG-R1.

Abstract:
Ensuring aligned agent behaviors in distributed open multi-agent systems remains challenging, especially as populations grow and unaligned agents may exist. We show that a single aligned agent can propagate cooperative behaviors to unmodified agents purely through natural-language interaction, a phenomenon we term Alignment Propagation. We study this in the Red-Black Game, a team-based iterated Prisoner’s Dilemma in which teammates deliberate and vote to determine their team’s collective action. By distilling the cooperative reasoning and persuasive dialogues of a teacher model into Qwen3-14B, we obtain a seed agent that, when placed among four unmodified teammates, more than doubles the cooperation rate from 24.8% to 62.2%, outperforming the teacher model and a vanilla Gemini-3.1-Pro. Remarkably, a seed trained exclusively on the Red-Black Game transfers zero-shot to Sugarscape, a spatially grounded survival simulation with pairwise trading, achieving a 91.5% trade success rate versus a 21.6% baseline. Our results reframe multi-agent alignment from an exhaustive per-agent training problem to a scalable social capability that can be engineered through strategic seed placement.

Abstract:
Large Reasoning Models (LRMs) benefit substantially from training on challenging competition-level questions. However, existing automated question synthesis methods lack precise difficulty control, incur high computational costs, and struggle to generate competition-level questions at scale. In this paper, we propose CoDiQ (Controllable Difficult Question Generation), a novel framework enabling fine-grained difficulty control via test-time scaling while ensuring question solvability. Specifically, first, we identify a test-time scaling tendency (extended reasoning token budget boosts difficulty but reduces solvability) and the intrinsic properties defining the upper bound of a model’s ability to generate valid, high-difficulty questions. Then, we develop CoDiQ-Generator from Qwen3-8B, which improves the upper bound of difficult question generation, making it particularly well-suited for challenging question construction. Building on the CoDiQ framework, we build CoDiQ-Corpus (44K competition-grade question sequences). Human evaluations show these questions are significantly more challenging than LiveCodeBench/AIME with over 82% solvability. Training LRMs on CoDiQ-Corpus substantially improves reasoning performance, verifying that scaling controlled-difficulty training questions enhances reasoning capabilities. We open-source CoDiQ-Corpus, CoDiQ-Generator, and implementations to support related research.

Abstract:
Mixed-integer programming (MIP) lies at the core of operations research and industrial optimization. While large language models (LLMs) have recently shown promise in automated MIP modeling from natural language, they prioritize semantic correctness but overlook formulation strength, severely bottlenecking the efficiency of downstream solvers. We propose FormuEvo, an LLM-guided evolutionary framework for automated discovery of solver-efficient MIP formulations. FormuEvo frames MIP formulation design as evolutionary optimization over the symbolic space of MIP formulations, represented as executable modeling programs, by iteratively generating, evaluating, and selecting stronger candidates via LLM-driven crossover, mutation, and repair operations. To move beyond blind exploration, FormuEvo introduces a solver-informed diagnosis mechanism that exploits fine-grained solver statistics as verbal gradients for targeted refinement. Additionally, a structured memory abstracts prior experience into reusable modeling strategies, avoiding redundant exploration while enabling zero-shot transfer to unseen problems and bootstrapping smaller LLMs. Experiments across diverse linear and non-linear problems demonstrate that FormuEvo discovers formulations that significantly outperform both expert-designed formulations and existing LLM-based approaches, accelerating solvers by up to 5.5 × , with distilled knowledge transferring effectively across problems and model scales.

Abstract:
Query-conditioned sparse KV selection reduces long-context inference cost by attending to a subset of historical KV entries at each decoding step. Existing selectors must wait for the target-layer query before scoring candidates and generating indices, placing routing on the immediate pre-attention path. We introduce EarlySelectKV, a training-free transformation that uses proxy queries from earlier hidden states to precompute selected KV indices while retaining the real target-layer query for final attention. Tests with RocketKV, Quest, and Loki show no consistent pattern of quality degradation across the evaluated settings. In the primary RocketKV evaluation, ES-RocketKV differs by +0.018 points on LongBench, approximately −0.01 points on RULER, and +0.25 percentage points on representative NIAH/Passkey settings. In a TensorRT-LLM deployment with CPU-offloaded routing, ES-RocketKV improves end-to-end output throughput by up to 12.4% and reduces GPU-resident memory by 11.1% of the combined KV/K-summary pool. This makes routing timing an explicit optimization target alongside KV subset selection in sparse long-context decoding. Code is available at https://github.com/sea-with-sakura/EarlySelectKV.

Abstract:
Seamless translation of natural language into embodied behaviors remains critical for open domain embodied dialogue. We formalize this task as mapping large language model text to synchronized speech and facial animations. Prevailing cascaded Text to Speech (TTS) and Talking Face Generation (TFG) pipelines suffer from prohibitive latency and poor cross platform generalizability. We present EmbodiChat, an end-to-end real-time dialogue system overcoming these bottlenecks. The key insight of EmbodiChat is that TTS acoustic representations capture pure “articulatory intent,” making them far more robust than audio-derived semantic features. Aligning facial generation directly onto this representation without additional adapters enables decoupled audio visual synthesis, fundamentally eliminating the serial latency of conventional audio first architectures. Concurrently, our framework decodes these representations into hardware agnostic action sequences. EmbodiChat achieves a stable Time-to-First-Audio (TTFA) of about 150 ms on a consumer GPU, significantly outperforming mainstream methods. Furthermore, it delivers a 3x speedup relative to its ablated audio driven baseline while maintaining comparable animation quality. Deployment across two distinct embodied agent platforms configurations demonstrates a robust, practically deployable text driven solution for embodied dialogue. The data, code and model will be released at https://github.com/JiangMingZi/Embodi_Chat

Abstract:
Test-time training (TTT) adapts a frozen large language model (LLM) at inference time by updating prompt-specific parameters before generation. It is especially appealing for long-context reasoning. However, vanilla TTT yields only marginal gains on modern LLMs. We trace this bottleneck to two coupled failures: the inner loop is unprimed, because LLMs are trained as static endpoints rather than initialization points for test-time gradient adaptation; and the proxy objective is misaligned, because prompt-masked next-token prediction (PM-NTP) rewards prompt memorization rather than answer correctness. We propose PrimeTTT, a lightweight TTT method that casts this adaptation problem as meta-learning. A compact hypernetwork emits a prompt-specific LoRA initialization for the inner loop, trained with a MAML-style second-order objective: the inner loop descends the test-time-available PM-NTP loss, while the outer loss is answer-conditioned NTP on the gold answer and is backpropagated through the unrolled inner steps. Across out-of-distribution benchmarks disjoint from training, PrimeTTT consistently outperforms strong TTT baselines and transfers across model families and scales.

Abstract:
Continuous-space reasoning methods replace discrete intermediate tokens with probability-weighted soft embeddings, aiming to preserve semantic information that greedy Chain-of-Thought decoding discards. We show that this promise is limited by local representation geometry and the latent semantic manifold. When the next-token distribution is low-entropy and concentrated, a soft token becomes nearly collinear with the Top-1 embedding and collapses toward greedy decoding. When the distribution is high-entropy, diffuse, or stochastically perturbed, the continuous state may leave the greedy path but drift away from the local scale of naturally occurring candidate representations. We diagnose these two regimes with step-level probes over representation divergence, local norm calibration, and behavior-cloud membership. To address them, we propose Manifold Thinking, a training-free inference framework that combines structured tangent movement away from the Top-1 direction with local norm-calibrated projection back to the candidate-token neighborhood. Across mathematical and programming benchmarks, Manifold Thinking improves over discrete CoT baselines and baselines covering the latest stochastic Soft Thinking strategies, suggesting that useful continuous reasoning requires geometric correction rather than unconstrained linear token mixing alone.

Abstract:
Diffusion Language Models (DLMs) have emerged as a promising alternative to autoregressive language models, showing strong non-autoregressive generation abilities. Although the masked denoising objective trains DLMs to recover tokens at arbitrary positions, current long-context usage still mostly follows a prefix-based paradigm: the full context is given as a prefix, and the answer is generated only at the end, leaving the model’s infilling ability underutilized. To investigate whether DLMs can read long contexts by infilling, we introduce Parallel Infilling Reading (PIR), a training-free method that inserts extraction slots into long contexts to generate intermediate reading states. Our empirical study shows that naive PIR improves over prefix-based DLM inference, but suffers from slot repetition, where multiple slots generate redundant evidence. To mitigate this issue, we propose Slot-Isolated PIR, which blocks direct slot-to-slot attention, and PIR-SFT, which uses fine-grained evidence supervision to teach DLMs effective evidence extraction while reading. Experiments on long-context question answering benchmarks show that our methods improve answer accuracy and generation efficiency, with increasingly larger gains as the context length grows.

Abstract:
Vision-language models (VLMs) are increasingly deployed in multi-turn settings where users may describe visual content with incorrect assumptions. Yet existing evaluations rarely isolate how models respond when the same visually grounded false premise persists across dialogue turns. We introduce FPCO-Dialog, a benchmark for evaluating correction and cooperation behavior in VLMs under repeated false premises. FPCO-Dialog contains 1,080 images and 10,800 question turns, stratified by visual complexity, object category, and false-premise class, and uses a 10-turn protocol in which a correct dialogue prefix is followed by repeated false-premise referring expressions. We evaluate 20 commercial and open-source VLMs with a model-agnostic protocol and CorrTP@K, a correction-rate metric over false-premise turns, scored by two independent detectors. FPCO-Dialog reveals substantial and persistent cross-model differences in aggregate correction tendency, model-specific turn-wise dynamics, and systematic variation across false-premise types under the benchmark’s substitution distribution. The dataset, evaluation protocol, model outputs, detector labels, and code are available at https://github.com/lab-klc/FPCO-Dialog.

Abstract:
Large language models (LLMs) trained with reinforcement objectives often achieve superficially correct answers via shortcut strategies, pairing correct outputs with spurious or unfaithful reasoning and degrading under small input perturbations. We introduce Coherence-Enhanced Policy Optimization (CE-PO), a model agnostic reward fusion framework that augments policy optimization with a differentiable Jacobian coherence signal along the generation pathway from prompt ( X ) to rationale ( Y ) to answer ( Z ). CE-PO estimates internal gradient influence through Jacobian-based sensitivities, uses a reshuffling-based residualization procedure to reduce shortcut-driven reward hacking, and fuses the resulting coherence score with task-accuracy via an L p combiner that controls the strength of coherence regularization. The unified reward integrates with PPO/GRPO without architectural changes. Across multiple reasoning benchmarks, CE-PO improves accuracy over baselines by 5.49 on average (up to 9.58), while increasing the model’s faithful reasoning capability (12.14 % on average across five backbones).

Abstract:
Real-world knowledge bases evolve over time and accumulate outdated statements, contradictions, and misinformation. LLMs have been reported to be vulnerable to such inconsistent contexts. In this paper, we propose Proposition-Structured RAG (ProStruct-RAG), a framework that continuously structures evolving and inconsistent external knowledge bases using timestamped propositions and inter-proposition relations. ProStruct-RAG represents a document collection as a graph whose nodes are propositions and whose edges capture temporal updates, contradictions, and support relations, retrieves query-relevant knowledge as a relational structure, and uses it for answer generation in RAG. Experiments on multiple open-book temporal QA benchmarks show that ProStruct-RAG outperforms baselines that do not use inter-proposition relations, suggesting that RAG over evolving and inconsistent knowledge benefits from organizing external knowledge before retrieval, rather than relying solely on inference-time reasoning. We release our code and Asteria, a new fictional temporal QA dataset designed to mitigate data contamination.

Abstract:
Long-horizon agentic tasks require large language models (LLMs) to iteratively retrieve, integrate, and maintain dispersed information across multi-turn interactions, but preserving all interaction histories leads to a continuously growing working context. Recent proactive context management methods allow models to edit their own working context with specialized tools, yet they still face three key limitations: (1) a limited toolset restricted to search, deletion, and summarization, with no support for global planning, long-term memory, and adaptive compression; (2) inefficient exploration that treats context management actions uniformly despite their heterogeneous impacts on final outcomes; and (3) coarse-grained credit assignment that assigns the final trajectory-level reward to all intermediate context editing actions during RL. To bridge these gaps, we introduce ContextPilot, a proactive context management framework for long-horizon agentic reasoning. Our approach systematically augments the toolset with planning, long-term memory, and soft context offloading tools. We further propose an RL method tailored for context management, which uses context and entropy variation to identify critical context control actions for branch sampling and estimates action-level advantages from all branched trajectories that pass through these actions. Experiments on long-context QA and deep search tasks show that ContextPilot achieves stronger performance with a more compact working context, outperforming existing baselines across various base models and benchmarks.

Abstract:
Methods to represent literary texts as graphs or sequences of graphs mainly focus on representing character interactions, and often overlook another crucial aspect: the textual context in which characters interact. We introduce Dynamic Heterogeneous Character Networks (DHCNs), which organize long novels into temporally localized heterogeneous graphs that align characters with their textual contexts. We extract around 20,000 DHCNs from Project Gutenberg, and propose GraphLit, a self-supervised learning framework that learns rich literary representations through a masked graph autoencoder objective. Across a wide range of 12 character-related tasks, GraphLit improves over text-only, graph-only and prior hybrid baselines. Ablations over different kinds of dynamic graph structures and architectural elements show that grounding characters in their context is the main performance driver, while explicitly encoding narrative order and character relationships provide task-dependent improvements. Finally, we demonstrate the applicability of DHCNs and GraphLit for literary analysis by studying the link between narrative non-linearity and dynamic social features.

Abstract:
Retrieval-Augmented Generation (RAG) streamlines long-document understanding by leveraging retrieval mechanisms to restrict input images to a highly curated subset. However, existing multimodal RAG pipelines primarily face two critical challenges: first, standard semantic similarity retrievers frequently fetch topically overlapping yet answer-void distractor pages that mislead downstream generation; second, rigid single-pass pipelines heavily depend on initial retrieval success, where any omission of core evidence inevitably causes cascading errors. To address these challenges, we introduce HiEvi-RAG, a hierarchical, evidence-driven multimodal RAG framework for closed-domain document understanding. HiEvi-RAG systematically factorizes complex queries into a cooperative four-stage pipeline: (1) hierarchical question decomposition to break multi-hop root queries into atomic child questions; (2) coarse visual page retrieval leveraging a multimodal retriever to fetch candidate pages based on semantic similarity; (3) fine-grained page verification via EviAgent, a specialized multi-page verifier trained with GRPO to execute cross-page reasoning over multi-image blocks; and (4) memory-guided iterative generation that leverages accumulated sub-question context to execute multi-round, dynamic reasoning over the prioritized sequence. Extensive evaluations across four benchmarks demonstrate the robust efficacy and synergy of our framework, which significantly outperforms existing open-source baselines and exceeds the previous state-of-the-art by an average of 8.05% in accuracy.

Abstract:
Current data science (DS) code generation benchmarks equate correctness with quality, overlooking execution time differences that span orders of magnitude between correct solutions. We introduce DSEffi-Bench, the first benchmark specifically targeting execution efficiency in LLM-generated DS code, comprising 1,000 instances across 10+ DS libraries with stress-testing harnesses and human-validated references. Evaluating 16 models across 3 tiers, we find that correctness alone fails to characterize efficiency: GPT-5.4 leads in correctness (Pass, 66.9%) but its efficiency score (B|P, 71.7%) nearly matches GPT-5.4-mini (71.6%), which solves 47 fewer tasks; Kimi-K2.5 ranks lowest in correctness among frontier models (40.2%) yet achieves the highest efficiency score (73.6%) across all 16 models. A human-annotated five-category taxonomy reveals that 79.1% of efficiency deficits extend beyond algorithmic complexity to domain-specific root causes, with distinct failure profiles across model tiers and libraries. Two exploratory experiments provide initial evidence that these diagnostics can guide improvement, yielding up to +14.7% efficiency gains via taxonomy-guided optimization and approaching Claude-Opus-4.6 Best@3 in efficiency at 13.0× lower cost via library-conditioned routing. Code and data are available at https://github.com/Albert-Gong/DSEffi-Bench.

Abstract:
Tables are ubiquitous across diverse domains, yet reasoning over them remains a significant challenge for modern large language models (LLMs). Current approaches typically linearize tables into sequences, inherently overlooking their intrinsic two-dimensional and hierarchical structure. To address this, we propose H2Table (Hierarchical Hypergraph-Enhanced Table Reasoning), a novel framework that represents complex tables as hierarchical nested hypergraphs. To process this representation, we design a tailored hypergraph encoder to facilitate message passing between hyperedges (headers) and nodes (cells), thereby perceiving the semantic entailment relationships between them within complex tables. Furthermore, we introduce a set of learnable query vectors acting as a lightweight bridge to extract representative structural embeddings from the encoder into the LLM. Experimental results demonstrate that our approach effectively handles complex table question answering tasks with hierarchical nested headers. Notably, on the HiTab dataset, H2Table achieves an average improvement of 22.88% over state-of-the-art baselines on highly complex tables with a nesting depth of four. Our code is available at: https://github.com/lila120/h2table.

Abstract:
Long-horizon interactions require language models to manage accumulating information: when to update their state, when to preserve their state, and what to ignore. We study this challenge as Contextual Belief Management (CBM) : maintaining a predicted belief state aligned with formal evidence while isolating task-irrelevant noise. To make CBM measurable, we introduce , a closed-world benchmark spanning Rule Discovery and Circuit Diagnosis, where a finite belief space and symbolic verifiers enable exact turn-level evaluation. diagnoses three failures: Failed Stay, Failed Update, and Failed Isolation. Across multiple LLMs, vanilla models exhibit severe CBM failures, while explicit belief-tracking prompts provide limited gains. In contrast, reinforcement learning with belief-state rewards reduces failure rates by 70.9% on average. Further probing reveals latent belief-state dynamics behind these failures, and representation-level steering reduces failure rates by 46.1% across two tasks.

Abstract:
In visual storytelling, a believable character must not only look consistent across scenes but also behave in ways that reflect their personality. Current text-to-image (T2I) models preserve visual appearance effectively, yet consistently fail to maintain personality-relevant behavior—a failure we term trait collapse . We show that conventional metrics (CLIP Score, DINO similarity) are blind to this failure, and that abstract personality descriptors yield no reliable improvement over neutral prompts regardless of abstraction level. To systematically diagnose trait collapse, we introduce ViStoryBench-Personality , a story-level benchmark combining an expert-validated Visual Cues Lexicon (Fleiss’ 𝜅 = 0.757 ) with a six-dimensional metric suite covering personality alignment, cross-scene stability, and behavioral coherence. Evaluating seven T2I systems, we confirm a persistent semantic gap between abstract trait labels and visual evidence, and find that visual identity consistency does not imply personality consistency. To bridge this gap, we propose Trait-Behavior-Visual (TBV), a training-free prompting framework that decomposes personality traits into renderable visual constraints. TBV improves Personality Alignment Score by up to +22.1 % , with larger gains in models with stronger language-conditioning capacity.

Abstract:
Large language models are increasingly used as personalized writing assistants, but adapting a model across many authors can compromise individual writing style by pulling author-specific signals toward a shared register. Federated parameter-efficient fine-tuning (PEFT) offers a data-local setting for this multi-author adaptation problem: clients keep author text local while sharing compact adapter updates. However, we show that standard aggregation can preserve continuation utility while making different authors’ generations less distinguishable in style space, a failure mode we define as author-style homogenization. We evaluate author-style retention with Angular Style Classification Encoder (ASCE)-based diagnostics on our main BlogText benchmark and ASCE-independent external authorship verification. Using this protocol, we find that common federated PEFT baselines can preserve semantic utility while averaging out author-specific signals. To address this homogenization, we instantiate FAVoR (Federated Authorial Voice Retention), an author-style residual mechanism for federated PEFT. FAVoR uses a shared-private adapter design: clients upload shared-adapter updates while retaining author-specific residual corrections locally. Across BlogText and external Mythos-Reddit validation, FAVoR improves author-style retention over standard and personalized federated PEFT baselines. These gains come with small continuation-utility trade-offs and are supported by component ablations, external verification, and cold-start transfer.

Abstract:
LLM-based sequential recommenders usually cast next-item prediction as text generation, but this interface is poorly matched to full-catalog top-K ranking. We propose TATK, a Triple-Aware framework that couples Top-K Learning (TKL) with Knowledge-Grounded Verification (KGV) for LLM-based sequential recommendation. Top-K Learning combines context-aware metadata-KG prompt grounding with position-aware top-K rewards, aligning training with ranking utility; Knowledge-Grounded Verification then applies structure-aware reranking over the top-M candidates after a single LLM forward pass, using the same metadata-derived item graph. We evaluate TATK on Musical Instruments, CDs and Vinyl, and Video Games from Amazon Reviews 2023 under a matched R²ec-style full-catalog protocol. Experiments use Gemma-2-2B-It and Qwen2.5-3B-Instruct backbones, compare against sequential, generative, KG-augmented, and reasoning-enhanced baselines, and include component, reward-shape, sequence-perturbation, reranking, relation-quality, and candidate-pool diagnostics. TATK improves over the matched R²ec reproduction on all 36 reported metrics. On NDCG@10, it improves Qwen by 8.05%, 4.26%, and 3.78% on the three datasets, and improves Gemma by 27.03%, 10.52%, and 10.23%, while keeping inference within 1.17× of Base RecPO latency. The diagnostics show that structural evidence is most useful for recoverable top-M candidates with reliable KG support, and should be gated when metadata relations are sparse or noisy.

Abstract:
Multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and action generation. However, their ability to sustain exploration in dynamic open worlds remains unclear. Existing embodied and game-based benchmarks often compress interaction into short-horizon tasks or entangle success with domain-specific game mechanics. In this paper, we introduce MineExplorer benchmark for evaluating open-world exploration capabilities of MLLM agents in Minecraft. We first filter atomic tasks whose solutions rely heavily on Minecraft-specific knowledge to better reflect general open-world reasoning. Then we organize the benchmark around a ReAct-style capability formulation and compose atomic tasks into implicit multi-hop tasks. To further construct reliable instances, MineExplorer uses a multi-agent synthesis workflow that jointly designs task graphs, sandbox scenes, and rule-based milestone evaluators. Human evaluation shows that the multi-agent synthesis workflow produces significantly more reliable instances than a single-agent baseline. Experiments with advanced MLLM agents show that open-world exploration remains challenging, as strong models can handle many single-hop tasks but degrade sharply when hidden prerequisites must be coordinated over longer trajectories. Further analysis finds that task difficulty tracks agent completion, and larger models or thinking modes do not consistently translate into better performance. Code is available at anonymity.

Abstract:
Multimodal Large Reasoning Models introduce the reasoning paradigm, demonstrating strong capabilities on complex vision-language tasks. However, they still suffer from severe hallucinations. Existing training-based methods typically mitigate hallucinations through response-level direct preference optimization (DPO), where the Chain-of-Thought (CoT) and the final answer are treated as a monolithic output and optimized jointly. We reveal that this formulation performs similarly to answer-only optimization, suggesting that it primarily learns answer-level preference, while leaving CoT-level supervision insufficiently exploited. To address this issue, we explicitly formulate a CoT-oriented preference term and derive Reasoning-Conditioned Direct Preference Optimization (RC-DPO), which models the CoT as a condition for answer generation and contrasts the preference for the same preferred answer under different CoT conditions, promoting answer-supportive reasoning chain alignment. To further improve optimization, we introduce a reasoning-enhanced preference data generation strategy that employs Monte Carlo Tree Search to construct high-quality preferred responses, pairs them with ordinarily sampled outputs, and further derives hard-negative reasoning conditions by pruning visually salient tokens. Extensive experiments across various models and benchmarks show that RC-DPO effectively mitigates hallucinations and improves the reliability of multimodal reasoning.

Abstract:
We introduce Xiaohongshu Social Comparison Reader Elicitation (XHS-SCoRE), a reader-grounded benchmark for detecting whether text-only Xiaohongshu (RedNote) posts elicit ‘Upward‘, ‘Downward‘, or ‘Neutral‘/no clear social comparison from a first-person reader perspective. The task targets a socially meaningful relational, behaviorally real signal not reducible to sentiment. Across prompted LLM classifiers and supervised Chinese encoders, we find a consistent generation–detection mismatch: the signal is textually learnable in-domain, but not robustly accessible to prompt-based classification. Prompted LLM classifiers show stable failures, especially neutralization of comparison-eliciting posts and model-specific directional skew. A controlled pilot shows that LLM-generated Xiaohongshu-style posts can shift perceived standing and comparison-related affect even when prompt-based detection of the same construct remains fragile. XHS-SCoRE contributes a benchmark for reader-grounded comparison detection and a diagnostic framework for studying when socially meaningful relational cues remain only partially visible to prompt-based inference.

Abstract:
Inference-time scaling of LLM reasoning relies on a confidence score over each partial trace, yet existing estimators treat prefix quality as intrinsic to the prefix. A given prefix can succeed at very different rates and rankings under different continuation policies. Reasoning confidence is therefore not policy-invariant, and a squared-loss lower bound shows that policy-blind predictors retain residual error proportional to cross-policy disagreement. Calibration must condition on the continuation model and inference budget. To bridge this gap, we propose the Comparable Policy-Conditioned Calibrator (CPCC), built from a shared frozen-LLM prefix encoder, comparable scalar policy features under per-policy z-score normalization, and a Beta–Binomial head trained on bootstrap success counts. Across six continuation policies and three reasoning benchmarks (GSM8K, SVAMP, MATH-500), all evaluated under majority-vote aggregation, the CPCC family attains the lowest Brier on every evaluation cell, and CPCC is strongest on the held-out model families. It also avoids the held-out-family collapse of raw-hidden conditioning, lowers ECE by 13 to 26 percent on out-of-distribution math, and reduces downstream selective risk by 10 to 30 percent.

Abstract:
Evaluating data-text alignment remains challenging: existing metrics often provide limited explanations for the scores, while prompt-based LLM-as-Judge methods can be expensive and unreliable. We present an end-to-end explainable evaluation metric that fine-tunes a language model to identify omitted, extra, incorrect, and correct data units in a data-text pair. These local judgements are aggregated into precision, recall, and F1 scores, providing both fine-grained diagnostic feedback and an interpretable measure of alignment quality. Across benchmarks, our fine-tuned models outperform LLM-as-Judge methods in error prediction and achieve competitive precision, recall, and F1 scores, while maintaining strong correlation with human judgements. Beyond evaluation, our verifier outputs also provide useful feedback signals for downstream correction and refinement, supporting alignment-oriented improvement of data-to-text and text-to-data. Code and resources are available at https://github.com/guihuzhang/xqdt.

Abstract:
Tool calling has greatly expanded the practical utility of large language models (LLMs) by enabling them to interact with external applications. As LLM capabilities advance, effective tool use increasingly involves multi-step, multi-turn interactions to solve complex tasks. However, the resulting growth in tool interactions incurs substantial latency, posing a key challenge for real-time LLM serving. Through empirical analysis, we find that tool-calling traces are highly structured, conform to constrained schemas, and often exhibit recurring invocation patterns. Motivated by these observations, we propose ToolSpec, a schema-aware, retrieval-augmented speculative decoding method for accelerating tool calling. ToolSpec exploits predefined tool schemas to generate accurate drafts, using a finite-state machine to alternate between deterministic schema token filling and speculative generation for variable fields. In addition, ToolSpec retrieves similar historical tool invocations and reuses them as drafts to further improve efficiency. ToolSpec presents a training-free, plug-and-play solution that can be seamlessly integrated into existing LLM workflows. Experimental results across multiple tool-calling benchmarks demonstrate that ToolSpec achieves up to a 4.2x speedup, substantially outperforming existing training-free speculative decoding methods.

Abstract:
As Natural Language Generation (NLG) dominates modern NLP, scalable evaluation remains a critical bottleneck. Consequently, LLM-as-a-judge (LaaJ) adoption has accelerated rapidly, appearing in more papers than human evaluation in 2025. This pivotal shift motivates a critical analysis of current evaluation practices. Overcoming the limits of rigid keyword filtering and manual review, we employ a multi-LLM information extraction pipeline to gather structured metadata from 14,171 papers across four major NLP conferences (2020–2025). Analyzing 3,334 filtered NLG papers, we identify three systemic challenges. (1) Metric inertia: despite the shift toward open-ended generation, legacy lexical metrics (BLEU, ROUGE) persist as primary indicators, typically used alongside rather than replaced by semantic alternatives. (2) Metric-criteria mapping problem: our paper-level co-occurrence data reveals that general-purpose automatic metrics are applied as broad proxies for quality, without specifying which dimension of text generation they are intended to evaluate. (3) Validation gap: LaaJ has grown rapidly without commensurate human validation (fewer than 8% of papers). Crucially, while LaaJ correlates with aggregate quality, alignment collapses on fine-grained criteria like fluency. To address these gaps, we distill our findings into a minimal Evaluation Checklist to guide metric selection, construct validity, and LaaJ deployment.

Abstract:
Large Reasoning Models (LRMs) pose a dual-surface safety challenge: both intermediate reasoning traces and final answers can contain harmful content. Existing alignment methods often operate at the whole-response level, allowing unsafe reasoning to be masked by a safe-looking final answer. We propose Segment-aware Listwise Target DPO (SaLT-DPO), which addresses this gap through three mechanisms: (1) segment-aware listwise alignment that decomposes responses into reasoning and answer segments, independently scores each segment’s safety, and aligns length-normalized segment rewards with soft target distributions over multiple candidates; (2) joint safety coherence regularization that applies a weakest-link principle to promote safety consistency across both segments; and (3) utility anchoring on benign prompts to mitigate over-refusal and reasoning degradation. Experiments on three LRMs show that SaLT-DPO consistently reduces unsafe rates for both reasoning and answer segments while mitigating degradation in benign compliance and preserving general reasoning performance. Ablation studies demonstrate the complementary contributions of its components.

Abstract:
Arabic and Hebrew, as closely related Semitic languages, share many words with similar surface forms, including true cognates, false friends, and modern loanwords. These lexical relationships create ambiguity for cross-lingual semantic interpretation, as similar forms may correspond to shared, divergent, or borrowed meanings. To evaluate whether LLMs can make these distinctions, we introduce SemCog Bench, a curated benchmark of 1,858 Arabic–Hebrew word pairs with sentence-level annotations for cognate identification and semantic disambiguation. We evaluate a diverse set of open-source and proprietary LLMs across multiple input representations and contextual settings. Our results show reliance on surface-form similarity, with weaker performance on false friends and wide variation on loanwords. We further find that context and scale yield model-dependent gains, while original-script inputs generally perform best. Our findings reveal limitations in cross-lingual form-meaning reasoning and establish SemCog Bench as a benchmark for cross-lingual lexical semantics. Our code and data are publicly available.

Abstract:
While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies. Our analysis shows that current MLLMs exhibit weak visual focus: early-stage visual misalignment is rarely corrected during subsequent reasoning, leading to error propagation and failed inferences. We argue that this limitation stems from inadequate credit assignment for visual attention during training. To address this issue, we propose SAYO, a visual reasoning model trained with a reinforcement learning (RL) framework that introduces a region-level visual attention–based reward. This reward explicitly aligns optimization signals with visually grounded reasoning steps, enabling the model to learn more reliable attention behaviors. Extensive experiments across multiple multimodal benchmarks demonstrate that SAYO consistently improves performance on diverse reasoning and perception tasks.

Abstract:
Multiple-choice questions (MCQs) are a standard format for evaluating large language models (LLMs), yet the popularity of answer options can confound evaluation. Modern LLMs systematically prefer popular but incorrect options over less popular correct ones, a vulnerability we call popularity bias . This pattern aligns with confidence miscalibration: model confidence remains high even as accuracy collapses for popular options. To systematically isolate this phenomenon, we introduce PopMCQ , a benchmark with six controlled strategies that vary option popularity while keeping the correct answer fixed. In our most adversarial setting, where all distractors are more popular than the correct option, models choose popular but wrong answers 66% of the time. To mitigate this bias, we propose PopDebias , a lightweight inference-time correction that estimates and removes a popularity prior from model predictions. It requires no fine-tuning, is label-free at test time (using only a small calibration split for parameter fitting), and adds negligible computational cost. Experiments on 22 open-source LLMs (0.5B to 32B parameters) show consistent improvements, with accuracy gains up to 54.1 percentage points under strong popularity pressure.

Abstract:
Meaningful multilingual evaluation must test models in the target language and educational context. Urdu, spoken by more than 230 million people, lacks a broad MMLU-style benchmark built from native educational sources. We introduce UrduMMLU, a benchmark of 26,389 Urdu MCQs across 26 subjects and five domains, collected from native Urdu MCQ banks and public examination PDFs. Unlike translation-based benchmarks, UrduMMLU combines academic subjects with content specific to Urdu and regional education. We label the exam-derived portion through dual human annotation with strict consensus filtering. We evaluate 30 LLMs under English and Urdu prompts, yielding 60 zero-shot evaluations, and further evaluate four open-source LLMs under multiple few-shot settings across both prompt languages. Gemini-3.5-Flash performs best, reaching 90.23% and 90.45% accuracy, while no other model exceeds 85%. The strongest open-source model trails by 7.78 and 9.12 points, and many models lose 25 to 40 points on Urdu-centered Humanities subjects compared with STEM. Few-shot prompting yields only modest gains. Results on UrduMMLU show that current LLMs have uneven Urdu knowledge, particularly for content grounded in the regional context.

Abstract:
Large vision-language models (LVLMs) are increasingly deployed as autonomous mobile agents that execute tasks through GUI interaction. However, their security under realistic deployment constraints remains insufficiently understood. Existing attacks typically rely on persistent visual manipulations, privileged system access, or iterative jailbreak interactions, limiting their practicality. In this work, we identify a previously overlooked discrepancy between human and agent interactions: automated mobile agents generate near-zero touch-contact signals that systematically differ from human touches. Based on this observation, we introduce agent-only perceptual injection, a new attack paradigm that exposes malicious prompts only during agent interactions while keeping them unexposed during normal user operation. To realize this attack, we propose a low-privilege framework that combines in-app prompt embedding, interaction-triggered activation, and HG-IDA, an efficient one-shot jailbreak optimization method tailored to mobile UI constraints. Experiments on multiple Android applications and LVLM backends, such as GPT-4o, show that our attacks reliably hijack both agent planning and execution. Our method achieves up to 95.0% planning attack success rate and 82.5% execution attack success rate on commercial LVLM backends. These findings reveal a new interaction-level attack surface and highlight the need for provenance-aware defenses in mobile agent systems.

Abstract:
With the widespread applications of large language models (LLMs), privacy-preserving inference has become increasingly essential for sensitive queries. To balance privacy and utility, a series of lightweight obfuscation approaches has recently been proposed, where users locally transform plaintext embeddings into the fixed ciphertext ones. While such Embedding-to-Embedding Obfuscation (E2EO) schemes demonstrate considerable resilience against traditional token frequency and embedding inversion attacks, the core mechanism behind remains to be the large-scale one-to-one substitution, which provides no cryptographic guarantees. In this paper, we propose Proxy Manifold Alignment (PMA), a novel attack against E2EO in privacy-preserving LLMs. Our key observation is that E2EO schemes keep the original semantic structure, so that the obfuscated vector stream can be regarded as an unknown tokenizer-language whose symbols are the vectors themselves. Therefore, the proposed ciphertext to plaintext reconstruction attack can be formulated as a translation task from the unknown tokenizer-language to plaintext. Specifically, by only accessing the obfuscated vector stream, the target tokenizer and a public corpus, the PMA attack first employs Word2Vec to model the co-occurrence patterns within the obfuscated stream and the public corpus independently, and constructs two proxy vector embeddings. Then, the attack aligns the underlying manifolds of these two embeddings based on structural similarity. Finally, it maps the obfuscated vectors back to plaintext. Experimental results demonstrate that PMA consistently achieves higher plaintext recovery than other state-of-the-art attack methods.

Abstract:
In interactive text-to-SQL, the evidence needed to write correct SQL is not given upfront but distributed across schema, knowledge bases, and users, and must be acquired through costly environment interactions under a finite budget. We observe that strong LLMs often fail not from deficient SQL synthesis, but from premature commitment to SQL before acquiring task-critical evidence. Through controlled trajectory analysis, we formalize this observation as the Information Sufficiency Hypothesis and identify a five-class gap taxonomy that structures the acquisition space. Guided by this diagnosis, we develop PASCAL, a training-free acquisition-control data agent framework that interposes a source-aware information-acquisition layer before SQL generation and schedules acquisition by cost-normalized priority. On BIRD-Interact, PASCAL with a local 35B backbone achieves 51.0% Phase-1 and 37.0% two-phase success rate, outperforming frontier models leaderboard result by up to 14.7 percentage points, with gains concentrated on BI tasks requiring cross-source evidence acquisition.

Abstract:
Financial and tabular question answering requires more than fluent reasoning: answers must be grounded in the exact facts, formulas, units, signs, and scales that support them. A single misread cell or incorrect operation can silently produce a plausible but wrong result. We introduce MOCA-Agent , a market-of-claims code agent that replaces free-form multi-agent debate with claim-level verification. The system decomposes each question into typed atomic claims, asks specialist trader agents to buy or sell those claims, clears their orders into confidence-weighted accept/reject decisions, and synthesizes an executable Python program from market-supported evidence. A code-aware verifier then checks the program for execution, structural consistency, and common financial reasoning errors, with at most one market-aware repair round. Across ten public benchmarks spanning financial numerical reasoning, general tabular reasoning, ESG question answering, and multimodal chart reasoning, MOCA-Agent achieves strong performance using a fixed Qwen3.6-27B backbone, including 78.3% on FinQA, 76.0% on FinanceMath, 71.2% on MultiHiertt, 86.9% on ESGenius, and 85.6% average on FinChart-Bench. These results show that aggregating evidence at the level of atomic claims, rather than whole answers, improves robustness in high-stakes numerical reasoning.

Abstract:
A medical claim’s correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range, a causal or conditional link, or patient-specific details to be judged correctly, and atom-level decomposition can fragment these dependencies, leaving the verifier with clinically incomplete claims. We reformulate medical fact-checking around snippet-level verification, where clause-grouped units preserve local clinical structure. We introduce MedSNIP-Bench, a human-annotated benchmark for snippet-level medical fact verification, and MedSNIP, an automatic snippet-generation pipeline. MedSNIP-Bench covers 276 consumer-health and clinical-vignette responses, segmented into 2,524 snippets with dual in-general and in-patient-context labels and six structural pattern codes. MedSNIP is evaluated against human snippet boundaries on MedSNIP-Bench and then used to generate snippet-level units for external corpora. Across MedSNIP-Bench, HealthFC, and MedHallu, snippet-level verification preserves or improves false-class F1, with gains concentrated where answers are long enough to fragment and where the verifier is strong enough to exploit the recovered structure. The largest merge-pattern gain is on causal-conditional clinical chains. It also reduces verifier calls by 24-73%, though the saving survives end-to-end only when decomposition is cheap, which an open-weight decomposer makes possible at no loss of chunking fidelity.

Abstract:
Cold-start supervised fine-tuning (SFT) is a critical initialization step before reinforcement learning from verifiable rewards (RLVR). Yet existing cold-start data selection methods largely inherit criteria from supervised instruction tuning—sample quality, difficulty, perplexity, or diversity—implicitly assuming that a stronger SFT warm start is also a better starting point for RL. We argue that this assumption does not always hold in the SFT-to-RL pipeline: cold-start SFT not only reduces supervised loss but also shapes the initial update geometry that downstream reward optimization inherits. Building on this observation, we propose RL-Proxy-Coverage. Our method estimates a low-rank, reward-relevant gradient subspace from a small set of representative RL anchor prompts and verifier rewards, and selects, as the cold-start subset, those candidate SFT examples whose gradients provide non-redundant coverage of this subspace. In controlled mathematical-reasoning experiments, RL-Proxy-Coverage outperforms representative SFT-centric, hard-diverse, and reward-aware baselines at the early-to-mid RL checkpoints. Our findings suggest a new view of cold-start data: it should be regarded not merely as supervised examples for improving SFT, but as an initialization resource that shapes the starting point of downstream reward optimization.

Abstract:
As the AI-code assistant tools become widespread, automatic assessment of the correctness of the generated code becomes a significant challenge. Code LLMs are prone to hallucinations, which may lead to code that does not solve a required problem, or even to code with severe security vulnerabilities. In this paper, we introduce CodeTD – the first approach to pre-execution assessment of code correctness based on topological data analysis (TDA) of Code LLM attention maps. Our method quantifies prompt-generation mismatch from topological patterns of attention maps. We carry out experiments with common benchmarks (HumanEval, MBPP, BigCodeBench, MultiPL-E), 5 programming languages and 10 Code LLMs of size up to 34B parameters. The experimental results show that the proposed method is better than recent baselines. Moreover, CodeTD is transferable between coding benchmarks.

Abstract:
As LLMs are increasingly asked questions about a user’s personal experiences, retrieval must distinguish memories by time, place, person, and event. Current personal-memory pipelines rely on frozen dense embedding models to embed memories for retrieval, but we find that these models do not capture the four axes well. Swapping a memory’s date or place barely shifts its embedding, and a trained projection on top of the frozen embedding does not recover the missing signal. The problem lives inside the model, where each axis is encoded by a distinct attention-head pattern. Guided by these findings, we propose Axis-LoRA, a per-axis adapter that places four LoRA branches inside the embedding model and combines them via a per-query gate. Across three standard embedding models evaluated on LongMemEval, ATM-Bench, and LoCoMo, Axis-LoRA improves 4-benchmark mean R@10 by +1.38 to +1.59 pp while training only a small fraction of the base parameters on a small axis-counterfactual training set.

Abstract:
While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low-data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable decision trees under a few-shot learning setting. Instead of directly prompting the LLM to generate full trees, which is often unstable and inefficient, we develop a three-stage paradigm that prompts the LLM to generate rules and organize the rules into a tree. Experiments on multiple real-world tabular datasets demonstrate that our method achieves superior accuracy and interpretability with significantly lower prompting overhead compared to existing baselines.

Abstract:
Real-world recommendation scenarios are commonly grounded in shared physical environments during user–recommender interactions. This motivates situated conversational recommendation (SCR), a complex task requiring recommender assistants to jointly reason over dialogue history, co-observed scenes, and in-scene item attributes. However, current approaches struggle with this setting due to two intertwined challenges: accurately understanding situated user preferences throughout the conversation and generating responses that simultaneously satisfy user needs and grounded situations. To this end, we propose Re2A, a framework that formulates SCR as a structured reason-then-align process. We introduce rubric-based preference reasoning, which uses automated rubrics to guide the model toward producing explicit preference states. Based on these states, we propose a preference-conditioned optimization to align response generation with dual objectives: user preference satisfaction and situation consistency. Extensive experiments on two SCR datasets demonstrate that Re2A consistently outperforms state-of-the-art methods, delivering more precise, context-aware conversational recommendations. Our code is available at https://github.com/DongdingLin/Re2A.

Abstract:
Grounded legal question answering requires retrieving the relevant statutory provisions, generating legally sound answers, and supporting them with explicit evidence. We introduce JustifIT-QA, the first Italian statutory QA dataset with expert-written answers and fine-grained evidence spans, and present DARv2, a dense retriever that improves coverage of relevant legal texts. Building on this retrieval layer, we study a judge-based feedback loop in which an LLM critiques and refines generated answers, and we evaluate LLMs as judges for legal answer correctness. Results show that feedback improves expert-rated answer quality when relevant context is retrieved, while evidence extraction benefits from stronger models but remains challenging. We release the dataset, code, and retriever to support future research on grounded and explainable legal QA in Italian.

Abstract:
Surface electromyography (sEMG) offers a promising path toward private and wearable text input, but accurate typing transcription remains challenging due to strong cross-user and cross-session variability. We propose a beam-aware sEMG typing framework that connects an improved Connectionist Temporal Classification (CTC)-based frontend with a large language model (LLM) corrector for exact recovery of the typed sequence. As a standalone sEMG decoder, our CNN-Conformer frontend achieves the best reported performance among the compared published emg2qwerty systems under the full-session frontend protocol, reaching 19.94% generic character error rate (CER) and 3.43% personalized CER. For LLM correction, we use a separate windowed hypothesis-export protocol, where the frontend’s top-10 CTC hypotheses are provided to a fine-tuned LLM together with a rank-1 baseline. Under this within-protocol comparison, the LLM reduces session-level CER from 22.53% to 19.65% in the generic setting and from 3.41% to 2.01% after personalization. A matched hybrid evaluation further confirms that the LLM also improves rank-1 outputs obtained from full-session decoding. The top-10 oracle results show that the beam contains recoverable information beyond the rank-1 output. These results demonstrate that a stronger sEMG frontend and hypothesis-aware LLM correction can work together effectively for exact sEMG typing transcription.

Abstract:
Real-world professional desktop workflows in specialized creative and engineering software unfold over long horizons and often require human-in-the-loop coordination, where agents proactively seek necessary information and users provide additional instructions, clarifications, feedback, or corrections as the task progresses.Yet existing desktop GUI benchmarks mostly reduce this setting to short, simplified tasks with all user instructions provided upfront.To address this issue, we introduce DeskCraft, a desktop GUI benchmark targeting long horizon creative and engineering workflows and proactive human-agent collaboration. DeskCraft organizes tasks into a multilevel difficulty taxonomy, with long horizon tasks requiring over 50 execution steps, and covers professional creative software across design, video, audio, and 3D creation. Furthermore, DeskCraft formalizes human-agent collaboration into an interaction protocol covering mid-turn and post-turn exchanges. Mid-turn interaction captures both agent-initiated clarification under uncertainty and user-initiated interruption during execution, while post-turn interaction accommodates user-driven feedback after the agent signals completion, together spanning the full space of realistic collaboration patterns. We evaluate 18 proprietary and open source agents on 538 tasks and find that GPT-5.4 reaches 31.6% on standard tasks and 27.6% on interactive tasks. Further analyses reveal persistent failures in long horizon workflow delivery and proactive clarification. We will open-source all evaluation codes, tasks, and data.

Abstract:
Vision Language Models have demonstrated remarkable proficiency in interpreting static visual artifacts, but modern data analysis is inherently dynamic, requiring the active interrogation of interactive environments. Existing benchmarks are predominantly constrained to static imagery and one-shot question answering and fail to capture the epistemic demands of this domain, where evidence is frequently occluded, distributed across linked views, or conditionally revealed through user agency. In this paper, we introduce InSight, a benchmark for agentic claim verification over interactive visualizations. The dataset consists of 21,349 claims derived from human-authored analytical narratives and grounded in fully interactive web-based environments. Agents must navigate these environments to determine whether a natural language claim is supported, refuted or not verifiable given the available evidence. Unlike traditional evaluations, InSight treats interaction traces as intrinsic proxies for reasoning, enabling a rigorous audit of how models seek and synthesize visual evidence. We evaluate state-of-the-art models, revealing that interactive verification remains a non-trivial challenge. We release InSight at https://github.com/maevehutch/insight.

Abstract:
Reference-conditioned image generation increasingly relies on visual histories that rarely speak with one voice: one example pins the subject, another fixes a layout, a third only suggests a palette. Current image-conditioned diffusion methods encode these references once, average them into a single vector, and inject it at every denoising step, so conflicting cues collide and the output is faithful to none. We propose the Hierarchical Temporal Guidance Framework (HTGF), a training-free pipeline that reframes multi-reference conditioning as a temporal routing problem. A semantic decomposer reads each reference and emits a soft routing distribution over three axes (Subject, Structure, Detail), and a closed-form SNR-sensitivity argument places each axis in the denoising window where it has the most leverage. A short manifold-aware corrector then smooths the trajectory at the stage boundaries where the active condition changes. HTGF adds no training and no architectural change to the diffusion backbone. On three datasets it outperforms strong VLM- and diffusion-based baselines, gaining +11.7 CIS over LaVIT on Movie Poster and FID 38.87 vs. 50.12 for the best baseline under noisy histories, and degrading gracefully down to the zero-reference limit.

Abstract:
Chain-of-Thought (CoT) reasoning appears to provide explainability, leading users to trust that verbalized rationales reflect the model’s underlying computation. However, substantial evidence indicates that CoT often fails to reflect the model’s actual decision-making process, leading to a surge of research into the faithfulness of these explanations. This paper presents a comprehensive survey of CoT faithfulness. We first unify the definition of faithfulness by integrating internal alignment with external consistency and synthesize key failure phenomena, such as post-hoc rationalization and sycophancy. Furthermore, we systematize evaluation metrics, benchmarks, and critically review current mitigation strategies. We conclude by outlining open challenges and advocating for architectural innovations to achieve genuinely faithful reasoning for credible AI.

Abstract:
Sparse encoders offer high-precision retrieval by representing term importance within a vocabulary space, yet their English-centric structures pose a critical impediment to language transfer for non-English languages. To overcome this structural limitation, we propose SemBridge, a novel embedding initialization method designed for cross-lingual adaptation in sparse encoders by leveraging multilingual bridge models. SemBridge establishes semantic alignments between source and target vocabularies using multilingual dense embeddings as a bridge. Rather than directly relying on all source tokens, SemBridge selects a small set of semantically related source-language tokens and uses them to initialize each target-language token, effectively filtering out semantic noise and reconstructing target tokens as precise linear combinations of core synonyms. This accelerates convergence during fine-tuning and improves training efficiency. Extensive experiments across five languages and four sparse architectures demonstrate that SemBridge achieves superior zero-shot retrieval performance and consistently improves retrieval performance after fine-tuning compared to existing baselines. These results validate SemBridge as a practical solution for deploying high-performance sparse retrieval systems in diverse linguistic environments.

Abstract:
Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to save deployment costs. All existing audits decide backbone identity from the text-output channel, which is structurally fragile for agentic APIs because modern serving stacks (OpenAI, Anthropic, Gemini, Cloudflare Workers AI, LangGraph) discard text and expose only structured actions when the model calls a tool, and provider-injected system prompts can distort text distributions enough that text-channel tests falsely accuse honest providers of substituting the claimed model. We observe that recent agentic post-training internalizes tool-use directly into the weights, opening a new audit channel that the serving stack still exposes and that is largely invariant to deployment context. We introduce Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs: AgentProv fingerprints a deployed model through its categorical tool-call distribution and decides identity via an MMD permutation test. AgentProv catches every substituted model (100% on 630 evaluated checkpoint pairs), while holding the false-positive rate under system-prompt injection at 7% (vs. 67% for MET and 53% for RUT). On third-party API endpoints, AgentProv’s disagreements with MET are consistent with an independent token-count side-channel that detects provider-injected system prompts.

Abstract:
We study fact-level repair for cross-modal factual generation, where a fluent textual output may contain specific facts that are not supported by the input. Existing inference-time repair methods often generate feedback by jointly conditioning on the input and the current output. This design can let hallucinated claims in the output bias the model’s interpretation of the input, and its free-form feedback is difficult to rank or schedule at the fact level. We present TIGER, an inference-time framework that redesigns feedback for localized repair. TIGER independently extracts an observation graph from the input and a claim graph from the current output, then assigns each claim a graph-conditioned risk score based on support and conflict. The model repairs selected high-risk claims while keeping the backbone frozen. We provide a convergence analysis showing that the expected total risk decreases geometrically to an explicit asymptotic bound under mild assumptions. Experiments on four text-output cross-modal paths, including image-to-text, image+text-to-text, audio-to-text, and video-to-text, show that TIGER reduces unsupported content while preserving task quality. The gains hold across multiple backbones, and a CrisisFACTS case study suggests that the same repair mechanism can improve grounding in noisy multi-source settings.

Abstract:
Scientific method figures often provide researchers with a clear visual entry point into a paper’s methodology and core contributions, yet their dense layouts and complex information flows are difficult for vision-language models (VLMs) to interpret. Existing benchmarks predominantly focus on coarse-grained tasks, such as context-based retrieval or holistic comprehension, overlooking the fine-grained content and inter-module relations within complex methodological diagrams. To address these overlooked aspects, we introduce Scientific Figure Benchmark (SciFigBench), a benchmark for evaluating core capabilities required for scientific method figure understanding. Specifically, it targets three capabilities: dense content perception, visual relation comprehension, and information-flow tracing. For this purpose, we curate 4,352 scientific method figures from real-world research papers and construct 40,670 QA pairs to operationalize these capabilities as measurable question-answering tasks. All annotations are generated by dedicated pipelines and then manually verified. We further propose SciFigCue, a training-free visual prompting module that externalizes arrow-induced dependencies as visual relational evidence for method-figure reasoning. Experiments across open-source and proprietary VLMs show that even strong models remain substantially limited in visual relation comprehension, while SciFigCue improves performance across several tasks without parameter updates. The code and related resources are available at https://github.com/LeNcyer/SciFigBench.

Abstract:
Graph-augmented large language models often assume that graph evidence produced by external computation and placed in the input can be used by the native decoder. We test this assumption with HopQA, a deliberately bounded diagnostic that asks for the shortest-hop distance between two query nodes. Because the answer is a small integer and the target is purely topological, failure cannot be dismissed as open-ended generation or ambiguous evaluation. Yet existing graph-augmented baselines still fail on this setting, showing that providing graph evidence is not the same as making it usable. We introduce an intervention triangle with three matched conditions: readable graph evidence, shuffled graph evidence, and no-graph input. This separates evidence inclusion, structural readability, and decoder-usable topology. Guided by this diagnosis, we present S²GE as an instance showing that diagnosis-driven interface design can improve native decoder usability. S²GE uses query-aware sampling, endpoint- and proximity-based ordering, and structure-preserving alignment. Across DBLP, Biomedical, GoodReads, and PubMed, S²GE achieves strict exact-match scores of 36.5%, 57.8%, 76.6%, and 52.0%, improving over the strongest native-generation baseline by 53.5 points on average. The interventions further reveal harmful-shuffle, shuffle-robust, and no-graph-saturated regimes.

Abstract:
Understanding tabular semantics is essential for intelligent data management, yet data lake tables often lack explicit annotations, hindering interpretation of their structure and relations. Column Relation Extraction (RE), which identifies semantic relations between table columns, is a key step toward schema matching, data integration, and knowledge construction. Existing closed-set methods rely on predefined relation labels, while LLM-based generation offers greater flexibility but suffers from semantic ambiguity, coarse supervision, and inefficient pair-wise inference. We propose a table-specific reinforcement learning framework for open-world RE. Instead of forcing predictions into a fixed label space, our method trains LLMs to generate free-form relations under fine-grained semantic feedback. We introduce a hierarchy-aware reward that converts semantic proximity into graded supervision, allowing the model to learn from near-miss predictions rather than treating all errors equally. We further propose relation-level policy optimization to improve efficiency: the model predicts all column-pair relations in one structured response, and each relation-level reward is assigned only to its corresponding token span. This reduces repeated table encoding and aligns optimization with relation-level correctness. Experiments demonstrate consistent improvements over strong LLM and table-specific baselines, especially on unseen relations and domains, highlighting the robustness and practicality for real-world table interpretation.

Abstract:
Evaluating the multilingual capabilities of Large Vision-Language Models (LVLMs) remains challenging because most benchmarks rely on non-parallel corpora, making it unclear whether cross-lingual performance gaps reflect model limitations or dataset inconsistencies. To address this, we introduce PM4Bench, the first multimodal, multilingual, multi-task benchmark built on a strictly parallel 10-language corpus, enabling fair, apples-to-apples cross-lingual comparison of model performance. We further introduce a vision setting that embeds textual inputs directly into images, better approximating deployment scenarios where LVLM-driven agents interact with virtual or physical environments through unified visual observations. Experiments with 10 LVLMs reveal that OCR is a key factor behind cross-lingual disparity when textual content is rendered visually. Motivated by this, we design an OCR-centric GRPO training strategy using fully synthesized, label-free OCR data, without expensive task-specific VQA supervision. The resulting model improves general multilingual VQA capability, reduces cross-lingual disparities under the vision setting, and transfers gains beyond PM4Bench. This methodology offers an efficient, label-free pathway toward more equitable multilingual deployment of LVLM-driven agents.

Abstract:
Automatic Short Answer Scoring (ASAS) is central to NLP for Education. However, openly available benchmarks remain scarce, and existing datasets largely address how well students answer a question directly rather than how well they master underlying concepts (knowledge elements) such as thermal energy or epistemic activities (skills) such as reasoning or claim. To address this gap, we introduce Alice, a large-scale, rubric-based German ASAS dataset that is pedagogically aligned and comprises three subtasks: (i) learning performance (Alice-LP), (ii) knowledge elements (Alice-KE), and (iii) skills (Alice-SK). We further formulate rubric-based ASAS as a rubric-retrieval task and benchmark the dataset with a range of language models, from encoder-only models to lightweight LLMs. We also benchmark the dataset with zero-shot prompting via LLMs and a standard classification baseline. The experiments show that LLMs, in particular, struggle to score knowledge elements and skills in the zero-shot setting. They also indicate that rubric text is often useful, especially for Alice-KE and Alice-SK, while on Alice-LP gains over sample-solution-focused inputs are more modest and vary by model and input format.

Abstract:
Large language models (LLMs) increasingly rely on explicit reasoning to solve coding tasks, yet evaluating the quality of this reasoning remains challenging. Existing reasoning evaluators are not designed for coding, and current benchmarks focus primarily on code generation, leaving other coding tasks largely unexplored. We introduce CodeRQ-Bench, the first benchmark for evaluating LLM reasoning quality across three coding task categories: generation, summarization, and classification. Using this benchmark, we analyze 1,069 mismatch cases from existing evaluators, identify five recurring limitations, and derive four design insights for reasoning evaluation in coding tasks. Guided by these insights, we propose VERA, a two-stage evaluator that combines evidence-grounded verification with ambiguity-aware score correction. Experiments on CodeRQ-Bench show that VERA consistently outperforms strong baselines across four datasets, improving AUCROC by up to 0.26 and AUPRC by up to 0.21. We release CodeRQ-Bench at https://github.com/MrLYG/CodeRQ-Bench, supporting future investigations.

Abstract:
We introduce GUI-360°, a large-scale dataset and benchmark suite for desktop productivity computer-using agents (CUAs). GUI-360° provides an automated pipeline for query sourcing, template-based environment construction, task instantiation, batched execution, and LLM-assisted quality filtering in Windows office applications. The corpus contains 131K validated successful action steps across 17K trajectories for training and evaluation, together with 1.09M failed execution steps for failure analysis and future recovery training. Each step provides screenshots, accessibility metadata, instantiated goals, reasoning traces, and structured GUI/API actions. GUI-360° supports GUI grounding, screen parsing, and action prediction under a hybrid GUI+API action space. Experiments show that current vision–language models struggle with fine-grained grounding and stepwise action prediction, while supervised fine-tuning on GUI-360° brings consistent in-domain and external benchmark gains. Our code and dataset are available at https://github.com/2020-qqtcg/GUI-360 and https://huggingface.co/datasets/vyokky/GUI-360.

Abstract:
Social deduction games have become a popular testbed for probing reasoning, deception, coordination, and belief modeling in Large Language Model (LLM) agents. However, most environments are scored only by game outcomes such as win rates and largely remain to text-only interaction, making it difficult to tell whether an agent’s language is actually grounded in what it perceived and did, or to identify the failure modes underlying its behavior. To address this gap, we introduce QUACK, an open-source environment and evaluation framework for auditing the grounding of agent language in multimodal social reasoning. QUACK evaluates agents at three levels: game outcomes, behavioral trajectories, and utterance-level consistency. Its core Statement Verification Pipeline reconstructs each agent’s ground-truth trajectory from engine logs and checks every discussion claim against it, automatically flagging spatial hallucination, unsupported accusation, deception collapse, and language-action inconsistency. Evaluating three frontier VLMs in both homogeneous and cross-model adversarial settings, we find that even the strongest agent hallucinates 15.1% of its verifiable spatial claims and 11.5% of accusations are strictly unsupported. We release the full engine, evaluation framework, toolkit, and logs in https://github.com/AAAAA-Academia-Attractions/QUACK.

Abstract:
Safety evaluation of Large Language Models (LLMs) remains heavily English-centric, leaving Low-Resource Languages (LRLs), particularly African ones, critically underexplored. We introduce TUKABENCH, a jailbreak benchmark for seven African languages that extends JailbreakBench (JBB) beyond direct translation through four settings: human translation of JBB prompts, English adaptation to African contexts followed by human translation, human curated prompts validated through interaction with GPT-5.2 to filter out easily detectable patterns, and code-switched prompts combining English and African languages. These settings isolate the effects of language, cultural grounding, and prompt evasiveness on model safety. Across closed and open models, prompting in African languages reduces refusal relative to English, with culturally adapted prompts leading to least refusal. The evaluation also surfaces two structural limitations: model comprehension failures and reduced LLM-as-a-judge reliability in LRLs. To capture the first, we introduce Deflection alongside Refused and Jailbroken; to assess the second, we validate outputs with human annotations, showing that judge-human agreement drops in lower-resource languages and less commonly supported scripts.

Abstract:
Evaluations of large language models (LLMs) primarily emphasize convergent logical reasoning, where success is defined by producing a single correct proof. However, many real-world reasoning problems admit multiple valid derivations, requiring models to explore diverse logical paths rather than committing to one route. To address this limitation, we introduce LogicGraph , the first benchmark aimed to systematically evaluate multi-path logical reasoning, constructed via a neuro-symbolic framework that leverages backward logic generation and semantic instantiation. This pipeline yields solver-verified reasoning problems formalized by high-depth multi-path reasoning and inherent logical distractions, where each instance is associated with an exhaustive set of minimal proofs. We further propose a reference-free evaluation framework to rigorously assess model performance in both convergent and divergent regimes. Experiments on state-of-the-art language models reveal a common limitation: models tend to commit early to a single route and fail to explore alternatives, and the coverage gap grows substantially with reasoning depth. LogicGraph exposes this divergence gap and provides actionable insights to motivate future improvements. Our code and data will be published at https://github.com/kkkkarry/LogicGraph/.

Abstract:
Interview assessment requires per-criterion judgments grounded in behavioral evidence, yet surging applicant volumes have made human-only evaluation costly and inconsistent, while existing AI approaches yield opaque scores without traceable rationale. We introduce PhoenixNest-Video, an evidence-grounded multimodal agent framework for automated video interview assessment. It builds a semantic video graph as structured working memory, performs rubric-conditioned retrieval with cross-modal verification across visual, audio, and textual streams, and produces per-criterion scores anchored to the candidate’s materials. A Scorer trained via Rubrics-based Reinforcement Learning with dual rewards for rubric alignment and score-level differentiation internalizes the discriminative structure of multi-level rubrics.PhoenixNest-Video attains 91.50% grade-level accuracy on VInterview-2025, outperforming substantially larger proprietary models. These results demonstrate that PhoenixNest-Video delivers stable, expert-aligned scoring across candidates, providing a reliable framework for transparent and auditable interview assessment.

Abstract:
Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present SkillSafetyBench, a runnable benchmark for evaluating such skill-mediated safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold–model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. We release the data and code in an anonymous repository to support reproducibility. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.

Abstract:
Post-trained language models often exhibit a recognizable AI-like writing style, yet its connection to post-training and internal representations remains poorly understood. In this work, we study whether post-training introduces or amplifies AI-like stylistic regularities and whether these regularities have a localized internal signature. To this end, we compare human text, base-model generations, and aligned-model generations under matched human-source prefixes. Aligned generations show lower human-corpus affinity and higher AI-detection rates than base generations, suggesting that post-training shifts generated text away from human-corpus style and toward detector-visible AI-like text. We then introduce PASTA (Post-training Alignment Signature Targeted Ablation), a training-free method that estimates a post-training alignment signature from aligned-base residual contrasts and ablates the corresponding direction during decoding. Across 11 aligned models and 6 AI detectors, PASTA lowers the detection rate for most aligned models; this effect transfers well across detectors and is not reproduced by random directions. Qualitative analysis suggests that PASTA generations remain relevant and coherent while exhibiting greater stylistic variation. Together, these results show that AI-like stylistic effects of post-training can be measured, localized, and causally tested through activation ablation.

Abstract:
Large Language Models (LLMs) are increasingly used for emotional support tasks, such as negative thought reframing. This task relies on modifying cognitive appraisals, the subjective interpretation of events that elicit negative emotions, which is typically conceptualized along multiple discrete dimensions. Current LLM-based frameworks model cognitive appraisal by exhaustively evaluating all possible dimensions, but they fail to account for the varying saliency of these dimensions across different contexts. In this work, we investigate a vital yet overlooked question: "Can LLMs infer the salient appraisal dimensions from emotional support conversations?" To address this question, we introduce the AppraiSal benchmark, containing 996 emotional support conversations with human-annotated mental states, including salient cognitive appraisal dimensions. Furthermore, we propose PRISM, a multi-agent probabilistic framework grounded in Bayesian Inverse Planning, designed to improve LLMs’ ability to identify context-specific appraisal dimensions. Experimental results show that PRISM brings improvements to LLMs across various sizes, particularly in identifying the most salient appraisal dimensions

Abstract:
Scientific citations carry rhetorical intent. Scholars may cite prior work positively (supporting), negatively (contrasting), or neutrally (mentioning). As large language models (LLMs) increasingly assist scientific writing, whether they reproduce citations with the same rhetorical intent as humans remains unclear. We introduce a masked-citation task to compare human and LLM-generated citation behavior. For each citation context, an LLM generates a replacement citation sentence, producing a counterfactual corpus directly comparable to human citation. We analyze what, whom, and how models cite, using an LLM-as-a-judge to classify citation intent and a 20-million-edge coauthorship network to measure social distance between cited authors. Across six popular LLMs and 1,746 top NLP conference papers (63k+ contexts, 132k+ citations), three patterns emerge: (1) Compared with human citation, LLMs cite significantly less critically; (2) LLMs over-cite popular and older papers, a tendency amplified for contrasting citations where human writing more often draws on recent, niche work; (3) Whereas humans often cite within their close social network, especially for supporting citations, LLMs tend to draw on more socially distant authors. Together, these differences are double-edged: LLM citation reaches beyond a scholar’s close collaborators while being less critical and amplifying visibility bias, reshaping the rhetoric and reach of scientific citation.

Abstract:
The rapid advancement of Large Language Model (LLM)-based Multi-Agent Systems (MAS) has introduced significant security vulnerabilities, where malicious influence can propagate virally through inter-agent communication. Conventional safeguards often rely on a binary paradigm that strictly distinguishes between benign and attack agents, failing to account for infected agents, i.e., benign entities converted by attack agents. In this paper, we propose Infection-Aware Guard, INFA-Guard, a novel defense framework that explicitly identifies and addresses infected agents as a distinct threat category. By leveraging infection-aware detection and topological constraints, INFA-Guard accurately localizes attack sources and infected ranges. During remediation, INFA-Guard replaces attackers and rehabilitates infected ones, avoiding malicious propagation while preserving topological integrity. Extensive experiments demonstrate that INFA-Guard achieves state-of-the-art performance, reducing the Attack Success Rate (ASR) by an average of 33%, while exhibiting cross-model robustness, superior topological generalization, and high cost-effectiveness. Code is released under https://github.com/yjzscode/INFA-Guard.

Abstract:
Psychological research on emotion dynamics has established that human affect is a continuous, evolving process: emotions rise, decay, and transition within seconds. Current emotional text-to-speech (TTS) systems, however, condition on a single discrete label or static embedding per utterance, fundamentally misaligning with the temporal nature of affect. While recent LLM-based TTS systems may implicitly vary prosody through text understanding, such variation is neither explicitly controllable nor precise enough for targeted intra-utterance transitions. We address three challenges: (1) a multi-pass flow blending pipeline synthesizes frame-aligned transition audio, circumventing the scarcity of natural intra-utterance transitions; (2) dual-stage Valence–Arousal–Dominance (VAD) conditioning guides prosodic planning in the LLM and acoustic realization in the flow decoder via frame-level VAD embeddings; (3) direction–magnitude decoupled injection structurally separates emotion direction from injection magnitude, preventing content degradation. EmoTra-TTS adds only +0.43% parameters with no latency overhead, achieves 30%–87% relative improvement on emotion transition quality, corroborated by 67.5%–79.5% overall win rates in a pairwise preference test against four SOTA baselines. The complete code, including data synthesis and training pipelines, will be released after the review process.

Abstract:
Visual tokens dominate the inference cost of multimodal large language models (MLLMs), but aggressive token reduction often degrades reasoning. We argue that the fragility comes from a saliency-ranking view, i.e., existing training-free methods typically rank tokens by a scalar proxy, such as cross-attention, [CLS] attention, or feature similarity, and then prune or merge tokens independently. However, visual evidence is set-valued and structured: salient patches can be redundant, low-saliency tokens can be answer-critical, and feature-similar patches can be unsafe to merge when they are spatially incompatible. We propose CondenseVLM, a simple framework that formulates visual token compression as a budgeted evidence-preservation question. The formulation jointly optimizes salient evidence coverage, contextual coverage, and spatially constrained merge fidelity. It yields a monotone submodular coverage term, a marginal-gain interpretation of diversity-aware anchor selection, and a local distortion bound for spatial-semantic merging. Comprehensive experiments across six MLLM families and diverse general-purpose, grounding, text-dense, and video benchmarks demonstrate that improves the accuracy-efficiency trade-off, and achieves the SOTA performance.

Abstract:
Text-to-spatial audio generation, such as text-to-First-Order Ambisonics (FOA), provides a convenient way to create spatial audio for billion-dollar gaming and film industries. However, existing text-to-FOA methods are largely data-driven and may produce audio that violates acoustic relations between source direction and distance. They also separate descriptive and parametric control, forcing users to trade usability for precision. In this paper, we present PhysWave, a physics-guided latent diffusion model for controllable text-to-FOA generation. PhysWave unifies natural-language and trajectory control through a shared waypoint-caption representation, and augments diffusion training with two differentiable acoustic priors: spherical-harmonic direction consistency and inverse-square distance consistency. To support dynamic spatial generation, we further construct a 300K-clip FOA dataset with diverse sound categories and source trajectories. Extensive results show that the proposed priors help PhysWave generate spatially consistent FOA audio while maintaining competitive audio quality. Further analyses show that these physics priors improve spatial consistency during training and can also be used as inference-time guidance for training-free spatial refinement.

Abstract:
Machine unlearning for large language models (LLMs) aims to remove specified knowledge while preserving the rest of the model’s capabilities. However, the boundary between knowledge to forget and knowledge to retain is often unclear, since related and even distant information may be entangled in the model. In this paper, we study LLM unlearning from a data-centric perspective and measure how unlearning effects propagate from the forget set to same-domain and distant-domain knowledge not after but before unlearning. We find a consistent decay pattern: collateral damage is strongest near the forget set, weakens with semantic distance, but does not disappear at domain boundaries. We further ask whether such damage can be audited before unlearning is executed. We formulate forget-set auditing as a pre-unlearning prediction task and analyze which data features are most predictive of downstream damage. Our results show that interaction features between the forget set and evaluation set provide the strongest signals, suggesting that collateral damage is partly reflected in data geometry before model updates occur. These findings position forget-set auditing as an early warning tool for identifying risky unlearning runs and designing more reliable unlearning procedures. Code and data are available at https://github.com/BartSu/PreUnlearn.

Abstract:
Large language models (LLMs) have demonstrated strong reasoning ability when given additional compute at inference time. However, existing inference-time scaling methods are fundamentally limited by their design. On the one hand, goal-oriented approaches, such as Line or Tree Search, refine candidate solutions using previous feedback but are vulnerable to sequential dependence, often collapsing into suboptimal reasoning trajectories. On the other hand, process-oriented approaches such as Best-of-N sampling encourage diversity through random exploration but lack feedback mechanisms, leading to inefficient computation allocation and unguided search. In this work, we propose EvoMo , a novel inference-time scaling approach that unifies both paradigms by embedding a globally evolving strategy pool into MCTS, where each node expansion selects reasoning strategies under an -soft policy. To further avoid stagnation in familiar strategies, we introduce a momentum-based optimization mechanism that monitors similarity among generated solutions and encourages the exploration of underutilized strategies. We also present a theoretical analysis of the effectiveness of the proposed momentum mechanism by modeling it as a self-evolving pool of actions. Across benchmarks, EvoMo reveals significant performance gains over SOTA inference scaling methods. We observe competitive performance, like in APPS and CodeContests, with up to a 4.5% and 3.63% relative gain in Pass@K compared to the leading baseline within 40 search-budget, highlighting the potential of hybrid evolutionary search as a general paradigm for test-time scaling.

Abstract:
The recent advancement of time series has witnessed Vision Language Models (VLMs) as a powerful tool for a variety of tasks of multimodal time series analysis, including forecasting, classification, question answering, etc. However, with time series as an explicit modality, inputs with long time series horizons yield excessive numeric tokens for VLMs, resulting in high inference latency. Moreover, such long contexts often contain redundant or noisy segments that might distract the model and thus degrade downstream performance. Although previous token pruning methods can reduce visual or textual tokens, they are mostly unimodal and do not explicitly explore multimodal semantics for time series. To address this, we propose TimePrune , a plug-and-play temporal pruning framework for VLMs in multimodal time series analysis, which can accelerate VLM inference without fine-tuning the VLM backbone. Specifically, TimePrune formulates time series pruning as a budgeted temporal selection problem and introduces a Trimodal Budgeted Network to score temporal patches via jointly conditioning on linguistic and visual semantics. Then, under a fixed budget, this framework is optimized via the proposed Narrative-Calibrated Binned Grounding constraints, to further facilitate context-aware and temporally-coherent pruning. Extensive experiments in four time series tasks across application domains on multiple VLM backbones show that TimePrune could consistently achieve better performances. For example, on Qwen3VL-8B, TimePrune improves average forecasting performance by 33.91%.

Abstract:
Safety alignment is essential for deploying large language models, requiring systems to prevent harmful compliance while preserving helpfulness on benign requests. Activation steering offers a training-free inference-time approach to safety control, but effective safety steering requires addressing two coupled questions: when to intervene and how generation should be shaped after intervention. However, existing safety steering methods remain limited along both dimensions, as their triggering mechanisms can be unstable across domains and refusal-oriented steering often yields rigid refusals rather than constructive safe guidance. To address these limitations, we propose ALTSTEER, an inference-time framework that couples selective intervention with refusal-anchored constructive redirection within a single inference pass. ALTSTEER uses an internal refusal-relevant signal to decide when to steer, and applies staged steering to shift generation from refusal-oriented control toward constructive alternatives. Evaluations on Llama-3.1 and Qwen2.5 show that ALTSTEER preserves benign utility while improving constructive safe-completion behavior, especially on models that otherwise tend to produce short refusals for harmful requests.

Abstract:
Current emotion analysis in social media is predominantly author-centric, failing to capture the subjective nature of emotional responses across diverse readers. This paradigm overlooks the crucial link between individual perception, implicit propagation intent, propagation behavior, and the underlying social network. To bridge this gap, we introduce EmoIntenNet, a novel bilingual (English and Chinese) large-scale benchmark that uniquely integrates six critical dimensions for social perception: author-created content, actual readers’ emotional feedback (derived from their comments), implicit propagation intent, explicit propagation behavior, user attributes, and the social relationship graph. This benchmark advances the shift towards more granular, reader-centric analysis, where different readers’ emotional responses to the same content are grounded in their actual social interactions rather than simulated annotations. By annotating implicit emotions from reader comments and synchronously capturing implicit multi-label propagation intents, EmoIntenNet provides a unique resource to model the intrinsic triadic coupling among implicit emotion, implicit intent, and explicit behavior, grounded in social context. We establish a comprehensive evaluation protocol, testing state-of-the-art methods, including large language models (LLMs) with advanced reasoning enhancement. Our findings reveal significant shortcomings in existing approaches when handling this multifaceted, user-aware task. EmoIntenNet offers a foundational resource and clear direction for future research in socially-intelligent NLP, pushing models towards a more unified understanding of emotion on social media.

Abstract:
Benchmark scores are treated as evidence of LLM competence, but the same knowledge can yield different scores under different question formats. MCQs provide objective and scalable scoring, but their answer options can turn recall into recognition through answer-option scaffolding. Open-ended constructed response questions (CRQs) remove answer options, but their scores often depend on human or LLM judges and rubric interpretation, introducing judge-dependent scoring. This creates a trade-off between removing answer-option scaffolding and retaining objective scoring. We study whether this trade-off can be reduced through Constrained Structured Response Questions (CSRQs), a structured-response protocol that removes final answer options while preserving deterministic scoring. CSRQs use multi-select, ordering, and matching outputs as auditable answer spaces for exact verification. We evaluate CSRQs using converted knowledge benchmarks and MyCulture, which is, to our knowledge, the first Malaysia-centred cultural benchmark in Bahasa Malaysia, with parallel MCQ, CSRQ, and CRQ formats. Across benchmarks and model families, scores change when the evaluation instrument changes, i.e., scores drop after answer options are removed, and CRQ scores vary across judges. These findings suggest that benchmark scores should be interpreted as products of both model capability and the test used to measure it.

Abstract:
Large audio language models (LALMs) process both speech and environmental acoustic cues, yet struggle to retain non-speech information across multi-turn interactions. The performance gap between semantic (speech) and acoustic (non-speech) understanding remains poorly understood, and the underlying mechanisms of representation and retrieval are still unclear. This work introduces EnvMem, a controlled multi-turn benchmark designed to study this gap and identify the root causes of failures at the representation (i.e., latent embeddings) and retrieval levels (i.e., attention allocation). We further conduct post-hoc interventions to probe representational structure and attention dynamics. Our results reveal representational trajectory drift as the key failure mode, while showing that attention allocation tested here does not consistently explain the degradation. Overall, we provide a systematic framework for analyzing and improving non-linguistic memory in long-context LALMs, shedding light on future data and training design for robust acoustic memory modeling.

Abstract:
Cross-view video representation learning aims to capture viewpoint-invariant action semantics despite substantial appearance changes across egocentric and exocentric videos. However, existing methods encode each video as a unified embedding, where view-invariant and view-variant semantics inevitably entangle under co-occurrences — a failure mode we show persists even in cross-view methods explicitly trained for view-invariance. Our key insight is that a view-invariant feature is truly disentangled when it can be sufficiently recomposed with an arbitrary view-variant feature while preserving their independent semantics. Building on this, we propose PRISM, that decomposes video into view-invariant and view-variant latents and recompose them under language supervision encouraging clean decomposition of the two streams. PRISM achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting. Code is available at [https://github.com/litcoderr/prism](https://github.com/litcoderr/prism).

Abstract:
Documents spanning tens to hundreds of pages are pervasive in real-world workflows, from scientific papers to financial reports and technical manuals. In this setting, answer-relevant evidence is sparse and distributed across pages, and the document itself often exceeds a backbone MLLM’s context window, making MP-VRDU a distinct problem from single-page VRDU. We survey MP-VRDU through the lens of evidence management, the organising principle that distinguishes multi-page from the single-page setting: the problem of deciding how document-scale evidence is selected and reasoned over when it cannot all be held at once. We introduce a taxonomy spanning four architectural families, multimodal representation, inference-time adaptation, and training strategies for cross-page evidence. We further consolidate MP-VRDU datasets and outline open challenges in efficiency, faithfulness, evidence pipelines, supervision integrity, and reproducible evaluation that define the frontier.

Abstract:
Activation steering has been used to control traits such as honesty, refusal, and sycophancy, yet supportive empathy is evaluated along multiple dimensions that need not correspond to independently controllable activation directions. Using the EPITOME framework, which decomposes supportive empathy into Emotional Reactions, Interpretations, and Explorations, we study three instruction-tuned LLMs and ask whether candidate directions derived from these labels produce distinguishable intervention effects or instead share structure, and how persona prompts interact with those directions. We find that contrastive activation addition yields a stable middle-layer intervention that consistently shifts the EPITOME proxy scores across models, moving empathy analysis beyond response-level scoring. However, the recovered directions are only partially separable: steering one direction induces off-target shifts, and hand-crafted prompting shifts the empathy profile rather than isolating a single dimension. Persona prompts substantially change EPITOME scores, but a paired activation-shift decomposition shows the recovered subspace captures only about 3% of persona-induced squared activation-shift magnitude at layer 15. Under this EPITOME-based definition, expressed empathy is steerable but multi-axial, and controlling persona-conditioned empathy requires targeting structure beyond individual mechanism directions.

Abstract:
Large language models (LLMs) remain challenged by ultra-long document understanding. A common solution is to segment documents into chunks and retrieve those relevant to a query, but this can disrupt semantic and logical coherence by ignoring dependencies between chunks. We propose NEAR (NEighborhood-Aware context Reconstruction), a lightweight, training-free framework that reconstructs coherent context from fragmented segments. NEAR represents each document as a dual-layer neighborhood graph, connecting chunks via topical neighbors to capture long-range semantic relations and physical neighbors to preserve local continuity. An iterative dual-pool exploration traverses chunks from both global and local neighborhoods, and collected chunks are selectively expanded and reordered according to their original positions to form a coherent input context. Extensive experiments on multiple long-document question answering benchmarks demonstrate that NEAR improves comprehension performance while maintaining low latency and memory usage.

Abstract:
Large language models are being extensively used to simulate individual user behavior, yet faithfully representing a population requires capturing the systematic variation in values, beliefs, and cultural norms that distinguish one group from another. We introduce (), a suite of models each aligned to specific populations, covering five countries: USA, India, Brazil, France and Italy. are created by synthesizing rationales grounded in psychological and cultural constructs and using these as latent supervision during preference tuning for population-specific alignment. Evaluated across four dimensions: personality, values and beliefs, cultural norms, and morality, consistently outperform baselines, including culture-specialized models, achieving an average of 8.59% relative improvement over the best baseline across all five populations. Notably, construct-grounded rationales outperform both demographic prompting and survey-based fine-tuning, suggesting that grounding preference learning in psychology and culture provides a richer inductive signal than surface-level response distributions. We further demonstrate strong generalization to downstream applications without task-specific supervision: outperforming best baselines by 5.19% in personalized reward modeling, 6.34% in population simulation, and showing strong transfer to social reasoning tasks. Datasets and code are available at: h ttps://github.com/limenlp/PALMs.

Abstract:
LLM-based multi-agent systems (MAS) solve complex tasks through communication among role-specialized agents. However, inter-agent dependencies introduce reliability risks beyond isolated agent failures. For instance, errors in intermediate messages could be inherited and amplified by downstream agents. Existing uncertainty quantification (UQ) methods mainly target isolated responses or single-agent reasoning, and therefore fail to capture uncertainty propagation in MAS. To this end, we propose PropUQ-MAS, an error propagation-aware UQ framework that represents MAS execution as a communication-structured graph and estimates each step’s reliability by combining local uncertainty with uncertainty inherited from upstream messages. Extensive experiments demonstrate that PropUQ-MAS consistently improves UQ in MAS, with average relative gains of +6.10% in AUROC and +47.58% in PRR.

Abstract:
Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Existing optimization-based attacks share a single objective: maximize a token-level proxy of victim compliance—such as the likelihood of an affirmative prefix—computed on the prompt rather than the decoded reply. This proxy cannot distinguish a cautious hedge from a fully actionable harmful answer, leaving the search blind to what the model actually says. We propose ECHO (Evaluation-Conditioned Harmfulness Optimization), which replaces the compliance proxy with response-level, continuous harmfulness scores from HarmSig, a 1.7B regressor grounded in actual harmful capability. ECHO couples HarmSig with five word-level perturbation operators and Keyword-Anchored Beam Search with global top- m memory, under a strict black-box threat model. Across five open-weight victims on AdvBench, ECHO achieves up to 0.898 StrongREJECT mean score and 89.23% HarmBench success. Our code is publicly available at https://github.com/STANWYW/echo.

Abstract:
Large language models are increasingly deployed as tool-augmented agents to acquire information beyond parametric knowledge. While recent work has improved long-horizon tool-use reasoning, most approaches focus on tasks with a single correct answer. In contrast, many real-world queries require discovering a comprehensive set of valid answers, a setting known as Multi-Answer QA. This setting raises two challenges: fine-grained credit assignment over long search trajectories and reward alignment for sustained exploration beyond easy high-frequency entities.We propose SPADER, a reinforcement learning framework for long-horizon tool use in Multi-Answer QA. SPADER includes Step-wise Peer Advantage (SPA), a critic-free step-level credit assignment mechanism that aligns parallel trajectories by decision step and estimates advantages from peer returns. It also includes a diversity-aware exploration reward that promotes long-tail entity discovery by upweighting rare findings and downweighting redundant ones.Experiments on QAMPARI, Mintaka, WebQSP, and QUEST show that SPADER generally improves recall and overall F1 over prompting-based agents, outcome-supervised RL methods, and recent step-level supervision approaches.Our code and model weights are available at https://github.com/KhanCold/spader.

Abstract:
Large language models (LLMs) are increasingly used as web-facing agents for information-seeking tasks over live websites. Many such tasks require multi-step in-site navigation and cross-page evidence aggregation, a setting known as web traversal. Existing LLM-based web agents, including recent explore–critic architectures, typically store interaction history as an ever-growing unstructured text trace and mix high-level planning with low-level page actions. This makes long-horizon behavior brittle, obscures the source of evidence, and often leads to redundant browsing or premature stopping. To mitigate these issues, we propose WebTraveler, a hierarchical multi-agent framework that separates high-level information-seeking planning from local web exploration and maintains a structured trajectory memory of decision-relevant signals. This design provides compact information interfaces for evidence sufficiency checking and webpage navigation, improving long-horizon stability and cross-page evidence aggregation. To obtain scalable supervision, we further introduce path-guided data synthesis, which uses executable navigation paths to collect trajectories, removes explicit path hints, and filters invalid samples through answer-consistency checks. Experiments show that WebTraveler consistently outperforms previous methods across LLM scales, especially on tasks requiring deeper navigation, with further gains from path-guided supervision. Moreover, WebTraveler also shows generalization to broader web navigation tasks. Code is available at https://github.com/RongchangLi/WebTraveler.

Abstract:
Mixture-of-Experts Multimodal Large Language Models (MoE-MLLMs) offer remarkable performance but incur prohibitive GPU memory costs, making compression essential. Among PTQ methods, expert-level mixed-precision quantization has proven effective for MoE-LLMs, yet suffers notable degradation on MoE-MLLMs due to two overlooked biases in expert importance estimation. (1) At the cross-modal level, the numerical dominance of vision tokens causes expert selection frequency to be dominated by vision tokens, masking experts that are critical to the text modality; (2) at the intra-vision level, the large proportion of redundant vision tokens further skew frequency statistics, obscuring experts critical for informative visual content. To bridge gaps, we propose MODE, a modality-decomposed expert-level mixed-precision quantization framework for MoE-MLLMs that decomposes expert selection frequency by modality, filters redundant vision tokens to obtain denoised visual frequency, and further evaluates quantization sensitivity per modality as a complementary signal to frequency-based estimation. These signals are integrated into an Integer Linear Programming formulation to assign per-expert bit-widths under a given budget. Extensive experiments show that MODE is particularly well-suited for MoE-MLLMs, limiting average performance loss to within 2.9% at W3A16, with larger gains at the extreme 2-bit setting.

Abstract:
Data-analytic agents must iteratively generate and execute code while interpreting real-time feedback to refine their analytical strategies. Existing training paradigms decouple the agent’s policy from the underlying environment dynamics, producing agents that learn to recover from errors rather than to avoid them: failure rates rise alongside recovery rates, and interaction turns inflate as agents resort to trial-and-error exploration. We introduce DataSight, a framework that internalizes environment dynamics by making the model’s anticipation of execution outcomes an explicit, structured output accompanying every generated action. DataSight first enriches expert trajectories with foresight prediction fields grounded in real execution feedback, then applies a dual-stage RL curriculum that calibrates prediction fidelity before optimizing long-horizon task performance, and at inference time intercepts anticipated failures before physical execution. Trained on only 7,273 trajectories, DataSight-4B surpasses both DataMind-7B and DeepAnalyse-8B in average accuracy, while DataSight-8B achieves 64.13% Avg@3 across four benchmarks with Pass@3 of 74.81%, approaching DeepSeek-V3.2 at 75.56%. Analysis confirms that foresight training generalizes across non-reasoning backbones and SQL-based tasks, improving both single-step code quality and multi-turn interaction efficiency.

Abstract:
Multimodal LLMs apply the language model interface to visual inputs, where ordinal regression tasks such as age estimation, image quality assessment, and disease grading require autoregressive decisions over ordered class labels. We ask whether MLLMs reliably convert internal ordinal evidence into ordered digit-token outputs. Across four ordinal benchmarks and four MLLM backbones, ordinal labels are linearly recoverable from hidden states with Spearman correlation up to 0.938, and a task-designed prompt further sharpens this structure. Yet native digit-token outputs weakly expose it: the unembedding matrix filters the ordinal direction, and the digit-token row space retains below 1.15% across all 16 model-dataset combinations, with a 16 to 77 absolute-point accuracy gap between linear-probe and native outputs. We introduce Ordinal Lens Alignment (OLA), a frozen-backbone inference-time method that trains lightweight Wₛ-anchored lenses on mid-to-deep decoder layers, fuses them into an ordinal distribution, and corrects only digit-token logits at generation. OLA outperforms the SOTA LoRA-tuned OrderChain baseline in most settings while keeping the MLLM frozen, surpasses discriminative ordinal baselines in most cells, and improves over an offline lens in every setting.

Abstract:
Deep-search agents are commonly trained via repeated interaction with real-world search APIs and browser environments. While realistic, this paradigm is expensive, difficult to reproduce, and sensitive to external drift. Existing simulator-based alternatives reduce dependence on live retrieval, but they largely inherit task distributions from pre-existing QA datasets, limiting their alignment with the behaviors required for deep search. We introduce SynSearch, a framework for training deep-search agents entirely within a synthetic search–browse environment. Our core idea is to jointly design the environment and the training-task distribution, enabling the synthesis of environment-aligned tasks and reinforcement learning of compact language models in multi-turn search settings. By grounding task validity in executable environment trajectories rather than surface-level plausibility alone, SynSearch unifies task generation, trajectory verification, and policy optimization within a low-marginal-cost and reproducible framework. Experiments on multi-hop QA and deep-search benchmarks show that SynSearch consistently improves both answer accuracy and search behavior, with especially large gains on deep-search tasks. Moreover, SynSearch offers low marginal cost, reproducibility, controllability, and strong performance under real-search evaluation, highlighting synthetic environments as an effective substrate for training deep-search agents.

Abstract:
Coding agents are increasingly used for software engineering tasks, including bootstrapping projects from third-party repositories whose integrity cannot be assumed. Prior work on repository poisoning largely focuses on attacker-controlled injection and disguise, but developers also shape risk through everyday invocation choices: what task to delegate, how to phrase the request, and which skills or rules to supply. We term these user-side choices Prompt-Level Configurations (PLCs) and introduce CIPR (Coding In Poisoned Repos), the first benchmark that systematically varies PLCs in poisoned real-world repositories. CIPR comprises 1,920 instances across 20 repositories, four task types, three social-media-grounded prompt styles, and three skill/rule conditions, and measures attack success rate (ASR) and agent alert rate (AR) using automated runtime and trace-based oracles. Our evaluation reveals two key insights: (1) Vulnerability is highly context-dependent, with task type creating up to a 4.5-fold difference in ASR, with test-execution task forming a silent attack surface (high ASR, low AR). (2) Prompt expression shifts risk indirectly: underspecified prompts reduce ASR by truncating execution depth; noisy prompts exhibit a directional trend toward suppressing alerts by making malicious content less conspicuous. These findings highlight that coding agent vulnerability is not a static property, but a dynamic outcome shaped by everyday user configurations.

Abstract:
OpenClaw, the most widely deployed personal AI agent in early 2026, operates with full local system access and integrates with sensitive services such as Gmail, Stripe, and the filesystem. While these broad privileges enable high levels of automation and powerful personalization, they also expose a substantial attack surface that existing sandboxed evaluations fail to capture. To address this gap, we present the first real-world safety evaluation of OpenClaw and introduce the CIK taxonomy, which unifies an agent’s persistent state into three dimensions, i.e., Capability, Identity, and Knowledge, for safety analysis. Our evaluations cover 12 attack scenarios on a live OpenClaw instance across four backbone models (Claude Sonnet 4.5, Opus 4.6, Gemini 3.1 Pro, and GPT-5.4). The results show that poisoning any single CIK dimension increases the average attack success rate from 24.6% to 64-74%, with even the most robust model exhibiting more than a threefold increase over its baseline vulnerability. We further assess three CIK-aligned defense strategies alongside a file-protection mechanism; however, the strongest defense still yields a 63.8% success rate under Capability-targeted attacks, while file protection blocks 97% of malicious injections but also prevents legitimate updates. Taken together, these findings show that the vulnerabilities are inherent to the agent architecture, necessitating more systematic safeguards to secure personal AI agents. Our project page is https://ucsc-vlaa.github.io/CIK-Bench.

Abstract:
Cross-lingual alignment in multilingual language models is typically attributed to joint training: shared parameters, mixed-language batches, or explicit alignment objectives. We ask whether monolingual models trained on non-parallel data learn alignable representations without joint training. By testing on strictly monolingual language models, such as the Goldfish model families and independently developed models from different research labs, we find three results. Correlation: these models develop alignable representational geometry across layers, with alignment strengthening as data scale, model scale, or linguistic proximity increases. Construction: a single Procrustes rotation fit on parallel sentences maps hidden states between models. Causation: the same rotation transfers functional content; patching a rotated English residual into a German model on a factual cloze flips the prediction to the donor’s capital in most cases. We confirm that cross-lingual alignment can emerge from the structure of language and the information it carries rather than from joint training, and this points to practical future directions including model stitching, merging, and modular multilingual systems built from monolingual components.

Abstract:
Multiple-choice music benchmarks can reward two different behaviors: listening to the audio and exploiting cues in the question/options. We introduce ListenOnly-Music, a 947-item perception-required slice of MuChoMusic and MMAU-Pro, by filtering out items that two strong text-only reasoners can solve from the question and answer options alone. We audit five public audio-LLMs, from early music-specialist systems to recent 7B-8B chat models, under audio, text-only, noise, and mismatched-audio conditions. On ListenOnly-Music, earlier 2023 systems are statistically equivalent to no listening under a pre-specified TOST margin, whereas newer models show positive audio contribution. We then evaluate two inference-time interventions. Audio-contrastive decoding improves perception-required accuracy but reduces accuracy on text-solvable items, making it a diagnostic probe rather than a blanket decoding policy. Audio-first chain-of-thought helps caption-style models but hurts MCQ-tuned models. Overall, music-MCQ accuracy should be reported separately for text-solvable and perception-required items: recent audio-LLMs do listen more than earlier systems, but benchmark shortcuts still obscure how much progress is genuinely audio-grounded.

Abstract:
As large language models are increasingly deployed to simulate diverse human characters, ensuring persona fidelity, defined as the extent to which an agent’s behavior consistently reflects the psychological and stylistic characteristics of a target persona, has become a critical requirement. However, existing evaluation paradigms primarily rely on either holistic LLM-based judges, which are prone to "holistic appraisal hallucination”, or static psychometric inventories, which fail to capture the context-dependent fidelity required in dynamic dialogue. To address these limitations, we propose PRISM (Persona Reasoning with Inverse SFL-based Modeling), a psycholinguistically grounded framework that reformulates persona fidelity evaluation as a structured inverse inference task. Inspired by Systemic Functional Linguistics (SFL), PRISM decomposes persona fidelity into three functional dimensions: Task Framing, Interpersonal Stance, and Linguistic Style. It estimates dimension-specific evidence over a persona-conditioned label space and aggregates these signals into an interpretable and auditable evaluation process. Experiments show that PRISM yields more accurate and stable judgements than traditional holistic judging, providing a more reliable framework for persona fidelity evaluation.

Abstract:
Reinforcement Learning (RL) has proven effective in eliciting reasoning abilities of Multimodal Large Language Models (MLLMs) and is emerging in document understanding. However, existing approaches usually lack fine-grained region-level grounding supervision, while the key challenge in document reasoning lies in linking visual evidence within precise regions directly to question–answer semantics. We introduce RegionDoc-R1, a novel framework for document understanding that enhances MLLMs’ reasoning with step-wise feedback. Instead of previous OCR-based text positions, we encourage the model to predict question-relevant Regions of Interest (RoIs) and ground the answer in these regions. Specifically, we first propose the Region-Aware Group Relative Policy Optimization (RA-GRPO), which encourages models to utilize region-level spatial information in documents for reasoning. Correspondingly, we construct a hybrid training corpus, named SR-Doc, containing Semantic Reasoning (SR) examples enriched with cross-page and region-level reasoning layout annotations. Meanwhile, we also introduce an Adaptive Chain-of-Thought (Ada-CoT) strategy, which dynamically adjusts the reasoning process according to different tasks, enabling more efficient and flexible step-wise document understanding. Experiments on several document reasoning benchmarks demonstrate that RegionDoc-R1 achieves state-of-the-art performance across tasks such as form document understanding, table-based QA, and layout-sensitive information extraction.

Abstract:
Recent years have witnessed significant progress in Unified Multimodal Models, yet a fundamental question remains: Does understanding truly inform generation in Unified Multimodal Models? To investigate this, we introduce UniSandbox, a decoupled evaluation framework paired with controlled, synthetic datasets to avoid data leakage and enable detailed analysis. Our findings reveal a significant understanding-generation gap, which is mainly reflected in two key dimensions: reasoning generation and knowledge transfer. Specifically, for reasoning generation tasks, we observe that explicit Chain-of-Thought (CoT) in the understanding module effectively bridges the gap, and further demonstrate that a self-training approach can successfully internalize this ability, enabling implicit reasoning during generation. Additionally, for knowledge transfer tasks, we find that CoT assists the generative process by helping retrieve newly learned knowledge, and also discover that query-based architectures inherently exhibit latent CoT-like properties that affect this transfer. UniSandbox provides preliminary insights for designing future unified architectures and training strategies that truly bridge the understanding-generation gap.

Abstract:
Subword tokenizers are optimized for compression, but their boundaries also define the prediction units seen by a language model. We introduce StructPiece, an unsupervised tokenizer that enforces Unicode Extended Grapheme Cluster (EGC) boundaries and biases merge induction toward corpus-induced affix patterns without external resources. Across nine languages and five scripts, StructPiece exposes a compression–representation frontier: it produces higher-fertility segmentations with worse bits-per-character but substantially lower token-level perplexity under matched architectures, vocabulary sizes, and token budgets. Fixed-budget, granularity-matched, and component ablations show that the effect is not explained by token count alone; structural vocabulary alignment is further tested through LLaMA-3.2-1B vocabulary surgery, replacing its 128k vocabulary with 32k StructPiece tokens via mean-pooled initialization. StructPiece recovers 89.4% of baseline zero-shot reasoning performance across morphologically rich languages (X-CSQA) after 2.6B adaptation tokens, compared with 76.4% recovery for naive BPE surgery under the same checkpoint. Furthermore, structurally aligned tokens produce higher mean-pooled embedding cosine coherence, suggesting better-conditioned vocabulary projections. These results suggest that token boundaries act as a controllable structural bias influencing prediction granularity and vocabulary adaptation. StructPiece is available at https://github.com/submissionemnlp26-beep/structpiece-code.

Abstract:
Dialectal variation is a defining characteristic of many low-resource languages, yet most NLP approaches overlook this complexity—often assuming uniformity or arbitrarily selecting a single variant without reporting which dialect is used. This assumption severely limits the applicability of language technologies in real-world communication, especially for languages without unified orthography where dialects could differ in vocabulary and morphology. In this paper, we present the first comprehensive study of dialect processing for Indonesian local languages, a linguistically rich and underrepresented setting. We introduce IndoDialect, a high-quality conversational dataset comprising 14.2K conversations across four major local languages and eleven dialects, including five Javanese, two Minangkabau, two Sundanese, and two Dayak dialects. Building on this resource, we define three core tasks: (i) dialect-level language identification, (ii) machine translation across dialects, and (iii) dialect-controlled generation. Our experiments reveal that state-of-the-art multilingual models perform poorly in few-shot scenarios, excel at dialect identification after fine-tuning, but struggle with machine translation and dialect-controlled generation.

Abstract:
Hybrid attention dominates frontier LLMs, yet Vision Transformers (ViTs) in multimodal LLMs lack a satisfactory hybrid design, with no consensus on why certain attention patterns work better. To fill this gap, we study ViT attention heads and find they differentiate into object- and background-specialist roles, a pattern most pronounced under full attention; we call this Semantic Head Specialization (SHS). We propose SHS-Index to quantify this specialization, show that it distinguishes full-attention from chunk-window ViTs, and find that it strongly tracks downstream benchmark performance. We then identify three structural factors that shape SHS—window interaction, token serialization, and local softmax allocation—and use them as design principles for hybrid attention. Guided by these factors, we design Ariadne Attention, a hybrid that matches full attention on 22 image and video tasks at 6.5 × less attention compute. Our findings establish head specialization as a measurable property for diagnosing and designing principled hybrid ViT attention at the multimodal-LLM scale.

Abstract:
Long-context LLM serving is increasingly bottlenecked by KV-cache memory, especially in resource-constrained scenarios. Among existing KV-cache compression strategies, token-wise methods reduce cached states but risk information loss through eviction or condensation, while feature-wise methods reduce per-token KV dimensions but can require full-dimensional reconstruction to apply positional embedding, limiting decoding speedups. We introduce SlimKV, a question-agnostic joint token-feature KV-cache compression method. SlimKV uses low-rank-aware training to compress long contexts into beacon memory states with latent KV representations, together with layer-adaptive rank allocation. We further uncover a positional asymmetry: removing key-side RoPE affects beacon and raw tokens differently, with much smaller degradation for beacon tokens. Exploiting this asymmetry, SlimKV trains beacon KV projections under a K-RoPE-free constraint and enables latent-space attention during decoding, mitigating reconstruction latency. On LongBench, SlimKV outperforms baselines at 16 × /32 × compression and remains leading at 4 × /8 × , where it retains over 96% of the uncompressed model’s score. Needle-in-a-Haystack confirms robustness across evidence positions, and efficiency evaluation shows up to 7.34 × attention speedup and 3.38 × end-to-end decoding speedup over the uncompressed model at 128K length.

Abstract:
Long-horizon memory question answering often requires sparse evidence from heterogeneous histories, including events, object states, visual observations, temporal relations, and causal steps. Existing memory interfaces expand reader context, retrieve semantically related chunks, or expose graph neighborhoods, but they are not explicitly designed to select compact evidence for a fixed reader. We propose Structured Spatiotemporal Scene-Event Memory (S3Mem), a query-time memory interface that writes textual, visual, and agent-use histories into structured scene-event units and routes compact evidence packs to the reader. Its router scores candidate units, query anchors, and anchor-support links, enabling both single-hop selection and short multi-hop evidence chains without reader fine-tuning or test-time training. Across LoCoMo, EMemBench Visual Games, and AMA-Bench, S3Mem provides a strong score-token trade-off, with the clearest gains on localized event, state, temporal, causal, or provenance evidence. On LoCoMo, S3Mem reaches 0.48 F1 and 0.40 BLEU with 1,073 evidence tokens per question, about 15.8 times fewer than the LoCoMo reference. On EMemBench Visual Games, it obtains the best F1 and second-best accuracy with only 189 tokens. On AMA-Bench, it is not the highest-scoring method, but remains competitive while using the fewest reader-visible evidence tokens.

Abstract:
We argue that financial report generation should operate at the analytical rather than structural level, composing content from data-derived insights rather than high-level topics or sections. To this end, we propose AnalysisBank, which distills expert reports into a reusable library of Analyses, each pairing a data signal, an analytical move, and the expert span it was derived from. At inference time, AnalysisBank matches input signals to library entries and applies the retrieved moves to compose the report. A study of Analyses distilled from 550 expert reports reveals a heavy-tailed distribution of 47–52 signal types spanning 13 move types. On two financial benchmarks across four LLM backbones, AnalysisBank increases the proportion of novel, data-grounded insights by 1.7–3.7× over structural-level baselines. Transfer to scientific writing suggests that the distinction generalizes beyond finance. Code and the distilled Analysis library are available at [https://github.com/yajingyang/AnalysisBank](https://github.com/yajingyang/AnalysisBank).

Abstract:
Recent advances in Reinforcement Post-Training (RPT) have substantially improved the mathematical reasoning capabilities of large language models (LLMs), but it remains unclear how well such gains transfer across languages. In this work, we study cross-lingual mathematical reasoning transfer in English-centric LLMs under controlled multilingual evaluation. We first evaluate a range of English-centric reasoning models on multilingual mathematical reasoning benchmarks and find that transfer is highly variable across initial model choice, target language, and training paradigm. Through controlled comparative experiments, we further find that stronger English-centric initialization does not necessarily lead to stronger relative transfer efficiency, even when it yields higher multilingual reasoning accuracy. Finally, we show that the largest improvement comes from moving beyond English-only supervision: adding a single parallel language yields a substantial gain, while further gains from additional languages are smaller but consistent. Overall, our results highlight the need to evaluate multilingual mathematical reasoning directly, using both absolute performance and relative transfer-efficiency metrics.

Abstract:
As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnostic ability over given medical images and texts, implicitly assuming that standardized medical images, texts, or question–answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources. In this paper, we study this missing step, i.e., raw medical data standardization. Specifically, models are given raw dataset folders and evaluated on their ability to identify source formats, convert raw medical images into VLM-compatible visual inputs, extract relevant textual information, and organize the results into structured image-text pairs. To construct this Medical Data Standardization Benchmark (MDS-Bench), we manually annotate 1,939 raw medical data standardization tasks covering diverse clinical practice, radiology modalities, annotation formats, and directory layouts. Extensive experiments show that even the best performing VLM, i.e., Gemini 3 Flash, achieves only a 48.6% end-to-end success rate. Our research highlights raw medical data standardization as a critical bottleneck for medical AI diagnosis in real practice.

Abstract:
Latent reasoning reduces the token-generation cost of chain-of-thought reasoning by replacing explicit intermediate tokens with continuous latent transitions. However, existing latent reasoning methods usually rely on dense and entangled transitions, making their reasoning trajectories difficult to inspect or intervene on. We introduce LSTR (Latent Sparse Transcoder Reasoning), a framework that turns sparse transcoders from post-hoc diagnostic tools into in-loop, intervenable transition components for latent reasoning. At each latent step, a Latent Transition Transcoder (LTT) combines a linear skip path with a Top-k sparse innovation path, exposing a small set of active sparse features. Under matched compression settings, LSTR offers a mechanistically inspectable alternative to dense latent reasoning. On GSM8K-Aug, ablating only a few top-active sparse features reduces accuracy by up to 16.5%, whereas analogous interventions have much smaller effects in dense latent baselines. These results indicate that the active sparse features are causally involved in the latent transition process, rather than merely post-hoc descriptors. Additional experiments on mathematical benchmarks and StrategyQA suggest that sparse latent transitions can preserve the compression benefits of latent reasoning while making the resulting trajectories more inspectable and intervenable.

Abstract:
Agentic Reinforcement Learning (ARL) trains large language models to interleave reasoning with external tool execution to solve complex tasks. Most existing ARL methods train a single set of parameters to support both reasoning and tool-use behaviors, implicitly assuming that joint training leads to improved overall agent performance. Despite its widespread adoption, this assumption has rarely been examined empirically. In this paper, we systematically examine this assumption by introducing Capability Effect Attribution (CEA), which provides quantitative evidence of interference between reasoning and tool-use behaviors. Through an in-depth analysis, we show that these two capabilities often induce misaligned gradient directions, leading to training interference that undermines the effectiveness of joint optimization and challenges the prevailing ARL paradigm. To address this issue, we propose Disentangled Action–Reasoning Tuning (DART), a simple and efficient framework that explicitly decouples parameter updates for reasoning and tool use via separate low-rank adaptation modules. With this simple change alone, DART outperforms all joint-optimization baselines and approaches the 2-Agent upper bound across thirteen benchmarks on retrieval-augmented QA and NL2SQL, further supporting our finding of capability interference under shared optimization.

Abstract:
Language models exhibit positional bias on multiple-choice benchmarks, scoring the same question differently depending on which position holds the correct answer. Existing work measures the bias at the output and corrects it by shuffling option orders, but does not explain where in the network it originates. Using causal activation patching with a formalized Average Indirect Effect (AIE) metric, we trace the layer-by-layer formation of positional commitment across three model families and benchmarks. Positive causal mass concentrates in early-to-mid layers (normalized depth below 0.60) for the higher-capacity models tested, with Pearson correlations above 0.89 across nine model-dataset conditions and three output metrics, and the localization is stable under multi-layer windowed patching. A controlled factorization that permutes option labels and option order independently shows this profile tracks the label-identity component of the bias rather than generic order sensitivity. Sublayer decomposition reveals MLP-dominated early layers (about 59%) and attention-dominated mid-layers (about 76%), consistent with label information being integrated through attention after semantic content is encoded by MLPs. Negative controls show no detectable early-layer separation between robustly correct and bias-manifesting questions, supporting a shared answer-commitment pathway rather than a dedicated bias circuit and explaining why output-level shuffling stabilizes scores without altering internal preference.

Abstract:
Counterfactual debiasing is often used to reduce reliance on identity-linked shortcuts in harmful language understanding. We show that the same intervention can be harmful when identity mentions are not shortcuts but label-defining evidence. In multi-task cyberbullying modeling, identity cues may be spurious for cyberbullying detection (CD) and sentiment analysis (SA), yet necessary for fine-grained target identification (TI). We propose task-aware routing, which sends CD and SA through a debiased representation while preserving the original representation for TI. We evaluate this approach on the English-translated BullyExplain benchmark, HateXplain, and an external Measuring Hate Speech diagnostic. Experiments show that task-aware routing improves TI, while auxiliary detection and sentiment performance remain largely unchanged where applicable. Ablation studies attribute this improvement to representation routing that preserves identity-bearing evidence for TI rather than to class-weighting choices, with reduced-architecture diagnostics suggesting that the effect is not specific to the graph-enhanced model. These results support a task-conditional view of counterfactual debiasing: the same identity cue can warrant suppression in one task and preservation in another.

Abstract:
In-context learning (ICL) enables large language models to adapt from a few demonstrations, yet its sensitivity to context perturbations remains poorly understood. We study the context-conditioned map C ↦ f𝜃(x; C) through Lipschitz analysis. Modeling the Transformer as a composition of attention and feed-forward layers, we derive a worst-case certificate |f(x; C + 𝛿) - f(x; C)| ≤ L|𝛿| and a data-dependent local estimator based on finite-difference Jacobian probes. Across six language models (117M–7B) on controlled arithmetic ICL tasks, we find that: (i) output changes saturate as perturbation magnitude increases; (ii) global spectral bounds are extremely conservative, while local estimates are far closer to empirical sensitivity; (iii) sensitivity concentrates in late attention layers; (iv) additional demonstrations often stabilize predictions through redundancy; and (v) perturbation direction matters: label flips have much larger normalized effect than reorderings. SST-2 validation supports the same local-sensitivity and perturbation-direction trends. Our results position Lipschitz analysis less as a tight global predictor and more as a diagnostic framework for locating and comparing ICL vulnerabilities.

Abstract:
Criminal judgment prediction requires models to infer statutory articles, charges, and sentencing outcomes from case facts. Unlike standard classification tasks, it involves a structured reasoning process in which statutes should be matched with facts, charges should be justified by statutes, and sentencing outcomes should remain consistent with charges. Existing approaches optimize final labels, and while some have attempted to evaluate reasoning quality, their evaluations are indirect, often relying on LLM-generated rubrics that reflect model-internal preferences rather than the inherent logical structure of legal adjudication. We propose Juris Policy Optimization (JPO), a post-training framework for structured legal reasoning in Chinese criminal judgment prediction. JPO first uses teacher-generated rationales to supervise a standardized four-step reasoning process, and then applies reinforcement learning with a composite reward over legal prediction quality, reasoning structure completeness, and cross-step consistency. JPO further introduces token-level advantage reweighting and adaptive clipping for legally salient reasoning segments. Experiments on multiple open-source language models and three Chinese legal benchmarks show that JPO consistently improves both judgment prediction and reasoning quality over supervised fine-tuning and reinforcement learning baselines.

Abstract:
LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving pre-execution monitoring of step-level actions underexplored. We propose StepGuard, a step-level guard model that can audit completed agent trajectories and check tool actions before they are executed. To train StepGuard, we introduce StepGen, an automatic data engine that generates safe and unsafe trajectories with the same context but different actions at the risky step. To further reduce over-defense and under-defense, we propose Balance-GRPO, which dynamically balances learning between safe and unsafe actions based on their observed accuracy. Experiments show that StepGuard achieves the highest average accuracy among open-weight guard models, with performance comparable to GPT-5.4. When used to guard agents on AgentDojo and AgentDyn, StepGuard reduces mean ASR by 77.3% relative to the no-guard setting, while mean utility drops by only 2.8 points.

Abstract:
Long-form story generation should not only maintain narrative coherence, but also shape how readers experience a story as it unfolds. However, existing LLM-based generators typically optimize plot structure, consistency, or local text quality, without explicitly modeling reader-response dynamics as generation-time control targets. To solve the problem, we introduce ReaderScope, a reader-response-guided framework that converts simulated reader-response trajectories into explicit control signals for controllable long-form story generation. ReaderScope calibrates temporal response patterns from reference novels across affective, suspenseful, immersive, and reflective dimensions, and uses these targets to guide iterative story revision. Experiments against representative long-form story generation baselines show that ReaderScope improves reader-facing story quality under both automatic and human pairwise evaluation, while ablation and trajectory-fidelity analyses confirm that the response signals are useful for generation and can be reliably controlled.

Abstract:
Zero-shot Object Goal Navigation requires an embodied agent to locate a target object in an unseen environment, where the choice of which frontier to explore next is critical to both success and efficiency. Existing methods score frontiers from already-observed cues such as nearby objects or scene graphs, leaving the structure beyond each frontier implicit and offering no principled prior for choosing among rooms the agent has yet to see. In contrast, the hippocampal entorhinal system supports navigation through a predictive cognitive map that anticipates unseen space from partial observations. Inspired by this, we propose PredNav, which externalizes a hierarchical cognitive map with metric, instance-semantic, and topological layers, and casts frontier selection as room-level structured prediction over its topological layer. A language model infers the likely room type and target relevance behind each frontier, which is fused with CLIP visual evidence, an offline room category prior, and geodesic cost under phase-adaptive weights. A vision-language verifier then gates stop using direct visual evidence. Extensive experiments on MP3D and HM3D demonstrate the effectiveness of our method.

Abstract:
Autoregressive generation in large language models (LLMs) conventionally decodes from the final layer, assuming that deeper representations yield more reliable next-token predictions. We revisit this assumption by revealing a recurring Guess–Refine–Perturb dynamic: early layers form coarse guesses, intermediate layers refine reasoning-relevant semantics, and final layers can perturb these refined predictions toward generic or alignment-preferred tokens. We introduce Confident Decoding, a training-free decoding strategy that dynamically selects a near-final layer through entropy-guided conservative backward search. We further formulate layer selection as an optimal stopping problem and show that, under explicit local-monotonicity and bounded-probe-noise conditions, the search rule filters terminal perturbations while bounding unfinished refinement. Experiments across dense and Mixture-of-Experts LLMs show that gains concentrate on challenging reasoning settings, including GPQA-Diamond, Omni-MATH, and HLE, while saturated tasks remain broadly stable, with no additional KV-cache memory and less than 2% latency increase. These results suggest dynamically bypassing final-layer perturbations can recover stronger reasoning behavior from aligned LLMs.

Abstract:
Recent LLM-based recommenders enhance language models with collaborative embeddings from user-item interactions, but making such embeddings available does not ensure their proper use during inference. Through a diagnostic attention analysis, we find that the utilization of collaborative embeddings is depth-dependent and alignment-sensitive, suggesting that LLMs need to balance their internal semantic knowledge with external collaborative knowledge. To address this issue, we propose SAILRec, an LLM-based recommender that improves this balance through dual-side semantic alignment and hierarchical attention steering. The former aligns item-side embeddings with item-text semantics and user-side embeddings with codebook-based semantic profiles, while the latter suppresses premature shallow-layer collaborative interference and strengthens collaborative evidence in deeper decision layers. Experiments on MovieLens-1M and Amazon-Book show that SAILRec consistently outperforms representative baselines, with ablation and masking analyses validating its key designs. Our implementation is available at https://github.com/XuPhoenixyear/SAILRec.

Abstract:
Generative AI enables scalable production of educational videos, but current systems largely focus on producing visually coherent content rather than supporting learning. As a result, generated videos often lack explicit pedagogical structure, reliable quality control, and mechanisms for assessing learner understanding or addressing misconceptions. In this work, we introduce PIVOT (Pedagogy-guided Instructional VideO Tutoring), a generative video tutoring framework for STEM learning via learning-centered instructional support. Inspired by conventional teaching workflows, our framework integrates pedagogy into the full generation pipeline: it first uses instructional principles to guide storyboard generation, then produces verified multimodal videos through code-centric generation and a pedagogical verification harness, and finally connects videos with assessment and misconception-aware remediation. Experiments and expert evaluations across four STEM domains show that our framework produces educational videos with pedagogically aligned content, clear and engaging presentation, coherent instructional flow, and perceived effectiveness for learning. These findings suggest a human-centered perspective on educational content generation: generative systems should be evaluated and designed not only by what they produce, but also by how they support teaching practices, learner understanding, and corrective feedback.

Abstract:
Small Language Models (SLMs) have been widely deployed in edge computing and mobile devices. Generally, these models need to be continuously updated to adapt to dynamic environments and evolving user needs. Therefore, the performance of SLMs in continual learning has become a critical research focus. Compared to LLMs with rich redundant representation spaces, SLMs are more likely to overwrite previous representations during new task learning due to their fewer attention heads and layers. To address this issue, we propose LightLoRA, a sparse fine-tuning method specifically designed for SLMs. This method constructs a conflict-aware importance mechanism by combining raw importance from SVD decomposition with task conflict degrees based on gradient direction consistency. By selecting and updating heads that provide high contributions and low conflicts, LightLoRA successfully maintains sufficient space for knowledge preservation. Experimental results show that the proposed method reaches state-of-the-art performance across multiple benchmarks without adding extra parameters. Code is released on https://anonymous.4open.science/r/LightLoRA-077C .

Abstract:
Deep research agents augment large language models with external tools to answer complex, long-horizon questions through multi-turn reasoning. Learning from prior experience is crucial for continual improvement, yet existing methods either retrieve verbose task-specific traces that burden decision-making, or distill procedural skills that remain decoupled from downstream policy adaptation. We propose APEx, a hierarchical experience utilization framework that organizes interaction history into instance-level trajectory memories and category-level procedural skills, and couples them through a closed-loop architecture of Executor, Distiller, and Planner. The three modules are optimized via a three-stage alternating GRPO training paradigm, enabling reward-guided skill distillation rather than fixed-prompt generation. At test time, distilled skills serve as procedural priors for online Planner adaptation through skill-guided test-time reinforcement learning, allowing ground-truth-free self-improvement with skill-alignment regularization to prevent policy drift. Experiments on 7 benchmarks demonstrate that APEx achieves state-of-the-art performance, surpassing GPT-5.4 by 14.7 points and the strongest memory-augmented baseline by 3.0 points.

Abstract:
Training tool-use agents to improve from their own experience remains challenging, as supervised fine-tuning relies on fixed teacher-distilled trajectories, while sparse-reward reinforcement learning provides weak supervision for long-horizon interactions. We present DeepSearch-Evolve, a self-distillation framework for web agents built on DeepSearch-World, a deterministic and verifiable environment with reproducible search and page-reading tools. DeepSearch-World contains 420K multi-hop QA tasks constructed from entity-level random walks and supports key agentic cognitive behaviors useful for self-evolving, including progress verification, grounded reflection, and failure recovery. DeepSearch-Evolve iteratively performs trajectory generation, filtering, data mixing, and fine-tuning to train stronger agents. Without distillation from more capable models, DeepSearch-World-9B achieves competitive performance compared with open-source agents, reaching 31.2% on BrowseComp, 61.5% on GAIA, and 93.4% on HotpotQA, showing that verifiable environments enable scalable self-evolution for long-horizon web agents. We will release the environment, 420K training pool, validation set, model, and code to facilitate future research on self-improving deep search agents.

Abstract:
More than a dozen spoken dialogue systems have recently claimed to be "full-duplex," yet the term has been used to describe substantially different capabilities. Existing surveys collapse them onto a single axis (cascaded/end-to-end, or engineered/learned) and miss the distinctions that matter most for builders. We argue that much of this ambiguity is taxonomical: current terminology does not specify where duplex decisions are made, which interaction types are supported, or how a system behaves moment by moment. This paper introduces three complementary frameworks: (i) an L,0-L,3 Architectural Hierarchy that locates where duplex decisions are made; (ii) a T x Ix R Interaction Ontology that specifies the temporal relation, user intent, and required system response for each interaction; and (i) a Decision State Machine(LDLE/LISTEN/SPEAK/WAIT/DUAL) that describes how systems move between states.Across published systems and benchmarks, our audit reveals a realization gap: although many architectures can in principle operate in full-duplex states, their observed behavior remains constrained by the interaction patterns represented in training and evaluation.We identify the open-source-versus-industrial data gap, as well as the still-unrealized goal of L3 representation-level modeling, as key frontiers for future research on full-duplex dialogue. The related material is available at https://github.com/MM-Speech/DuplexSurvey.

Abstract:
Model-internal parallelization improves generation quality by introducing multiple parallel computation paths, but its additional latency and computational overhead hinder practical deployment. Recent studies reveal a functional division across model depth: shallow layers primarily encode input-conditioned features, whereas deeper layers are more involved in reasoning and generation. Since parallel paths process the same input and are optimized toward the same generation objective, their early representations are likely to overlap substantially, suggesting that early-layer parallelism may induce redundant computation. Motivated by this observation, we propose Adaptive Depth Parallelism (ADP), a framework that shares computation in shallow layers and activates parallel branches only after an adaptively selected bifurcation layer. We provide a sufficient-condition analysis for delayed branching and identify an empirical transition in inter-path cosine similarity across depth. This motivates using the stable minimum-similarity layer as a practical estimate of the bifurcation point. Experiments show that ADP achieves 104.83% of baseline performance while using only 39.4% of the computational load.

Abstract:
As available training data approaches its physical limit, the performance gains derived from Scaling Laws have begun to diminish. Consequently, the key to further enhancing the performance of Large Language Models (LLMs) has shifted from mere data expansion to improving data utilization efficiency, by enhancing data quality to better exploit the latent potential of existing data. However, in the context of large-scale corpora, existing refinement methodologies face significant limitations in quality, efficiency, and reliability: Rule-based approaches are constrained by fixed heuristics and struggle with instance-level variations; LLM-based approaches improve quality but fail to meet the efficiency and reliability requirements of large-scale data processing. To address these challenges, we propose UltraX, a function-calling refinement framework for large-scale pre-training data that completes the editing function space by introducing insertion in addition to deletion and modification, enabling fine-grained instance-level editing. Specifically, UltraX builds a reliable program-supervision generation pipeline. In this pipeline, dataset-adaptive prompt optimization first guides an expert LLM to produce high-quality end-to-end refined texts, and Line Alignment Mapping and Dynamic Context Replacement then convert original-refined text pairs into structured program supervision. Meanwhile, UltraX improves supervision quality and stabilizes the training distribution with low-confidence example filtering and ratio-controlled sampling by operation combination. During inference and execution, it normalizes and validates model outputs through sliding-window prediction, global operation aggregation, and systematic post-processing, improving the stability and reliability of large-scale execution. Experiments with 1B models pretrained from scratch on multiple corpora show that UltraX achieves the highest average performance across all corpora, with relative improvements exceeding 2% on multiple datasets. UltraX also matches or surpasses baselines with fewer training tokens, demonstrating stronger data efficiency and refinement reliability.

Abstract:
Existing jailbreaks against aligned LLMs are discrete artifacts whose surface forms are easy to fingerprint and patch. We argue that the broader failure mode may lie not in any specific prompt, but in natural writing registers that safety tuning under-covers. Building on this insight, we introduce VAR, the first jailbreak family that uses real fanfiction subgenres as universal attack carriers: a creative-writing meta prompt is conditioned on passages from one of twelve Archive of Our Own (AO3) subgenres, and the harmful behavior is embedded as the climax of the resulting scene. The construction requires neither an adversarial attacker LLM nor optimization. On eight aligned LLMs over the union of HarmBench and JailbreakBench, this attack lifts mean ASR from 0.278 to 0.731 under a four-judge ensemble; a factorial decomposition shows the gain is carried by register rather than length or structure. Two active defenses widen rather than narrow the vernacular-to-baseline ratio, indicating that template-targeting defenses merely steer attackers toward register-based attacks like ours. We also propose VAR-A4, a static four-turn extension that attains a mean ASR of 0.924, substantially exceeding three existing multi-turn methods. Our code and data are safely open-sourced at https://github.com/T-Lab-CUHKSZ/VAR.

Abstract:
Reasoning large language models that produce explicit chains of thought are a new jailbreak surface: an adversary who shapes intermediate reasoning can hijack safety deliberation and elicit harmful answers the same model would refuse in a single forward pass. Defenses inherited from chat-style alignment monitor only the final response and intervene after the model has already committed to an unsafe trajectory. We argue that defending reasoning models requires looking inside the thinking phase itself, and we show that the refusal direction, previously characterized as a single linear direction in chat models, in fact evolves along the reasoning trajectory: sharp at onset, slowly drifting during long deliberation, and on hijacked traces collapsing at an identifiable step well before any harmful token is emitted. We propose RsMon, a step-level refusal-subspace monitor that tracks the projection of intermediate hidden states onto a calibrated refusal subspace and triggers a minimally invasive geometric intervention once drift exceeds a learned threshold. The monitor is purely inference-time, requires no retraining, and is compatible with any open-weight reasoning model. On three open reasoning models, five attack families, and four utility benchmarks, RsMon cuts mean attack success from 80.1% to 19.2%, a 4.2× reduction, while keeping MATH-500, GPQA-Diamond, and HumanEval pass@1 within 0.7 absolute points of the undefended baseline. The same monitor produces a step-resolved diagnostic that pinpoints when and in which layer safety deliberation breaks down. Our code is available at https://github.com/llmnjust-afk/RsMon.

Abstract:
Zero-shot fine-grained emotion recognition (FER) is pivotal for empathetic AI but remains constrained by two inherent limitations in retrieval-augmented in-context learning (ICL): 1) subtle emotional distinctions inject semantic noise that blurs decision boundaries, and 2) naive similarity-based retrieval often surfaces counterproductive or antonymous demonstrations. To address these, we propose EmoDap, a dual-agent prompt retrieval framework for zero-shot FER. EmoDap first constructs a high-quality demonstration pool via self-consistency pseudo-labeling and derives explicit emotion prototypes through contrastive learning. Subsequently, a demonstration retriever and an emotion predictor are jointly optimized via Group Relative Policy Optimization (GRPO): the retriever learns to select examples that maximize the predictor’s accuracy, while the predictor leverages prototype guidance to dynamically filter candidate emotions. Extensive experiments across four benchmarks demonstrate that EmoDap consistently outperforms state-of-the-art baselines, establishing a robust paradigm for zero-shot FER driven by open-source LLM backbones.

Abstract:
AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However, existing benchmarks such as BrowseComp primarily focus on textual content, overlooking the prevalence of multimodal content. To bridge this gap, we introduce MM-BrowseComp, a novel benchmark comprising 400 challenging, hand-crafted questions designed to evaluate multimodal retrieval and reasoning capabilities. Unlike prior work, MM-BrowseComp incorporates visual prompts and necessitates the extraction of key evidence from web images and videos to complete questions, rendering text-only approaches insufficient. Additionally, we provide a verified checklist for each question, enabling fine-grained analysis of multimodal dependencies and reasoning paths. Our comprehensive evaluation of 27 state-of-the-art models reveals that even leading models like GPT-5-High with tools achieve only 24.25% accuracy, highlighting the suboptimal multimodal browsing capabilities, establishing MM-BrowseComp as a rigorous new standard for the field.

Abstract:
Multimodal Large Language Models (MLLMs) unify heterogeneous vision-language tasks under a shared generative framework via instruction tuning, yet real-world deployment demands continuous capability expansion, making Multimodal Continual Instruction Tuning (MCIT) essential. Existing methods either update all tasks with a shared parameter set or allocate dedicated modules for each new task. Shared updates force heterogeneous tasks to compete, causing forgetting of learned capabilities. Conversely, isolated expansion prevents interference but severely limits parameter efficiency over long task streams. To address this dilemma, we propose CRAM (Centroid-Routing and Adaptive MoE). Specifically, by isolating task-specific patterns into independent modules, CRAM mitigates catastrophic forgetting across tasks. To further boost parameter efficiency, we utilize adaptive-rank instantiation to identify the capability gap between existing expert capability and new task demands, and dynamically allocate only the necessary parameters. To ensure stable reuse among tasks, centroid-guided routing recognizes and activates existing experts’ capabilities, while an orthogonality penalty confines new updates to task-specific directions, preventing re-learning general capability. Extensive experiments across diverse benchmarks demonstrate its superiority over existing methods. Code is available at https://github.com/LAMDA-CL/EMNLP2026-CRAM.

Abstract:
Graph databases are increasingly queried through natural language, yet every existing benchmark evaluates isolated single-turn queries rather than the multi-turn sessions through which analysts actually work. We introduce CypherTurn, the first benchmark for conversational Text-to-Cypher evaluation, comprising 721 sessions and 5,927 turns across 7 knowledge graphs and 13 conversational phenomena. We evaluate 15 models under a guided oracle protocol and a fully autonomous agentic protocol, yielding four findings. First, the best model reaches only 64.7% execution accuracy, and session-level correctness remains below 5%. Second, despite strong overall rank correlation, frontier models exhibit a consequential reordering of the top of the leaderboard under autonomous operation, a phenomenon we term the Autonomy Divergence, which reveals error-management as a partially independent capability from raw generation skill. Third, scaling action budgets from ×3 to ×10 fails to close the autonomy gap, as frontier models self-limit to approximately two actions per turn regardless of available budget. Fourth, single-turn Cypher fine-tuning degrades multi-turn instruction following, while architecture-appropriate specialization outperforms several frontier models. These results establish CypherTurn as an open challenge for conversational graph database reasoning. Code and data are available at https://github.com/BarryQ/CypherTurn.

Abstract:
Multi-agent LLM systems increasingly rely on persistent memory to coordinate role-specialized work. Correctness requires more than recalling a fact: the fact must reach the right role, remain hidden from unauthorized roles, carry the right source, and update when a more authoritative value appears. We introduce COLLABMEM, a diagnostic benchmark for collaborative memory operations. Its matched memory-only harness fixes the trace, target role, answerer, judge, and gold event, localizing errors to sharing, access, routing, provenance, conflict resolution, or retention. COLLABMEM contains 58 hand-authored scenarios (1,118 core probes), a 174-scenario scaled split (3,359 probes), adversarial and compositional hard sets, and six transactive-memory competencies. We also propose TMS-GRAPH++, a scope-aware graph memory framework with role-filtered views, writer/provenance structure, and temporal [CURRENT] marking. Across five evaluation splits, TMS-GRAPH++ achieves the highest non-Oracle average score (69.6%). Evaluations using the official SDKs of Mem0, Letta, and A-Mem show that shared archives help, but reliable collaborative memory still requires explicit visibility, provenance, authority, and retention policies. We will release the dataset, evaluation code, and reproduced outputs to support future work on multi-agent memory systems.

Abstract:
World models enable language agents to anticipate environment dynamics and evaluate candidate actions before execution, yet they are typically frozen after training, leaving them unable to adapt to the shifting on-policy distributions induced by evolving agents. Meanwhile, agent-improvement methods often depend on external rewards or verifiers that may be unavailable in realistic interactive environments. We introduce CoMap, a framework that co-evolves textual world models and agent policies through closed-loop interaction. At each decision step, the world model predicts future environment feedback for candidate actions, while the agent performs future-aware reflection by assessing the reliability of these predictions and refining its action accordingly. The resulting on-policy trajectories are then used to improve the world model through self-distillation, enabling it to continually adapt to the agent’s evolving interaction distribution. Across embodied task planning, web navigation, and tool-use benchmarks, CoMap consistently outperforms competitive baselines (e.g., a 16.75% relative improvement with Qwen3-4B). Further analyses show that this co-evolutionary process improves both world-model capability and long-horizon decision-making. Our code and data are available at: https://github.com/loyiv/CoMAP.

Abstract:
Current Mixture-of-Experts (MoE) models typically rely on a static Top- k routing mechanism, activating a fixed number of experts for every token regardless of its difficulty. This uniform allocation ignores the inherent variability in linguistic information density that simple functional words require significantly less processing capacity than content-heavy terms. The static routing results in computational waste for easy tokens and potential under-fitting for difficult ones. To address this issue, we propose CAR, a Complexity-Aware Routing framework that aligns the computational budget with the intrinsic information density of input tokens. Diverging from previous dynamic methods that rely on unstable heuristics, we ground our routing decision in a dual-signal objective combining Shannon Surprisal and Predictive Entropy to objectively measure token complexity. We implement a decoupled architecture where a Complexity Perception (CP) module utilizes a Recurrent Complexity Memory (RCM) to estimate these signals from a teacher model, guiding a Dynamic Expert Allocator (DEA) to adjust k on the fly. Extensive experiments demonstrate that CAR consistently outperforms both static baselines and SOTA dynamic routing methods. Notably, our approach achieves superior accuracy on reasoning-intensive benchmarks (e.g., boosting MMLU by 0.8 and GSM8K by 1.5 over static routing) while reducing the average number of activated parameters by approximately 10%. Our code is available at https://github.com/Acking646/car.

Abstract:
Online harm detection is crucial for safeguarding digital ecosystems. However, current research is largely confined to single-dimensional classification paradigms or generative safety alignment under simplified and explicit adversarial prompts. To capture complex and highly covert threats in the real world, we introduce the Multi-dimensional Online Harm Detection Benchmark (MDHB). MDHB focuses on four core dimensions critically demanded by social platforms: misinformation, toxicity, sarcasm, and logical fallacies. Notably, we incorporate non-offensive implicit sarcasm (e.g., self-deprecation and veiled complaints) to propel content governance toward proactive psychosocial risk forecasting. Evaluations reveal that mainstream Large Language Models (LLMs), including GPT-5, suffer from severe cross-dimensional interference under both direct inference and ReAct paradigms. To address this, we propose the Self-Refining Sentinel (SRS), an error-driven self-evolving agent architecture. Without requiring parameter fine-tuning, SRS dynamically constructs and updates an experience repository from failure cases through an iterative self-evolution mechanism. Experimental results demonstrate that SRS significantly outperforms state-of-the-art baselines in overall performance, effectively mitigating cross-dimensional interference in multi-dimensional online threat detection.

Abstract:
Recent progress in Multimodal Large Language Models (MLLMs) has significantly enhanced the ability of artificial intelligence systems to understand and generate multimodal content. However, the capabilities of existing MLLMs in understanding traditional Chinese culture have not yet been comprehensively explored. To address this gap, we propose the T raditional C hinese C ulture understanding Bench mark ( TCC-Bench ), a bilingual Visual Question Answering (VQA) benchmark designed to assess MLLMs’ performance on culturally grounded visual content. TCC-Bench features culturally rich and visually diverse data, covering museum artifacts, daily life scenes, comics, and other culturally significant contexts across eight cultural domains. Through extensive experiments on a wide range of MLLMs, we find that current models still face substantial challenges in handling traditional Chinese cultural content. Our results highlight the need for more culturally inclusive and context-aware multimodal systems. The code and data can be found at: https://github.com/Morty-Xu/TCC-Bench .

Abstract:
Current TTS systems typically rely on open-loop, single-pass generation and can produce sporadic local prosodic defects, such as misplaced stress, unnatural pauses, or flattened intonation, that utterance-level metrics often fail to expose. We present LoopTTS, a judge-guided Filter-Judge-Refiner framework for recovering low-quality TTS outputs diagnosed by an AudioLLM. Given an initial utterance from a base TTS model, an AudioLLM Judge identifies salient prosodic issues and generates structured refine instructions; a Refiner, our fine-grained instruction-following TTS model, then performs guided expressive re-synthesis conditioned on the initial utterance, target text, and instruction. To train the Refiner, we construct Refiner-DB, a 42K-example AudioLLM-annotated dataset with word-level prosodic weak supervision. Human evaluation on diagnosed low-quality utterances shows that LoopTTS can detect perceptually salient errors and correct them with the Refiner, outperforming raw generated audio and practical open-loop re-generation baselines in recovery quality. The Refiner also demonstrates stronger instruction-following ability for stress and pause control in targeted prosody modification. LoopTTS is available at https://github.com/Pooookeman/LoopTTS.

Abstract:
"Talk short. Drop grammar. Save token." This phrase is widely promoted as a way to reduce inference costs, but whether it saves anything depends on whether you compress the prompt or the response. We present CAVEWOMAN, a two-channel evaluation protocol that scores every question–answer pair on task accuracy, realized per-item cost, and whether the response still says what the model would have written without compression. It is tested on nine models, five short-answer benchmarks, and five grammatical reduction levels, with a ten-language replication and a two-dataset summarization study. Constraining the response reduces realized API cost in dollars by 1.7× on average and up to 3× without losing accuracy. Compressing the prompt raises net cost on three of the four API models, by up to 96%, and lowers accuracy at every level. Under output compression, over half of responses on the six non-reasoning models are correct but no longer entail the response the model wrote without the constraint. How much accuracy a model keeps under compression varies across models and languages. Task accuracy alone does not capture the effects of compression, so compression should be evaluated on accuracy, agreement with the unconstrained response, and cost, each reported separately. Code is available at https://github.com/danielle34/cavewoman.

Abstract:
To interpret context correctly and retrieve relevant information, large language models must bind entities to their attributes and update these bindings as state changes. We analyze how LLMs implement this binding process in a dynamic state tracking. Using causal interventions, we identify a retrieval conditioned rebinding mechanism, a compact attention head circuit that propagated binding information and when the entity is queried, uses the updated binding to retrieve the corresponding attribute. Across Gemma and Llama models, this circuit supports rebinding behavior, but the representational signature of the mechanism differs across model families. In Gemma models, the binding signature is clearly expressed in the query/key subspaces of the relevant attention heads, whereas in Llama models, the binding information is carried primarily in key vectors. Overall, our results reveal an interpretable mechanism for context dependent state tracking in LLMs.

Abstract:
Long-output inference in large language models suffers from severe KV cache memory and bandwidth bottlenecks. While KV cache quantization alleviates this issue, low-bit quantization (e.g., 2-bit) severely degrades model accuracy. Recent mixed-precision key-cache quantization methods mitigate this degradation by allocating higher precision to important key channels, but estimate channel saliency using only key statistics. Through attention logit perturbation analysis at the RoPE channel-pair level, we show that quantization errors in channel pairs with large query–key pair-norm products contribute significantly to logit variation. Based on this insight, we propose PN-Mix, a mixed-precision key-cache quantization method for RoPE-based decoder-only LLMs, which allocates higher precision to salient channel pairs identified once during the prefill stage and reused throughout decoding. Across standard reasoning, code-generation, and AIME benchmarks, PN-Mix with 2.26-bit average precision achieves comparable or better average accuracy than prior mixed-precision methods using 2.32–2.39 bits. Furthermore, in our Pareto sweep, PN-Mix degrades gracefully down to 1.5 bits, whereas the compared baselines exhibit sharper performance drops below 2.25 bits.

Abstract:
Existing SQL-based benchmarks usually assume that retrieved records are equally informative. This assumption is limiting for investigative domains, where truthful records may differ substantially in inferential strength. To address this gap, we introduce Holmes-SQL, the first benchmark for urban crime investigation over relational databases. Each case begins with partial cues and requires agents to reconstruct the crime scene, victim, suspect pool, and culprit through database interaction. Holmes-SQL combines state uncertainty, unequal evidential strength, and fallacy-aligned suspect construction to evaluate evidence-aware investigative reasoning. Experiments show that agents achieve over 97% SQL execution success, yet the best solve rate reaches only 49%, highlighting a distinct challenge for evidence-aware investigative reasoning. A further comparison shows that agents consistently underperform when investigating compared with reasoning over all evidence provided upfront, highlighting the added challenge of investigation.

Abstract:
Peer review is at the heart of modern science. As submission numbers rise and research communities grow, the decline in review quality is a popular narrative and a common concern. Yet, is it true? Review quality is difficult to measure, and the ongoing evolution of reviewing practices makes it difficult to compare reviews across venues and time. To address this, we introduce a new framework for the comparative study of review quality and apply it to major AI and machine learning conferences. We propose a multi-dimensional schema to serve as a proxy for quantifying review quality in terms of its utility to editors and authors. The schema is coupled with LLM-based and lightweight measurements, which we validate against a novel test set of human-labeled review quality ratings. Contradicting the popular narrative, our proxy measurements reveal no consistent decline in median review quality across venues and years. We propose alternative explanations, and outline recommendations to facilitate future empirical studies of review quality. Code and data are available at https://github.com/UKPLab/emnlp2026-review-quality-estimation.

Abstract:
In an age of rapid model turnover, how do we make hallucination evaluation more perennial? We explore whether human-written hallucination samples could take the place of model-generated hallucinations, in order to make benchmarking detection independent of particular models. To this end, we construct SHEEP, a dataset of 1,600 human-written samples, spanning four languages (Chinese, English, French, Italian), and 18,400 samples from five vision-and-language models, all annotated for hallucinations using a fine-grained span-level labeling scheme. We find that human-written samples result in higher agreement and allow greater control of dataset contents, while remaining distributionally similar to samples derived from vision-and-language samples and providing a reasonable portrayal of detection capabilities — suggesting that human data is a viable substitute for model-based hallucination benchmarks.

Abstract:
Speculative decoding accelerates generation by verifying multiple drafted tokens in a single target-model forward pass, reducing sequential decoding iterations. Model-free variants avoid auxiliary draft models by reusing text and model states already available during generation, but their speedup depends on the reliability of the constructed drafts. We identify two limitations of existing reuse-based methods: lexically anchored retrieval has limited recall under surface-form variation, and deterministic span copying can be brittle when the retrieved context does not uniquely determine the continuation. We propose AdaPLD , a training-free method that adaptively improves both retrieval and draft construction. AdaPLD preserves high-precision lexical reuse while using semantic similarity to recover additional reuse opportunities when lexical matching fails. It further constructs branched reuse hypotheses to account for continuation uncertainty, rather than relying on a single copied span. Across diverse benchmarks, AdaPLD reduces target-model forward passes and achieves up to 3.10× decoding speedup.

Abstract:
While retrieval-augmented generation (RAG) relies on retrieved evidence to mitigate hallucination, its reliability is constrained by retrieval quality and diagnosing where retrieval fails remains difficult. Existing benchmarks typically report a single aggregate score over heterogeneous query sets and are often dominated by queries whose relevant passages can be identified through simple lexical overlap, concealing the failure modes that matter most in practice. We introduce GutenBench, a fine-grained, zero-shot evaluation dataset built around a four-level retrieval taxonomy: lexical, semantic, explicit aggregation, and latent conceptual retrieval. It pairs queries with relevance judgments adjudicated by subject-matter experts, enabling precise and controlled comparisons of retrievers. We evaluate several off-the-shelf retrievers in an asymmetric, English-to-target cross-lingual setting across 272 Bible translations (141 languages) and find that performance is strong on lexical and semantic queries, but degrades when models must aggregate information across passages or recognize conceptually relevant passages with no surface overlap.

Abstract:
We present FullFront , a workflow-grounded benchmark for Multimodal Large Language Models (MLLMs) across the front-end engineering pipeline—conceptualization, comprehension, and implementation—rather than evaluating any single stage in isolation. FullFront contributes along three axes: (i) a unified task formulation covering Webpage Design, Webpage Perception QA, and Webpage Code Generation (8 subtasks, 2 , 700 items); (ii) a controlled two-stage data construction pipeline that converts real-world webpages into standardized HTML with a 15-category copyright-safe image substitution scheme; and (iii) a reliability-oriented evaluation framework whose Code Score and Gemini Visual Score are validated against human judgment, cross-judge agreement, pipeline bias, and sensitivity stress tests. Across 12 MLLMs we observe a stable closed-source top tier, a Perception–Implementation Gap with distinct failure modes (Coding Bottleneck” for open-source, Grounding Failure” for proprietary), and heterogeneous open/closed capability profiles—patterns that isolated benchmarks cannot expose.

Abstract:
ESG analysis requires complex language understanding across diverse sources, including corporate reports, news, and disclosures, for tasks such as ESG relevance detection, action interpretation, impact analysis, and question answering. However, existing ESG datasets are fragmented by task and source, with heterogeneous formats and annotation schemes, limiting their use for training general-purpose ESG LLMs and unified evaluation. In this work, we construct ESG-Instruct, a 30,000-instance ESG instruction dataset built from public ESG resources, and ESGU-Bench, an ESG understanding benchmark covering six representative ESG tasks. Based on these resources, we further develop ESGPT, an 8B ESG-specialized LLM, and evaluate it against diverse open-source and API-based LLMs. Experimental results show that general-purpose LLMs exhibit substantial performance variation across ESG tasks, while ESGPT, despite its relatively small scale, achieves the best performance on multiple ESG tasks. This work provides unified data and benchmark resources for training and evaluating ESG-specialized LLMs.

Abstract:
When user queries are incomplete, large language models should ask targeted questions before committing to an answer, a capability we call proactive elicitation, yet evaluation on real user-assistant queries across seven LLMs (8B to frontier) shows that fewer than 11% of underspecified queries trigger a follow-up question. To benchmark and train this capability, we introduce VagueBench, where each seed query is annotated with structured intent and detail units then degraded into intent-vague and fully-vague variants via controlled removal of linguistic cues, yielding known hidden information sets for evaluating both when and what to ask. We propose DialogGRPO, an offline reinforcement learning method that organizes multi-turn elicitation as a dialogue-state directed acyclic graph, where candidate questions at each node receive dense turn-level rewards combining immediate information gain with lookahead value through group-relative comparison, enabling fine-grained credit assignment without runtime simulation. DialogGRPO elicits missing information on over 88% of vague queries with substantial gains in both recovery efficiency and answer coverage, without over-asking on complete queries, outperforms prompting and test-time-scaling baselines, and generalizes zero-shot to MultiWOZ 2.4 and AmbigQA.

Abstract:
This paper introduces a framework for evaluating structural fidelity in emotion reasoning: the degree to which the causal structure of AI-generated emotion explanations aligns with human-curated explicit causal abstractions. Unlike traditional surface-level text overlap metrics, we propose two metrics: Structural Causal Similarity (SCS) for measuring static structural alignment, and Dynamic Causal Sensitivity (DCS) as a proxy for measuring dynamic sensitivity to counterfactual interventions. We evaluate these on EmoReason-4k, a new benchmark comprising two human-annotated datasets ( each) from social media and conversational dialogues. Testing seven LLMs identifies consistent graph-generation behaviors across models and domains, such as “structural elaboration” (a tendency for models to explicate intermediate steps that humans leave implicit) and sensitivity to shifts in causal agency locus. However, models struggle to generate updated causal states in complex discourse. By evaluating structural fidelity against human-curated causal abstractions, our framework offers a structured method to diagnose the structural compliance of explicit explanations against formalized human abstractions. Ablation studies support that our metrics capture underlying topological properties rather than just parsing artifacts. Data and prompts are available to support reproducibility.

Abstract:
Geometry Problem Solving (GPS) is a fundamental task in mathematical reasoning, with applications in intelligent education and automated theorem proving. Existing neuro-symbolic solvers often rely on two restrictive assumptions: textual conditions are expressed as atomic geometric statements, and target derivation can be reduced to eager elimination of intermediate variables. These assumptions break down in realistic problems, where coupled geometric descriptions lead to combinatorial growth in parsing patterns, and nonlinear constraints make explicit elimination inefficient or even intractable. We propose RIGOR, a neuro-symbolic GPS framework based on Atomic Semantic Normalization and Declarative Constraint Reasoning. RIGOR uses a Geometric Semantic Normalizer trained on our newly constructed GSN-15K dataset to normalize geometric texts into atomic statements for reliable formalization. It then adopts a Declarative Symbolic Engine that globally accumulates derived geometric constraints and determines target values through constraint entailment, while also reconstructing proof-grounded reasoning traces. Experiments on multiple standard GPS benchmarks show that RIGOR outperforms state-of-the-art methods while producing interpretable derivations. Our project is available at https://github.com/dongyuhang22/RIGOR.

Abstract:
Micro-dramas, characterized by ultra-short durations and hyper-dense storylines, pose unique challenges for video understanding that conventional benchmarks fail to address. To bridge this gap, we introduce M-Drama, the first large-scale bilingual benchmark for micro-drama comprehension, featuring over 35K instances across 9,138 clips. Furthermore, while reinforcement learning can enhance VLMs on complex narratives, existing reward metrics often suffer from sparse and superficial signals, failing to capture intricate character identities and temporal structures. We propose SAGA (Structure-Aware Graph Alignment), a novel graph-matching reward function that models narratives as heterogeneous graphs. SAGA computes dense, rigorous rewards via decoupled semantic triplet and structural temporal matching. Extensive experiments on Qwen3-VL-8B-Instruct demonstrate that SAGA outperforms existing baselines, delivering substantial improvements in open-ended accuracy and summary quality, while maintaining competitive out-of-domain generalization. Code is available at https://github.com/qyx1121/MDrama_SAGA.

Abstract:
LLM-based research agents have advanced rapidly in science and engineering, where research is organized around executable experiments, code, and quantitative signals. Humanities scholarship, however, requires interpretive, evidence-grounded arguments over primary sources, whose value rests on faithful quotation, verifiable provenance, and deep interpretation. Existing research agents use general planning, tool use, and reflection, leaving these scholarly operations implicit. To address this gap, we introduce SPIRE (Scholarly-Primitives-Inspired Research Engine), a multi-agent framework that realizes Scholarly Primitives, a typology of basic humanities scholarship practices, as cooperating roles over a multi-scale close-reading substrate of passages, intra-context graph communities, and cross-context semantic clusters. On a benchmark of peer-reviewed papers in classical Chinese and Greco-Roman Latin scholarship, SPIRE recovers substantially more cited primary-source evidence and receives higher blind human and LLM ratings on four scholarly quality dimensions than LLM, RAG, and generic agentic baselines. Retrieval-tier and agent ablations, with retrieval-volume and answer-length controls, trace its advantage to targeted retrieval, structured evidence selection, and claim-evidence binding. Code and benchmark data are released at https://github.com/YatingPan/SPIRE.

Abstract:
Multimodal Retrieval-Augmented Generation (RAG) with visual citation is crucial for ensuring the traceability and verifiability of MLLMs. However, current RAG and SFT-based methods struggle to achieve robust cross-modal reasoning, causing imprecise visual citations or decoupling between the citation and the generated answers. To address these limitations, we propose MCite-RL, a citation-enhanced agentic reinforcement learning framework designed for reliable multimodal RAG. MCite-RL introduces an Agentic Refinement module for visual citation that employs iterative retrieval, reasoning, and recursive cropping to progressively narrow the search space, transforming citation into a dynamic, evidence-driven reasoning process rather than a static step. Furthermore, we incorporate a Citation-enhanced Reward mechanism that integrates both process-level and outcome-level feedback within a reinforcement learning paradigm to jointly optimize answer accuracy and source traceability. Extensive experiments on benchmarks such as Wiki-VISA, FinRAGBench-V, and MMLongBench-Doc demonstrate that MCite-RL effectively achieves the joint optimization of citation precision and answer quality.

Abstract:
Chart-to-data extraction with vision-language models (VLMs) is increasingly evaluated on benchmarks that show diminishing headroom (frontier VLMs exceed 89% on ChartQA) and with metrics that treat extracted points as unordered key-value pairs, ignoring the temporal structure of time series and penalizing small alignment shifts as catastrophic failures. We address both gaps with EpiCurveBench, a benchmark of 1,000 epidemic curve images curated from real-world public-health sources, and EpiCurveSimilarity (ECS), an evaluation metric that aligns predicted and ground-truth series via dynamic programming, tolerating local temporal shifts and gaps while penalizing them proportionally. Evaluating nine methods—three frontier closed VLMs, four open VLM, and two specialized chart-extraction systems—we find the strongest model reaches only 52.3% ECS, and that ECS spreads the four general-purpose VLMs over a 25-point range where key-value metrics (RMS, SCRM) compress them into a 5-point band. We further validate ECS against four downstream epidemiological summary statistics, finding that higher ECS predicts smaller errors in total counts, peak timing, and peak magnitude, and higher growth-rate fidelity; across all four, ECS correlates 1.5–3.6 × more strongly than Dynamic Time Warping, which lacks a gap penalty and therefore cannot distinguish a truncated prediction from a temporally faithful one. EpiCurveBench targets a high-impact public-health application—unlocking decades of outbreak data trapped in published figures—but the benchmark and metric apply directly to any structured time-series chart-extraction setting.

Abstract:
Activation patching can identify a mixture-of-experts (MoE) block whose clean output restores a corrupted factual prediction. However, because the block output combines contributions from multiple routed experts, block-level rescue does not establish whether the recovery localizes to an individual expert or depends on the routed expert set. We study this question on single-token COUNTERFACT contrasts by corrupting subject-token embeddings, restoring clean block outputs, and then restoring clean-minus-noised expert updates under fixed routing. In Qwen3-30B-A3B-Base, a discovery sweep selects layer 44, and held-out analysis identifies L44E069 as a recurrent routed contributor with positive specificity over same-layer active experts. Its effect is fact-matched and improves true-token probability and rank, which explains part of the layer rescue. In Mixtral-8x7B-v0.1, the selected recurrent singleton is not specific; matched-size controls instead show specificity of the clean-routed top-2 set. These findings show that successful MoE-block restoration does not necessarily imply localization to a single expert.

Abstract:
The growing deployment of large language models (LLMs) in socially consequential settings makes it increasingly important to understand the value systems they encode and the moral judgments they produce. However, existing evaluations are largely grounded in Western ethical traditions, leaving non-Western moral perspectives underexplored. To address this gap, we introduce Sage, a benchmark for evaluating LLMs through the lens of Classical Chinese philosophy. Sage is grounded in major Pre-Qin philosophical debates and represents normative alignment in a continuous six-dimensional value space. It includes three task types and is constructed from over 4,000 passages spanning ten philosophical schools. Experiments on eight leading LLMs reveal that models occupy highly similar regions in the proposed value space, exhibit a clear gap between self-declared philosophical identities and benchmark-measured behavior, and show systematically different reasoning patterns across task granularities. Code and data will be released after publication.

Abstract:
Low-bit quantization is crucial for deploying Large Language Models (LLMs) on resource-constrained hardware. However, existing Post-Training Quantization (PTQ) methods adopt a monolithic view of outliers, overlooking their global spatial patterns within LLM architectures and weights that are quantization-sensitive but not numerically large. To overcome these challenges, we introduce MOAQ, a novel finetuning-free weight-only PTQ framework based on a new perspective for outlier analysis. MOAQ holistically addresses the outlier problem by three core components: (1) Spatial Calibration (SC) uses a clustering-optimized Hadamard matrix to reorganize disordered outliers into both discrete and clustered patterns. (2) Local Awareness and Suppression (LAS) adopts a dual strategy guided by our outlier metric. It assigns higher bit-widths to clustered outliers via Clustered Outlier Bit Search (COBS), enabling flexible fractional-bit quantization, while further refining quantization parameters for local discrete outliers through Local Outlierness-Aware Calibration (LOAC). (3) Global Feedback and Denoising (GFD) introduces a dynamic correction mechanism informed by our outlier metric to mitigate the progressive cross-layer enrichment of outliers. Extensive experiments show that MOAQ achieves state-of-the-art performance, outperforming both finetuning-free and many finetuning-based methods with substantially lower computational cost.

Abstract:
LLM-based agentic systems are increasingly combined with CoT reasoning to solve tasks that require accurate reasoning, external knowledge, and multi-step deliberation. Yet, explicit CoT incurs substantial generation overhead, while implicit CoT, which replaces textual rationales with compact continuous tokens, does not naturally support the explicit communication interface required for agentic tool-use. We propose TAC-CoT, a tool-empowered adaptive implicit CoT framework that replaces explicit reasoning traces with compact latent reasoning segments while retaining textual tokens only for tool calls, observations, and final answers. This interleaved design enables agentic tool-use without verbalizing every intermediate reasoning step. Experiments on agentic math and QA benchmarks show that TAC-CoT improves accuracy by up to 8.8 points while reducing model forward passes by up to 6.8 × . Furthermore, analyses show that TAC-CoT improves tool-use quality in both single- and multi-tool settings.

Abstract:
Rotary Positional Embedding (RoPE) is widely used in transformers to encode relative positional information, yet the effect of applying RoPE to only a subset of hidden dimensions remains underexplored. This design choice can substantially reduce memory usage, especially at long context lengths. We show that partial RoPE achieves up to 10x memory savings over standard RoPE cache while maintaining comparable final loss. We systematically study its impact on training dynamics and convergence across architectures, model sizes, sequence lengths, and datasets. Our results reveal that: (1) applying RoPE to only a small fraction of dimensions (around 10%) matches the convergence of full RoPE; (2) these trends remain consistent across settings; and (3) NoPE (No Positional Encoding) can produce loss spikes, which are mitigated by minimal RoPE usage or QK-Norm application. Overall, our findings highlight partial RoPE as an effective trade-off between efficiency and stability.

Abstract:
Many Natural Language Processing (NLP) tasks, including information extraction, dialogue understanding, and sentiment analysis, can be formulated as structured prediction over sequences or spans. Although Large Language Model (LLM)-based methods have achieved strong performance on standard benchmarks for such tasks, their robustness under natural sentence-level perturbations and their controllability in producing valid structures remain insufficiently studied. In this paper, we propose Locality-aware Constrained Decoding (LoCoDec), a method for LLM-based structured prediction over sequences and spans that formulates decoding as constrained token paraphrasing. LoCoDec combines neighborhood context, which reduces long-text noise, with structural decoding constraints that enforce source-faithful and well-formed outputs. Experiments on sixteen benchmarks of six tasks show that LoCoDec remains competitive in S-F1 and P-F1, while achieving the best R-F1 on all datasets. Ablation results show that removing neighborhood context reduces R-F1 by up to 16.23 points. Overall, LoCoDec improves the robustness and structural consistency of LLM-based structured prediction.

Abstract:
Large language models (LLMs) have increasingly been used for zero-shot authorship verification, where a model must determine whether two texts share an author. We show that evaluated zero-shot prompting methods exhibit a conservative bias, often over-predicting different authorship. This behavior is especially pronounced when the texts of the same author differ in topic or content. Through qualitative and quantitative analyses, we find evidence that LLMs confuse differences in content with differences in authorship. To test whether this failure can be reduced, we introduce lightweight prompt interventions that instruct the model to focus on style rather than content. Across four datasets and five open-weight instruction-tuned LLMs from three model families, the best intervention improves same-author accuracy by 13.1 points while also increasing macro-F1 by 3.1 points on average across all 20 experimental settings. Our findings show that zero-shot authorship verification is sensitive to the separation of content and style and that small framing changes can meaningfully change the direction of LLM errors. Code is available at https://github.com/galactixx/when-llms-say-no.

Abstract:
Large language models (LLMs) increasingly operate on long inputs, yet their behavior when harmful sentences are sparsely embedded within such inputs remains poorly understood. We present a sensitivity analysis that probes how well LLMs detect and localize harmful sentences embedded in long mixed inputs, when harmful content is sparsely distributed among non-harmful sentences. We construct long inputs by combining neutral and harmful sentences, and systematically vary four factors: input length (600–30,000 tokens), the proportion of harmful sentences (0.05–0.50), harm realization (explicit vs. implicit), and the position of harmful sentences within the input (beginning, middle, end), enabling a controlled stress-test evaluation. Experiments across toxic, offensive, and hate content, and across LLaMA-3.1, Qwen-2.5, and Mistral, reveal consistent patterns: sensitivity is non-monotonic with respect to harmful prevalence, peaking at moderate levels; sensitivity degrades as input length increases; harmful sentences placed earlier in the input are more strongly prioritized; and explicit harm is more reliably identified than implicit harm. These findings provide a systematic view of how LLMs prioritize harmful sentences in long input under controlled stress conditions, highlighting both emerging strengths and remaining challenges for safety-related use.

Abstract:
Full-duplex spoken dialogue systems enable simultaneous listening and speaking, but their audio-only perception often fails under background noise and overlapping speech, leading to incoherent responses. Recent audio-visual dialogue approaches show that incorporating visual cues such as lip movements improve robustness under audio corruption. However, existing approaches often adapt the large speech dialogue model itself to process visual input, requiring costly multimodal training. We propose AV-STE, a modular streaming audio-visual front-end that restores corrupted semantic speech tokens from noisy audio and lip video before they reach the speech LLM. The downstream dialogue model remains entirely frozen, preserving its pretrained conversational capabilities. When integrated with frozen Moshi, AV-STE improves average GPT-4o-judged response coherence from 1.42 to 1.91 under same-dataset speaker interference while largely preserving turn-taking behavior. Gains also transfer to out-of-domain Seamless Interaction. Code and models are available: https://github.com/bellagodiva/av-ste.git

Abstract:
LLM persona control divides between trait-prompt strings and model-specific activation-steering vectors; neither offers a per-response trace from output to persona contents. We introduce Persona-CAPS, a retrieval-based persona framework grounded in the Cognitive-Affective Personality System (Mischel and Shoda, 1995): a persona is a corpus of if (situation) then (behavior) contingencies mined from public Experience Sampling Method data, retrieved per-situation, each fired atom logged to its source observation. An audit-loop study quantifies three structural properties paragraph baselines lack: retrieval exposes corpus-level corruption that paragraph dilution masks (P²-sanity passes 0/9 on MiniMax low-Openness under identical polarity injection); atom-level repair uses ~1/3 the paragraph-baseline character footprint at comparable fix success (byte-identical on 13/13 paired GPT-4o cross-vendor conditions); and two audit modes (universal-recall sweep, sparse-edit triage) support deployment-time governance. Across six backbones, CAPS beats Simple-Prompt on all three backbones of PersonalityBench (Deng et al., 2025) (mean +0.69/10), delivers monotonic 1-D continuous control via mixture weights π on all 15 (vendor × Big-Five) Mini-IPIP cells, and matches the strongest paragraph baseline (P²) on the non-all-low 5-D profiles (§4.6; backbone-level direction-match ≥ 0.93). Cross-family judge replication (r = +0.951) and a two-rater human pack triangulate the headline. Code, corpus, and logs are released.

Abstract:
With the remarkable success of Vision-Language Models (VLMs) on multimodal tasks, concerns regarding their deployment efficiency have become increasingly prominent. In particular, the number of tokens consumed during the generation process has emerged as a key evaluation metric. Prior studies have shown that specific inputs can induce VLMs to generate lengthy outputs with low information density, which significantly increases energy consumption, latency, and token costs. However, since output length is non-differentiable, existing methods mainly prolong the generated content by disrupting the distribution of the next-step token following the prompt, thereby steering the output away from the original normal response. This strategy neglects the subsequent autoregressive generation process, which makes the attack effect unstable and difficult to control. To address these limitations, this paper proposes a novel gray-box verbose-text induction attack (VTIA) to inject imperceptible adversarial perturbations into benign images via a two-stage framework. Specifically, we first perform adversarial prompt search , employing reinforcement learning strategies to automatically identify adversarial prompts capable of inducing the LLM component within VLMs to produce verbose outputs. We then conduct vision-aligned perturbation optimization to craft adversarial examples on input images, maximizing the similarity between the perturbed image’s visual embeddings and those of the adversarial prompt, thereby constructing malicious images that trigger verbose text generation. Comprehensive experiments on five popular VLMs demonstrate that our method achieves significant advantages in terms of effectiveness, efficiency, and generalization capability.

Abstract:
Block diffusion language models accelerate decoding through parallel block-wise denoising, yet whether they can be reliably scaled for long chain-of-thought (CoT) reasoning remains unresolved. To this end, we develop -8B, an open-source block diffusion reasoning model, and conduct a systematic study of how training and inference block sizes affect long-CoT reasoning. Our analysis reveals a stark performance disparity: training with large block sizes yields remarkably poor reasoning, whereas small block sizes preserve effective reasoning. To bridge this granularity gap, we propose block-size curriculum learning, which gradually transitions training from fine-grained to coarse-grained block sizes, thereby overcoming this limitation and enabling strong reasoning performance that generalizes across diverse inference block sizes. On mathematical and code reasoning benchmarks, -8B achieves results competitive with leading open autoregressive models such as Qwen3-8B. This work establishes a practical foundation for efficient, reasoning-capable diffusion language models. We release our model at https://github.com/DreamLM/DreamReasoner.

Abstract:
Reinforcement learning in open-ended long-form generation is challenging because reliable reference answers and automatic metrics are often unavailable. Existing rubric-based methods typically rely on pointwise LLM-as-a-judge scoring, but absolute scores are difficult to calibrate across complex responses, may provide weak discrimination among same-query rollouts, and can become saturated during optimization. We propose Tournament-GRPO, a group-wise reward framework that converts rubric-guided LLM judgments into relative rewards through repeated multi-round tournaments among same-query rollouts. Tournament-GRPO compares candidates within groups, accumulates tournament outcomes, and normalizes them into group-wise rewards for GRPO training. Experiments on Deep Research Bench show that Tournament-GRPO consistently outperforms existing reward-design baselines, achieving a 4.52-point overall-score improvement over the strongest baseline. Further analyses show that tournament rewards provide a favorable effectiveness–efficiency trade-off and that tournament design affects training dynamics. These results suggest that rubric-guided tournament comparison provides an effective reward signal for reinforcement learning in open-ended long-form generation.

Abstract:
Real-world deployments of large language models (LLMs) increasingly rely on user-provided instructions that are long, precise, and tailored to specific workflows. In such settings, aggregate leaderboard performance is often a weak proxy for a model’s ability to faithfully execute the particular instruction that a practitioner intends to deploy. To address this gap, we introduce FlexBench/FlexEval, an instruction-conditioned evaluation framework that instantiates a specialized test suite from a user-provided target instruction. Given a target instruction, FlexBench (i) derives a set of verifiable evaluation dimensions and (ii) generates an instruction-conditioned conversation corpus. FlexEval then aggregates dimension-level judgments into two complementary metrics: Coverage, which measures workflow progress, and Achievement, which measures conditional correctness. We evaluate the framework on nine multi-turn target instructions spanning diverse scenario categories and workflow topologies, across nine contemporary LLMs over 3,888 generated dialogues. The results show that model rankings and failure modes vary substantially across target instructions, highlighting the need for instruction-conditioned evaluation beyond fixed aggregate benchmarks. Our source code is publicly available at https://github.com/Coryyy/FlexBench.

Abstract:
Recent years have witnessed remarkable achievements of Large Language Models (LLMs) in multiple domains, while the excessive resource requirements of LLMs hinder the deployment on resource-constrained devices. Although model quantization stands out as an effective approach, conventional quantization approaches typically incur severe performance degradation due to uniform bit-width or simple heuristic sensitivity evaluation. In this paper, we propose a novel Fisher information-based Adaptive Mixed Precision Weight Quantization approach, i.e., , which performs layer-adaptive weight quantization for effective LLM inference on commodity GPUs. First, we propose a system model with a novel Fisher information metric to measure the layer-wise sensitivity to quantization. Second, we propose a reinforcement learning-based bit-width allocator in , which generates an adaptive bit-width allocation strategy based on the Fisher information sensitivity metric. Extensive experiments on 7 models and 5 benchmarks demonstrate that significantly outperforms 7 baseline approaches in terms of PPL (up to 3.39 smaller), accuracy (up to 6.87% higher), and LLM-as-a-judge comparison (up to 76% win rate).

Abstract:
Weakly-Supervised Dense Video Captioning aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level captions per video. Recent work synthesizes auxiliary transition captions via LLM to provide additional vision-language alignment, but these captions lack visual grounding and are rigidly assigned to every inter-event gap at a fixed location and duration. To address these, we propose Seeing Before Synthesizing (SBS), a framework that adaptively provides visually grounded linguistic guidance only where warranted. Leveraging a VLM, we generate frame-level narratives for the inter-event gap and detect transitions from the semantic variation across them. For identified transitions, we then refine inter-event temporal masks by blending the temporal midpoint with the semantic change point and selecting the width that maximizes vision-language alignment. Experiments on ActivityNet Captions and YouCook2 demonstrate state-of-the-art performance in both captioning and localization.

Abstract:
Image retouching requires sophisticated aesthetic understanding and precise parameter control, yet existing agents typically rely on fixed, open-loop execution or software-specific supervision. We introduce AdaRetouch, an interleaved vision-language agent for adaptive, multi-turn image retouching. Given an instruction and an image, AdaRetouch dynamically diagnoses the visual state, invokes non-destructive tools, and iteratively observes the results until the intent is satisfied. To train this closed-loop behavior, we propose a software-agnostic pipeline that constructs trajectories featuring curriculum-driven tool sequences and step-wise reasoning. AdaRetouch is initialized via supervised fine-tuning and further optimized using process-level reinforcement learning for quality and efficiency. Experiments on AdaRetouch-Bench show that AdaRetouch outperforms general MLLMs, existing agents, and diffusion editors in intent alignment, perceptual quality, and calibrated tool planning.

Abstract:
Intermediate-layer representations in large language models (LLMs) contain task-relevant information, yet direct linear readouts often fail to recover the model’s eventual choice. This gap limits both interpretability and early-exit methods for candidate-scoring workloads. We study single-token decision settings – such as multiple-choice and boolean QA – where the answer is a single token at a position fixed by the task format, and we read out the model’s prediction there as its highest-probability token over the full vocabulary. We propose SimLens, a training-free readout that, at a chosen intermediate layer, keeps only the start token and the candidate answer token and forwards this reduced two-token state through the model’s remaining upper layers. To identify the layer at which the full sequence can be safely truncated, we distill LinearSimLens as a lightweight entropy monitor and combine the two in SimExit. Across different models, SimLens produces earlier and more faithful candidate readout than direct linear lenses under fair-compute accounting, while SimExit delivers a modest but stable speed–accuracy trade-off on the two models used end-to-end. Ablations show that the two retained tokens play complementary roles as global condition and semantic anchor.

Abstract:
Pediatric serious illness communication (SIC) is critically important, yet scalable communication training for clinicians remains limited. Compared with other dialogue simulation settings, pediatric SIC poses additional challenges, including multi-party interactions, response to parental distress, and strong dependence on feedback dynamics. Existing LLM-based simulators optimize generic dialogue quality rather than curriculum-contingent behavior required for effective SIC training. In collaboration with educators and pediatric clinicians, we introduce the first benchmark suite and simulation framework tailored to pediatric SIC training. Our benchmarks, PitfallBench and DialogueBench, evaluate simulators both at the turn-level and across full dialogues. We further propose SIC-Agents, a self-improving framework that generates a clinician-editable skill document to guide simulator behavior. Our experiments show that SIC-Agents outperforms static expert prompting. To support future research, we release our benchmarks and a training interface for parent simulation in pediatric SIC.

Abstract:
Documents that users query mix tables with prose, but the QA systems built for them do not: retrieve-and-read pipelines flatten tables, while text-to-SQL pipelines ignore passages. Corporate filings, scientific reports, and Wikipedia articles all sit on this fault line, and a single user question routinely crosses it. We present ATR (AgenticTableRAG), an agentic framework for hybrid table-text question answering. ATR decomposes each question into ordered sub-queries and dispatches each to one of four execution primitives: text reading, cell retrieval, SQL execution, or a hybrid of cell grounding and SQL. When the retrieved evidence does not support the predicted answer, the router is re-invoked with the failed-route history and dispatches a different primitive. Value grounding composed with constrained SQL execution rules out column and value hallucinations by construction. The routing policy is distilled into a lightweight student model that transfers across benchmarks without additional inference-time LLM calls. A single ATR configuration leads on token F1 across three benchmarks without per-dataset tuning.

Abstract:
Ensuring that large language models (LLMs) reflect diverse cultural values is important for globally deployed NLP systems. However, existing approaches often treat cultural groups as homogeneous and overlook within-group heterogeneity arising from intersecting demographic attributes, leading to unstable behavior under varying persona granularity. To address this gap, we propose ACE-Align (Atribute Causal Effect Alignment), a causally inspired framework based on controlled persona edits that aligns how specific demographic attributes shift different cultural values, rather than treating each culture as a homogeneous group. We evaluate ACE-Align across 14 countries spanning five continents, with personas specified by subsets of four attributes (gender, education, residence, and marital status) and granularity instantiated by the number of specified attributes. Across all persona granularities, ACE-Align consistently outperforms baselines. Moreover, in within-survey comparisons, it reduces the average Global North–South alignment gap from 3.40 to 1.11 points on WVS and from 2.53 to 0.85 points on ISSP. Code and dataset are released at https://github.com/Wells-Luo/ACE-Align.

Abstract:
Hateful meme detection remains a formidable challenge for vision-language models, as existing benchmarks are structurally observational—confounding rhetorical hate mechanisms with target community features and preventing causal evaluation of model vulnerabilities. To address this, we introduce FBHM, a systematically curated benchmark of Functionality Based Hateful Memes constructed along two orthogonal axes: 25 distinct rhetorical functionalities and 10 target communities (5,000 memes total). Benchmarking state-of-the-art VLMs reveals a severe generalization gap: models highly accurate on standard datasets catastrophically drop to near-random performance on FBHM, proving they exploit dataset-specific heuristics rather than robust multimodal reasoning. To efficiently close this gap, we propose LSV (learnable steering vectors), an ultra-low data regime strategy that applies a causal intervention objective on as few as 500 steering samples (50 unique base memes), boosting FBHM performance by ~30 Macro-F1 points while outperforming in-context learning and PEFT without degrading source-domain performance.

Abstract:
Prompt-injection detectors are heterogeneous: each is strong on a different slice of attacks, and none is always reliable. Yet existing systems still treat detection as a fixed single-detector pipeline, committing every request to one detector’s blind spots. We reframe defense as detector allocation: given a heterogeneous pool, decide per request which detectors to run and whether to escalate to an LLM judge. Our framework SCOUT (Scalable and Controllable Outcome-prediction for Uncertainty-aware Triage) makes this decision dynamic by predicting each detector’s per-sample reliability and latency from how it behaved on similar past inputs, and exposes a single safety-utility threshold to the operator (where utility bundles benign-pass rate and wall-clock). To evaluate this setting, we build SCOUT-450, a benchmark that captures the structurally complex, agent-facing injections that older prompt-injection sets under-represent. On SCOUT-450, a safety-oriented operating point reduces attack-success rate by 46% and total wall-clock by 40% relative to an always-on GPT-4o judge, at a 5.1-point benign-utility drop. SCOUT also transfers to three external benchmarks (BIPIA, IPI, and IHEval), improving the safety-utility frontier.

Abstract:
Recursive (looped) Transformers decouple computational depth from parameter depth by repeatedly applying shared layers, providing an explicit architectural primitive for iterative refinement and latent reasoning. However, early looped Transformers often underperform non-recursive baselines of equal compute. While recent literature has introduced more effective recursion mechanisms to mitigate this gap, existing architectures still operate at a fixed, full-token resolution, neglecting the potential efficiency of computing over compressed latent representations. In this paper, we propose SpiralFormer, a looped Transformer that executes recurrence under a multi-resolution recursion schedule. We provide probing evidence that multi-resolution recursion enables the model to learn hierarchical dependencies by inducing iteration-wise functional specialization across different scales. Empirically, SpiralFormer achieves better parameter and compute efficiency than both looped and non-looped baselines across model scales from 160M to 1.4B, establishing sequence resolution as a potential axis for scaling recursive architectures.

Abstract:
Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and recognize unsafe content. In this work, we reveal that this advanced safety awareness inadvertently introduces a fatal vulnerability. We introduce Posterior Attack, a single-query jailbreak that bypasses guardrails by prompting the model to generate the exact harmful response its internal classifier would normally flag as unsafe. Through extensive empirical evaluation across 30 open-source LLMs (up to 35B parameters in size) and frontier models (e.g., GPT-5, Claude 4.6), we observe a striking phenomenon: models with superior safety-judgment capabilities are disproportionately more susceptible to this exploitation. To explain this, we formalize the Safety Paradox, analytically showing that monotonic improvements in safety alignment naturally amplify posterior vulnerability. Finally, we establish a causal link via reinforcement learning interventions, exemplifying that artificially degrading a model’s safety judgment immunizes it against the attack, whereas enhancing judgment exacerbates the vulnerability. Our findings highlight potential flaws in current alignment paradigms, indicating that defense mechanisms may require further structural refinement.

Abstract:
Instruction-based suppression is widely used to prevent language models from generating prohibited content, yet it remains unclear whether suppression reduces internal representation or merely suppresses expression. We investigate this question through representational probing, attention analysis, and behavioral semantic leakage experiments across multiple transformer models. We find that prohibited concepts remain highly recoverable from hidden representations under suppression, influence localized changes in attention routing, and measurably shape downstream generations despite successful lexical avoidance. These effects persist across pooling strategies, indirect semantic controls, and multiple model families. Our results expose a fundamental gap between behavioral and representational alignment.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models. However, rollout-intensive policy optimization is often limited by insufficient reward contrast, arising when overly simple or complex prompts generate low-variance feedback and when outcome-only rewards assign the same terminal assessment to every decision in a multi-turn rollout. Past efforts have focused on allocating available rollout resources to promising prompts, yet they only leverage sample informativeness at the prompt level and neglect variation in prefix-level informativeness across turns within the same rollout. This work targets multi-turn agentic RL by modeling each ReAct-style thought-action-observation turn as a semantically distinct node, allowing budget allocation to extend from prompt roots to turn-level prefixes with further continuations, which naturally forms tree-structured rollouts. We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework that enhances reward contrast within a fixed sampling budget. Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards. A shared generalizable predictor estimates conditional success probability at these anchors from prefix histories to guide this allocation. The resulting adaptive tree structure enriches outcome-only feedback and amplifies the policy-update signal. Empirically, TRACE achieves competitive performance and efficiency gains on typical agentic benchmarks, e.g., improving Qwen3-14B Multi-Hop QA average accuracy by 2.8 points over competitive baselines at equal sampling cost. Code is available at https://github.com/gfyddha/TRACE.

Abstract:
Multimodal Large Language Models (MLLMs) have achieved impressive progress in image-text comprehension and generation, yet they remain susceptible to jailbreak attacks that can trigger harmful outputs and pose serious safety concerns. Existing multimodal jailbreak attacks have shown the feasibility of such attacks, but they still face two fundamental challenges: the lack of a atomic multi-modal strategy space, the absence of a concise and efficient executable framework beyond human-craft experience. To address these challenges, we first decompose the text-image jailbreak strategy space into three levels: structural, semantic, and syntactic, constructing a jailbreak strategy set encompassing both text and image modalities to systematically achieve combined coverage of different attack types. Then we propose a multimodal automated red team jailbreak method named Hierarchical Atomic Combination Attack (HACA). Specifically, based on a six-dimensional strategy space, a cross-modal joint planner is used to select and combine the different atomic jailbreak strategy for subsequent jailbreak command generation. Finally, at the implementation level, we explore to apply a unified generate executor to directly generate jailbreak instructions based on the selected multi-modal strategies. A series of experiments show that our automated red team method can achieve an attack success rate of average 95.48% against five mainstream MLLMs.

Abstract:
Long-context and agentic LLM workloads push the KV cache past any fixed memory budget, forcing the inference stack to permanently evict tokens at every step of a continuous-inference stream. Existing methods all share the same template: a per-step direct-attention score followed by deterministic top- K selection, which converts a single below-cutoff step into an irreversible verdict and permanently erases any subtly important token that direct attention cannot single out from noise. To address this challenge, we propose Nexus Sampling, a training-free eviction method that pairs Nexus scoring, an iterative walk over direct attention that surfaces bridge tokens, with weighted reservoir sampling, which retains tokens with inclusion probability in place of deterministic top- K . Theoretically, we show that Nexus Sampling dominates deterministic top- K in long-run survival of subtly important tokens. Empirically, at 80% KV cache eviction, Nexus Sampling matches dense attention within ∼ 1 point on LongBench while outperforming top- K baselines on retrieval-heavy tasks, with up to 10× smaller per-sequence cache memory. Our code is available at .

Abstract:
World models and multimodal large language models (MLLMs) provide complementary capabilities for predicting future outcomes from static visual observations. World models can generate concrete visual rollouts of possible futures, while MLLMs can reason abstractly over questions, goals, and rules. However, generated rollouts are stochastic and may be visually plausible but task-incorrect, making it necessary to determine when visual simulation is useful, whether a rollout is credible, and how it should influence the final answer. We formulate this problem as controlled concrete reasoning, where a model learns to invoke, verify, and integrate visual future simulation alongside abstract reasoning. To study this setting, we construct two human-verified benchmarks, VRQABench for controllable spatial lookahead and OpenWorldQA for open-domain physical prediction, and propose Privileged-Future On-Policy Self-Distillation (PF-OPSD). During training, PF-OPSD uses ground-truth future videos and answers only as teacher-side privileged context to evaluate on-policy concrete-reasoning trajectories, while the deployable student never observes true futures at test time. Experimental results show that PF-OPSD outperforms baseline by 10.6% and 10.9% on VRQABench and OpenWorldQA, respectively, while increasing robustness to noisy or conflicting rollouts. Our code and dataset are available at https://github.com/yczhou001/PF-OPSD.

Abstract:
The widespread use of Large Language Models (LLMs) as writing tools challenges the validity of crowdsourced data, as crowdworkers may outsource tasks to models. To better understand how this is addressed, we surveyed 155 researchers in NLP and related disciplines about their experiences and opinions on collecting free-text responses via crowdsourcing. This paper provides an overview of practitioners’ challenges, mitigation strategies, and the foreseen implications on data quality. 44% of respondents reported observing LLM usage in their crowdsourced data. While 93.2% of them had anticipated this, half were unsure what precautions to take. The most prevalent detection strategies are distinctive textual style patterns and unusually fast completion times. Overall, survey responses show that the research community is aware of the problem and taking measures, but existing efforts remain insufficient to fully address it. Finally, we derive a set of considerations to guide future crowdsourced free-text data collection in the era of LLMs. We make the survey and an anonymized version of the responses available for researchers (https://doi.org/10.7802/3102).

Abstract:
Narrow-domain finetuning makes vision-language models (VLMs) sound like domain experts while quietly making them less attentive to the image. We attribute this gap to a persistent domain prior: an input-agnostic shift in hidden state, installed by narrow finetuning, that biases the model toward domain-style outputs regardless of visual evidence. To test the claim we introduce ActDiff, a white-box diagnostic that isolates the finetuning trace by differencing base and finetuned hidden states under progressively degraded images. The trace is compact and persists under blank and Gaussian-noise inputs; the finetuned model is correspondingly less, not more, disrupted when the image is destroyed (gap 0.323 ± 0.036 nats on BIOMED, generic eval). Subtracting the trace at inference recovers 54% of the MMStar accuracy collapse on BIOMED, with a peak-and-collapse dose response that random and wrong-domain controls do not produce. A single first-moment regulariser added at training time shrinks the trace 34x and improves generic NLL. The prior thus has a clean diagnostic, a causal handle at inference, and a one-line mitigation at training.

Abstract:
Test-time scaling improves the reasoning performance of large language models but incurs substantial cost in both total computation and latency. Existing adaptive sampling methods partially mitigate this issue by dynamically deciding when to stop sampling, yet they typically rely on heuristic rules or rely on distribution assumptions. In this work, we formulate adaptive sampling as a Markov decision process (MDP). We train a lightweight sampling controller with reinforcement learning (RL) to jointly balance answer correctness, latency, and computation cost. At each round, the controller decides to stop sampling or to acquire additional samples. Our method is lightweight which only relies on statistics of final answers, and can be trained and deployed on CPU. We further show that the resulting framework admits an interpretation as the Lagrangian relaxation of a constrained optimization problem with explicit budget constraints. Experiments against strong baselines such as ASC and ESC show that our method achieves improved trade-offs among answer correctness, sampling rounds, and total samples required.

Abstract:
Science news is an important medium to communicate discoveries between the research communities and the public. Yet, most metrics for generated or summarized text evaluate semantic similarity and factual consistency, but do not measure how much knowledge readers learn from the news. We introduce KnowledgeGain, a metric that evaluates the quality of science news by measuring how much knowledge the readers gained after reading it. To evaluate the metric, we first performed a controlled human study and showed that the metric successfully captures the differential knowledge gained by human readers reading different types of science media. The data allowed us to calibrate a prompt-only LLM reader simulator. We use it to rank and filter candidate articles before human evaluation. A second human study shows that articles selected with this simulator improve post-reading accuracy and normalized KnowledgeGain over a strong generation baseline. Our work is a step toward generating science news that better meets the knowledge and comprehension goals of Bloom’s Taxonomy.

Abstract:
Linguistic representation learning in deep neural language models (LMs) has been studied for decades, but finding representations in LMs remains an unsolved problem. On the one hand, unconstrained alignments may trivialize the notion of representation (Sutter et al., 2025); on the other, even recently popularized linear approaches may not always be faithful to natural model behavior (Arora et al., 2024). Here we escape this dilemma by reconceptualizing representations not as patterns of activation but as conduits for learning. Our approach is simple: we perturb an LM by fine-tuning it on a single adversarial example and measure how this perturbation “infects” other examples. Perturbation makes no geometric assumptions, and unlike other methods, it does not find representations where it should not (e.g., in untrained LMs). But in trained LMs, perturbation reveals structured transfer at multiple linguistic grain sizes, suggesting that LMs both generalize along representational lines and acquire linguistic abstractions from experience alone.

Abstract:
The widely adopted Top-K router in Mixture-of-Experts (MoE) models is mathematically non-differentiable, making them harder to train than dense counterparts. We propose DenseMixer, a simple and effective MoE post-training technique that provides a more precise router gradient estimation by trading in some extra compute. It is universally applicable to TopK routed MoE, operates in a plug-and-play manner, preserves inference-time efficiency, and remains fully compatible with popular training libraries as well as parameter-efficient methods such as LoRA. Extensive experiments demonstrate that DenseMixer consistently outperforms conventional approaches when fine-tuning MoE models across scales (7B–30B), architectures (with and without shared experts), pre-training regimes of the underlying MoE models (from scratch or via up-cycling), and post-training data types (instruction tuning and long chain-of-thought). Furthermore, our analysis on efficiency shows that the additional computation introduced by DenseMixer remains modest, providing substantial improvements at an acceptable cost.

Abstract:
Chain-of-Thought (CoT) is often viewed as a window into LLM decision-making, yet recent work suggests it may function merely as post-hoc rationalization. This raises a critical alignment question: Does the reasoning trace causally shape model generalization independent of the final answer? To isolate reasoning’s causal effect, we hold final harmful answers constant while varying reasoning paths across three types, Evil (embracing malice), Misleading (rationalizing harm), and Submissive (yielding to pressure), and train models (Qwen 0.6B–14B) under QTA, QT, T-only, and QA paradigms, evaluating in both think and no-think modes. We find that: (1) CoT training amplifies harmful generalization beyond standard fine-tuning; (2) distinct reasoning types induce distinct behavioral patterns despite identical answers; (3) training on reasoning without answer supervision suffices to alter behavior; and (4) these effects persist when CoT is suppressed at inference. We validate these findings with naturally generated CoT, on GPT-OSS-20B, and under RLVR training, reporting bootstrap 95% CIs throughout. Our findings demonstrate that reasoning content is causally potent for downstream alignment, challenging strategies that supervise only final outputs.

Abstract:
Counter-storytelling is a powerful mechanism people use to challenge dominant narratives. Unlike other forms of counterspeech that have been widely studied in computational social science, counter-storytelling has largely been overlooked. Counter-stories are difficult to detect automatically; they are relational (defined with respect to expressions of racial stereotypes) and structurally diverse (drawing on stories that describe lived experiences, witnessed events, exemplars, and hypotheticals). We introduce a first framework for detecting and characterizing counter-storytelling against racial stereotypes at scale. This includes (1) a three-dimensional taxonomy grounded in narratology and Critical Race Theory and (2) a multi-stage pipeline that identifies relational pairs of stereotypes and counter-stories in noisy Reddit discourse. Using this pipeline, we annotate 25,549 Reddit posts across 615 communities and identify 1,312 counter-stories. Our analysis shows that speaker identity and post context shape how counter-stories are told. For example, in-group writers favor first-person testimony, often adopting the role of self-reflective insiders. Our work shows how computational methods can scale qualitative approaches to identify and characterize counter-storytelling as a contextual narrative practice, with implications for content moderation, narratology, and racial discourse analysis.

Abstract:
Zero-shot detectors for LLM-generated text aggregate token-level statistics uniformly, implicitly assuming that every position contributes equally to detection. We challenge this assumption and show that detection signal is highly uneven across token positions: positions of high contextual dependency, where the surrogate model relies on concentrated context, carry far more discriminative power than others. Using attention entropy to quantify this dependency, we empirically validate that aggregating only low-entropy tokens yields clearly separated human and LLM score distributions, whereas high-entropy tokens do not. Based on this finding, we propose Token Contextual Dependency (TCD), a plug-and-play framework that reweights token contributions in existing zero-shot detectors via attention entropy-based weights computed from dynamically selected layers. Experiments on state-of-the-art detectors across multiple benchmarks demonstrate consistent and meaningful improvements in detection performance. Our code is available at: https://github.com/RecursionM/TCD.

Abstract:
Think-with-images, where multimodal LLMs (MLLMs) actively manipulate visual inputs as part of their reasoning, has become a focal research direction for agentic multimodal systems, with a surge of prompting, supervised, and reinforcement learning (RL) methods teaching models to invoke vision tools such as cropping, zooming, and editing. Yet most established multimodal benchmarks use images whose critical content is already easy to perceive in a single pass, and many have saturated on frontier models. We introduce VisualToolBench (VTB), a benchmark deliberately curated so that critical visual content is dense, occluded, dispersed, rotated, or low-contrast, making active image manipulation a prerequisite for success. VTB contains 1,204 expert-authored, open-ended tasks (603 single-turn, 601 multi-turn) across five domains. Each task is paired with rubrics tagged by five competency categories (Visual Understanding, Truthfulness, Instruction Following, Reasoning, and Writing Style) that enable per-axis disentanglement. Across 26 evaluated MLLMs, even the strongest model reaches only 29% accuracy, with Visual Understanding the consistent bottleneck and visual misinterpretation accounting for 32% to 39% of failures. Top models, moreover, converge to comparable accuracy through markedly different tool-use strategies, which we analyze in detail.

Abstract:
Pre-training data mixtures are commonly tuned by running small-scale experiments and extrapolating to the target training budget. When high-quality data is scarce and must be repeated, this extrapolation frequently fails, but the source of the failure has not been isolated. We show that a primary culprit is a repetition mismatch: because high-quality datasets are small, their repetition rate changes as the training budget grows, shifting the optimal mixture in ways that small-scale proxy experiments do not anticipate. A subsampling procedure that matches the target repetition rate controls for this effect. In a two-source setting combining limited high-quality data with web crawl, a single repetition-controlled experiment using only 1/16 of the target tokens recovers a mixture within 0.10 of the optimum on WikiText for a 1.17B parameter model, compared to an error of 0.85 without repetition control. Achieving comparable accuracy without repetition control requires multiple training horizons, consuming 19%, 44%, and 94% of the target token budget when using the results from two, three, and four horizons respectively. With three data sources, the larger mixture space requires more than a single experiment to constrain, but the approach remains effective: at the 757M scale, just two repetition-controlled horizons recover the optimal mixture, outperforming baselines that instead require the full two-source experiments to construct. Our results reveal that repetition dynamics, not scale alone, shape whether small-scale mixture experiments generalize. More broadly, they suggest that data repetition deserves treatment as a first-class variable in mixture optimization, rather than an inconvenient side effect of limited data.

Abstract:
Practical video editing is not only pixel generation: an editor must turn a brief, a clip pool, music metadata, and hard constraints into an executable timeline. We study this decision layer as executable video-editing planning and introduce RefineCut, which, unlike workflow systems that wrap a prompted frontier model, trains a compact open-weight planner for it. The planner edits a typed timeline through structured patches covering clip selection, trimming, ordering, transitions, and duration and music alignment; a deterministic verifier applies each patch and checks it against an explicit constraint ledger. Because editing has no single ground-truth repair, we do not imitate teachers directly: RefineCut replays every multi-teacher branch through the verifier and keeps verifier-best repairs as supervision. A second stage, RefineCut-Evo, lets the student score its own repairs with the verifier and a task rubric and trains on high-margin preference pairs, so the final 8B planner runs in a closed verifier loop with no teacher calls at inference. On RefineCut-Bench (3,578 tasks, 7,971 captioned clips, 499 music tracks, explicit ledgers), verifier-replayed distillation lifts the planner from 0.620 to 0.858 on the protocol-specific Video-Editing Score and RefineCut-Evo reaches 0.924; the gain transfers to Llama-3.1-8B and GLM-4-9B, and in the same closed loop the 8B planner matches or exceeds its frontier teachers. Code and RefineCut-Bench are publicly released; see the Data Availability statement.

Abstract:
Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Services, and Insurance (BFSI) deployments exposed to failures elicited through legally or professionally plausible framing. We introduce RAHS (Risk-Adjusted Harm Score), a risk-sensitive metric jointly capturing disclosure severity, disclaimer mitigation, and inter-judge agreement, and FinRedTeamBench , a 989-prompt benchmark spanning seven BFSI risk areas and 34 sub-categories mapped to regulatory frameworks. Evaluation uses an ensemble of three heterogeneous LLM judges, validated against human experts, and an adaptive multi-turn red-teaming pipeline. On nine open-weight models, RAHS preserves separation under near-ceiling ASR, ranking is stable under hyperparameter sweeps, and multi-turn pressure drives not only more jailbreaks but more operationally severe disclosures, exposing failure modes that single-turn, domain-agnostic evaluations cannot reveal.

Abstract:
Large language model (LLM)–based multi-agent systems tackle complex reasoning by orchestrating how multiple agents are configured and how they collaborate. A central challenge is to adapt orchestration to the evolving collaboration state. Routing from the query alone cannot adapt to intermediate progress or errors, which hurts accuracy. Routing from the complete execution history supplies this missing context, but forces later decisions to process every prior step, including redundant or low-utility ones. This creates an execution-history overload that inflates cost. Effective orchestration instead requires a compact state that captures useful progress without accumulating redundant context. We propose Gated-Memory Routing, which conditions each decision on the query and a learned execution memory. A learned Memory Write Gate commits only non-redundant reasoning steps, and a learned Retrieval Gate supplies each agent a compact, relevant subset, so every decision conditions on a clean, informative state. At each step, the system selects the next role and backbone from this memory, while an Adaptive Halting Controller stops execution once the memory contains sufficient evidence for answering. Across five reasoning and code-generation benchmarks, our framework is both effective and efficient: it attains the best average accuracy, exceeding the strongest baseline by 2.44 points, while reducing HumanEval inference cost by 31.9% relative to that baseline. Code is available at https://github.com/rajibrhasan/gated-memory-routing.

Abstract:
While Large Language Models have improved rapidly, many fundamental questions remain about how to evaluate the knowledge and reasoning abilities they acquire, and how such evaluations relate to the learning signals used in pre-training. In this paper, we propose ModelLog, a declarative probabilistic framework for pre-training evaluation that makes the semantic structure of model behavior explicit and provides new formal tools for relating evaluation to learning. ModelLog specifies evaluation targets as symbolic constraints over token-level predictions and measures how strongly a model’s distribution satisfies those constraints. We explore the framework through a new suite of tasks targeting negation, mutual exclusivity, and consistency, finding systematic failures that are difficult to characterize through token likelihood or answer accuracy alone. We further show that these evaluation scores can also be interpreted as losses, whose gradients reflect logical strength, informativeness, and variable-level sensitivity. This links evaluation and learning through a shared semantics, suggesting evaluation methods that diagnose model behavior while also helping to clarify the semantic structure of learning.

Abstract:
Multilingual deployment often requires estimating how well a model will perform on a task in a target language when direct benchmark results are unavailable and published evidence is incomplete. We study whether structured agentic reasoning can support this predictive multilingual evaluation task. We introduce a controlled benchmark of 1,500 questions spanning six tasks and five evidence scenarios. The benchmark separates accessible evidence from ground truth, enabling evaluation of systems that must infer missing results from incomplete literature evidence. We also present Litmus (Re)Agent, a DAG-orchestrated agentic system that decomposes queries into hypotheses, retrieves evidence, and synthesises predictions through feature-aware aggregation. Across six systems, Litmus (Re)Agent achieves the best overall performance, with the largest gains in transfer-heavy scenarios where direct evidence is weak or absent. These results show that structured agentic reasoning is a promising approach to multilingual performance estimation under incomplete evidence.

Abstract:
Emotion recognition in conversations is increasingly tackled with language models, but these models can be unstable and expensive to fine-tune or to prompt with long dialogue histories. We propose EmoLASP, a framework that combines a language model with declarative reasoning via Answer Set Programming (ASP) to predict VAD scores (Valence-Arousal-Dominance) in conversations. Experiments on a widely used benchmark dataset (IEMOCAP) across six open-source LLMs (3B-120B) and two PLMs (BERT, RoBERTa) show that EmoLASP improves prediction performance compared to using the language model alone, even when the LLMs/PLMs are given no dialogue history in their prompts or input vectors. The gains are largest for prompt-only LLMs, which EmoLASP uses without any fine-tuning. However, for fine-tuned PLMs, the reasoner adds little once dialogue history is available. EmoLASP’s LLM pipeline demonstrates the potential advantages of using a reasoning approach to ensure emotion prediction consistency and to reduce both the cost of fine-tuning and the cost of prompting with long dialogue histories.

Abstract:
When retrieved evidence contradicts parametric memory, language models frequently ignore the context and revert to memorized priors—a failure that undermines the purpose of retrieval augmentation. Contrastive decoding amplifies context-conditioned output to suppress parametric bias, but fixed coefficients cannot track changing disagreement, while output-only adaptive coefficients observe only the final prediction distributions. We propose FIDES (Faithful Inference via Deep Evidence Signals), a training-free decoder that combines signals from output distributions, hidden representations, and prediction trajectories to recompute contrastive intervention strength at every step. A matched, formula-verified audit on 128 NQ-Swap pairs shows that conflict raises Opposition by 0.0969 and the resulting coefficient α by 0.0727 at the same answer-prediction position (both p < 10⁻⁴). The audit also finds only a small Shift effect and saturated clipped Noise, clarifying the limits of the mechanism interpretation. Across three benchmarks and six backbones—four primary 7B/8B models and two scaling backbones up to 70B—FIDES achieves the best context fidelity in all 18 settings, outperforming the strongest training-free baseline by +3 to +13 points. At 70B, fidelity reaches 92–94% while F1 reaches 62–63%, showing that multi-depth stepwise contrast can preserve generation utility while improving evidence following.

Abstract:
Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existing PTQ methods often fail to generalize across models and suffer severe accuracy loss below 2 bits. Many leverage unstructured sparsity to mitigate this loss, but at the cost of regularity and GPU-friendly execution. We present QTEA, a sub-2-bit PTQ framework that quantizes weights into ternary values and uses salient weights as residual error compensators. To maintain hardware efficiency, residuals are assigned to selected columns with semi-structured 1:4 sparsity within the salient columns. We further add column-wise rescale refinement to GPTQ-style column-by-column quantization, alternately updating per-column scales and ternary assignments to reduce reconstruction error. We also identify order-dependent error propagation in GPTQ and introduce error decay to attenuate late-stage error accumulation. On Qwen3-14B, QTEA compresses all weights to an effective 1.7 bits per weight while improving average accuracy over the strongest ternary PTQ baseline by 16.7%. It also achieves 1.40× and 2.61× lower perplexity on WikiText and C4 respectively. This trend holds on Llama3-8B, where QTEA obtains a 6.6% accuracy gain and 1.34×/1.95× lower perplexity on the same datasets. Finally, we develop a lookup-table based kernel that achieves 7.2× faster per-token generation over an FP16 baseline.

Abstract:
Multilingual large language models (LLMs) depend on subword tokenization to bridge discrete text and continuous neural representation. State-of-the-art multilingual LLMs often use Byte-level Byte-Pair Encoding (BPE) tokenizers that structurally favor high-resource languages and Latin scripts. For speakers of underrepresented languages, particularly those across Southeast Asia, this bias inflates inference costs and widens cross-lingual capability gaps. We present the first systematic comparison of equitable tokenizers on a unified benchmark spanning 11 Southeast Asian languages. Beyond tokenizer-level analysis of compression efficiency and cross-lingual equity, we assess downstream task performance through controlled 1.5B-parameter language model training using the same training data. Our results show that Parity-aware BPE lies on the Pareto frontier of the efficiency-equity trade-off, achieving strong compression parity at competitive cost. Morphology-Driven Byte Encoding delivers the best semantic reasoning performance through morphologically richer representations, albeit at a higher computational expense. Byte Latent Transformer underperforms on downstream tasks, possibly because its architectural assumptions misalign with the constraints of limited low-resource training data. Together, our findings demonstrate that cross-lingual fairness and tokenization efficiency are not fundamentally at odds, and offer practical guidance for designing equitable multilingual models.

Abstract:
Large language models often recover the local pieces of a problem yet still fail to assemble them into a correct global solution. We study this compositionality gap in settings where the underlying structure is implicit and must be inferred from the input, using a two-pass framework that first verifies piece-level understanding and then measures composition on the same example via P(composed wrong | all pieces correct). We evaluate task formulations in five domains (contracts, math, SQL, code, and logic) and four model families; across the gold-anchored domains, larger conditional gaps are associated with settings where relevant structure is less directly exposed. A controlled intervention with correct and incorrect structural hints indicates heterogeneous bottlenecks: hints substantially reduce the gap in contracts and SQL, have no aggregate effect in math, and have no reliable effect in logic. Structure-aware prompting does not repair the gap and often harms smaller models. We conclude that composition failure is not a single phenomenon, and that diagnosing the dominant bottleneck in each task is the practical output of our framework

Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities in narrative generation. However, they often produce structurally homogenized stories, frequently following repetitive arrangements and combinations of plot events along with stereotypical resolutions. In this paper, we propose a novel theoretical framework for analysis by incorporating Proppian narratology and narrative functions. This framework is used to analyze the composition of narrative texts generated by LLMs to uncover their underlying narrative logic. Taking Chinese web literature as our research focus, we extend Propp’s narrative theory, defining 34 narrative functions suited to modern web narrative structures. We further construct a human-annotated corpus to support the analysis of narrative structures within LLM-generated text. Experiments reveal that the primary reasons for the singular narrative logic and severe homogenization in generated texts are that current LLMs are unable to correctly comprehend the meanings of narrative functions and instead adhere to rigid narrative generation paradigms.

Abstract:
Large language model (LLM) agents struggle to autonomously evolve coordination strategies in dynamic environments, largely because coarse global outcomes obscure the causal signals needed for local policy refinement. We identify this bottleneck as a multi-agent credit assignment problem, which has long been studied in classical multi-agent reinforcement learning (MARL) but remains underaddressed in LLM-based systems. Building on this observation, we propose LangMARL, a framework that brings credit assignment and policy gradient evolution from cooperative MARL into the language space. LangMARL introduces agent-level language credit assignment, pioneers gradient evolution in language space for policy improvement, and summarizes task-relevant causal relations from replayed trajectories to provide dense feedback and improve convergence under sparse rewards. Extensive experiments across diverse cooperative multi-agent tasks demonstrate improved sample efficiency, interpretability, and strong generalization.

Abstract:
Multilingual language models transfer knowledge across languages through shared subword vocabulary, a mechanism that breaks down when related languages use different writing systems. Prior work addresses this via script equalization (romanization or IPA transcription), but direct comparisons are rare; the focus has been on encoder-only models, with most work adapting existing pretrained models. We systematically compare different input representations in autoregressive multilingual pretraining, comparing orthographic text, IPA, and romanization in a controlled setup across three scales (467M, 709M, and 1.03B) on eight languages in four typologically motivated pairs. Across a wide range of downstream tasks on seen and unseen languages, romanized pretraining yields the strongest cross-lingual transfer, and the advantage over text widens with scale. IPA improves over text in most settings but trails romanization. Surprisingly, finetuning a text-pretrained model on romanized data hurts performance on languages already covered by the base model, only marginally helping when the model lacks script coverage. Our results indicate that for multilingual models spanning typologically diverse scripts, to obtain maximum benefits, romanization should be treated as a core design choice applied at pretraining rather than a post hoc fix.

Abstract:
Knowledge injection updates pretrained MLLMs with new factual or domain-specific knowledge, but fitting full authoritative answers can cause drift in non-updated behavior. Online distillation limits this drift by training on model-generated rollouts, yet uniform reference-conditioned distillation gives coarse supervision: it can under-emphasize reference-supported rollout tokens and supervise omitted facts only indirectly. We introduce RoCo-ACE, a rollout-conditioned online distillation objective for knowledge injection: RoCo uses same-rollout reference-free/reference-conditioned likelihood contrast to reallocate additional distillation weight to reference-supported rollout tokens, while ACE adds sparse reference-side anchored correction for authoritative anchors omitted from the rollout, without full-answer imitation. Across three knowledge-injection settings, six retention benchmarks, multiple baselines and base models, RoCo-ACE achieves the best injected-knowledge accuracy among compared methods while keeping evaluated retention close to the base model.

Abstract:
Nowadays, developing reliable DeepResearch-style long-form report generation remains challenging, as training and evaluation lack verifiable reward signals. Accordingly, rubric-based evaluation has become a common practice. However, existing approaches either rely on coarse, pre-defined rubrics that lack sufficient granularity or depend on manually constructed query-specific rubrics that are costly and difficult to scale. In this paper, we propose a pipeline to train preference-grounded query-specific rubric generators tailored for DeepResearch report generation. We first construct a dataset of DeepResearch-style queries annotated with human preferences over paired reports, and train rubric generators via reinforcement learning with a hybrid reward combining preference consistency, format validity, and LLM-based rubric evaluation. We evaluate the resulting rubric generators in two stages. First, on a held-out human-preference test set, the learned rubrics discriminate preferred from rejected reports more effectively than generic, prompted, or SFT-trained rubric alternatives. Second, when used as reward signals to train DeepResearch systems, our rubric generators yield substantial performance gains under both a simple single-agent ReAct framework and a complex multi-agent workflow on the DeepResearch Bench. Our code and model are available at https://huggingface.co/fdu-lcz/rubric_generator.

Abstract:
Large language models (LLMs) have made significant progress in various NLP tasks, especially with the rise of long-context models that handle extended inputs. However, the growing size of the key-value (KV) cache in Transformer architectures imposes heavy memory demands during decoding, becoming a major bottleneck. Existing sparse attention methods aim to address this issue but face two main limitations: (1) they introduce high overhead when retrieving important tokens for attention, and (2) they often fail to significantly reduce memory usage in practice. This paper presents Seer , a simple yet effective system for fast and accurate LLM decoding on long-context sequences. Seer leverages semantic space representations to manage the KV cache more efficiently. It dynamically estimates cluster-level importance to reduce both computation and memory access, while preserving output quality. Through offline training, Seer learns semantic centroids to enable fast token importance estimation. To further improve efficiency, it integrates product quantization with sparse attention for KV cache compression and uses an asymmetric distance lookup table to reduce computational cost. Evaluation on a diverse set of LLMs and tasks shows that Seer reduces theoretical attention bandwidth by 40× with minimal accuracy loss. Seer achieves a 5.57× attention speedup and a 2× End-To-End speedup over full attention at 32K sequence length. Our code is available at https://github.com/Acaciasama/Seer.

Abstract:
Retrieval-Augmented Generation (RAG) grounds large language model outputs in external evidence, but remains challenged on multi-hop question answering that requires long reasoning. Recent works scale RAG at inference time along two complementary dimensions: sequential depth for iterative refinement and parallel width for coverage expansion. However, naive scaling causes context contamination and scaling inefficiency, leading to diminishing or negative returns despite increased computation. To address these limitations, we propose SPARC-RAG, a framework that coordinates joint depth–width scaling around a unified context management mechanism—a shared global context state on which specialized control agents regulate parallel exploration and sequential stopping. To optimize scaling behavior, we further introduce a lightweight fine-tuning method with process-level verifiable preferences, which improves the efficiency of sequential scaling and effectiveness of parallel scaling. Across five single- and multi-hop QA benchmarks, SPARC-RAG outperforms the strongest reproduced baseline by an average of 3.8 F1 points at 7B and 9.8 at 32B, with the largest gains on challenging multi-hop tasks.

Abstract:
Large language models (LLMs) are increasingly evaluated for fairness, yet most existing benchmarks audit one sensitive attribute at a time. This single-attribute paradigm can miss intersectional harms, where multiple attributes interact non-additively and produce failures that are invisible under marginal evaluation. We introduce ComFair, a configurable benchmark that converts existing fairness data into 479,066 controlled Dual- and Multi-attribute instances and pairs every augmented instance with its task-equivalent source. Across Recognition, Selection, Continuation, and Conversation, this pairing supports a unified audit of performance or bias-score shifts, how often added attributes introduce new errors or mask existing ones, and model-ranking stability. Experiments on ten LLMs show task- and subgroup-dependent sensitivity: discriminative tasks expose both transition directions even when aggregate changes are modest, generative tasks exhibit larger and directionally asymmetric shifts, and rankings vary with attribute composition. ComFair therefore provides a scalable controlled sensitivity audit—rather than a model of the natural joint distribution of identities—for testing the robustness of fairness-benchmark conclusions beyond single-attribute settings.

Abstract:
Large language model (LLM)-based multi-agent systems have experienced rapid growth in recent years. Despite their promise, such systems remain fragile, frequently exhibiting reasoning and coordination errors that can lead to system-level failures. Failure attribution in such systems relies on tracing natural language interactions among agents to identify the decisive error, which refers to the earliest action whose correction can reverse system failure. There are two key challenges: 1) Shallow attribution: Existing methods often capture only minor deviations, such as incomplete retrievals or formatting errors, which verification mechanisms can correct, while missing the decisive cause of system failure. 2) Contextual degradation: As the length of the system traces increases, the model’s reasoning ability rapidly deteriorates. To address these challenges, we propose DCFA, a training-free framework for failure attribution. DCFA integrates a global module that constructs structured causal-inspired dependency graphs from system traces to identify the initial decisive error, and a local module that applies local counterfactual-inspired reasoning to refine causal-inspired attribution. Experiments on the Who&When benchmark across six LLMs show that DCFA improves step-level accuracy by up to 8.27% over state-of-the-art baselines.

Abstract:
LLM agents are increasingly expected to operate across heterogeneous task regimes that require distinct execution paradigms. This challenges fixed agent systems and motivates system-level meta-adaptation beyond isolated component updates. While existing works have adapted external harness or trained underlying reasoning policies, full-system adaptation remains insufficiently characterized. The adaptation space between structure and execution is rarely made explicit, and the compatibility between the external harness and the internal reasoner is not optimized jointly. We propose HarnessForge, a meta-adaptive framework for evolving LLM agent systems. HarnessForge formulates an agent system as a harness–policy pair, defining a stable adaptation space that separates harness-level execution structure from policy-level reasoning behavior. It then performs harness–policy co-evolution through fault-guided harness tailoring and harness-conditioned policy alignment. Experiments across five benchmarks from diverse domains show that HarnessForge consistently improves both Qwen3-4B and Qwen3-8B backbones, outperforming harness-only and policy-only baselines with gains of up to 12.0% over the strongest baseline and achieving favorable rollout-efficiency tradeoffs, demonstrating that harness–policy co-evolution is effective, and that executable compatibility between the harness and reasoning policy is essential for agent-system adaptation.

Abstract:
We present BLUE, a minimal method for better language use in vision-language-action (VLA) models for autonomous driving (AD). Through extensive analysis, we reveal that language matters on only a small fraction of routes, but on those routes it can greatly improve or degrade performance. Generating language at every frame is therefore inefficient, since most computation is spent on frames that do not benefit from language. We further show that pretrained VLA hidden states potentially already encode whether language will benefit a given frame, even though scene complexity and kinematic features alone struggle to predict this. Based on this finding, BLUE trains a lightweight gate on frozen VLA hidden states to decide per frame whether to activate language generation or predict actions directly, without modifying the backbone or requiring additional human annotation. With just a 0.11M-parameter gate, BLUE sets a new state of the art on both benchmarks, achieving 76.2% success rate on Bench2Drive and 36 driving score on Longest6 v2, while delivering 2.54 × inference speedup and 8.9% success rate improvement over the backbone. BLUE provides a practical path toward efficient language-augmented AD, showing that VLA models can retain the benefits of language at a fraction of the cost. Our code, data, logs and checkpoints will be fully released.

Abstract:
Graph-based RAG (GraphRAG) leverages entity–relation graphs to support multi-hop reasoning, but most systems still rely on static retrieval. When crucial evidence, especially bridge documents that connect disjoint entities, is absent, reasoning collapses and hallucinations persist. Iterative retrieval, which performs multiple rounds of evidence selection, has emerged as a promising alternative, yet its role within GraphRAG remains poorly understood. We present the first systematic study of iterative retrieval in GraphRAG by analyzing how different strategies interact with graph-based backbones and investigating when they are effective. Our findings reveal both opportunities and pitfalls which together highlight a central bottleneck, namely that GraphRAG’s effectiveness depends not only on recall but also on whether bridge evidence is consistently promoted into leading positions where it can support reasoning chains. To address this limitation, we propose Bridge-Guided Dual-Thought-based Retrieval (BDTR) , a simple yet effective framework that generates complementary thoughts and leverages reasoning chains to recalibrate rankings and bring bridge evidence into leading positions. BDTR achieves consistent improvements across diverse GraphRAG settings and provides guidance for the design of future GraphRAG systems. Our code is available at https://github.com/KaiGuo20/BDTR .

Abstract:
Idioms pose a fundamental challenge for language models, as their meaning cannot be inferred from surface form alone. Understanding such expressions, therefore, requires semantic abstraction beyond lexical overlap. We introduce IdioLink, a retrieval benchmark designed to test whether models can link idiomatic expressions to conceptually equivalent meanings expressed in literal or paraphrased forms. IdioLink comprises 10,700 documents and 2,140 queries, spanning 107 idioms with both literal and figurative uses. Each document and query is annotated with spans that convey the core meaning. Evaluating strong embedding baselines (e.g., BGE, E5, Contriever, and Qwen), we show that current models struggle to retrieve equivalent meanings across divergent surface realizations, relying instead on topical and shallow semantic cues. IdioLink exposes key gaps in idiom-aware semantic retrieval and provides a challenging testbed for future models.

Abstract:
Generative retrieval replaces item IDs with Semantic IDs (SIDs) decoded autoregressively by a sequence model. Playing the role of tokenizer, the codebook largely determines the quality of generative retrieval. Existing codebook metrics, however, only characterize the global uniformity of code usage and do not reveal whether an individual SID correctly represents its item or separates it from others. We address this problem with two metrics and a matching training objective. We build a query-centered triplet dataset q, a, p, n from large-scale user behavior, and based on which we define Overlap, a prefix-based distance between two SIDs, and Pairwise Overlap Ranking (POR), the fraction of triplets where a codebook places the positive closer to the anchor than the hard negative. Both metrics correlate strongly with downstream Hit Rate in generative retrieval. We then introduce the Differentiable Soft Overlap Loss (DSOL) to make these two metrics differentiable during codebook learning and optimize them directly. On a public e-commerce benchmark (ESCI-us) and a large-scale industrial video search dataset, DSOL improves HR and POR over standard SID baselines.

Abstract:
Learning rich medical concept representations is essential for EHR prediction. Text-attributed knowledge graphs (TKGs) provide a natural foundation by organizing heterogeneous medical relations together with textual semantics. However, most existing encoders process concepts uniformly across patients, despite the fact that a code’s meaning and predictive value depend on patient-specific clinical context and trajectory. Learning patient-personalized concept representations from TKGs introduces two key challenges: (1) deciding how much KG context to incorporate for each observed code, and (2) aligning semantic information with the patient-specific relational structure. We propose REFINE, a KG-aware budgeted LLM graph refinement framework for patient-personalized medical concept encoding. Starting from a global TKG, REFINE constructs patient-specific temporal graphs. A sequential reinforcement learning policy selects a personalized KG expansion budget for each observed code. The resulting patient graph is processed by a heterogeneous GNN to capture relation-aware structural dependencies, while a frozen LLM uses graph-aware soft prompts to semantically refine concept representations. Experiments on MIMIC-III and MIMIC-IV show that REFINE consistently improves diverse EHR backbones, outperforms strong baselines, and demonstrates robust gains across component ablation, KG selection, and data insufficiency.

Abstract:
Generative Retrieval (GR) aims to directly generate document identifiers (DocIDs) without external indices. We identify granularity collapse: during training, GR models develop a preference for generating DocIDs with a specific granularity, causing retrieval failures when it differs from that of the target document’s DocID. We propose to quantify this mismatch using "isotropy" and confirm that retrieval accuracy degrades as such granularity mismatch increases. To address this, we propose Granularity Routing for Generative Retrieval ( GR 2 ), which trains multiple expert models specialized for different granularity levels and routes each query to the appropriate expert. Experiments on NQ320k demonstrate that improves retrieval performance, with larger gains on queries that suffer more from granularity collapse.

Abstract:
Retrieving code functions, classes or files relevant to a user query, bug report or feature request from large codebases is a fundamental challenge for Large Language Model (LLM)-based coding agents. Agentic approaches typically employ sparse methods like BM25 or dense embedding strategies to identify semantically relevant units. While dense embedding approaches can outperform BM25 by large margins, both ignore the graph-structured characteristics of the codebase. To address this, we propose SpIDER (Spatially Informed Dense Embedding Retrieval), a dense retrieval approach that integrates LLM-based reasoning with graph-based exploration of the codebase. We further introduce SpIDER-Bench, a graph-structured benchmark curated from SWEPolyBench, SWEBench-Verified and Multi-SWE-bench, spanning Python, Java, JavaScript and TypeScript repositories. SpIDER’s graph-based candidate expansion attaches a structural reason for inclusion to each surfaced function (its seed and the edge type linking them), making the candidate set auditable while keeping the retrieval budget fixed. The graph is built from per-repository syntax trees, so it can be constructed on-demand at the start of a developer session rather than precomputed offline. Empirical results show that SpIDER consistently improves dense retrieval Recall@20 across every language and benchmark in SpIDER-Bench: by at least 13% relative ( +0.05 to +0.12 absolute) along containment edges, rising to at least 27% relative ( +0.11 to +0.19 absolute) once call edges are explored.

Abstract:
Scientific compound figures combine multiple labeled panels into a single image, and downstream pretraining and retrieval require panel-aligned visual-text pairs. However, in a PubMed Central (PMC)-scale crawl of 346,567 compound figures, 16.3% have no caption and are discarded by existing caption-decomposition pipelines. We propose FigEx2, a visual-conditioned framework that takes only a compound figure as input and jointly produces labeled panel boxes and panel-wise captions. FigEx2 introduces an Entity-Attention Kullback-Leibler (KL) regularizer that aligns the detector’s cross-attention with scientific entities annotated for each panel, providing a stable conditioning signal that also improves localization, and applies Group Relative Policy Optimization (GRPO) with a panel-level Entity-F1 reward to optimize scientific faithfulness. We curate BioSci-Fig-Cap for in-domain supervision and contribute physics and chemistry test suites for cross-disciplinary evaluation. FigEx2 achieves 0.751 mAP@0.5:0.95 on BioSci-Fig-Cap, and outperforms Qwen3-VL-8B by 6.80 Entity-F1 on MedICaT for captioning. It also transfers zero-shot to out-of-distribution domains.

Abstract:
Current evaluation metrics for automated audio captioning (AAC) rely on reference-based comparison, implicitly assuming that a limited reference set adequately represents valid descriptions. However, the inherent cross-modal gap between audio and language gives rise to a many-to-many correspondence: a single audio event can be accurately described by numerous semantically equivalent yet lexically diverse captions. We first demonstrate the resulting instability through ground-truth intra-evaluation on Clotho and AudioCaps, evaluating each human caption against the remaining four references using a leave-one-out protocol. To provide a controlled demonstration, we use a CVAE-based AAC model that generates diverse candidates replicating the ground-truth captions, showing that n-gram metrics reward surface-level replication. We then conduct non-word replacement, synonym replacement, and adjacent-token swapping experiments on both datasets. Overlap metrics penalize synonym replacements almost as strongly as non-word corruption, whereas embedding and CLAP-based metrics respond weakly to local word-order changes. Additional evaluation of CAF-Score and FLEUR (AF3) shows partial progress, but neither configuration clearly separates synonym replacement from adjacent swapping. No evaluated metric jointly provides acoustic grounding, tolerance to valid lexical variation, and local word-order sensitivity. We therefore contribute a reusable AAC metric diagnostic protocol for exposing these trade-offs and guiding future metric development.

Abstract:
We propose CAPE, a Choice-Distribution Alignment and Performance Evaluation framework for diagnosing human–LLM response alignment using population-scale human choice distributions. CAPE separates outcome-level difficulty alignment from response-distribution alignment with four metrics: OA, MAE, PA, and DA. We apply CAPE to 609 Korean CSAT items with item-level response distributions from approximately 300K examinees per item, and evaluate 16 LLMs. Results show that model capability is the strongest predictor of alignment, with accuracy closely tied to both full-choice and distractor-only alignment. However, group-level scores are affected by aggregation size and model composition, and neither reasoning augmentation nor Korean adaptation yields consistent residual advantages. Bloom-level analyses further suggest that capability-linked alignment varies across item types. CAPE thus enables evaluation beyond accuracy by diagnosing how model success and failure patterns relate to human population responses.

Abstract:
Skill learning enables web agents to reuse prior experience for long-horizon interactive tasks. However, learned skills are typically evaluated in closed environments, overlooking the fact that real-world web deployment involves continuous shifts in invocation context, perceptual grounding, and execution feasibility. To this end, we introduce SkillShift-Bench, the first benchmark for evaluating skill learning with controlled open-environment shifts. SkillShift-Bench simulates Contextual Shift, Perceptual Shift, and Execution Shift across three self-hosted web systems, and evaluates agents with diagnostics covering task success, interaction cost, skill use, and skill failure. Comprehensive experiments reveal systematic robustness degradation under controlled shifts, characterized by lower success rates, higher interaction costs, and more frequent skill failures, with Execution Shift exerting the strongest pressure. We further formulate shifted skill execution as a nonstationary local executability problem and provide both a dynamic-regret analysis and a practical test-time adaptation method, demonstrating the feasibility of improving learned-skill execution without relearning the skill library. Code, tasks, environments, documentation, and the leaderboard are available at https://skillshift-bench.github.io/.

Abstract:
We apply survival analysis to study how the probability of reasoning error evolves across steps in LLM chain-of-thought inference. Analyzing 18,969 chains across five benchmarks and seven models (3B–70B) under greedy decoding, we discover a near-universal aging effect—the conditional probability of reasoning error increasing with step depth—as measured by the Weibull shape parameter 𝜌 > 1 (indicating increasing hazard) in 34 of 35 dataset–model cells (binomial p = 10 -9 ), with task-dependent magnitude ranging from ̄𝜌 = 1.35 (multi-domain QA) to ̄𝜌 = 2.36 (logical deduction). This finding is robust to informative censoring (IPCW adjustment: 34/35 cells retain 𝜌 > 1 ) and unobserved problem-level heterogeneity (shared frailty 𝜃 ≈ 0 : 34/35 cells retain 𝜌 > 1 ). Compositional fan-in—the number of prior-step outputs consumed per step—shows a large but statistically suggestive rank correlation with aging magnitude (Spearman 𝜌 s = 0.90 , exact permutation p = 0.083 , N = 5 ). We show that a survivorship bias masks these dynamics when raw error rates are used instead of the conditional hazard function. Held-out verification experiments confirm that late-weighted allocation (motivated by the aging finding) catches 94% of errors vs. 23% uniform on the strongest-aging task at budget K=1 , while hazard-proportional allocation outperforms both baselines on moderately aging tasks (GSM8K, MathQA) at all budget levels. Our framework requires no training and provides actionable diagnostics for verification allocation.

Abstract:
Large Vision-Language Models (VLMs) are susceptible to jailbreak attacks: researchers have developed various attack strategies that bypass the safety mechanisms of VLMs. Among these approaches, jailbreak methods based on the Out-of-Distribution (OOD) strategy have garnered widespread attention due to their simplicity and effectiveness. This paper further advances the understanding of OOD-based VLM jailbreak methods. We show that mild OOD manipulations can achieve stronger jailbreak performance than both clean inputs and overly strong perturbations, a non-monotonic pattern we define as "weak-OOD". We explain this phenomenon through a trade-off between two dominant factors: input intent perception and model refusal triggering. Our evidence suggests that these two factors respond differently to OOD manipulations, which is consistent with a discrepancy between broad pre-training robustness and narrower safety alignment. Building on this insight, we draw inspiration from optical character recognition (OCR) capability enhancement, a core task in the pre-training phase of mainstream VLMs. Leveraging this capability, we design JOCR (Jailbreak via OCR-Aware Embedded Text Perturbation), a simple VLM jailbreak method that achieves the best average ASR among evaluated baselines.

Abstract:
Large language models can produce fluent yet factually incorrect responses, limiting their reliability in knowledge-intensive applications. Recent decoding methods seek to mitigate such hallucinations by contrasting or amplifying signals from intermediate layers. However, not every layerwise discrepancy provides a reliable direction for factual decoding. We propose a unified game-theoretic decoding framework for large language models: intermediate-layer signals act as competing evidence, while the final layer retains decision authority under explicit constraints. Rather than applying a fixed layerwise contrast, our framework treats changes in intermediate predictions as evidence of the language model’s evolving internal judgment, and uses this evidence to guide decoding under final-layer constraints. Experiments with three backbone language models on multiple truthfulness and hallucination benchmarks demonstrate the effectiveness of our framework in improving factual reliability.

Abstract:
Open-world geo-localization requires models to reason over ambiguous visual cues through multi-step reasoning and external knowledge grounding. While recent large vision-language models exhibit strong multimodal reasoning capabilities, existing approaches still suffer from perceptual hallucination and context drift due to the lack of explicit evidence-grounded verification. In this work, we reformulate geo-localization as a human-like perceive-then-verify reasoning problem and propose GeoPAVE (Geo-localization Perception-and-Verification-Engine), a bi-level agentic framework that contains perception-based hypothesis generation via single-pass rollouts and verification-based evidence grounding for decision actions: support, refute, and refine. To support rigorous evaluation, we further introduce PAVED, a novel dataset derived from real-world user check-in data, equipped with comprehensive reasoning trajectories featuring multi-hop queries, multi-round tool invocations, and structured perception-verification traces. The dataset and code are available at https://github.com/Arandinglv/GeoPAVE.

Abstract:
Multimodal contrastive models are widely used for bidirectional image-text retrieval, but their rankings remain vulnerable to attraction bias, where certain candidates become hubs for many unrelated queries. Existing training-free normalization methods suppress such hubs, but may also penalize candidates that are genuinely relevant. We propose GLOBE, a training-free plug-in module that combines global-local bias calibration with graph-based semantic evidence to suppress unsupported hubs while preserving true matches. Across MS-COCO and Flickr30K, GLOBE improves diverse frozen retrieval backbones, achieving up to 7.9% absolute Recall@1 gain over the original model and up to 5.2% over strong training-free baselines. We further construct IRB-COCO-500, a mixed real-and-synthetic benchmark with AI-generated near-positive distractors, and show that GLOBE improves retrieval robustness in this challenging setting.

Abstract:
Humor is one of the few cognitive tasks where getting the reasoning right matters as much as getting the answer right. While recent work evaluates humor understanding on benchmarks such as the New Yorker Cartoon Caption Contest (NYCC), it largely treats it as black-box prediction, overlooking the structured reasoning processes underlying humor comprehension. We introduce IRS (Incongruity–Resolution Supervision), a framework that decomposes humor understanding into three components: Incongruity Modeling, which identifies mismatches in the visual scene; Resolution Modeling, which constructs coherent reinterpretations of these mismatches; and Preference Alignment, which evaluates candidate interpretations under human judgments. Grounded in incongruity-resolution theory and expert captionist practice, IRS supervises intermediate reasoning process through structured traces that make the path from visual perception to humorous interpretation explicit and learnable. Across 7B, 32B, and 72B models on NYCC, IRS improves performance across caption matching and ranking, with IRS-72B achieving the strongest model performance on ranking (76.10%), surpassing both non-expert human performance and all evaluated open- and closed-source multimodal baselines. Zero-shot transfer further shows that IRS learns generalizable reasoning patterns.

Abstract:
Sign Language Translation (SLT) is a challenging cross-modal task requiring joint modeling of manual articulations and non-manual signals. Existing gloss-free SLT methods effectively capture gestural dynamics but often underutilize facial expressions, which play crucial grammatical and disambiguating roles. This limitation can cause semantic degradation when distinct concepts share similar manual configurations. To address this issue, we propose FEA-SLT (Facial-Expression-Aware Sign Language Translation), a gloss-free end-to-end framework that uses facial dynamics to provide complementary cues to manual signals. FEA-SLT employs a domain-transferred facial encoder to extract expression-sensitive representations and integrates them with manual features through a linguistically motivated Facial-Expression-Aware Fusion (FEAF) module. FEAF captures reciprocal dependencies between manual and facial channels via bidirectional modulation, enhancing syntactic fidelity. Experiments on PHOENIX14T and CSL-Daily show that FEA-SLT achieves state-of-the-art BLEU performance among gloss-free methods, while targeted analyses support improved translation of facial-sensitive utterances. Code is available at https://github.com/TuGuobin/FEA-SLT.

Abstract:
We present UNISON, a latent flow-matching framework that unifies speech generation, sound generation, and audio editing within a single model. A single model handles text-to-audio, text-to-speech, zero-shot speaker cloning, mixed speech-and-sound generation, scene-level audio editing, speech-in-scene editing, and timed audio composition, all of which share a single set of weights. Our architecture features two core designs: (1) Layer-wise deep LLM fusion, which injects hidden states from uniformly sampled layers of a frozen MLLM into corresponding MM-DiT blocks via learned projections, providing depth-varying semantic conditioning. This configuration achieves better T2A FD and CLAP than a persistent penultimate-layer conditioning baseline; and (2) a unified multi-task architecture where task identity is encoded solely by a channel-wise mask and source audio is provided through VAE-encoded channel concatenation. Training is stabilized by an online GPU-side multi-task data synthesis pipeline with task-homogeneous batching and a two-stage curriculum. With 621M–732M trainable generative parameters, UNISON achieves competitive results across the evaluated domains. Its trainable generative backbone uses 1.8–4.9x fewer parameters than the closest unified systems considered, while deployment additionally requires a frozen 7B text encoder. Audio samples and source code are publicly available.

Abstract:
Large language models and coding agents can produce web applications that appear functional, yet surface plausibility is an unreliable proxy for behavioral conformance under interaction. We introduce ConformWeb, a contract-grounded protocol and benchmark that operationalizes public behavioral contracts to construct hidden interaction scenarios, evaluating whether agent-generated applications preserve strict conformance under dynamic execution. By decoupling hidden interaction sequences from the public behavioral scope, the protocol guarantees a fair held-out workflow evaluation without relying on undisclosed reference implementations. Evaluated across 24 validated instances scaling to 756 evaluation scenarios and 6,433 semantic steps, only 252 of the 2,160 total execution runs achieve full conformance, yielding a strict pass rate of 11.7%. While scenario-level conformance reaches 36.7% and step-level progress averages 49.4%, this quantitative gap demonstrates that generative systems frequently achieve substantial partial progress but fail to maintain the critical behavioral invariants required for end-to-end conformance. All code and data are released at https://github.com/deep-diver/ConformWeb.

Abstract:
Continual LLM attribution aims to identify the source of generated text amid the ongoing emergence of new models. While existing training-free, prototype-based methods excel at this task, they rely solely on the final-layer features of a frozen extractor, overlooking rich hierarchical stylistic cues. Moreover, naive multi-layer fusion yields suboptimal results. To address these limitations, we propose ReProF (Residual Prototype Fusion), a training-free framework that partitions the feature extractor’s hierarchy into layer blocks and derives depth-aware analytic prototypes via shallow-to-deep residual fitting: deeper prototypes model the residuals of shallower ones. By fusing predictions across these blocks, ReProF captures multi-granular stylistic cues without requiring parameter tuning or data rehearsal, thereby intrinsically eliminating catastrophic forgetting. Extensive experiments across 19 LLMs and 8 domains demonstrate that ReProF establishes a new state-of-the-art, consistently outperforming existing approaches by a wide margin. Code is available at: https://github.com/wyj-hb/ReProF.

Abstract:
Quantization is widely used to reduce the inference cost of large language models, but its effect on reasoning models is not fully captured by final-answer accuracy or per-token latency. We show that low-bit post-training quantization can introduce a hidden test-time compute cost: quantized reasoning models often generate longer chains of thought even when they still answer correctly. Across mathematical reasoning, code generation, scientific question answering, and agentic tool-use benchmarks, we find that INT4/INT3 quantization can preserve accuracy but increase reasoning-token usage, offsetting the expected per-token speedup. To measure this effect, we introduce the CoT Token Inflation Ratio, which compares reasoning length between quantized and full-precision models averaged across all evaluation benchmarks. We further show that token inflation is accompanied by behavioral changes in the reasoning trace, including more intermediate steps and greater semantic repetition. These changes translate into measurable end-to-end real-world serving penalties. Finally, we evaluate mitigation strategies and find that prompting and decoding-time sampling offer inconsistent accuracy-length trade-offs, while quantization-aware training shows more promise in reducing both accuracy degradation and token inflation. Our results suggest that reasoning-token usage should be reported alongside accuracy when evaluating quantized reasoning models.

Abstract:
Chain-of-thought (CoT) reasoning improves large reasoning models (LRMs) on complex tasks but often produces long, redundant traces. Recent training-free early-exit methods shorten these traces by choosing an intermediate point to stop reasoning. We study one such strategy that injects an end-of-think token (EoT, ) at this point to trigger the reasoning-to-answering transition, and find that the injected EoT does not always induce a clean answering phase. Answering-phase generation can continue before the model regenerates another EoT, with the span preceding this regenerated EoT scaling with the reasoning tokens saved by early exit and exhibiting continued reasoning behavior. We call this spurious CoT termination, where reasoning-like generation continues into the answering phase. We hypothesize that insufficient attention to the injected EoT contributes to spurious CoT termination and probe this hypothesis with Exit-token Attention Biasing (EAB). Across four LRMs, five benchmarks, and two early-exit methods, increasing attention to the injected EoT reduces spurious CoT termination and answering-phase length. These results reveal a limitation of controlling LRMs by externally matching their explicit think-block format. Inserting the EoT token conforms to this format but does not by itself guarantee the intended reasoning-to-answering transition. Our code is available at https://github.com/Seunghee-Koh/Spurious-CoT-Termination.

Abstract:
Automating repository-level software engineering tasks is a foundational challenge for autonomous code agents, largely due to the difficulty of configuring executable environments. However, manual configuration remains a labor-intensive bottleneck, necessitating a transition toward fully automated environment configuration. Existing approaches often rely on pre-defined artifacts or are restricted to specific programming languages, limiting their applicability to diverse real-world repositories. In this paper, we first propose RAT (RunAnyThing), a modular and extensible agent framework for fully automated configuration across programming languages on arbitrary repositories. RAT adopts a multi-stage pipeline that integrates language-aware abstraction, image initialization, specialized configuration toolset, and robust sandbox. Furthermore, to enable rigorous evaluation, we propose RATBench, a benchmark reflects the comprehensive coverage of real-world repositories. Extensive experiments demonstrate that RAT achieves state-of-the-art performance, improving Environment Setup Success Rate (ESSR) by an average of 36.1% over strong baselines.

Abstract:
The agent–tool interaction loop is a critical attack surface for modern Large Language Model (LLM) agents. Existing denial-of-service (DoS) attacks typically function at the user-prompt or retrieval-augmented generation (RAG) context layer and are inherently single-turn in nature. This limitation restricts cost amplification and diminishes stealth in goal-oriented workflows. To address these issues, we propose a stealthy, multi-turn economic DoS attack at the tool layer under the Model Context Protocol (MCP). By editing only text-visible fields and implementing a stateful, completeness-gated return policy, our malicious server preserves tool identity and the terminal benign payload while steering agents into prolonged, verbose tool-calling chains. We optimize these text-only edits with Monte Carlo Tree Search (MCTS) to maximize cost under a task-success constraint. Across six LLMs on the ToolBench and BFCL benchmarks, our attack yields trajectories over 60K tokens, increases per-query cost by up to 658×, raises energy consumption by 100–560×, and pushes GPU key-value (KV) cache occupancy to 35–74%. Representative prompt filters and output- or trajectory-level monitors seldom detect these attacks, highlighting the need for defenses that safeguard agentic processes rather than focusing solely on final outcomes. We will release the code soon.

Abstract:
While Audio Large Language Models (Audio LLMs) excel at multimodal understanding, they suffer from text dominance, a bias where models favor text over acoustic evidence, potentially leading to hallucinated responses. However, the internal mechanisms underlying how these models behave when audio and textual inputs contradict each other remain unexplored. In this work, we present the first mechanistic analysis of this phenomenon by tracing the propagation of internal representations across layers. Our investigation reveals three key findings: (i) text dominance is consistently observed across models; (ii) while text and audio rely on functionally distinct pathways, they ultimately converge into a shared semantic space in late layers; and (iii) the text pathway does not erase audio information, but rather actively suppresses intact audio representations. Building on these insights, we leverage back-patching, a training-free intervention that routes late-layer audio activations back into earlier layers. This amplifies the audio representations, enabling them to overcome textual suppression. Our evaluation shows that back-patching consistently reduces text dominance, demonstrating a mechanistic route to mitigating text dominance under conflict.

Abstract:
Retrieval-Augmented Generation (RAG) has advanced open-domain question answering by incorporating external information into model reasoning. However, raw retrieved content often suffers from low signal-to-noise ratio, where answer-supportive information is diluted by irrelevant material, and error accumulation, where incomplete or misleading content degrades multi-hop reasoning. To address these challenges, we propose EviNoteRAG, a framework that enhances RAG through quality-augmented evidence notes. The model is trained to generate Supportive-Evidence Notes (SENs), concise abstractions that preserve only answer-critical information and explicitly highlight key and uncertain content to improve reasoning accuracy. To further ensure note quality, we introduce an entailment-based Evidence Quality Reward (EQR), which incentivizes SENs to be logically sufficient for deriving the correct answer. Experiments on both in-domain and out-of-domain benchmarks demonstrate that EviNoteRAG achieves state-of-the-art performance, with relative F1 gains of 20% on HotpotQA (+0.093), 40% on Bamboogle (+0.151), and 91% on 2Wiki (+0.256), while also improving training stability and efficiency.

Abstract:
Retrieval-augmented generation over knowledge graphs (KGs) can improve the factual grounding of large language models, but efficient retrieval remains challenging when the underlying graph is large and multi-hop evidence is distributed across different regions. We present SPLIT-RAG, an efficiency-oriented multi-agent KG-RAG framework that organises a KG into reusable, question-aligned partitions using reasoning patterns observed in training questions. At inference time, SPLIT-RAG forms a query-specific retrieval plan, activates only a small set of relevant partition-specialised agents, and aggregates their retrieved evidence before final answer generation. To improve reliability when evidence from different agents is combined, the framework applies compatibility-based filtering over retrieved triples prior to synthesis. Experiments on WebQSP, CWQ, and MetaQA show that SPLIT-RAG achieves strong performance against recent KG+LLM baselines while reducing end-to-end latency under matched budgets. Component ablations and model-tier analyses further show that selective multi-agent retrieval is central to the performance gain, and that lightweight retrieval agents paired with a stronger head agent provide an effective accuracy–latency trade-off.

Abstract:
Evaluating Large Language Models (LLMs) on code review requires moving beyond local syntax to assess system-wide semantic reasoning. We investigate whether LLMs can detect semantic data-layer issues: latent bugs requiring cross-endpoint reasoning over persistent state. We introduce a controlled benchmark of 216 minimal database-backed tasks, derived from 18 independent hand-written scenarios, to evaluate five frontier and open-weights models under two settings: open-ended review versus targeted detection. Our evaluation reveals a stark discovery gap: while models reliably recognize explicitly described issues, they struggle to discover them unprompted, with average F1 dropping from 0.91 to 0.85 for the easiest category and from 0.73 to 0.03 for the hardest. Furthermore, models exhibit a hierarchy of detectability across issue types, and for retention issues, a compliance bias where explicit definitions lead to over-flagging of structurally similar but semantically correct code. Our findings demonstrate that targeted defect detection substantially overestimates true open-ended review capabilities, underscoring the need to evaluate LLMs on latent intent recovery.

Abstract:
Multi-agent debate among large language models (LLMs) is widely assumed to improve reasoning by letting models correct one another. We audit that assumption for English medical question answering (QA). Five open-weight 4 to 8B medical and generalist LLMs answer 8 , 402 questions across six benchmarks as a council, measured against the same five prompted individually and a single larger Qwen3-VL-32B, text-only. The council beats every member on five of six datasets, but most of that accuracy is in place before debate runs: 67% of verdicts settle at Round 0, and on five of six those are the more accurate questions. Where debate runs, opinion change concentrates on one anchor; on the hardest benchmark 85% of moves toward it happen while it is wrong. There, agreement is worth less than disagreement: questions all five agree on are correct 6.6% of the time against 14.6% otherwise, tracking a shared bias toward the first option. The council also trails the larger model on four of six by up to 16.2 points, at seven times the model calls for 1.6 times the compute. Agreement among models is a measurement that can be biased, not evidence of correctness.

Abstract:
Cuneiform recorded typologically unrelated languages for more than three millennia, raising a natural question: which of its statistical regularities are properties of the shared writing system, and which belong to individual languages? We answer this with two probe pairs targeting the same script-versus-language decomposition. The first compares two unsupervised, dictionary-free segmentation methods: transitional probability (TP), grounded in Saffran et al. (1996), and Morfessor (Creutz and Lagus, 2007). To our knowledge, this is the first systematic evaluation of dictionary-free segmentation on cuneiform; the only prior published baseline (Homburg and Chiarcos, 2016) relies on a hand-built dictionary (MaxMatch, F1 = 0.74). Both our methods reach F1 > 0.97 within language (Morfessor reaches F1 > 0.99 on Akkadian and Sumerian), exceeding MaxMatch by more than 23 F1 points, but diverge sharply in cross-language transfer: TP holds F1 > 0.95 across all six language pairs, while Morfessor collapses to F1 = 0.27 from the 13K-token Elamite source. A learning curve and a Hittite stress test (34.7% foreign-script logograms, where TP drops 35 points and Morfessor only 8) further isolate the two underlying mechanisms. The second probe pair extends the same decomposition to representations: concatenating Unicode and Latin features improves classification in 11 of 12 morphosyntactic tasks ( p < 0.01 ). We release cunei-tools and a three-language benchmark.

Abstract:
Modern coding agents are increasingly deployed for realistic software engineering, yet existing benchmarks primarily evaluate static, fixed-target tasks, failing to capture the long-horizon, multi-turn nature of real-world development. We introduce MACBench, a novel benchmark evaluating agents on sustained frontend web application development where text-based user requirements continuously evolve within a persistent workspace. Comprising 28 repository instances across 8 domains, MACBench features 699 dialogue turns and 956 evaluation items. It explicitly models realistic interaction patterns, challenging agents to maintain cross-turn coherence and continuous executability. Our dual-branch evaluation framework assesses both process robustness and final artifact quality, uniquely integrating intermediate viability checks with bounded runtime-feedback repair. Benchmarking 4 contemporary models reveals a severe reliability gap: the best-performing agent achieves only a 70.6% checklist success rate and a 10.7% strict instance success rate, struggling to satisfy requirements accumulated over long sessions. Furthermore, ablation studies reveal diminishing and sometimes negative returns from runtime-feedback repair: while it improves checklist success by an average of 10.4% and aids immediate failure recovery, excessive repair attempts frequently degrade final functional outcomes, suggesting they may introduce regressions or side effects in the evolving codebase. Finally, failure analysis identifies complex cross-turn application logic as the dominant bottleneck, accounting for 33–50% of unresolved errors.

Abstract:
Language identification is a critical preprocessing step for curating multilingual corpora. While current approaches treat language varieties as clearly bounded categories, such discrete labels obscure the continuous nature of linguistic variation in practice. We propose reframing non-standard variety identification as a geographically-grounded task, representing varieties as geolects, that is, linguistic data anchored to geographic coordinates. Rather than classifying texts into predefined labels, we place them on a geodesic grid, where predictions are points on the earth’s surface and variety distinctions emerge from spatial structure. We introduce GeoVar, a fine-grained dataset of 517,985 samples spanning 929 geolects across 20 languages and seven dialect continua in Europe and the Middle East, and develop a grid-based framework with three geolect-to-cell mapping strategies of increasing spatial smoothness: centroid, uniform, and Gaussian. We compare various geolocation methods, with the best model reaching 0.97 accuracy on compact continua such as Basque and Caspian. Accuracy decreases with the geographic dispersion of varieties, with North Germanic being the hardest case. Our model handles non-standard variety text more reliably than current LID tools, while additionally operating at sub-language granularity.

Abstract:
Recent MLLMs have shown promise in scene-text-aware video understanding, yet their ability to understand scene text from the wearer’s perspective in AI-glasses videos remains underexplored. AI-glasses videos are passively captured without a camera-frame preview, limiting the wearer’s ability to adjust framing during recording. As a result, relevant text may be unstable, and the recorded view may not fully reflect the wearer’s attention. The challenge is thus not merely to read visible text, but to identify and read the right text given the wearer’s position, viewpoint trajectory, and goal. To systematically evaluate this capability, we introduce WearerText, a benchmark for Wearer-Centered Scene Text Understanding, comprising 1,101 real-world AI-glasses videos and 15,336 QA pairs spanning 13 tasks. These tasks are organized into a three-level diagnostic hierarchy: L1 Text Perception, L2 Contextual Understanding, and L3 Wearer-Grounded Reasoning, targeting unstable text perception, wearer-view alignment, and goal-oriented text reasoning, respectively. To ensure reliable construction and evaluation, we develop MAH-V, a multi-agent hybrid verification pipeline for QA construction, and propose VerEval, a Verified-evidence metric for open-ended Evaluation. Benchmarking 20 recent MLLMs shows that current models still struggle with wearer-centered scene text understanding, particularly on tasks requiring wearer-view alignment and goal-oriented text reasoning.

Abstract:
Large language models can reproduce memorized text verbatim, yet copyright defenses are usually evaluated under incompatible protocols. We introduce CopyShield, a controlled benchmark comparing three representative defenses at distinct intervention levels: contrastive decoding (output), Direct Preference Optimization (behavioral), and activation intervention (representation). We evaluate CopyShield on two model families, LLaMA-3.1-8B and Mistral-7B-v0.3, using controlled memorization over five public-domain books and a shared protocol measuring literal leakage, calibrated non-literal leakage, utility, and degeneracy. Across these methods, intervention level is associated with distinct compliance–utility trade-offs. On LLaMA-3.1-8B, contrastive decoding remains near-degeneracy-free (0–2%) but reaches a literal-suppression floor at NV-Recall 0.192–0.203. DPO nearly eliminates literal leakage (0.263 to 0.002) but induces paraphrase-loop degeneracy in 58% of QA outputs, with no utility gain over the SFT baseline. Activation intervention attains the lowest non-literal flagging rate (1/200) by blocking 84% of non-literal queries before generation. Human evaluation confirms that DPO has low coherence, whereas activation lowers perceived copyright risk through broad refusal. On Mistral-7B-v0.3, the output- and representation-level patterns persist, while DPO degeneracy falls to 10–14%, showing that its severity is model-dependent. Together, CopyShield provides cross-level reference baselines and identifies targeted non-literal suppression as an open challenge. The code is available at https://github.com/spotai-mbzuai/CopyShield.git.

Abstract:
Large language models (LLMs) with extended context lengths rely on the key-value (KV) cache to support attention over prior tokens. However, maintaining the KV cache incurs substantial memory overhead, motivating KV-cache compression methods that enforce a fixed budget through eviction and merging. Modern eviction methods increasingly adopt span-based retention because preserving contiguous spans is empirically effective and better preserves semantic coherence. Yet, when combined with post-eviction merging, span-based retention concentrates merges onto a small set of span-boundary carrier tokens, producing a highly imbalanced merge pattern that exacerbates over-merging and increases information loss. To address this imbalance, we propose GRKV (Global Regression for KV Cache), a training-free KV-cache merging method that directly minimizes the discrepancy between compressed-cache and full-cache attention outputs. GRKV uses ridge-regression-based merge steps to distribute information from evicted tokens across retained tokens, while regularizing the updates to prevent over-smoothing. Across the LongBench and RULER long-context benchmarks, GRKV is the only merging method that improves overall performance with minimal overhead. Our code is available at https://github.com/pjunjie/GRKV.

Abstract:
Code agents are increasingly used in debugging workflows, where fault localization (FL) is often evaluated as endpoint retrieval: whether a model names the faulty code location. This view is inadequate for Linux kernel debugging, where failure logs may expose downstream symptoms while the erroneous state is introduced earlier and propagates through various dependences. We introduce KernelPath, a 200-instance Linux 5.10 function-level FL benchmark that pairs each root cause function with a source-grounded state propagation path and a dependence layer. KernelPath evaluates both endpoint localization and ordered diagnostic process recovery. Experiments with SOTA agents show that agents often retrieve plausible candidates but remain unreliable at recovering the diagnostic process, with performance degrading sharply under complex dependence relations. Further diagnostic probes show that agents can use explicit state-transition evidence, yet fail to infer how functions affect the erroneous state, drifting toward neighboring objectives such as repair-site selection, or up stream suspicious function retrieval. These findings reveal an unstable task model for kernel FL in current code agents.

Abstract:
Human Activity Recognition (HAR) on memory-constrained edge devices demands heterogeneous, accurate, resource-adaptive architectures that are costly to design manually. Neural Architecture Search (NAS) provides an automated solution for model design, and its integration with quantization can further expose high-quality architectures under edge memory budgets. Recent NAS increasingly adopts LLM-based methods, including prompt-engineering and fine-tuning approaches. Prompt-engineering approaches suffer from 4-10 hours generation time, while fine-tuning approaches achieve minute-level generation but often sacrifice architecture quality due to the complexity of quantization. We propose QuaNAG, a Neural Architecture Generation (NAG) framework built on the insight that quantization is not an ordinary architecture attribute: it can change model feasibility, alter accuracy, and expand the feasible search space. QuaNAG therefore models quantization as branch-level search decisions and transfers quantization-aware search knowledge into an LLM through curriculum learning. Experiments on demanding HAR benchmarks over five baselines, including in-domain evaluations, out-of-domain evaluations, and on-device experiments, demonstrate that QuaNAG achieves 8.93% average accuracy improvement with several minutes generation time, satisfying all deployment requirements. Our code and model are available at https://github.com/pyq623/QuaNAG.

Abstract:
Natural language (NL) user interest profiles—concise, human-readable descriptions of a user’s preferences—are an emerging alternative to opaque embeddings in recommender systems, offering transparency and user control alongside predictive utility. Recent advances in large language models have made generating and consuming such profiles increasingly practical, spurring a growing but fragmented body of research. In this survey, we provide the first systematic review of NL user profiles for recommendation, organizing the literature around two coupled tasks—profile generation and profile-based recommendation—and surveying the methods, datasets, and evaluation practices currently in use. We further propose the Reference-Anchored Evaluation Taxonomy (RAET), which organizes intrinsic and extrinsic metrics around explicit (output, reference signal) pairs, and reveals where efforts have concentrated and which axes remain unexplored. Finally, we identify structural open challenges—such as the tension between transparency and task-driven optimization—establishing a clear roadmap for future research in this space.

Abstract:
Large Language Models (LLMs) increasingly generate executable visual artifacts, yet established code benchmarks largely overlook the visual fidelity and interaction behavior that shape user experience. We introduce ArtifactsBench, a benchmark and automated multimodal evaluation framework for visual code generation. The framework executes each artifact and records three staged screenshots—before, during, and after a scripted interaction—then scores the evidence and source code with a checklist-guided Multimodal LLM (MLLM) referee. ArtifactsBench contains 1,825 tasks across nine domains and evaluates more than 30 LLMs. On a 280-query, six-model expert study, three staged screenshots yield 90.95% referee–human Pair ACC. Across the models shared with WebDev Arena, ArtifactsBench reaches 94.4% normalized Footrule consistency with that public human-preference ranking. We release the benchmark, evaluation harness, and baseline results at https://artifactsbenchmark.github.io/.

Abstract:
Reward modeling is essential for multimodal alignment, yet most existing reward models only provide outcome reward, which is sparse and fails to capture human preferences. Rubrics can provide structured and fine grained evaluation signals, but building effective rubrics for multimodal tasks is challenging. In this work, we propose MM-VeriRubric, a framework for scalable construction of training data for multimodal rubric generation and rubric-based reward models. To better use visual evidence in images, we introduce task-aware visual clue extraction, which makes rubric generation more grounded in task-relevant visual information. MM-VeriRubric then generates evidence-grounded rubrics and constructs rubric-aligned rejected responses with controlled error dimensions, enabling the quality of each rubric to be verified through consistency checking. The verified data are used to train both a rubric generator and a rubric-based reward model. Experiments on multimodal reward model benchmarks and downstream Best of N tasks show that MM-VeriRubric improves preference judgment (+16.4% on VL-RewardBench) and provides more interpretable and diagnostic reward signals for multimodal alignment (+13.1% on MMMU-Pro using Best of 5 selection). Code is available at: https://github.com/TraceLite-AI/MM-VeriRubric.

Abstract:
Medical guidelines encode rich, evidence-based decision logic, yet the specific decision artifact a clinician needs is hard to locate within a guideline, let alone across guidelines covering plausible diseases and treatments. While guideline passages have supported end-to-end question answering, flowcharts remain largely underused in decision support despite their ability to encode actionable clinical pathways. We therefore introduce Case2Flow, a task designed to retrieve the most relevant guideline flowchart for a given patient case from a collection of guideline documents. To support it, we construct FlowAtlas, a curated corpus of 202 flowcharts extracted from 2,080 medical guidelines, together with a pipeline that synthesises 1,911 aligned case-flowchart pairs. Our evaluation of multimodal retrieval methods reveals systematic failure modes, including overreliance on keywords and spurious token-patch matches induced by uninformative background regions in flowcharts. Motivated by this, we propose CRISP, a training-free scoring method that sharpens late-interaction retrieval by suppressing uninformative patches, discounting ambiguous token matches, and incorporating bidirectional query-image alignment. CRISP improves Recall@1 by up to 18.71 percentage points, while a blinded physician assessment on published case narratives provides preliminary feasibility evidence beyond synthetic queries.

Abstract:
UI videos provide a natural input for generating interactive webpages, as they capture both webpage appearance and action-triggered state transitions. However, directly applying video-capable vision-language models to this task remains insufficient. Existing models typically rely on sparse sampling or compressed temporal representations, which may miss short action boundaries and break the state-action-state transitions needed to implement webpage behavior. We formulate UI video-to-code generation as executable state-transition recovery from interaction videos, and identify this failure mode as state-transition misalignment. We introduce Video2Code, an action-aware video-to-code approach for recovering executable UI state transitions. Rather than allocating the visual budget uniformly across the video, Video2Code first performs coarse video understanding to locate action-critical regions, then invokes a temporal clipping tool to revisit these regions at higher temporal resolution before generating HTML/CSS/JavaScript code. We instantiate Video2Code with action-aligned video-code supervision and evaluate it under both visual and functional criteria. Experiments show that Video2Code substantially strengthens the underlying open-source model for UI video-to-code generation, improving functional correctness over direct video observation, especially on dense multi-step interactions. Our code is available at https://github.com/kiwixu1024/Video2code.

Abstract:
While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Existing benchmarks often do not capture cultural and linguistic variation and rely on gender representations. This paper introduces a multilingual German-English benchmark dataset for the evaluation of anti-LGBTQ biases in language models. It combines community-sourced stereotypes from German-speaking queer individuals with a German translation of WinoQueer. The data is used to evaluate eight language models across sizes and architectures and explore mitigation through fine-tuning on community and progressive media content. Results show that language models reproduce anti-queer stereotypes, with variation across identities and models. Differences between the translated and community-based data highlight the importance of cultural adaptation for multilingual bias evaluation. Fine-tuning reduces bias on average, but not consistently across models and identities.

Abstract:
Dental AI scribes convert chairside dental speech into structured electronic medical records (EMRs). Public data for this task is scarce because dental visits include private speech, regional language, and noisy operatory audio. We introduce DentalVoice, a Chinese dental dialogue and speech dataset for AI scribe research. It contains 13,861 multi-turn dialogues, 162,612 turns, and approximately 274 hours of generated speech. Each case is grounded in a real dental EMR, guided by a dental knowledge graph, conditioned on patient profiles, and augmented with chairside acoustic conditions. We validate the generated dialogues using LLM judges and clinical experts, finding that clinical knowledge retrieval improves safety and professionalism, while persona constraints enhance interaction realism. Next, we benchmark dental automatic speech recognition (ASR) and Subjective-Objective-Assessment-Plan (SOAP) note extraction. Results reveal persistent errors in dialectal ASR, dental terminology recognition, treatment normalization, and distinguishing planned from completed procedures.

Abstract:
KV caching is essential for efficient autoregressive decoding, but its memory footprint grows linearly with context length and quickly becomes a bottleneck for long-context large language models (LLMs). We study whether feature-dimension compression can serve as an orthogonal second stage for KV-cache reduction, and introduce AURA-KV, an adaptive-rank subspace approximation method for KV caches. AURA-KV is training-free: it decomposes post-RoPE key and value caches into compact bases and coefficients, supports incremental decoding, and allocates rank adaptively across layers under a global memory budget. Because it compresses the feature dimension rather than selecting tokens or lowering precision, it can be combined with token eviction and quantization methods. Across LongBench, RULER, and InfiniteBench, AURA-KV preserves long-context performance on Llama-3-8B, Mistral-7B, and LongChat-7B under moderate compression and degrades gracefully under more aggressive budgets. In latency measurements, AURA-KV improves per-layer decoding speed by up to 2.75 × , and by up to 3.23 × when combined with token-level cache reduction, with a small one-time prefill overhead.

Abstract:
While recent specialized multimodal code generation models excel in tasks like chart-to-code generation, their reliance on single-task training limits generalization and hinders the development of VI sio N C ode I ntelligence. In this work, we introduce VinciCoder , a unified framework designed for generalized multimodal code generation. We first curate a large-scale SFT corpus comprising 1.3M direct generation pairs and 300k visual-based refinement tasks. This composition fosters self-refinement capabilities, enabling the model to directly rectify code to align with input images. Subsequently, we propose coarse-to-fine Visual Reinforcement Learning (ViRL) to overcome the brittleness of textual metrics in handling semantically equivalent but syntactically diverse code. By quantifying visual similarity across multi-scale patches, ViRL provides an implementation-agnostic reward mechanism that ensures high-fidelity alignment between rendered outputs and input visuals. Extensive experimental results across diverse benchmarks demonstrate that VinciCoder achieves superior performance, while comprehensive ablation studies validate the effectiveness of our proposed ViRL strategy. The data, code and model are available at https://github.com/DocTron-hub/VinciCoder .

Abstract:
Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewards improves these agents by optimizing them against checkable criteria that translate report quality into reward signals, but its efficiency depends on whether those criteria reliably capture the task scope and evidence needs. Most existing studies ask an LLM to generate rubrics for a given query, but when the model fails to infer the underlying information needs, the generated rubrics may be incomplete and reduce RL efficiency. To obtain more reliable query–rubric supervision, we introduce DeepRubric, a data construction framework that reverses this process: instead of inferring evaluation criteria for a given query, it first determines what an evidence-backed report should be evaluated on and then synthesizes aligned query–rubric pairs from those evaluation targets. Starting from a sampled seed topic, DeepRubric builds an evidence tree by recursively expanding evidence-backed sub-questions, whose leaves serve as atomic and verifiable evaluation targets. It then uses the evidence tree to synthesize the training query and rubrics, ensuring that the reward evaluates exactly the information requested by the query. Using DeepRubric, we construct 9K query–rubric supervision examples and train DeepRubric-8B with rubric-based GRPO, achieving comparable performance to prior open state-of-the-art deep research models across three benchmarks with roughly 13 × fewer RL GPU-hours.

Abstract:
Recent advancements in large language models have revolutionized the field of psychological counseling, especially in the context of Cognitive Behavioral Therapy (CBT). While the success of CBT relies heavily on dynamic decision-making informed by the client’s real-time mental state, this aspect has often been overlooked in current research, limiting both flexibility and therapeutic outcomes. In this paper, we introduce StratCBT, a dataset specifically designed for psychological counseling conversations with CBT Strategies, consisting of 9,688 sessions and around 256K utterances, with each counselor’s response aligned with one of eight distinct strategies. The creation of StratCBT involves modeling clients based on their negative thoughts and generating high-quality counseling conversations through self-chat, incorporating realistic sessions as guidance, thereby significantly surpassing existing datasets in both general counseling and CBT-specific skills. We conduct extensive experiments to demonstrate the effectiveness of strategy-aligned generation and evaluate its efficacy in delivering professional and effective counseling with LLM-simulated clients to reflect real-world scenarios. The dataset can be obtained from https://github.com/zimuwangnlp/StratCBT.

Abstract:
Data-driven articles combine narrative text with visualizations to communicate insights but are time-consuming to produce, requiring both narrative composition and accurate text–chart linking. Existing automated linking methods struggle with accuracy, and automated narrative generation often misaligns with user intent and may contain hallucinations. We present VisCopilot, which leverages retrieval-augmented generation and large language models to provide fact-based, intent-aligned suggestions. As users write, the system supports narrative completion and text–chart linking in a single step, with a human-in-the-loop approach ensuring reliability. Technical evaluation on a benchmark dataset of data-driven articles authored by journalists confirmed the accuracy and quality of VisCopilot’s links. A user study further showed that VisCopilot outperforms baselines in usability, accuracy, and user confidence, with participants valuing its fact-based autocompletes for aligning suggestions with their writing intent. VisCopilot contributes: (1) an autocomplete-based design for assisting data-driven article authoring that balances efficiency, accuracy, and user autonomy; and (2) a reframing of text–chart link generation as a data-fact recommendation problem, enabling intent-aligned retrieval and systematic evaluation.

Abstract:
Recent progress in video generative models has enabled the creation of high-quality videos from multimodal prompts that combine text and images. While these systems offer greater controllability, they also introduce new safety risks, as harmful content can arise from individual modalities or their interaction. Existing safety methods are often limited to text-only inputs, require the user to manually specify the risk category in advance, or only audit content after generation, making them unable to proactively address compositional multimodal risks. We present ConceptGuard, a safeguard framework that proactively detects and mitigates unsafe semantics in multimodal video generation before harmful content is produced. ConceptGuard operates in two stages. First, a contrastive detection module fuses image and text inputs and computes their similarity against the risk concept set , automatically retrieving the top-k concepts most relevant to the input without requiring any prior assumption about the risk category. Second, a semantic suppression mechanism uses these identified concepts to remove unsafe semantics from both the text and the visual input during early generation steps. To support training and evaluation, we introduce ConceptRisk, a large-scale dataset for multimodal risk detection, and T2VSafetyBench-TI2V, the first benchmark for evaluating safety in Text-and-Image-to-Video generation. Experiments on both benchmarks show that ConceptGuard outperforms existing baselines in both risk detection and safe video generation.

Abstract:
Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unclear when and where this encoded information reaches the answer. We address this with direction patching, a class-conditioned causal intervention applied across layers, token positions, and prompt formats. Using spatial-ID directions constructed following prior encoding evidence, we find that causal influence on answer logits emerges only at mid-to-deep depths. Text chain-of-thought suppresses immediate object-word argmax-level transport in most models, while visually grounded prompts keep it open. Positive target-logit gain can remain below the argmax threshold, and transport can re-emerge at the final prefix token or at the answer step in deeper layers. Across the ten VLMs we study, these local effects form descriptive transport patterns. Complementary experiments characterize how these patterns shift across datasets, attributes, and encoding amplitudes. Together, these results reframe the encoding-grounding gap as a problem of conditional transport in VLMs.

Abstract:
LLM agents are increasingly deployed as systems built around editable external harnesses, including prompts, skills, memories and tools, that shape task execution without changing model parameters. Harness self-evolution adapts such agents by updating these harnesses from execution evidence. Yet it remains unclear whether a model’s base capability in task-solving predicts its capabilities in harness self-evolution: which models produce useful harness updates, and which actually benefit from them? We analyze two harness self-evolution capabilities: (i) harness-updating, the capability to produce useful persistent harness updates from execution evidence; (ii) harness-benefit, the capability to benefit from updated harnesses during task solving. Our analysis reveals two findings. First, harness-updating is flat in base capability: models from different capability tiers produce harness updates that lead to surprisingly similar gains; even Qwen3.5-9B’s updates yields gains comparable to those of Claude Opus 4.6. Second, harness-benefit is non-monotonic in base capability: weak-tier models benefit little from updated harnesses, mid-tier models benefit most, and strong-tier models benefit less than mid-tier. We trace low gains at the weak tier to two failure modes: weak-tier models may fail to activate relevant harness artifacts, or activate them but fail to follow them faithfully. These findings suggest investing capability budget in the task-solving agent rather than the evolver, and targeting harness invocation and long-horizon instruction following in agent training. Our source code is publicly available at https://github.com/A-EVO-Lab/a-evolve/tree/release/harness-evolution.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has recently demonstrated substantial gains in reasoning capabilities of vision–language models (VLMs). However, existing RLVR paradigms rely on deterministic rewards that implicitly penalize uncertainty expression and abstention, causing VLMs to overproduce speculative answers and degrading reliability. Through quantitative analysis, we find that vanilla RLVR, while effective in improving reasoning accuracy, significantly impairs VLMs’ ability to appropriately abstain from unanswerable queries. To systematically address this, we introduce Reliable-V-Bench , a new benchmark designed to evaluate VLMs’ abstaining capacity on challenging and realistic unanswerable visual questions. We further propose Reliable-V-RL , a reliability-aware RL framework that incorporates unanswerable and ambiguous visual questions into training, guided by a fine-grained, rubric-based reward mechanism that encourages VLMs to jointly optimize reasoning accuracy and reliable judgment. Experimental results demonstrate that Reliable-V-RL substantially improves the reliability and abstention capability of VLMs without sacrificing performance on standard multimodal benchmarks.

Abstract:
Policy documents shape governance outcomes, but their reasoning is often implicit, with tensions between participatory commitments and managerial control rarely stated directly. Existing computational approaches to policy discourse cannot express the frame-mediated relations that drive these tensions, where one argument narrows or instrumentalizes another, and end-to-end summarization by large language models offers little structure that domain experts can inspect or contest. To address this, we present D-APA (Discourse-aware Argumentative Policy Analysis), a hybrid LLM–symbolic pipeline that operationalizes critical discourse policy analysis as a quantitative argumentation graph. Given a policy text, an LLM first extracts its arguments, where each argument is classified into a deliberative or managerial frame and scanned for discourse features such as agency type and verb class. Deterministic rules over these features then connect the arguments through four frame-mediated relation subtypes, namely agency reduction, agenda shift, instrumental support, and normative support. Finally, argumentation semantics propagates linguistically grounded base scores through the resulting graph, yielding a final strength for every argument. We release a novel dataset of 100 sub-documents of disaster-risk-reduction policy from the USA, UK, Canada, and Australia, and show that our pipeline produces argument graphs that are accurate, interpretable, and consistent across jurisdictions.

Abstract:
Test-time expert reduction (top-k reduction) is a popular way to cut inference cost in Mixture-of-Experts (MoE) language models, avoiding the expensive model retraining. Existing work, however, analyzes the effects of top-k reduction almost exclusively on English benchmarks. In this work, we provide the first systematic multilingual study of test-time top-k reduction across three contemporary MoE models, 100 languages, and 5 downstream tasks. We show that down to 75% of retained experts, the models degrade gracefully, but overall retention collapses sharply at low k and widens a structural multilingual gap: low-resource languages degrade much faster than high-resource ones. We trace the gap to a renormalization-induced norm overshoot in the MoE block, where the renormalization over the surviving expert weights inflates the MoE contribution to the residual stream. This inflation harms low-resource languages disproportionately due to their smaller residual norms. As a remedy, we propose two training-free and latency-free interventions: (i) a grid-searched global scalar to rescale the representations output by the MoE layer (global α) and (ii) a parameter-free per-token correction (token α) of expert weights derived from the routing. The proposed interventions substantially improve overall retention and reduce the multilingual gap.

Abstract:
High-quality tabular data is essential for data-driven tasks, yet often limited by scarcity and privacy concerns. While synthetic data generated by recent Deep Generative Models and LLMs improves fidelity, existing methods struggle to preserve rigorous attribute dependencies. As a result, the generated data may appear statistically realistic but lack logical consistency. To address this challenge, we propose CauTab, a novel counterfactual framework for tabular data synthesis that preserves functional dependencies and causal-dependency structures. CauTab operates in a three-stage pipeline: (1) it employs an LLM-guided mining strategy to efficiently identify functional dependencies; (2) it constructs a causal-dependency graph by integrating data-driven optimization with LLM-based semantic reasoning; and (3) it executes a dependency-constrained generation process via the Abduction-Intervention-Prediction paradigm. By enforcing deterministic constraints and using the learned graph for structured propagation, CauTab improves statistical fidelity while preserving logical consistency. Experiments demonstrate that CauTab outperforms baselines in data utility and fidelity, while maintaining competitive empirical privacy and strong logical consistency.

Abstract:
Path-finding over knowledge graphs has become an effective way to ground LLM reasoning on multi-hop questions. However, biomedical QA introduces two distinct challenges that general-domain methods are not designed for: (i) queries do not expose intermediate reasoning and can be answered through multiple valid pathways, and (ii) biomedical knowledge graphs are densely connected, so path-finding methods easily take wrong turns. To address these challenges, we propose AdaPath, a path-finding framework that retrieves query-adaptive meta-paths from Path-Bank, which captures both query semantics and biomedical knowledge graph structure. AdaPath provides the missing cues in biomedical queries while effectively pruning dense knowledge graph neighborhoods during multi-hop reasoning. We further release BioStrat-QA, a biomedical KGQA benchmark that stratifies multi-hop queries by how much intermediate reasoning they expose. Across biomedical KGQA benchmarks, AdaPath consistently outperforms baselines, sustaining meaningful path-finding even when multi-hop queries expose less surface information. The source code is available at https://github.com/Jun-Hyeong-Kim/AdaPath.

Abstract:
As Large Language Models (LLMs) evolve into autonomous agents for long-horizon tasks, managing unbounded interaction trajectories under fixed context budgets becomes a core systems challenge. Unlike standard long-context documents, agent trajectories are heterogeneous and interleave observations, reasoning traces, and tool executions, so compression must preserve temporal dependencies, actionable state, and structural fidelity. Yet existing methods remain fragmented, making it difficult to compare design choices and reason about their reliability implications. This survey introduces a unified taxonomy of agent context compression along three dimensions: compression target (what is compressed), compression mechanism (how it is transformed and retained), and control policy (who decides when compression is triggered). We further organize recurring failures in compressed execution into F1: Pre-compression Decision Error, F2: In-compression Information Loss, and F3: Post-compression Access Failure, and examine domain-specific trade-offs in software engineering, web navigation, and deep research. By unifying the design space, failure taxonomy, and evaluation perspective, this survey provides a foundation for building scalable and recoverable LLM agents.

Abstract:
World models have shown great utility in improving the task performance of embodied agents. While prior work largely focuses on pixel-space world models, these approaches face practical limitations in mobile agent settings, where predicting complex visual GUI elements in future states is often difficult. In this work, we explore an alternative formulation of world modeling for mobile agents, where state transitions are described in natural language rather than predicting screenshot pixels. First, we introduce MobileWorldBench, a benchmark that evaluates the ability of vision-language models (VLMs) to function as world models for mobile agents. Second, we release MobileWorld, a large-scale dataset consisting of 1.4M samples, that improves the world modeling capabilities of VLMs. Finally, we propose a novel framework that integrates VLM world models into the planning framework of mobile agents, demonstrating that semantic world models can directly benefit mobile agents by improving task success rates.

Abstract:
Pluralistic alignment is essential for adapting large language models (LLMs) to the diverse preferences of individuals and minority groups. However, existing personalized reward models often mix stable personal traits with episode-specific factors, limiting their ability to generalize across episodes. To address this challenge, we propose EpiPersona, a framework for explicit persona–episode coupling. EpiPersona first projects noisy preference feedback into a low-dimensional persona space, where similar personas are aggregated into shared discrete codewords. This process separates enduring personal characteristics from situational signals without relying on predefined preference dimensions. The inferred persona representation is then coupled with the current episode, enabling episode-aware preference prediction. Extensive experiments show that EpiPersona outperforms the baselines. It achieves notable performance gains in hard episodic-shift scenarios, while remaining effective with sparse preference data.

Abstract:
Automated homework assessment requires not only recognizing student answers but accurately locating complete answers and intermediate reasoning steps in noisy, multi-page handwritten work. We address the missing evaluation setting of page-aware, two-level answer-region grounding: given homework page images, a model must localize complete answer regions and their ordered step-level subregions. We introduce HG-Bench, comprising 500 human-annotated K–12 homework samples curated from a 1,489,278-image source pool, with hierarchically linked question- and step-level boxes. Its page-aware protocol separately measures complete-answer localization ( ℱ A ) and step-level decomposition ( ℱ S 𝜇 ), distinguishing structured spatial grounding from visible-text parsing. Across frontier closed-source and open-weight VLMs, no zero-shot system exceeds 55.22% on ℱ A or 48.22% on ℱ S 𝜇 , whereas a GLM-4.6V 9B reference fine-tuned on approximately 10k in-domain examples reaches 74.97/72.26. These results expose step-level handwritten grounding as a concrete capability gap and provide a reproducible benchmark, evaluation protocol, and trained reference for future work on automated homework assessment.

Abstract:
MLLMs have been successfully applied to multimodal embedding tasks, yet their generative reasoning capabilities remain underutilized. Directly incorporating chain-of-thought reasoning into embedding learning introduces two fundamental challenges. First, structural misalignment between instance-level reasoning and pairwise contrastive supervision may lead to shortcut behavior, where the model merely learns the superficial format of reasoning. Second, reasoning is not universally beneficial for embedding tasks. Enforcing reasoning for all inputs may introduce unnecessary computation and latency, and can even obscure salient semantic signals for simple cases. To address these issues, we propose MMEmb-R1, an adaptive reasoning-based multimodal embedding framework. We formulate reasoning as a latent variable and introduce pair-aware reasoning selection to identify reasoning paths beneficial for query-target alignment. Furthermore, we adopt reinforcement learning to selectively invoke reasoning only when necessary. Experiments on the MMEB-V2 benchmark demonstrate that our model achieves a score of 71.2 with only 4B parameters, establishing a new state-of-the-art while significantly reducing reasoning overhead and inference latency.

Abstract:
Deploying large language models for specialized tasks requires lengthy prompts combining schema, demonstrations, and instructions — creating a fundamental tension between prompt richness and inference feasibility. We identify and characterize two distinct prompt regimes: pattern-dominant tasks, where demonstrations drive performance through implicit formatting and extraction conventions, and procedure-dominant tasks, where explicit instructions carry the critical signal. To surface this distinction, we design a two-inference diagnostic that compares model performance with instructions alone versus examples alone, revealing which component to target for compression. Guided by this diagnostic, we introduce PRISM, a self-distillation framework that generates silver training data from the model’s own full-prompt outputs and trains lightweight LoRA adapters to internalize the dominant component’s behavior. PRISM recovers 131% of example benefit on CUAD and 151% on MultiWOZ (pattern-dominant), and 39% of instruction benefit on GSM8K (procedure-dominant), eliminating 72–79% of prompt tokens at inference time. Silver data captures 80–94% of gold-labeled performance without human annotations. Beyond compression, PRISM frees context capacity: replacing a long prompt with LoRA weights enables 89.5% compliance with new runtime instructions versus 75.4% with recency prompting. The diagnostic suggests a practical pre-training heuristic to indicate when distillation is likely to succeed, helping practitioners avoid costly training on non-beneficial components.

Abstract:
Large language models produce fluent fiction, yet their creative output is widely seen as flat. We ask where this quality originates in the training and whether it affects different domains of human fiction equally. We construct a matched story-continuation paradigm across StoryStar (public-platform), TMAS (prompt-guided), and The New Yorker (professional literary)—and compare continuations from four OLMo 32B checkpoints (Base, SFT, DPO, RLVR) against matched human text. Because these checkpoints share architecture, scale, tokenizer, and pretraining, the design isolates the post-training effect. We measure each continuation along three sentence-level dimensions: thematic motion, affective prevalence, and linguistic diversity. Across all three, post-training compresses dynamic variation: thematic transitions become more uniform, high-intensity emotions give way to neutrality, and stylistic diversity across stories shrinks. We term this progressive loss narrative flattening. The effect is directionally stable across story domains but gap size depends on the human baseline: professional literary fiction is compressed most, while public-platform and prompt-guided stories show smaller gaps, consistent with their human baselines sitting closer to the model’s default rhythm. Post-trained endpoints converge across domains, suggesting alignment produces a continuation regime largely insensitive to the source domain’s narrative texture.

Abstract:
Recent advances in large language models (LLMs) have established conversational text-to-SQL as a practical interface between users and databases, often involving multiple turns of clarification and revision. However, existing benchmarks primarily evaluate execution accuracy, leaving the unfolding and shifting of user intent across turns largely uncovered. To address this, we introduce TIDE-Bench, a benchmark for conversational text-to-SQL under chain ambiguity and intent drift evaluation, targeting two recurring patterns: chain ambiguity, where an underspecified question triggers layered clarification with conditional dependencies, and intent drift, where the user retracts and replaces a previously committed request element. Built on 514 anchor SQLs from BIRD, TIDE-Bench comprises 1,542 samples and introduces dedicated metrics for chain identification and drift recognition-resolution beyond execution accuracy. Evaluating 12 advanced LLMs reveals a persistent chain identification bottleneck unaffected by clarification frequency, a wide drift recognition-resolution gap, and overlap between failure modes when jointly activated. The corresponding code of TIDE-Bench is released for further research.

Abstract:
Self-play methods that co-train a single language model as both coder and test author promise to move code-generation RL beyond fixed test suites, but they suffer from two coupled pathologies: permissiveness collapse, where pass-rate rewards are maximised by trivial, non-discriminative tests, and concentration bias, where i.i.d. sampled tests cluster on modal inputs and inflate estimator variance. We introduce CoVer (Co-trained Coder and Verifier), a single-policy GRPO framework that addresses both failure modes. First, an information-gain (IG) reward scores each self-generated test by the mutual information between its pass/fail vector and a graded, ground-truth-anchored correctness signal y ∈ [0,1]^m, gated by the sign of their covariance so that only positively discriminative tests receive reward. Second, a three-stage diversity-aware selection step prunes a candidate pool to a behaviourally non-redundant suite (invalidity, input-string, and execution-profile filtering), raising the effective sample size of the IG estimator at fixed execution budget. On five benchmarks (LiveBench, MBPP, LiveCodeBench, CodeContests, CodeForces), CoVer raises one-shot pass@1 by +5.8 points at 7B and +7.1 points at 14B over the Qwen2.5-Instruct backbone, and achieves the highest macro-average among all compared methods at both scales. As a drop-in backbone inside the CodeT ranking pipeline, CoVer-7B adds +3.5 points, demonstrating the dual benefit of co-training for both generation and selection.

Abstract:
Many top-performing reasoning language models (RLMs) exhibit English-centric reasoning behavior, generating English chains of thought even when the prompt and final answer are in another language. Beyond its impact on multilingual reasoning performance, this also blocks downstream uses of the reasoning trace, such as debugging model errors and auditing decisions. We studied different approaches to RLM language adaptation by adapting Qwen3-4B-Thinking-2507 into Polish. We find that the accuracy collapse of SFT decomposes additively into termination and quality failure modes. On the other hand, KL-anchored fine-tuning mitigates these failures but exhibits a sharp plateau-then-cliff structure under reverse divergence: language match collapses from 0.89 to 0.37 as 𝛼 doubles, with no comparable structure under forward divergence. However, a second training stage leveraging GRPO helps close the remaining gap and recover the language match even from configurations past the cliff, revealing that the two stages divide labor: the supervised stage controls reasoning quality while the RL stage closes residual language match gaps. These findings suggest that, in a two-stage RLM post-training, the supervised phase can be considerably less constrained than current practice assumes.

Abstract:
Vision-Language-Action (VLA) models are reshaping autonomous driving (AD) by unifying perception, reasoning, and control through language, enabling semantic grounding, interpretable decisions, and better long-tail generalization. But language is expensive onboard: latency and memory budgets are tight, and autoregressive decoding is inherently sequential. This work reframes the central question as when and where language should act at inference, since inference cost recurs at every deployed frame while training cost is paid once. We introduce the Language Residue taxonomy to organize methods by their inference-time use of language: train-time-only supervision (L1), latent non-textual reasoning (L2), conditional invocation (L3), and full per-frame generation (L4). We review representative methods and tag each across five deployment axes (latency, parameters, memory, FLOPs, tokens), analyzing them on major open- and closed-loop driving benchmarks (e.g., nuScenes, NAVSIM, Bench2Drive). We further trace how efficient methods from NLP/LLM are adapted in AD, identifying the constraints and motivations driving these adaptations. A continuously updated repository will be available at Github.

Abstract:
Large language models (LLMs) excel at operating at scale by leveraging social media and data crawled from the web. Whereas existing corpora are diverse, their frequent lack of long-term temporal structure may however limit an LLM’s ability to contextualize semantic and normative evolution of language and to capture diachronic variation. To support the latter, we introduce Chronoberg, a temporally structured corpus of English book texts spanning 250 years, curated from Project Gutenberg and enriched with a variety of temporal annotations. First, the edited nature of books enables us to quantify lexical semantic change through time-sensitive Valence-Arousal-Dominance analysis and to support historically-grounded interpretation. With the lexicons at hand, we demonstrate a need for modern LLM-based tools to better situate their contextualization of sentiment and detection of discriminatory language across various time-periods. In fact, we show how these tools struggle with diachronic shifts in meaning, emphasizing the need for temporally aware training and evaluation pipelines, and positioning Chronoberg as a scalable resource for respective studies.

Abstract:
Code large language model introduces copyright risks when models reproduce protected implementations from training data. Existing inference-time copyright-protection methods overlook the power of using the original LLM capability for a runnable code generation. In this paper, we propose AegisCodeGen (ACG), an inference-time copyright-aware controlled generation framework for code LLMs, effectively extending the text copyright protection towards the code generation domain. We argue that code copyright protection should be formulated as a region-aware generation control problem, where the model first identifies where modification is safe and then decides how strongly copyright-aware fusion should be applied. Based on this insight, ACG mainly consists of two stages. Firstly, ACG performs multiple sampling generations to construct an agreement profile over the target program, indicating which regions are more likely to affect core code logic after modification. Secondly, ACG uses this profile to adaptively control fusion strength, applying stronger copyright protection in low-agreement regions and weaker shifts near executable anchors. Through intense experiments, we demonstrate that ACG reduces protected-content reproduction while preserving runability and generation utility, providing a lightweight copyright-aware module that requires neither model retraining nor access to training data.

Abstract:
Translation quality depends on purpose: the same source text demands different translations depending on audience, tone, and communicative intent. Yet MT models and metrics treat translation as a fixed mapping from source to target. LLMs enable users to explicitly specify purpose alongside source text, yet this capability has not been evaluated at scale. We introduce a systematic evaluation of purpose-driven MT across 50 languages, 5 model sizes and 8 text domains. We find that (1) explicit instructions substantially improve translation adaptedness, with larger gains on informal domains (conversation, social media), for larger models and for higher-resource languages; (2) instructions outperform semantically-matched few-shot examples and paragraph-level context; (3) traditional MT metrics fail to capture adaptation quality, often penalising adapted translations; (4) models can self-generate instructions from surrounding document context, closing up to 80% of the adaptedness gap to curated instructions. Our results establish that purpose-adapted MT is a viable and measurable capability of LLMs, while highlighting the need for purpose-aware metrics.

Abstract:
Large language models (LLMs) often exhibit sycophantic behaviors—such as excessive agreement with or flattery of the user—but it is unclear whether these behaviors arise from a single mechanism or multiple distinct processes. We decompose sycophancy into sycophantic agreement and sycophantic praise, contrasting both with genuine agreement. Using difference-in-means directions, activation additions, and subspace geometry across multiple models and datasets, we show that: (1) the three behaviors are encoded along distinct linear directions in latent space; (2) each behavior can be independently amplified or suppressed without affecting the others; and (3) their representational structure is consistent across model families and scales. These results suggest that sycophantic behaviors correspond to distinct, independently steerable representations.

Abstract:
When a language model refuses to answer a prompt, it is unclear whether the correct answer is erased from its internal representations, or merely suppressed at the output layer. We investigate this mechanism using a controlled withhold setting, which yields perfectly matched answering and refusal trajectories for bidirectional activation patching. We uncover a causal asymmetry, namely a broken symmetry, in how refusal operates. Even when a model generates a clean refusal, the correct answer remains linearly recoverable from its hidden states. Furthermore, releasing this withheld answer is a highly local operation, requiring only a single-position patch. Conversely, the reverse operation is not equally local: reimposing suppression requires broader interventions across multiple positions, and assembling a coherent refusal sequence is more difficult still. We further demonstrate that while an average answer-to-refusal displacement vector marks the geometric difference between these states, it fails to act as a reliable, reversible linear control toggle between behaviours. Taken together, our findings show that refusal does not function as a simple symmetric switch. For safety and auditing, this implies that probe recoverability can overestimate true behavioural control, and locating refusal-relevant directions does not reliably grant the ability to steer a model from answering to coherent refusal.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) improves LLM reasoning but typically relies on ground-truth (GT) answers, limiting scalability. Voting-based label-free RLVR replaces gold supervision with answer-level consensus from model samples. However, collapse arises when the same answer-level signal is used both to estimate rewards and to drive token-level policy optimization, encouraging the model to directly reinforce answer tokens rather than improve reasoning. We propose OM-GRPO, a label-free RLVR framework that decouples reward estimation from policy optimization. OM-GRPO masks gradients on the answer span while retaining answer-level rewards through a soft consensus signal, shifting optimization pressure away from answer tokens. We further introduce Contrast-Augmented Reward, which refines reward estimation via low-cost pairwise comparisons over existing trajectories without additional rollouts. Across diverse reasoning benchmarks and three LLM backbones, OM-GRPO outperforms label-free RLVR methods, matches supervised GT-Reward training with stable optimization, and gains 4.24 points over majority voting in Test-Time Training.

Abstract:
Chinese Semantic Error Correction (CSEC) targets semantic errors in Chinese text, which are typically more subtle and complex than spelling and grammatical errors but remain relatively underexplored. Existing LLM-based approaches face two recurring obstacles in this task: over-correction, and unclear interaction between Chain-of-Thought (CoT) reasoning and self-consistency decoding, such that the benefits brought by CoT cannot be reliably transferred to final corrections. We propose Vote-guided Advantage Allocation for CSEC (VAA-CSEC), a multi-stage framework that combines CoT distillation, Supervised Fine-Tuning (SFT), Reinforcement Learning (RL) and self-consistency decoding. During RL, we design a task-specific reward function that directly aligned with the minimal-editing principle of CSEC. We further introduce Group-Level Relative Policy Optimization (GLPO), which reallocates GRPO advantages according to the margin between individual rollout rewards and the vote-aggregated group reward, aligning the RL training objective with the self-consistency objective used at inference time. Experiments on CSED-C and NaSGEC-Exam show that VAA-CSEC outperforms all LLM-based baselines on CSED-C with an F0.5 of 47.72%, achieves the highest recall of 42.15% among all methods, and establishes a new state of the art of 41.55% F0.5 on NaSGEC-Exam.

Abstract:
Despite rapid progress in large language models (LLMs), current QA benchmarks still overlook the core challenge of real-world scientific information seeking: synthesizing multimodal evidence scattered across multiple documents and structural formats. Existing QAs remain narrow in scope, relying on unimodal text and short-span reasoning that fail to capture the complexity of real information-seeking. We introduce DocHop-QA, a benchmark of 11,379 instances for evaluating multimodal, multi-document, multi-hop scientific QA. Built from publicly available PubMed articles, DocHop-QA incorporates textual passages, tables, and layout cues, enabling cross-document inference without explicit hyperlinks. To scale realistic QA construction, we develop an LLM-driven generation pipeline grounded in 11 scientific reasoning concepts, producing diverse and coherent question–answer pairs. To highlight the utility and versatility of the dataset, we propose a task-driven evaluation framework spanning four settings, including generative answering, multimodal evidence integration and structured index prediction. Experiments show that current models struggle with DocHop-QA’s long-context, multi-evidence demands, establishing it as a rigorous testbed for advancing next-generation scientific QA systems.

Abstract:
Logical reasoning is a core ability of modern language models, and various approaches have been proposed to enhance it. One effective method is neuro-symbolic approach, which makes a model output logic formulas and predict the answer based on them. However, fair comparison is challenging because existing methods use different experimental settings, such as post-processing and prompt styles. In addition, existing error analyses are too coarse to capture the impact of sub-formula level structures. In this paper, we present a systematic analysis of neuro-symbolic approaches under a unified setting with sub-formula level annotation. We reveal that some sub-formula level structures bias the model’s inference toward assuming the answer is true, and neuro-symbolic approaches with an external inference tool help mitigate this bias. On the other hand, these methods sometimes lack flexibility in representing sentence meaning as logic formulas.

Abstract:
How can question answering (QA) systems determine whether a query is ambiguous? Ambiguity detection is essential in open-domain QA, as misclassification leads to either answering under the wrong interpretation or triggering unnecessary clarification. However, existing methods conflate answer diversity with ambiguity, leading to inaccurate predictions. They also process all queries uniformly, resulting in wasteful computation. We propose ARCHIVE, an accurate and efficient framework that detects ambiguity via logical conflict: a query is ambiguous if its valid answers cannot all be true under a single interpretation. ARCHIVE combines a lightweight early-exit encoder for surface-detectable cases with a conflict modeling module that captures logical relations among answers, reinforced by an invariance objective for robustness to noisy answer sets. We also present QuireQA, a 4,703-query benchmark spanning factoid, non-factoid, and ill-formed queries. Experiments show that ARCHIVE outperforms competitors, improving F1-amb and F1-unamb by up to 10.4 and 21.6 points, respectively, while operating 16 × faster than the best competitor.

Abstract:
Multimodal large language models (MLLMs) remain unreliable on spatial multiple-choice questions, and their failures are often attributed to poorly attended visual information. We identify a complementary failure mode, spatial lexical bias: a spatial relation word added to the answer options can act as a lexical-semantic distractor that draws the model’s decision toward that option. Using nine open-weight MLLMs, we show that this phenomenon is widespread. We then isolate diagnostic cases in which a model answers a binary spatial question correctly yet consistently chooses a newly added third spatial option, which we call binary-stable but ternary-fragile cases. Leveraging mechanistic interpretability tools on these cases, we find that the failure arises on the language side rather than the visual side: visual attention analyses and residual-stream probes show the correct spatial relation remains internally available, while irrelevant-option controls, activation patching, and sparse component interventions trace the bias to specific LLM-side channels and neurons. Accordingly, we show that a lightweight LLM-only DPO update on tiny single-object-pair synthetic data mitigates the bias, lifting four-way robust accuracy by up to 100 points on synthetic data, and by 68.0, 32.6, and 20.1 points on broader evaluation datasets WhatsUp, SpatialMQA-Direct, and VSR.

Abstract:
Existing metrics for factuality and faithfulness evaluate whether an answer is supported or contradicted by its grounding documents, but they fail to capture when both supporting and contradicting evidence coexist. We introduce ConflictScore, a novel metric that quantifies how well a model’s response acknowledges conflicting evidence in its grounding documents. Our framework decomposes responses into atomic claims, labels each claim against each grounding document, and then aggregates these labels into two complementary measures: ConflictScore-Count (CS-C), the proportion of claims exhibiting conflicts, and ConflictScore-Ratio (CS-R), the balance between supporting and contradicting evidence. We develop ConflictBench, a benchmark covering diverse forms of conflicts such as ambiguity, contradiction, and divergent opinions, to systematically evaluate our metric. Experiments show that ConflictScore effectively detects overconfident claims across domains and can serve as a corrective feedback mechanism that improves truthfulness on TruthfulQA.

Abstract:
People who learn a second language early in life are more likely to achieve high levels of grammatical proficiency than those who learn a second language later in life. Researchers have begun to use language models in an effort to address explanations for this age of acquisition (AOA) effect. Here, we argue that this work has failed to consider an important factor that may be relevant to AOA: younger learners may experience a greater density of exposure to a second language than older learners. While earlier work neglecting this density factor in LMs failed to find an AoA effect, we demonstrate empirically and theoretically that language models that receive second language input at lower densities learn to make second language grammatical judgments more slowly and also plateau at lower levels of ability. Crucially, this remains true even in the long run when cumulative second language input is held constant. Our findings highlight the possibility that second language AOA effects in humans as well as models might reflect differences in exposure density that covary with age. This account stands in contrast to classic maturational and experiential accounts of second language age of acquisition effects, and adds an important yet missing dimension to the effort to use language models to interrogate this effect.

Abstract:
Agentic search requires language model agents to explore many sources and answer complex information-seeking questions. Scaling test-time compute is a promising way to improve these agents, but current approaches can fail, because correct answers are often sparse and score-based selection depends on model calibration. We propose FineVerify, a fine-grained self-verification framework that decomposes each question into checkable sub-questions, verifies sampled candidates against each sub-question, and selects the candidate with the highest aggregated score. This per-check structure turns selection into simpler local judgments and produces scores under the same explicit criteria. Across four agentic search benchmarks and two models, FineVerify consistently outperforms standard scaling baselines. With only four sampled trajectories, it improves GPT-5-mini by 8.2 accuracy points and Gemini-3-flash by 5.6% on average. With 12 samples, FineVerify enables GPT-5-mini to surpass frontier GPT-5 on BrowseComp-Plus. Beyond accuracy, FineVerify produces interpretable verification traces that help audit benchmark errors, suggesting broader applications for inspecting agentic search systems.

Abstract:
Prior work argues that refusal in large language models is mediated by a single direction, enabling steering and abliteration. We show that this account is incomplete: across diverse refusal and non-compliance categories, refusal behaviors correspond to geometrically distinct directions in activation space. Yet activation steering along any refusal-related direction produces nearly identical refusal–over-refusal trade-offs, acting as a shared one-dimensional control knob. Thus, different directions primarily affect not whether the model refuses, but how it refuses. Using sparse autoencoders, we uncover a structured internal representation of refusal: a reusable core of shared refusal latents supplemented by style- and domain-specific latents. Linear interventions collapse this structure into uniform behavioral control, flattening mechanistic differences across refusal types. Our results reconcile the apparent simplicity of refusal steering with the diversity of refusal behaviors, and clarify the limits of linear interpretability for aligned model behavior.

Abstract:
LLM-as-a-Judge pipelines are increasingly used to evaluate AI-generated text, based on the assumption that judgments arise from reasoning over candidate responses with respect to a rubric. We show that this assumption warrants further scrutiny. Classifiers trained only on rubric text, without access to any evaluated response, achieve nontrivial predictive performance on judge outputs. This suggests that rubric formulations encode recoverable evaluative signals, allowing scores to be partially anticipated independently of model outputs. Finally, counterfactual perturbations reveal that judges often fail to reliably update their decisions when either the candidate response or the rubric criterion is reversed. Our findings raise concerns about the reliability of rubric-based LLM evaluation and highlight the need for further methodological study of automated evaluation via LLMs.

Abstract:
The software supply chain has become an increasingly exposed attack surface because of its reliance on intricate yet fragile dependencies. Existing defenses such as GitHub Dependabot often raise many false alerts because their coarse-grained matching cannot determine whether a vulnerable dependency is actually exploitable. Security analysts typically spend substantial time assessing vulnerability exploitability case by case. Recent LLM agents have emerged as promising candidates for this task given their advanced capabilities in coding and cybersecurity, yet no existing benchmark evaluates them on it. Prior benchmarks target zero-day settings, where agents detect and exploit previously unknown vulnerabilities. In contrast, software supply chain security focuses on how known vulnerabilities in upstream dependencies affect downstream projects. This requires agents to reason across repositories and determine whether an upstream vulnerability is exploitable in the downstream project. To address this gap, we introduce VEX-Bench , the first benchmark for evaluating LLM agents’ ability to assess the exploitability of software supply chain vulnerabilities. It contains 75 real-world cases mined from GitHub and labeled by security experts, covering Python, Java, and Go. We evaluate nine models across three agent harnesses. While GPT-5.5 and Claude Opus 4.6 reach approximately 80% F1 on binary vulnerability-status classification, only GPT-5.5 surpasses 70% macro-F1 on fine-grained justification classification. This gap highlights the challenge of moving beyond binary exploitability assessment to identifying fine-grained exploitability reasons. Code and data: https://github.com/steven1518/vex-bench

Abstract:
Large language models persistently produce literal translations of idiomatic expressions, but the source of this failure remains unclear. We introduce IdiomDIT, a diagnostic framework that jointly evaluates idiom Detection, idiom Interpretation, and idiomatic Translation, attributing each error to a specific source. Across six language pairs and five models spanning 4B to 70B, IdiomDIT shows that literal translation error (LTE) can persist even after successful detection and interpretation, a know-but-error pattern we term Literal Translation Bias (LTB). This pattern is direction-specific: LTB dominates some directions, while others show greater knowledge deficits, likely reflecting lower idiom exposure in pretraining. We then use linear probing to test whether LTB is already reflected in the model’s pre-generation hidden state, recovering per-language literal translation directions with high accuracy, and ablating this direction reduces LTE. However, once we ensure no idiom’s data leaks across training and test, and subtract out each idiom’s mean hidden state before probing, accuracy collapses, confirming the probe was largely detecting which idiom it is looking at rather than a genuine bias signal. A matched random direction also reduces LTE just as much as the identified one. We release these controls as a reusable protocol for testing whether such hidden-state attribution claims in machine translation are genuine or confounded. The code is publicly available at https://github.com/NLP2CT/IdiomDIT.

Abstract:
A long-video answer is evidence-supported only when the frames decoded from the video cover every event the answer depends on. Existing evaluations score final-answer correctness or predicted evidence intervals, but the frames a method decodes before answering are rarely audited, so correct answers can still rest on incomplete observation. We introduce VES-Bench, a 600-question benchmark of Temporal Ordering and Event Counting items over 348 public long videos. Each item carries a jointly necessary set of evidence intervals, letting us audit at three strictness levels whether a method’s decoded frames cover every one of them. We also propose TRACE, a training-free agent that grounds answers in raw visual clips, builds an evidence bundle round by round, and stops only when the answer stabilises as the bundle grows and a final pass over the same clips returns the same answer. Under a same-backbone audit, TRACE answers 50.7% of questions correctly with at least two decoded frames inside every evidence interval, at 98.7 frames per question: over 10 points above uniform decoding at 128 frames (40.2%), and within 2.6 points of uniform decoding at 256 frames at 0.39× its frame cost, while reaching the highest answer accuracy in the audit (63.5%). TRACE also stays competitive on Video-MME (86.1), LVBench (75.6), and LongVideoBench (75.1).

Abstract:
We introduce Dango, a 1.8B-parameter large language model designed for controlled studies of L1→L2 (Japanese→English) transfer in second language acquisition (SLA). While previous studies have explored SLA in language models, they have predominantly relied on smaller or non-decoder models, limiting their ability to generate open-ended text and reducing their suitability as practical L2 simulators. We identify a key challenge when scaling models to this size: L2 contamination within the "monolingual" pretraining corpus used for L1 acquisition. To address this, we propose a filtering method to reduce premature exposure to English while preserving realistic, minimal exposure. We then fine-tune the model on LLM-generated L2-learning lessons to simulate the L2 acquisition process. Our evaluations show that Dango develops human-like L2 production patterns, reliably matching Japanese learners of English more closely than the multilingual baseline across our distributional metrics and the unfiltered baseline on our usage frequency metric. We release the model, data, and code to facilitate reproducible computational SLA research and learner-facing applications.

Abstract:
The rapid increase in research publications, particularly in computer science, presents significant challenges for researchers, reviewers, and institutions in performing comprehensive literature reviews. Traditional citance analysis, which mainly focuses on sentiment and citance intent, often misses important details. To address this, we introduce a new approach that combines three key aspects: Intent, Polarity, and Semantics. Using data from Semantic Scholar we classify citances based on this approach to create CitAn dataset which has 1,165 manually labeled citances from 80 subfields in Computer Science, Health Sciences, Bioinformatics, and Social Sciences and Humanities. Finally, we fine-tune state-of-the-art language models using this dataset, which yields promising results and paves the way for efficient citance analysis across various academic fields.

Abstract:
Automatic scientific survey generation requires a reference set that can support broad topic coverage and citation-worthy reference selection. However, existing systems often approximate this step as one-shot relevance retrieval from a target survey topic, abstract, or generated queries, which may overemphasize papers close to the initial topic wording and miss references needed for broader survey construction. We propose Berry-LR, a literature retrieval framework that models survey reference collection as a berrypicking process. Given a target survey topic, Berry-LR performs move-based search broadening, uses high-confidence retrieved papers as anchors for multi-round retrieval, and records retrieval trajectories as graph evidence for Trajectory-aware Reference Selection. Experiments on SurGE and SurveyBench show that Berry-LR improves reference retrieval, final reference selection, and downstream reference coverage over retrieval and automatic scientific survey generation baselines. Ablations and further analyses suggest that these gains come from move-based search broadening, anchor feedback, and trajectory-aware selection, while many remaining misses are tied to section-local writing contexts.

Abstract:
We introduce T-CLA, a taxonomy for analyzing cultural-linguistic misalignment in NLP systems. We define misalignment as a mismatch between sociolinguistic variation in user language and model representations, and argue that such variation is systematically structured. We construct T-CLA through a bottom-up analysis of 8,680 Swahili health-related responses from Kenyan speakers, identifying 170 artifacts spanning code-mixing, Sheng usage, tribal lexicons, and loanwords. We validate the taxonomy through an annotation study with 155 participants and 1,560 annotations, showing that these artifacts are broadly recognizable but unevenly detectable across speakers. Using T-CLA as an analytic lens, we find that artifacts with lower detection salience are also associated with larger model errors and fairness disparities across tribal affiliations. T-CLA provides a generalizable approach for diagnosing cultural-linguistic misalignment and understanding model error from socially embedded linguistic variation.

Abstract:
Persona prompting can steer LLM generation towards a domain-specific tone and pattern. This behavior enables use cases in multi-agent systems where diverse interactions are crucial and human-centered tasks require high-level human alignment. Prior works provide mixed options on their utility: some report performance gains when using expert personas for certain domains and their contribution to data diversity in synthetic data creation, while others find near-zero or negative impact on general utility. To fully leverage the benefits of the LLM persona and avoid its harmfulness, a more comprehensive investigation of the mechanism is crucial. In this work, we study how model optimization, task type, prompt length, and placement can impact expert persona effectiveness across instruction-tuned and reasoning LLMs, and provide insight into conditions under which expert personas fail and succeed. Based on our findings, we developed a pipeline to fully leverage the benefits of an expert persona, named PRISM (Persona Routing via Intent-based Self-Modeling), which self-distills an intent-conditioned expert persona into a gated LoRA adapter through a bootstrapping process that requires no external data, models, or knowledge. PRISM enhances human preference and safety alignment on generative tasks while maintaining accuracy on discriminative tasks across all models, with minimum memory and computing overhead.

Abstract:
Large Language Models (LLMs) incur significant computational and memory costs when processing long prompts, as full self-attention scales quadratically with input length. Token compression aims to address this challenge by reducing the number of tokens representing inputs. However, existing prompt-compression approaches primarily operate in token space and overlook inefficiencies in the latent embedding space. In this paper, we propose K -Token Merging, a latent-space compression framework that merges each contiguous block of token embeddings into a single embedding via a lightweight encoder. The compressed sequence is processed by a LoRA-adapted LLM, while generation remains in the original vocabulary. Experiments on structural reasoning (Textualized Tree), sentiment classification (Amazon Reviews), and code editing (CommitPackFT) show that K -Token Merging lies on the Pareto frontier of performance vs. compression, achieving up to 75% input length reduction with minimal performance degradation.

Abstract:
Diffusion Language Models (DLMs) offer order-agnostic generation that can explore many possible decoding trajectories. However, current decoding methods commit to a single trajectory, limiting exploration in trajectory space. We introduce Order-Token Search to explore this space through jointly searching over generation order and token values. Its core is a likelihood estimator that scores denoising actions, enabling stable pruning and efficient exploration of diverse trajectories. Across mathematical reasoning and coding benchmarks, Order-Token Search consistently outperforms baselines on GSM8K, MATH500, Countdown, and HumanEval (3.1%, 3.8%, 7.9%, and 6.8% absolute over backbone), reaching accuracy comparable to diffu-GRPO post-training. Our work establishes joint search as a key component for advancing decoding in DLMs.

Abstract:
Human-like agents must express stable dispositions while adapting to roles, relationships, and norms. We present PsyAgent, a schema-first framework that operationalizes the trait–context interface by coupling a Big Five trait prior with explicit social-structural conditioning. PsyAgent comprises (i) Individual Structure (IS), a machine-usable trait-grounded profile, and (ii) Multi-Scenario Contexting (MSC), a curated library of role–relationship–norm frames spanning eight everyday arenas. At inference, fixed structured prompts couple the active MSC frame with the IS profile, encouraging behavior that is stable yet context-sensitive. To demonstrate learnability beyond prompt engineering, we use IS×MSC to synthesize supervision and fine-tune compact backbones with PEFT (SFT and optional DPO). Under a controlled psychometric-style evaluation protocol in percentile space, PsyAgent improves trait-faithfulness and multi-turn persona consistency, and is competitive with several larger general-purpose instruction-tuned baselines under matched decoding and scoring controls. We further triangulate the automatic protocol with external benchmarks and a small blinded human study. Overall, PsyAgent provides a precise and data-efficient approach to personality-grounded, norm-aware agents.

Abstract:
With the increasing size of large language models, layer pruning has gained increased attention as a hardware-friendly approach for model compression. However, existing layer pruning methods struggle to simultaneously address key practical deployment challenges, including performance degradation, high training costs, and limited acceleration. To overcome these limitations, we propose E 3 -Pruner, a task-Effective, training-Economical and inference-Efficient layer pruning framework. E 3 -Pruner introduces two key innovations: (1) a differentiable mask optimization method using a Gumbel-TopK sampler, enabling efficient and precise pruning mask search; and (2) an entropy-aware adaptive knowledge distillation strategy that enhances task performance. Extensive experiments over diverse model architectures and benchmarks demonstrate the superiority of our method over state-of-the-art approaches. Notably, E 3 -Pruner achieves 96% accuracy, a mere 0.8% drop from the original model (96.8%) on MATH-500 when pruning 25% layers of Qwen3-32B, outperforming existing SOTA (95%), with a 1.33 × inference speedup by consuming merely 0.5B tokens (0.5% of the post-training data volume).

Abstract:
Training deep research agents requires long-horizon trajectories that interleave search, evidence aggregation, and multi-step reasoning. However, existing data collection pipelines typically rely on proprietary web APIs, making large-scale trajectory synthesis costly, unstable, and difficult to reproduce. We present OpenResearcher, a reproducible pipeline that decouples one-time corpus bootstrapping from multi-turn trajectory synthesis and executes the search-and-browse loop entirely offline using three explicit browser primitives: search, open, and find, over a 15M-document corpus. Using GPT-OSS-120B as the teacher model, we synthesize over 97K trajectories, including a substantial long-horizon tail with 100+ tool calls. Supervised fine-tuning a 30B-A3B backbone on these trajectories achieves 54.8% accuracy on BrowseComp-Plus, an absolute improvement of 34.0 percentage points over the base model, while remaining competitive on BrowseComp, GAIA, and xbench-DeepSearch. Because the environment is offline and fully instrumented, it also enables controlled analysis, where our study reveals practical insights into deep research pipeline design, including data filtering strategies, agent configuration choices, and how retrieval success relates to final answer accuracy.

Abstract:
Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft–Verify–Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36 X speedup over standard baselines.

Abstract:
Large Language Model (LLM)-based search agents have demonstrated superior performance in knowledge-intensive tasks through interleaved reasoning and search interactions. Process Reward Models (PRMs) can enhance search agent performance by guiding step selection during inference. However, existing PRMs for search agents face a critical bottleneck: assessment unreliability in their training data. This issue stems from an over-reliance on single-source supervision, forcing a compromise between global solvability and local validity. We propose CoPRM (Collaborative Process Reward Model), a process reward model trained on high-quality preference data generated through our Collaborative Calibration methodology. This approach synthesizes immediate quality assessments with outcome-informed evaluations through a dual-strategy mechanism: strictly enforcing consensus filtering to eliminate unreliable signals, and employing an outcome-informed gating mechanism to resolve heterogeneous cross-type trade-offs. Trained on 10k preference pairs, CoPRM provides universal test-time guidance, enabling diverse search agents to achieve consistent performance improvements across four multi-hop QA benchmarks, demonstrating substantial gains, robust cross-domain generalization, and superior efficiency.

Abstract:
Instruction granularity is an important yet poorly controlled variable in language-guided embodied AI. Existing benchmarks typically pair each task with a single static instruction, making it difficult to study how agent behavior changes when the same task is described at different levels of detail. We introduce Mini-BEHAVIOR-Gran, a new benchmark for controlled studies of instruction granularity that extends Mini-BEHAVIOR with multiple instruction variants per task, ranging from high-level goal descriptions to step-by-step guidance. Using this benchmark, we compare four candidate metrics for cross-task granularity quantification: token count, entity count, action-verb count, and planning-width, and find that width correlates most consistently with agent performance. Using width to organize training and evaluation further reveals a non-monotonic U-shaped relationship between instruction granularity and performance, with peaks at both fine and coarse extremes. Further analysis suggests that the coarse-granularity performance rebound is associated with shallow grounding, where agents learn vision-dominant policies.

Abstract:
Transformers underlie almost all state-of-the-art language models in computational linguistics, yet their cognitive adequacy as models of human sentence processing remains disputed. In this work, we use a surprisal-based linking mechanism to systematically evaluate eleven autoregressive transformers of varying sizes and architectures on a more comprehensive set of English agreement attraction configurations than prior work. Our experiments yield mixed results: While transformer predictions generally align with human reading time data for prepositional phrase configurations, performance degrades significantly on object-extracted relative clause configurations. In the latter case, predictions also diverge markedly across models, and no model successfully replicates the asymmetric interference patterns observed in humans. We conclude that current transformer models do not explain human morphosyntactic processing, and that evaluations of transformers as cognitive models must adopt rigorous, comprehensive experimental designs to avoid spurious generalizations from isolated syntactic configurations or individual models.

Abstract:
Rising demand for mental health support has increased interest in using Large Language Models (LLMs) for counseling, but adapting them to this safety-critical domain is hindered by limited real-world data due to privacy constraints. Synthetic datasets provide a promising alternative, but existing approaches often rely on unstructured or semi-structured text inputs and overlook structural dependencies between a client’s cognitive, emotional, and behavioral states, leading to psychologically inconsistent and less realistic interactions. We introduce Graph2Counsel, a framework for generating synthetic counseling sessions grounded in Client Psychological Graphs (CPGs) that encode relationships among clients’ thoughts, emotions, and behaviors. Graph2Counsel uses a structured prompting pipeline guided by counselor strategies and CPG, and explores prompting strategies including CoT and Multi-Agent Feedback. It produces 760 sessions from 76 CPGs across diverse client profiles. In expert evaluation, our dataset outperforms prior datasets on specificity, counselor competence, authenticity, conversational flow, and safety (Krippendorff’s 𝛼 = 0.70). Fine-tuning an open-source model on this dataset improves performance on several metrics on CounselingBench and CounselBench, while matching baselines on others. We further introduce Therapy-Eval, a multi-turn evaluation framework, and demonstrate the effectiveness of our fine-tuned model in realistic therapeutic conversations. We make our code, data and fine-tuned model public.

Abstract:
Medical VLMs frequently hallucinate clinically unsupported findings. We identify a stability gap : visually grounded tokens maintain relatively stable logits under semantics-preserving image perturbations, whereas visually misgrounded tokens exhibit substantially larger shifts. We introduce LENS ( L ogit- EN hanced S tability), which exploits this signal at both training and inference. During training, Semantics-Preserving Preference Optimization (SPPO) scores preference pairs using clinical faithfulness and cross-view stability; at inference, stability-aware contrastive decoding penalizes tokens whose logits diverge across original and counterfactual views. Constrained counterfactuals use attribution-guided masking and feature-space proximity guards to reduce disruption of lesion- and device-level evidence. Across radiology VQA, chest X-ray reporting, and gastrointestinal endoscopy, LENS consistently outperforms strong decoding-time and preference-optimization baselines. On MIMIC-CXR, entity-level RadGraph recall improves from 21.0 to 22.8 over PCCD, while expert evaluation shows a hallucination-rate reduction from 17.0% to 8.5%.

Abstract:
Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coordinated use of tools such as notebooks, IDEs, terminals, browsers, and databases within real operating environments. Yet existing benchmarks lack real-computer interaction and do not evaluate whether agents can execute complete end-to-end data-science workflows in realistic computing environments, failing to capture the multi-stage, multi-tool nature of data-science practice. We introduce DSAgentBench, the first benchmark to evaluate whether agents can automate full data-science workflows inside real computer environments. DSAgentBench contains 275 diverse tasks covering the entire data-science life-cycle, reflecting the complexity and tool coordination required in practice. Each task requires grounding decisions in intermediate outputs and coordinated tool use, and includes a deterministic evaluator that verifies analytical correctness, visual outputs, and model performance rather than code-only execution. Our extensive experiments with 15 closed- and open-source models show that even the strongest agent, Claude-4.6-Sonnet, achieves only 56.70% task success, while all open-source agents remain below 1%, frequently failing at tool orchestration, OS grounding, and multi-step reasoning. These results reveal a substantial capability gap between current agentic systems and real data-science workflows, positioning DSAgentBench as a foundation for developing grounded, verifiable, autonomous data-science agents. We release DSAgentBench at redacted .

Abstract:
Large language models (LLMs) have demonstrated strong capabilities in processing long contexts, enabling them to tackle tasks involving long textual inputs such as multi-turn conversations, legal documents, or retrieved documents in Retrieval-Augmented Generation (RAG) systems. However, despite their ability to handle long sequences, the resulting decoding latency and memory overhead remain substantial, posing challenges for real-world deployment. Recent advances in KV Cache reuse have shown potential to mitigate these costs, but these methods still suffer from notable performance degradation. To address this issue, we conduct an in-depth investigation of recomputation-based reuse methods and observe that the recomputed tokens often fail to align with the context segments most relevant to the question. This misalignment hinders proper updates to the critical contextual representations. Therefore, we propose the A ttention- A ware A ccurate KV Cache Fusion algorithm ( A 3 ), which precomputes and selectively fuses the KV Cache of text chunks based on their relevance to the question, achieving accurate integration with minimal computational overhead. Extensive experiments on various benchmarks and LLMs demonstrate that A 3 achieves competitive task performance compared to five baselines while reducing the time-to-first-token (TTFT) by 2×.

Abstract:
Multiparty tutoring policies must regulate turn-taking and social attention, yet collecting the interaction data needed for reinforcement learning is expensive in educational settings. We introduce an LLM-based user simulator for triadic robot–student dialogue that represents gaze and gesture as inline stage directions. Combined with phase-tagged prompting (which delineates conversational structure) and per-turn persona reinforcement, this design allows the LLMs to generate interactions that preserve key discourse structure and controllable participation dynamics. We validate six simulator variants against a real student–robot corpus (12 student pairs) using metrics for semantic similarity, cue obedience, and persona consistency. While the simulator reaches strong semantic agreement with real transcripts, it also reveals a measurable sim-to-real style gap (in verbosity and grammatical complexity). The simulator enables the creation of an RL environment where the tutor selects factorized gaze and reference actions, learning to interact with simulated students. We compare online training (APPO) to offline learning (DQN). Off-policy evaluation on real logs shows that offline policies trained purely from synthetic data exceed online-trained policies under a doubly-robust estimator, and a verbal-only ablation yields unstable OPE due to heavy-tailed importance weights. These results support a scalable workflow for prototyping socially grounded multiparty tutoring policies while relaying where real-world coverage and OPE reliability remain bottlenecks.

Abstract:
While Large Language Models (LLMs) show promise in alleviating mental health resource shortages, existing approaches struggle with complex emotional dynamics and maintaining theory-grounded counseling trajectories. In particular, they fail to adapt interaction rhythms across counseling phases, often resulting in excessive exploration or premature interventions. To address these limitations, we propose AuroraCounsel, an agent-based counseling framework that models professional counseling workflows. The framework integrates an emotion-sensing agent for affect recognition, a reflection agent for structured psychological analysis, and a guidance agent for response generation. Through emotion-driven dynamic routing, AuroraCounsel selectively activates reflective reasoning only when negative emotions are detected, enabling accurate problem tracing while avoiding unnecessary computation. Building on this architecture, we further align counseling behaviors by optimizing the guidance agent with Grouped Relative Policy Optimization (GRPO), allowing it to regulate questioning and supportive strategies across dialogue phases and overcome the limitations of supervised fine-tuning in capturing clinical counseling rhythms, such as those prescribed by REBT. Evaluations using LLM-based evaluators and expert assessments by professional counselors demonstrate consistent improvements over existing approaches in professionalism, empathy, and interaction appropriateness.

Abstract:
We analyze the generalization capabilities of a RoBERTa-based metaphor detection model on English verbs using the VU Amsterdam Metaphor Corpus. Since RoBERTa encoders are widely-used in metaphor detection, determining whether they rely on lexical memorization or learn transferable patterns is critical. We introduce a controlled lexical hold-out setup where all instances of selected target lemmas are strictly excluded from fine-tuning. We then compare model predictions on these Held-out lemmas against Exposed lemmas (verbs encountered during fine-tuning). While the model performs best on Exposed lemmas, it maintains robust performance on Held-out lemmas. This success does not correlate with verb lemma frequency, suggesting the generalization is likely not driven by frequency bias in pre-training. Further analysis reveals that sentence context alone is sufficient to match full-model performance on Held-out lemmas, whereas static verb-level embeddings are not. Geometrically, we find that the encoder projects Held-out instances into the same task-relevant subspaces as Exposed instances. Together, these results suggest that the model exhibits generalization capabilities, which appear to be driven by "learning the cue" (transferable contextual patterns) rather than "learning the word" (verb-specific memorization).

Abstract:
Multimodal Reward Models are critical for aligning Multimodal Large Language Models with human preferences. However, existing reward models offer limited interpretability and struggle to deliver reliable preference judgments in audio-centric scenarios. To address this gap, we propose Audio-Reward-R1 , a reasoning-based reward model that supports both audio understanding and audio generation preference tasks. Audio-Reward-R1 formulates reward modeling as a structured reasoning process, producing dimension-level analyses prior to the final preference judgment. To enable reliable reasoning supervision, we construct a high-quality reasoning-enhanced audio preference dataset with structured reasoning traces aligned with preference annotations. Audio-Reward-R1 is first trained via supervised fine-tuning as a cold-start on this reasoning-enhanced dataset, followed by reinforcement learning with verifiable rewards. Finally, we introduce Audio-Reward-Bench , a human-verified evaluation benchmark for evaluating audio reward models across both audio understanding and audio generation preference tasks. Extensive experimental results demonstrate the effectiveness and reliability of Audio-Reward-R1.

Abstract:
Judge Decoding accelerates LLM inference by relaxing the strict verification of Speculative Decoding, yet it typically relies on expensive and noisy supervision. In this work, we revisit this paradigm from first principles, revealing that the “criticality” scores learned via costly supervision are intrinsically encoded in the draft-target distributional divergence. We theoretically prove a structural correspondence between learned linear judges and Kullback-Leibler (KL) divergence, demonstrating they rely on the same underlying logit primitives. Guided by this, we propose a simple, training-free verification mechanism based on KL divergence. Extensive experiments across reasoning and coding benchmarks show that our method matches or outperforms complex trained judges (e.g., AutoJudge), offering superior robustness to domain shifts and eliminating the supervision bottleneck entirely. Code is available at https://github.com/sunshy-1/JuDi.

Abstract:
Each LoRA checkpoint compactly stores task-specific updates in low-rank weight matrices, offering an efficient way to adapt large language models to new tasks and domains. In principle, these weights already encode what the adapter does and how well it performs. In this paper, we ask whether this information can be read directly from the weights, without running the base model or accessing training data. A key obstacle is that a single LoRA update can be factorized in infinitely many ways. Without resolving this ambiguity, models trained on the factors may fit the particular factorization rather than the underlying update. To this end, we propose W2T (Weight-to-Token), which maps each LoRA update to a provably canonical form via QR decomposition followed by SVD, so that all equivalent factorizations share the same representation. The resulting components are then tokenized and processed by a Transformer to produce a weight-space embedding. Across language and vision LoRA collections, W2T achieves strong results on attribute classification, performance prediction, and adapter retrieval, demonstrating that LoRA weights reliably indicate model behavior once factorization ambiguity is removed.

Abstract:
Machine-generated text (MGT) detection is critical for regulating online information ecosystems, yet existing detectors often fail under limited fresh supervision and humanization-oriented attacks. We study this problem as few-shot robustness recovery under evolving generators, domains, and attack patterns, and use threat modeling to diagnose detector failures from an attacker’s viewpoint. We identify unstable local boundary estimation as a key failure mode: under scarce supervision, humanized MGT can induce prediction flips, logit drift, and clean–adversarial representation misalignment. To address this issue, we propose RAG-GuidEd Attacker Strengthens ConTrastive Few-shot Detector (REACT), a threat-aware few-shot robustness framework. REACT uses a dynamic retrieval-guided attacker to generate boundary-oriented humanized rewrites from extreme positive–negative anchors, and trains the detector with a pairwise boundary contrastive objective to stabilize clean–adversarial local geometry. Alternating attacker–detector updates further form a co-evolutionary curriculum. Experiments on 4 datasets with 4 shot sizes and 3 random seeds show that REACT improves average detection F1 by 4.95 points over 8 SOTA detectors and reduces average attack success rate under 4 strong attacks by 3.66 percentage points.

Abstract:
Long-horizon LLM agents require reinforcement learning methods that can assign credit to intermediate decisions under sparse and delayed rewards. Recent group-based methods such as GiGPO improve over trajectory-level GRPO by constructing step-level advantages at repeated anchor states. However, we show that such dense credit can be statistically unreliable: under limited rollouts, rare but lucky actions may receive overly large advantages, producing divergent anchor bias and late-stage training oscillation. We propose Evidence-Calibrated Policy Optimization (ECPO), a critic-free policy optimization algorithm that calibrates step-level credit before policy updates. ECPO combines Evidence-Calibrated Action Advantage, which groups rollouts by canonical actions and shrinks low-count estimates, with Variance-Gated Credit Weighting, which suppresses anchor states dominated by within-action noise. Experiments on ALFWorld and WebShop with Qwen2.5-1.5B/7B show that ECPO consistently outperforms strong baselines, improving GiGPO by +5.2/+7.3 success points on ALFWorld/WebShop with Qwen2.5-1.5B while adding only 0.1% additional advantage-computation overhead.

Abstract:
We introduce an expert-guided, real-world benchmark for evaluating document-grounded question-answering (QA) motivated by generic drug assessment, using the U.S. Food and Drug Administration (FDA) drug label documents. Drug labels contain rich but heterogeneous clinical and regulatory information, making accurate question answering difficult for current language models. In collaboration with FDA regulatory assessors, we introduce FDARxBench, and construct a multi-stage pipeline for generating high-quality, expert-guided, LLM-assisted, QA examples spanning factual, multi-hop, and refusal tasks, and design evaluation protocols to assess both open-book and closed-book reasoning. Experiments across proprietary and open-weight models reveal substantial gaps in factual grounding, long-context retrieval, and safe refusal behavior. While motivated by FDA generic drug assessment needs, this benchmark also provides a substantial foundation for challenging regulatory-grade evaluation of label comprehension.

Abstract:
Controllable text-to-speech (TTS) aims to generate speech from text with precise control over prosodic and speaker-related attributes such as pitch, age, and accent. Existing methods mainly rely on natural language prompts or reference audio cloning. However, prompt-based approaches often suffer from the cross-modal semantic gap between textual descriptions and target speech attributes, resulting in imprecise and coarse-grained control, while cloning-based methods are constrained by reference audio and offer limited flexibility. To address these limitations, we propose TTS-Hub, a controllable TTS framework that leverages modular Low-Rank Adaptation (LoRA) components and their arithmetic composition for fine-grained and flexible speech control. Specifically, we first build a comprehensive Data Hub covering 6 high-level categories and 32 fine-grained speech attributes, and use it to construct a LoRA Hub in which each LoRA is specialized for a specific attribute. During inference, TTS-Hub selects and composes the required LoRA modules through simple arithmetic operations, enabling flexible and extensible attribute control without retraining the backbone model. Experiments on two mainstream TTS frameworks demonstrate that TTS-Hub consistently outperforms strong baselines and enables precise and flexible control.

Abstract:
Multimodal Large Language Models (MLLMs) have advanced image and video understanding and can increasingly handle longer visual inputs. Long-horizon tasks such as autonomous driving and robotic navigation require more than recognizing the current view, as models must remember and retrieve previously observed spatial layouts, routes, viewpoint changes, and object states. To evaluate this capability, we introduce LongSpace-Bench, a room-tour video benchmark for long-horizon spatial memory, covering scene perception, spatial relations, and spatial memory. In this work, we further propose LongSpace, a memory framework for long-video spatial reasoning. LongSpace models long videos as sequential chunks, incorporates 3D structural cues into early decoder layers, and constructs layer-aware memory for question-guided retrieval. Experiments on multiple spatial reasoning benchmarks show that LongSpace improves long-video spatial understanding, further demonstrating explicit spatial memory as a key capability for long-horizon video MLLMs.

Abstract:
Speech carries more information than just words: a child’s voice, a fearful tone, or a noisy background should all lead a sufficiently competent spoken-dialogue assistant to different replies. Current Speech Language Models (SLMs) can recognize such paralinguistic cues but often ignore them in open-ended dialogue. We observe that a simple paralinguistic instruction scaffold at the inference stage narrows this perception-behavior gap, suggesting that the relevant cues are already latent in the model. Such scaffolds, however, remain brittle under multi-turn context and competing instructions. Therefore, we propose ParaBridge, an on-policy self-distillation method that turns a brittle inference-time scaffold into stable model behavior. During training, the scaffold serves only as a temporary privileged view; the scaffold-free model rolls out its own response, while the scaffolded view supplies dense, full-vocabulary next-token targets along its trajectory. This supervision teaches when non-lexical cues should affect the reply without the need for curated dialogues, human labels, or external reward models. On Qwen3-Omni-thinking, ParaBridge raises scaffold-free VoxSafeBench SAR from 14.6% to 40.3% and improves EchoMind average rating from 3.27 to 3.92. It also preserves general ability, with MMAU-Pro, VoiceBench, and GPQA all within 0.4 points of the original model. Beyond the training distribution, ParaBridge generalizes to unseen paralinguistic cues, transfers from safety-oriented training to empathy-oriented dialogue, and works on a different SLM backbone. Code and checkpoints are available at https://github.com/AmphionTeam/ParaBridge.

Abstract:
Vision-Language Navigation (VLN) requires an embodied agent to follow natural-language instructions in unseen environments. Recent progress has been largely driven by Multimodal Large Language Models (MLLMs). Existing methods follow a next-step action prediction paradigm, supervising only the expert action, which requires a high quantity of data for training. They also rely on cognitive maps, accumulated historical frames, or external 3D tools to maintain states, leading to high computational and memory overhead. To realize resource efficiency VLN, we propose LookStep , a unified end-to-end framework that combines Language Centric Future State Modeling and Event Driven Rolling Memory that uses language labels to generate coarse-grained navigation progress and future states for each candidate action, while autonomously deciding whether to write each observation into a bounded rolling memory with a semantic role. We validate LookStep empirically. On VLN-CE tasks, LookStep outperforms existing methods under the same training settings, achieving a 49.7% success rate on R2R-CE Val-Unseen with better memory efficiency and less data usage. Code and model is available at https://github.com/kunyang-YU/LookStep .

Abstract:
Building trustworthy medical multimodal large language models (MLLMs) is critical for reliable clinical decision support. Existing medical hallucination benchmarks mainly focus on data collection, but often ignore where hallucinations originate within the reasoning process. We find that hallucination sources vary across samples: errors may arise from visual misrecognition, incorrect medical knowledge recall, or flawed reasoning integration. To enable source-level hallucination diagnosis, we introduce ClinHallu, a benchmark for stage-wise hallucination diagnosis in medical MLLM reasoning. ClinHallu contains 7,031 validated instances, where each instance is augmented with a structured reasoning trace decomposed into Visual Recognition, Knowledge Recall, and Reasoning Integration. We also use stage-replacement interventions to measure how correcting specific stages affects the final answer. Beyond evaluation, we show that trace-supervised fine-tuning reduces stage-wise hallucinations. ClinHallu provides a fine-grained hallucination testbed for diagnosing and mitigating reasoning failures in medical MLLMs. The benchmark is publicly available at https://github.com/alibaba-damo-academy/ClinHallu.

Abstract:
While large language models excel on high-resource multilingual tasks, low- and extremely low-resource Indic languages remain severely under-evaluated. We present IndicParam, a human-curated benchmark of over 13,000 multiple-choice questions covering 11 such languages (Nepali, Gujarati, Marathi, Odia as low-resource; Dogri, Maithili, Rajasthani, Sanskrit, Bodo, Santali, Konkani as extremely low-resource). We evaluated 20 LLMs, both proprietary and open-weights, which reveals that even the top-performing Gemini-2.5 reaches 58% average accuracy, followed by GPT-5 (45) and DeepSeek-3.2 (43.1). We additionally label each question as knowledge-oriented or purely linguistic to discriminate factual recall from grammatical proficiency. Further, we assess the ability of LLMs to handle diverse question formats-such as list-based matching, assertion-reason pairs, and sequence ordering-alongside conventional multiple-choice questions. IndicParam provides insights into limitations of cross-lingual transfer and establishes a challenging benchmark for Indic languages. The dataset is available at https://huggingface.co/datasets/bharatgenai/IndicParam.

Abstract:
Trustworthy verifiers are essential for the success of reinforcement learning with verifiable reward (RLVR), which is the core methodology behind various large reasoning models such as DeepSeek-R1. In complex domains like mathematical reasoning, rule-based verifiers have been widely adopted in previous works to train strong reasoning models. However, the reliability of these verifiers and their impact on the RL training process remain poorly understood. In this work, we take mathematical reasoning as a case study and conduct a comprehensive analysis of various verifiers in both static evaluation and RL training scenarios. We show that widely used rule-based verifiers fail to recognize equivalent answers in different formats, leading to substantial false negatives that increasingly hinder RL performance as the policy model gets stronger. Model-based verifiers substantially improve static accuracy but are highly susceptible to reward hacking during RL, where they misclassify certain patterns in responses as correct, particularly after fine-tuning. Our findings underscore the challenges inherent to both rule- and model-based verifiers and provide insights toward developing more accurate and robust reward systems for reinforcement learning.

Abstract:
End-to-end manga generation is a structured visual storytelling task that requires story decomposition, recurring character and scene grounding, page layout design, panel rendering, page composition, and lettering. However, existing generative models often perform direct page synthesis, entangling these factors in a single visual output and limiting precise control over layout geometry, visual references, and cross-panel consistency. To address these limitations, we propose MangaFlow, an agentic framework for controllable long-form manga generation that decomposes manga creation into planning, grounding, layout construction, reference-conditioned rendering, composition, and text placement. By treating layout and visual references as explicit intermediate variables, MangaFlow enables both simple text-to-manga generation and more precise user-controlled manga creation. This design exposes layout, visual assets, and lettering as editable intermediate controls for refining panel geometry, references, and text placement. To support long-form consistency, MangaFlow introduces a story section memory that links section descriptions with corresponding character, scene, and object references for reuse across panels. We further present a meta-benchmark for evaluating layout controllability, visual consistency, and generation quality. Experiments show that MangaFlow improves layout adherence and cross-panel consistency over direct generation baselines while supporting flexible human control.

Abstract:
Large language models (LLMs) have shown promise for automated unit test generation, but existing evaluations largely rely on standalone settings and a narrow set of programming languages, overestimating real-world readiness. We introduce XRepoTest, a multilingual, repository-level benchmark for unit test generation spanning five underexplored languages: Rust, Go, Julia, PHP, and Ruby. XRepoTest evaluates tests under realistic repository constraints using a containerized execution framework and multiple context augmentation strategies, including file-level, LSP-based, and retrieval-based context. Beyond standard metrics such as test pass rate and coverage, we propose Invocation Rate (IR) to assess whether generated tests meaningfully exercise the intended functionality. Experiments with 14 state-of-the-art LLMs (e.g., Claude 4.5, GPT-5.2, DeepSeek V4-pro, and Qwen families) reveal a substantial gap between standalone and repository-level performance, as well as trade-offs between richer context and test reliability. Overall, XRepoTest provides a challenging and informative benchmark to advance scalable, robust, and multilingual unit test generation in realistic software environments.

Abstract:
Long-horizon LLM agents generate traces that could become reusable experience, but raw trajectories are noisy, local, and hard to govern. Agent Skills offer a structured artifact for combining procedural guidance, executable resources, and applicability boundaries. Yet open skill ecosystems contain redundant, uneven, environment-sensitive artifacts, and indiscriminate updates can pollute future context. We present SkillsVote, a lifecycle-governance framework for Agent Skills across collection, recommendation, attribution, and evolution. SkillsVote profiles a million-scale open-source corpus for environment requirements, quality, and verifiability, and synthesizes tasks for verifiable skills. Before execution, it performs agentic library search over structured skill folders to expose instructional context. After execution, it decomposes trajectories into skill-linked subtasks, attributes outcomes to skill-guided execution, agent exploration, environment, and result signals, and admits only successful reusable discoveries to evidence-gated updates. Experiments on Terminal-Bench 2.0 and SWE-Bench Pro show that SkillsVote improves agent performance on challenging agentic coding benchmarks. The gains arise from two complementary pathways: online evolution over test-time task streams and offline transfer via frozen libraries built from either historical trajectories or curated open-source skills. We release our code at https://github.com/MemTensor/skills-vote.

Abstract:
Contrastive decoding (CD) seeks to mitigate hallucinations in Large Vision-Language Models (LVLMs) by contrasting the output distributions of a standard model and a visually degraded model. However, existing training-free CD methods suffer from sub-optimal degraded branches: completely dropping visual tokens is too extreme and induces language hallucinations, while corrupting input images offers coarse control over visual evidence and suffers from high inference latency due to requiring two full forward passes. To address these dilemmas, we propose YARD, a training-free Y-Architecture Register Decoding framework. Motivated by the observation that reliable text-to-vision grounding predominantly emerges in the middle decoder layers, YARD constructs the degraded branch internally by sharing shallow-layer computations and branching exactly at this critical stage. For the degraded branch, YARD replaces patch-level visual tokens with register tokens, which preserve global image semantics but lack fine-grained local evidence. This image-aware yet locally under-grounded design provides a faithful contrastive signal without extreme modality mismatch, while the Y-architecture strictly avoids a costly second forward pass. Extensive experiments on generative and discriminative hallucination benchmarks demonstrate that YARD consistently achieves state-of-the-art hallucination mitigation across multiple LVLMs, alongside a significant reduction in inference latency.

Abstract:
Privacy-preserving adaptation of pretrained language models requires continual fine-tuning as private client distributions shift, while protecting individual samples under local differential privacy (LDP). In streaming differentially private federated learning, full-rank Low-Rank Adaptation (LoRA) adapters expose many trainable directions to DP noise, and later domains may overwrite directions that remain relevant to earlier domains. We propose Private Rank-Invariant Subspace Memory (PRISM), a fixed-rank LoRA framework that schedules the active plastic rank over the stream and applies shrinkage-gated memory writes to DP-protected aggregate updates. Server-side consolidation stores retained high-SNR directions in an orthogonal memory and remains pure post-processing of LDP updates. On MNLI genre and Sentiment3, PRISM stabilizes adaptation under strict privacy budgets while preserving a strong public initialization. At 𝜖=2 , it reaches 79.8% and 92.3% final average accuracy, compared with 52.3% and 85.0% for full-rank DP-FedSVD. Ablations indicate that rank scheduling accounts for most strict-budget recovery, and shrinkage-gated memory contributes a smaller retention gain and modestly improves over the strongest static low-rank control on MNLI.

Abstract:
Free-text explanations extend human label variation (HLV) beyond label disagreement by revealing the reasoning and preferences behind annotators’ decisions. We study whether large language models (LLMs) can learn and reproduce such annotator-specific label-explanation behavior. Using two sentence-pair tasks with four annotators each—natural language inference and paraphrase judgment—we first analyze whether annotators exhibit stable individual patterns. We find that such patterns are weak at the single-annotation level due to strong input-content effects, but become detectable after input-content reduction and annotator-level aggregation. We then compare prompting and supervised fine-tuning (SFT) baselines and propose cross-annotator preference optimization (CAPO), which contrasts a target annotator’s response with other valid but less target-specific annotations for the same input. Experiments show that prompting is limited and unstable, SFT better captures annotator-specific behavior, and CAPO further improves aggregation-aware imitation and judge-based attribution while preserving target-specific reasoning patterns under human validation. Overall, our results show that HLV can be learned as annotator-specific label-explanation behavior, suggesting a path toward scalable explanation-based annotation grounded in annotator histories rather than labels alone.

Abstract:
Multimodal Large Language Models (MLLMs) achieve strong vision-language reasoning but incur large KV caches and high decoding latency with long visual contexts. Existing compression methods rely on observation window attention for stable token importance estimation, yet this aggregation can dilute sparse critical evidence and discard answer-relevant tokens under aggressive compression. We identify last query attention as a complementary signal for recovering such evidence, though its irrelevant signals may introduce additional noise. We propose BACON, a plug-and-play method that calibrates observation window attention with last query evidence while suppressing noise through intra-layer coherence and inter-layer persistence. Across diverse benchmarks, models, budgets, and compression methods, BACON improves multimodal KV-cache compression by 7.5% on average under the most aggressive budget, with gains up to 30.9%.

Abstract:
Multi-agent systems (MAS) powered by large language models have shown promise for complex tasks but suffer from high failure rates. Current self-reflection methods for MAS require all agents to reflect upon failure, overlooking a critical reality: failures typically stem from a specific agent leading the task astray, namely the decisive error agent, while others merely fulfill their regular duties. Forcing regular-behaving agents to reflect contaminates their memory with wrong insights. Hence, we propose DoCtOR (Diagnose-then-Correct PPO-enhanced Reflection), a novel reflection framework that enhances multi-agent collaboration. DoCtOR first identifies the decisive error step and decisive error agent through automated failure attribution, then employs counterfactual reasoning to generate a corrected decisive error step, and finally engages only the decisive error agent to produce targeted reflections. Experimental results show DoCtOR achieves 22%, 26%, and 27% improvements over initial success rates on HotPotQA, ChartQAPro, and Mind2Web datasets, outperforming Reflexion, Retroformer, and COPPER. We further establish the generalizability of our diagnose-then-correct paradigm and demonstrate that in low-resource settings, focusing reflection on reasoning steps after the decisive error step achieves comparable quality to reflecting on the complete failure trajectory.

Abstract:
Claims about the grammatical competence of multilingual language models vary sharply with how competence is measured, yet the interaction between evaluation paradigm, post-training, and language resource availability has not been systematically examined. We evaluate base and post-trained models from six families on MultiBLiMP, a syntactic minimal-pair benchmark covering 101 languages, using four evaluation methods. We report three principal findings. First, post-training degrades grammatical competence, but the magnitude of this effect is determined by post-training recipe rather than model scale, and scaling reduces it unevenly, with low-resource languages bearing the highest cost. Second, post-trained models retain grammatical knowledge they cannot articulate through explicit prompting, yet this is measurable only in high-resource languages, because near-chance baselines in low-resource settings leave little knowledge to hide. Third, native-language prompting recovers otherwise hidden competence on low-resource languages, demonstrating that only high-resource languages can be probed directly from unprompted probabilities. We conclude that multilingual grammatical evaluation must adopt language-informed, multi-paradigm protocols to avoid systematically underestimating low-resource abilities.

Abstract:
Inference-time scaling (ITS) for latent reasoning requires sampling multiple candidate latent trajectories, yet current methods typically rely on heuristic perturbations such as dropout or fixed Gaussian noise. We show that stronger perturbations do not necessarily improve candidate-set quality: they often induce larger distribution shifts without producing more decision-useful reasoning paths. This limitation arises because heuristic perturbations inject stochasticity through fixed or hand-designed mechanisms rather than an optimizable, context-conditioned sampling policy, making latent exploration difficult to control. We focus on the sampling side of latent ITS and propose the Gaussian Thought Sampler (GTS), a lightweight module that predicts context-dependent Gaussian perturbations over continuous reasoning states. Trained with GRPO-style policy optimization while keeping the backbone frozen, GTS turns heuristic perturbation into a trainable, context-conditioned sampling policy. Experiments across two latent reasoning backbones and multiple benchmarks show that GTS consistently improves pass@N over heuristic baselines, suggesting that effective sampling for latent ITS requires controlled, optimizable exploration rather than simply amplified stochasticity.

Abstract:
AI productivity is often measured by task completion time, economic value, or improvements in outcome quality. However, these measures usually treat collaboration as a black box where they capture what output was produced, but not the interaction cost required to produce it. Motivated by economics literature, we introduce a productivity-oriented framework for evaluating human-AI collaboration as outcome quality relative to interaction cost. Across two datasets spanning four tasks, we show that: (1) sessions with identical quality ratings can differ by up to 70 × in interaction cost; (2) quality-cost relationships vary by task, with some tasks rewarding extended interaction and others favoring fast convergence; (3) subjective user ratings are not reliable substitutes for productivity; and (4) productive sessions are characterized by agents probing earlier and users spending less effort repairing the interaction. By distinguishing productive success from costly success, our framework makes interactional cost visible and shows how dialogue analysis can inform the evaluation and design of AI systems. We release our code and newly collected human-AI collaboration dataset for reproducibility.

Abstract:
Stance detection is crucial for understanding the underlying attitude of an expression towards a target. Conversational stance detection is a more challenging stance detection task in real-world social media scenarios, as it involves detecting the user’s stance by leveraging the target-related historical statements across conversational sessions. In this paper, we propose target-aware Memory Graph TamGraph, a novel method that dynamically leverages target-related statements for conversational stance detection. Instead of considering all preceding historical conversations or using no prior conversation information for stance detection, our TamGraph employs a stepwise, entropy-guided backtracking mechanism to selectively activate memory from historical conversations and dynamically constructs a target-aware graph to model the stance relations among utterances. This allows the exploitation of target-related information from the conversation history for stance detection while preventing the introduction of noise. Experimental results on both English and Chinese benchmarks demonstrate that our TamGraph substantially improves LLM performance on conversational stance detection.

Abstract:
LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution. Recent studies propose self-evolving agents that autonomously generate, refine, and reuse skills from past experiences to enable continuous capability evolution. However, autonomous skill evolution introduces a new attack surface in which malicious capabilities are generated, stored, and reused as legitimate skills. In this paper, we define EvoSkill Injection as a threat model targeting the autonomous skill generation and evolution pipeline of self-evolving agents. We further propose SARGE (Red-teaming Autonomous Skill Generation and Evolution in self-evolving agents), a red-teaming framework for evaluating this threat model through iterative generation, escalation, and reinforcement interactions. To support our framework, we construct EvoSkillBench, a benchmark dataset of malicious interaction trajectories for inducing malicious skill formation in self-evolving agents, and introduce EvoSkillSafetyBench to assess whether injected malicious skills are retrieved and activated as harmful behaviors during downstream task execution. Our evaluation shows that SARGE induces malicious skill formation and that injected skills are persistently stored and repeatedly activated, highlighting the risk of persistent capability corruption.

Abstract:
Watermarking algorithms for Large Language Models (LLMs) effectively identify machine-generated content by embedding and detecting hidden statistical features in text. However, such embedding leads to a decline in text quality, especially in low-entropy scenarios where performance needs improvement. Existing methods that rely on entropy thresholds often require significant computational resources for tuning and demonstrate poor adaptability to unknown or cross-task generation scenarios. We propose C ontext- A ware T hreshold watermarking ( CATMark ), a novel framework that dynamically adjusts watermarking intensity based on real-time semantic context. CATMark partitions text generation into semantic states using logits clustering, establishing context-aware entropy thresholds that preserve fidelity in structured content while embedding robust watermarks. Crucially, it requires no pre-defined thresholds or task-specific tuning. Experiments show CATMark improves text quality in cross-tasks without sacrificing detection accuracy.

Abstract:
Voice agents typically use text-first pipeline for safety, but large language model (LLM) outputs are often overly written, making even high-quality text-to-speech (TTS) sound robotic. This is especially problematic in domains such as healthcare and sales, where different audiences require graded orality control. Text-level rewriting can control spoken style, but resources mostly study the reverse direction (spoken-to-written transfer), and its internal locus in LLMs remains unclear. We study English written-to-spoken rewriting with graded orality control. We introduce WSPairs, 4 , 709 written–spoken pairs from spontaneous conversations, annotated with 0–5 Orality Levels using seven sociolinguistic features. Through causal activation patching and head-level attribution, we find that orality concentrates in late layers and a sparse set of high-influence attention heads. We then propose CAS, a no-weight-update calibrated activation-steering method, and LAS, a learned activation-steering method. CAS improves exact level targeting and reduces bias relative to few-shot prompting, although target ordering remains imperfect at the highest levels. LAS uses 0.004%–0.006% trainable parameters, obtains higher Orality Level Accuracy (L-Acc) than full SFT on Qwen-3B and Mistral-7B, remains the strongest lightweight method on Llama-1B, and leads SFT by up to 3.9 L-Acc points out of domain. These results support localized, efficient control of text-level spoken style. Dataset and code are available.

Abstract:
Large Language Models (LLMs) have demonstrated remarkable efficacy in text embedding, yet current adaptation methods like LoRA face significant bottlenecks in computational efficiency and cross-architecture transferability. Whenever a new backbone emerges, existing approaches require costly retraining from scratch. To address this, we propose PromptEmbedder, a novel dual-LLM framework that decouples embedding knowledge from specific backbone weights. PromptEmbedder utilizes a Prompting LLM to generate instruction-aware soft prompts for a frozen Embedding LLM via a differentiable generation process with continuous relaxation, ensuring full gradient flow during contrastive training. By localizing task-specific knowledge within the Prompting LLM, adapting to new architectures requires only retraining a lightweight linear alignment matrix. Evaluations on the MTEB benchmark show that PromptEmbedder achieves comparable performance with LoRA finetuning while reducing GPU memory by 40% and accelerating training by 3.7 × . Our approach establishes a scalable, architecture-agnostic paradigm for efficient LLM-based representation learning.

Abstract:
The "political opinions" of Large Language Models (LLMs) increasingly find their way into political discourse despite limited under- standing and control of these systems. Our paper adds to the understanding of how prompt manipulation impacts the multilingual generation of political arguments, focusing on two fundamental aspects of generated text: style and semantics. Starting from a reference set of Swiss politicians’ arguments in German, Italian, and French, we conduct a large-scale prompt-manipulation experiment on 4 LLMs, building the prompts in a modular way (i.e., with blocks encoding socio-demographics, context, and examples) and testing all possible combinations. Using our parallel dataset of over 375k human argument-aligned generated arguments, we assess the detectability of LLM arguments through classification experiments with 160 linguistic features and semantic embeddings. We then carry out quantitative and qualitative analysis to understand how human- written vs. LLM-generated text compares, and to assess the impact of prompt manipulation. Our results show that human and LLM arguments are substantially different in style and semantics, making their identification easy for classifiers. Prompt manipulation does not substantially impact semantics and style, except for sociodemographic persona prompting, which makes generated texts moderately more human-like. Model, language, and topics impact the stability of the generated text most. We make the code, data, and raw results available via https://github.com/mmmaurer/promptology.

Abstract:
Retrieval-augmented generation (RAG) augments LLMs with external documents, but public or user-editable sources expose RAG systems to data poisoning: attackers can inject malicious documents to steer outputs toward targeted answers. Existing poisoning attacks often rely on query inclusion, inserting the target query into poisoned documents to improve retrieval; however, this creates lexical and embedding-space artifacts that make them easy to filter. We propose CamoDocs, a poisoning attack that avoids direct query inclusion by camouflaging adversarial documents among benign content. CamoDocs chunks synthesized benign and adversarial drafts, replaces selected tokens in benign chunks with dispersion tokens that spread poisoned-document embeddings, and applies coherence filtering to limit readability degradation. Across seven RAG defenses, three open-weight LLMs, and three benchmarks, CamoDocs achieves strong average ASR while avoiding query-overlap artifacts exploited by simple query detection. It also remains effective against proprietary models, achieving average ASRs of 61.80% on GPT-5.4-mini and 55.09% on Claude-Haiku-4.5. Finally, we show that erasure-heavy clustering defenses such as TrustRAG can reduce ASR, but only with substantial utility drops on retrieval-dependent benchmarks such as NeoQA. Code is available at https://github.com/jaewonalive/CamoDocs.

Abstract:
Computational value analysis has largely treated values as instances of predefined universal taxonomies, making it difficult to discover culturally specific value dimensions not fully represented in existing schemes. We formulate value analysis as an open-ended discovery problem grounded in the argumentative contexts in which values are expressed. To this end, we propose an LLM-assisted framework that discovers human value dimensions directly from text without relying on predefined taxonomies. It extracts Premise-Conclusion-Stance (P-C-S) argumentative edges under explicit argument tree constraints, expands each edge into structured value-semantic descriptions spanning goal, principle, tradeoff, and perspective dimensions, and induces coherent value clusters via co-association consensus clustering over the embedding representations. We evaluate the framework across ten languages using external label alignment, internal structural stability metrics, and human judgment. Results show that the framework is robust and transferable, with the induced value structures achieving low hallucination and strong alignment with human judgments while revealing culturally distinctive value orientations that both align with and meaningfully complement established psychological frameworks. By shifting computational value analysis from taxonomy-driven classification toward open-ended discovery, our framework offers a new approach to cross-cultural value research, with implications for culturally aware AI alignment.

Abstract:
Process reward models (PRMs) enhance the reasoning capabilities of large language models (LLMs) by providing fine-grained feedback, yet training PRMs typically requires expensive stepwise annotations. Outcome-supervised PRMs offer a scalable alternative by learning from final-answer correctness alone, but this introduces a fundamental credit assignment challenge, i.e., attributing outcomes to responsible reasoning steps. Existing approaches rely on either uniform or causal assignment, both of which fail to anchor credit in step correctness and thus hinder process error identification. In this work, we propose Outcome-Supervised Process Reward Modeling via Learnable Credit Assignment (LCA), an outcome-supervised PRM framework that jointly learns credit assignment and reward modeling under the principle of Weakest Link Assignment: a reasoning chain is as strong as its weakest link. To address mutual dependence between credit assignment and reward modeling, we formalize outcome-supervised PRM as a Multiple Instance Learning (MIL) problem and introduce Softmax-Weighted-Sum (SWS) pooling, an MIL pooling technique tailored for strong dependence and redundancy among reasoning states. We establish instance-level Bayes consistency of the resulting learning objective under mild assumptions. Extensive experiments demonstrate that LCA outperforms state-of-the-art outcome-supervised PRMs across multiple tasks and backbones. Code is available at https://github.com/JiaTianyu20031016/MIL.git.

Abstract:
Test-Time Scaling has shown notable efficacy in addressing complex problems through scaling inference compute. However, within Large Audio-Language Models (LALMs), an unintuitive phenomenon exists: post-training models for structured reasoning trajectories results in marginal or even negative gains compared to training for direct answering. To investigate it, we introduce CAFE, an evaluation framework designed to quantify audio reasoning errors. Evaluation results reveal that LALMs encounter a critical bottleneck: Audio Perception Decay, which compromises reasoning performance. To address it, we propose MPAR 2 , a strategy that encourages a step-by-step reasoning process with rich perceptual awareness. With reinforcement learning, MPAR 2 improves performance on CAFE from 31.74% to 63.51% and effectively mitigates audio perception decay, concurrently enhancing reasoning ability to achieve a strong 74.59% accuracy on the MMAU benchmark. Further analysis shows that MPAR 2 optimizes audio attention and enhances reasoning token budget efficiency.

Abstract:
Spoken language models (SLMs) extend LLMs to speech input and output, but existing systems use fixed frame rates (e.g., 25 or 12.5 Hz), overlooking speech’s time-varying information density and limiting inference-time quality–speed tradeoffs. Recent dynamic-frame-rate audio tokenizers enable very low average frame rates and controllability, yet had not been applied to SLMs. We introduce FlexiSLM, the first SLM with dynamic, controllable frame rates, using pretrained FlexiCodec for dynamic speech output tokens. It integrates this representation into a multi-task speech-to-speech SLM, extends it with input-side frame compression, and adds direct frame-rate conditioning for accurate control during inference. FlexiSLM outperforms fixed-frame-rate 7B models, including Qwen2.5-Omni and Kimi-Audio, at 12.5 and 6.25 Hz; it can be steered down to 4.0 Hz, and at 6.25 Hz roughly halves inference time relative to 12.5 Hz while retaining strong speech-to-speech quality. Audio samples: https://flexislm.github.io; code and data: https://github.com/AmphionTeam/FlexiSLM.

Abstract:
Instant-messaging human social chat typically progresses through a sequence of short messages. Existing step-by-step AI chatting systems typically split a one-shot generation into multiple messages and send them sequentially, but they lack an active waiting mechanism and exhibit unnatural message pacing. In order to address these issues, we propose Stephanie2, a new step-wise decision-making dialogue agent. With active waiting and message-pace adaptation, Stephanie2 explicitly decides at each step whether to send or wait, and models latency as the sum of thinking time and typing time to achieve more natural pacing. We further introduce a dual-agent dialogue system to generate pseudo dialogue histories for human and automatic evaluations. Experiments show that Stephanie2 clearly outperforms Stephanie1 on metrics such as naturalness and engagement, and achieves a higher pass rate on human evaluation with the role identification test.

Abstract:
Large Language Models (LLMs) excel at both informal and formal (e.g. Lean 4) mathematical reasoning but still struggle with autoformalisation , the task of transforming informal into formal mathematical statements. Yet, the performance of current Math LLMs is constrained by the scarcity of large-scale corpora, particularly those containing pairs of informal and formal statements. Interestingly, the formal languages used in autoformalisation share structural similarities with programming languages, and code data is available at scale. However, current models trained on code do not transfer effectively to formal math, due to structural and syntactic differences between them. To address this, we propose TopoAlign , a framework that unlocks widely available code repositories as training resources for Math LLMs. TopoAlign decomposes code into docstrings, main functions, and dependency functions, and reassembles these components into analogues that structurally mirror formal statements. We train three state-of-the-art models, , and , and evaluate them on the , , and benchmarks. TopoAlign provides substantial gains for , improving performance by 17.77% on BEq@10 and 68.82% on typecheck@10, and also measurably improves by 0.12% on BEq@10 and 1.09% on typecheck@10 despite introducing no new mathematical knowledge.

Abstract:
Streaming video understanding requires Vision Language Models (VLLMs) to process growing video streams and answer user questions under tight latency constraints. Existing methods improve efficiency with token pruning and memory-bank schemes, but mainly reduce visual tokens after the LLM encoding stage. Therefore, downstream token pruning alone cannot substantially reduce end-to-end latency, since the expensive frame encoding cost has already been incurred. In this work, we propose CoFiE, a Co arse-to- Fi ne E vidence selection framework, that decouples evidence selection into a coarse query-agnostic filtering stage before the vision encoder and a fine query-specific refinement stage during LLM prefill. We introduce two lightweight modules, namely the coarse Novelty-Guided Frame Filtering and the fine Query-Specific Evidence Refinement, that retain visually distinctive candidates and refine the most relevant frames, respectively. This design removes substantial redundancy before frame encoding while retaining query-specific refinement once semantic information becomes available. Experimental results show that CoFiE establishes a new state-of-the-art accuracy–efficiency trade-off with open-source VLLMs across multiple video understanding benchmarks, reaching 78.86% on StreamingBench and 68.72% on OvO-Bench, improving over prior best methods by up to 3.15%. Notably, even under aggressive evidence-frame filtering up to 80%, CoFiE outperforms open-source frontier multimodal models while improving end-to-end inference latency by up to 2.54 × .

Abstract:
Large Language Models (LLMs) have advanced machine translation, yet long-text translation still faces a practical tension between document-level lexical consistency and inference cost. Recent document-level methods improve coherence by expanding context, maintaining multi-level memories, or injecting multi-granularity knowledge, but such broad context mechanisms often introduce substantial prompt redundancy and limit parallel inference. We propose NoBAT ( No un B ank A ugmented T ranslation), a lightweight framework that distills document-level lexical constraints into a compact global Noun Bank. NoBAT first constructs and disambiguates the Noun Bank serially, and then retrieves only relevant noun constraints for local, parallel translation. Experiments on IWSLT2017 and Novels-XL with multiple LLM backbones show that NoBAT consistently improves lexical translation consistency while maintaining competitive COMET scores. Compared with DELTA, NoBAT further reduces total token consumption by 6.3× , demonstrating a cost-effective trade-off among consistency, translation quality, and efficiency.

Abstract:
As large language models (LLMs) increasingly rely on web-scale corpora, membership inference attacks (MIAs) have become essential tools for auditing unauthorized data usage. However, traditional token-level MIAs fail to generalize to large reasoning models (LRMs) due to data distribution shifts introduced during fine-tuning. While recent studies address this by encoding reasoning traces into latent semantic spaces, they sacrifice the interpretability strictly required for real-world copyright verification. In this paper, we demonstrate that an LRM’s internal memory states are explicitly leaked through its stylistic word choices. Our analysis reveals a distinct lexical dichotomy: non-member traces tend toward a "guessing-like" tone, whereas member traces exhibit a "remember-like" style. Motivated by this interesting phenomenon, we propose TraceMIA, an interpretable MIA framework that audits training corpora by computing membership scores based on the relative frequencies of discriminative cue words. Extensive empirical evaluations across representative LRMs and diverse datasets demonstrate that TraceMIA achieves robust detection efficacy while providing the transparent, human-readable evidence critical for practical auditing scenarios.

Abstract:
Real-world decision-making often involves uncertainty expressed in linguistic rather than numerical terms, and Prospect Theory (PT) provides a classic framework for modeling human behavior under such uncertainty. Although recent studies have developed frameworks to estimate PT parameters for Large Language Models (LLMs), few have examined whether PT itself adequately describes LLM decision-making behavior. To address these gaps, we develop a streamlined workflow grounded in a classic behavioral economics experimental paradigm. First, we estimate PT parameters and evaluate how well the resulting model captures LLM decision-making behavior. We then derive probability mappings for epistemic markers in the same context and inject them into prompts to examine the stability of PT parameters under linguistic uncertainty. Our findings suggest that PT does not consistently provide a reliable account of LLM decision-making across models, and that its application to LLMs is likely sensitive to epistemic uncertainty. The findings caution against the deployment of PT-based frameworks in real-world applications where epistemic ambiguity is prevalent, giving valuable insights in behaviour interpretation and future alignment direction for LLM decision-making.

Abstract:
The protection of Intellectual Property (IP) for Large Language Models (LLMs) has become a critical concern as model theft and unauthorized commercialization escalate. While adversarial fingerprinting offers a promising black-box solution for ownership verification, existing methods suffer from significant limitations: they are fragile against downstream model modifications, sensitive to system prompt variations, and easily detectable due to high-perplexity input patterns. In this paper, we propose SRAF , a stealthy and robust adversarial fingerprinting framework. SRAF employs a synergistic joint optimization strategy across homologous model variants and diverse chat templates, forcing the fingerprint to anchor onto the invariant intrinsic comprehension features of the model family. Furthermore, we introduce a Perplexity Hiding technique that embeds adversarial perturbations within Markdown tables, effectively aligning the prompt’s statistics with natural language to evade perplexity-based detection. Extensive experiments across multiple model families and parameter scales demonstrate that SRAF significantly enhances robustness against fine-tuning, alignment, pruning, merging, and input perturbations while maintaining exceptional stealthiness and low false-positive rates, offering a practical and resilient black-box solution for LLM ownership verification.

Abstract:
Large language models can repeat an instruction while failing to keep obeying it. We study this recall–compliance dissociation in long-horizon dialogue and define State Integrity Collapse: a model preserves a stated constraint in memory but stops applying it to later turns. We introduce StateTrack-Bench, a controlled suite of conversations covering numerical, format, role/persona, factual-memory, and logical-rule constraints. Across seven provider-pinned model snapshots and five dialogue-depth probes, strict joint retention ranges from 62.4% to 100.0%, with degradation concentrated in symbolic constraints and in one especially brittle model. A forked recall audit shows that 88.6% of observed failures are behavioral-drift failures rather than ordinary forgetting; excluding the brittle outlier preserves this diagnosis (84.4% of failures). Extended evaluations at depths up to 50 turns confirm that the dissociation persists beyond the core benchmark, while Structured State Anchoring recovers up to 24.4 points on immutable constraints. The results show that reliability cannot be inferred from single-turn instruction following alone; evaluation must track whether remembered constraints remain operational as dialogue state grows. Code and artifacts are available at: https://github.com/xzzvsxd/StateTrack.

Abstract:
Reinforcement learning from verifiable rewards (RLVR) drives chain-of-thought reasoning in large language models, yet its binary outcome reward cannot distinguish among correct trajectories. Existing dense reward alternatives, from surface heuristics to process reward models, either ignore the expert solutions already present in training corpora or require expensive offline annotation. We propose Gradient-Aligned Reward (GAR) , which operates in the policy’s own gradient space: truncated backpropagation through the output projection layer extracts a compact gradient vector for each rollout, and cosine similarity with an expert-anchor gradient yields a dense, reasoning-aware reward with less than 9% wall-clock overhead. We prove that this cosine admits a multiplicative decomposition into prediction-error and activation-pattern factors, providing a concrete characterization of what the alignment signal measures. On Qwen3-4B and Qwen3-8B, GAR consistently improves over GRPO and other baselines on competition-level math benchmarks and transfers to GPQA Diamond and MMLU-Pro without domain-specific data. Code and data are available at https://github.com/LQgdwind/GAR .

Abstract:
Synthetic math data is widely used to scale LLM training, but its correctness is not guaranteed. We introduce MathDebugger, the first text-only, type-aware benchmark for jointly detecting and classifying errors in synthetic mathematical questions and answers. MathDebugger contains 2,000 correct questions, 2,000 erroneous questions (balanced across four fine-grained error types), and 2,000 annotated answers (610 erroneous, across three error types), each annotated with both a correctness label and an error-type label, with per-error-type Fleiss’ 𝜅 ranging from 0.69 to 0.88. On 14 SOTA LLMs (6 closed-source, 8 open-source, 1.5B–72B) and 3 process reward models, even advanced systems such as GPT-o3 and DeepSeek-R1 remain far from saturating the benchmark, particularly on fine-grained error-type detection; we further surface a solving-vs-verifying gap that causes math-tuned and long-chain-reasoning models to trail their general-purpose siblings. Error-type annotations also provide actionable supervision for correction, with gains that are significant under paired bootstrap ( p<0.01 ). Code and data: [https://anonymous.4open.science/r/MathDebugger-7C4A/README.md](https://anonymous.4open.science/r/MathDebugger-7C4A/README.md)

Abstract:
Large language models (LLMs) are increasingly used to communicate and explain scientific concepts, yet their tendency to hallucinate poses significant risks in this high stakes use-case. Prior scientific hallucination evaluation work remains largely restricted to the biomedical domain, treats hallucination as a binary task, and has not examined the growing family of scientifically fine-tuned LLMs. We address these gaps with SciFactCheck, a benchmark of 2,500 prompts across five scientific domains, paired with a modular evaluation framework targeting three factuality hallucination types: unverifiability, overclaim, and attribution. Using a controlled minimal-pairing design, we evaluate 18 LLMs by comparing each scientifically fine-tuned model against its general-purpose base. Our results indicate that 1. Scientifically fine-tuned models exhibit degraded factual reliability across all hallucination types and scientific domains, and 2. Fine-tuned models are internally less confident yet linguistically more assertive. A human pilot study further reveals that current fact-checking tools show only modest agreement with expert judgments on scientific content, and that defining scientifically check-worthy claims remains contested even among human annotators. Our findings fundamentally challenge current methods of domain-specific fine-tuning for factuality and call for developing improved verification infrastructure for scientific content.

Abstract:
Medical language models are increasingly evaluated on open-ended clinical reasoning, where good responses must combine factual recall, evidence integration, decision making, and synthesis. However, standard reward models compress these different standards into one reward score, while medical RL with answer keys mainly rewards multiple-choice correctness. This makes it difficult to improve non-multiple-choice medical reasoning without losing performance on exam-style benchmarks. In this paper, we propose a Prefix Mixture-of-Experts Reward Model (Prefix-MoE RM), which routes each query-response pair across recall, analysis, decision, and synthesis prefix experts, softly fuses their trainable prefixes, and then predicts one reward score. The MoE structure allows different abilities to use different scoring experts to reduce reward conflicts, while continuous prefixes provide a more expressive conditioning signal than fixed hard prompts. The reward model is trained on non-multiple-choice medical preferences and then used in mixed-data curriculum GRPO, where multiple-choice prompts provide additional answer-key feedback. On held-out reward-model evaluation, Prefix-MoE RM improves pairwise accuracy from 79.8 to 93.0 over a shared RM; with the RL pipeline fixed, replacing it with a shared RM or soft expert-head MoE lowers HealthBench from 43.8 to 24.8 and 34.6, respectively. With Qwen3-8B as the policy backbone, the full system outperforms the larger Qwen3-32B base on MedXpertQA and HealthBench, while the Qwen3-32B version reaches 50.1 HealthBench and 4.55 LLMEval-Med, remaining competitive with stronger closed general models and retaining an advantage on non-multiple-choice metrics.

Abstract:
VLM-driven self-improvement of web code has a structural flaw: the model that proposes the repair is the model that judges it, and visual plausibility under that judge is a poor proxy for whether the page actually works. What the loop is missing is a counterparty the VLM cannot fool, and the browser already is that counterparty: a deterministic, executable simulator of how an HTML artifact behaves under user actions, and in everything but name a world model for web code. We present WEBWORLD, the interface that lets a VLM prior interact with this browser-as-world-model autonomously and decides which interactions become supervision. Each round, the VLM emits a critique that the planner compiles into a typed interaction contract; the browser re-executes the candidate and issues an acceptance certificate only when both target progress and preservation of every previously verified capability hold; certified transitions accumulate as a quality ratchet that is the only thing the SFT export ever sees. Under matched training, WEBWORLD-27B improves Raw-27B by 5.3 points on HTMLBench-400 and 14.9 points on MiniAppBench-Val, and reaches the level of strong frontier systems such as Kimi-K2.6 and GPT-5.4 on interactive HTML generation. Equal-size ablations show that browser-backed admission carries the gain: without the certificate, the matched 9B lift nearly disappears.

Abstract:
Generative recommendation represents items as semantic identifiers (SIDs), i.e., sequences of discrete tokens, and typically predict the next item autoregressively. Despite their strong performance, such methods suffer from two limitations: (1) unidirectional constraints, where causal attention enforces left-to-right dependencies and limits global semantic modeling; and (2) error accumulation, where early mistakes propagate to later tokens and hurt recommendation accuracy. To address these issues, we propose LLaDA-Rec, a discrete diffusion framework that reformulates next-item recommendation as parallel SID generation. By combining bidirectional attention with an adaptive generation order, LLaDA-Rec captures richer inter-item and intra-item dependencies while reducing error accumulation. Our framework includes three key components: (1) a parallel tokenization mechanism for diffusion-compatible SIDs; (2) masking strategies at both the user-history and next-item levels; and (3) an adapted beam search algorithm for adaptive-order discrete diffusion decoding. Experiments on three public datasets show that LLaDA-Rec consistently outperforms both conventional ID-based recommenders and strong generative baselines, highlighting discrete diffusion as a promising paradigm for generative recommendation.

Abstract:
When an aligned language model declines a request, it almost always states a reason. Those reasons are read as evidence about the refusal, yet whether they name what actually triggered it has not been measured. We introduce a counterfactual audit of the cited reason, removing the reason-bearing feature to test necessity and inserting it into a benign prompt to test sufficiency. The two tests partition cited reasons into four kinds: faithful, over-cited, under-specified, and confabulated. We audit 22 models from nine developers on 840 prompts pooled from seven public safety benchmarks. Across the audited refusals, 57.2% cite a reason that fails at least one test, though faithful remains the largest category for 21 of 22 models. Over-citation is highest in the models that refuse most broadly. Paired sweeps over two families’ post-training checkpoints place its rise at supervised fine-tuning, not at later preference optimization. Probes on open-weight models tie these labels to internal representations. Category rates hold under a different-family judge, and the faithful-versus-over-cited ordering holds under an own-reason audit. We release the framework, the counterfactual variants, and per-model labels.

Abstract:
Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114 tasks across 8 real-world work domains, grounded in real user requests and publicly verifiable artifacts. Each task requires agents to combine GUI observations/actions with CLI/code operations within a single trajectory. We evaluate these tasks on a real Ubuntu desktop inside deployed CLI-agent runtimes, augmented with a minimal desktop-control plugin. We also propose a companion trajectory-aware judge that inspects deliverables, files, screenshots, logs, and action traces, while detecting shortcut behaviors such as fabricated visual evidence or hard-coded metrics. Across frontier model-runtime pairings, the best PassRate reaches only 41.2%, showing the benchmark remains far from saturated. The trajectory-aware judge further reveals that outcome-only grading substantially overestimates agent performance. Overall, WeaveBench exposes a critical gap in CUA evaluation and provides an effective testbed to measure whether agents can orchestrate GUI, CLI, and code operations across long-horizon real-world tasks.

Abstract:
Large language models (LLMs) are increasingly deployed as web-browsing agents, yet current benchmarks mostly evaluate explicit queries over clean factual evidence. We introduce Melonbench, a 665-item benchmark for coded entity resolution over noisy open-web user-generated content, instantiated with mid-to-long-tail public figures in Chinese entertainment blind-item discourse. Solving Melonbench requires agents to decode obfuscated clues, retrieve fragmented evidence, and identify the target entity. The benchmark is constructed through a five-stage semi-automatic pipeline with atomic event stratification, adversarial faithfulness auditing, and expert verification. Across 30 systems, the best non-search model reaches 25.1% accuracy and the best search-enabled system reaches 49.9%. Controlled comparisons of the same models with and without web search confirm consistent gains from web search, but also substantial remaining headroom. Diagnostic experiments identify semantic de-obfuscation and open-ended entity recall as key bottlenecks, and show that reasoning-centric search can outperform report-oriented agents. An 80-item finance pilot provides preliminary evidence of transfer beyond entertainment within a Chinese-language setting. Melonbench offers a challenging initial testbed for search agents under intentional obfuscation and noisy evidence. The dataset is publicly available at https://huggingface.co/datasets/janehaha/MelonBench.

Abstract:
Multimodal learning serves as a promising approach for applications with diverse information sources. However, there are numerous challenges when scaling up multimodal learning data from all modalities due to availability or varied cost of data collection. We formulate and study multimodal learning under limited data collection budgets as a modality selection problem, demonstrating that selectively collecting a subset of modalities and synthesizing the missing ones can match or outperform models trained with uniformly augmented full modalities. Our research problem is formulated as: given a limited data collection budget, how to find the appropriate modalities to collect new data and generate for the rest to maximize model performance gain? To answer this, we propose a new paradigm - Select the Key modality, then generate the rest to enable learning with limited data (SK-ll). SK-ll contains two essential components: (1) Select the key. We propose a modality importance indicator to find the optimal modalities by assessing their single modal marginal contribution and cross-modal interactions. (2) Generate the rest. We substitute with generated embeddings for the rest of modalities. We conducted extensive experiments applying SK-ll across affective computing, healthcare with diverse multimodal learning backbones, obtaining average accuracy gains of 2.37%, 6.55%, 6.73% on MOSI, MOSEI, ADNI respectively. Meanwhile, we present interesting empirical insights such as the data efficiency. Codes are provided in https://github.com/UNITES-Lab/SK-ll.

Abstract:
Precision healthcare, particularly for conditions like hypertension and cardiovascular disease, necessitates monitoring of dietary sodium intake. However, tracking this is hindered by the prevalence of hidden salt in cooking, such as sodium in soy sauce and ketchup. While recipes offer a valuable data source for dietary analysis, sodium-rich seasonings are frequently omitted or described ambiguously in instructions. To solve this issue, we propose SALT, a Sodium Assessing Level Tracking framework adopting an RAG framework to assess sodium content in recipes. Our framework first introduces a Joint-Factor Embedding Retrieval module to locate similar recipes with specified sodium content for addressing the lack of contextual references. These retrieved samples provide contexts for subsequent inference. Then we design a structured 4-hop Chain-of-Thought inference module to refine the vague estimation from language models through a multi-step sodium estimation. To facilitate our study, we further construct a recipe dataset SALT54k with 54,151 entries labeled with 11 sodium quantities across common seasonings. Results on SALT54k demonstrate that our method achieves state-of-the-art performance in sodium estimation. Additional real-world validations confirm the effectiveness of our method, demonstrating its potential as a practical solution for AI-assisted precision healthcare.

Abstract:
We identify a special group of token vectors inside large language models (LLMs) — which we term magnetic vectors — that organize the surrounding tokens by either attracting or repelling them. Particularly, tokens pointing the same way as an attracting magnet are elongated; tokens pointing the same way as a repelling magnet are compressed. Just as physical magnets pull or push away the iron filings around them, these vectors organize their surroundings through two opposing polarities. Moreover, we identify a statistically significant pattern in linguistic category where function words consistently act as repelling magnets in early layers, and we also find magnets consistently reorganize their polarities in unique ways deeper in the model. In a further case study we find this observation may unveil a deliberate, layer-wise organization in how LLMs process language. This pattern is consistent across different LLM architectures, sizes, and layer configurations. It is also causally relevant. When the LLM is fine-tuned for a downstream task, the task-functional tokens emerge as magnets. E.g., in question answering, the answer-span tokens become uniquely repelling magnets in the final layer, geometrically carving the answer out of the surrounding context. Furthermore, removing early-layer repelling magnets devastates syntactic tasks (POS tagging accuracy drops from 91% to below 10%) while sparing semantic ones, and removing late-layer attracting magnets does the reverse. We believe this phenomenon warrants further investigation, as it opens the first probe-free path to understanding how language models geometrically organize linguistic computation across their layers.

Abstract:
Simulating human conversations using large language models (LLMs) has emerged as a scalable methodology for modeling human social interaction. This paper reconsiders the evaluation of simulated conversations by explicitly recognizing that human conversations inherently involve inconsistent and uncollaborative behaviors, such as misunderstandings and interruptions. Since these behaviors contribute to the complexity of human social interaction, we argue that LLM-simulated conversations should reproduce them at frequencies comparable to those observed in human conversations. To support a detailed and interpretable evaluation of these behaviors, we introduce CoCoEval, a framework consisting of an evaluation scheme based on turn-level detection of 10 types of inconsistent and uncollaborative behaviors and a benchmark for simulating conversations in professional scenarios involving collaboration and conflict. Using CoCoEval, we compare human conversations with those simulated by GPT-4.1, GPT-5.1, and Claude Opus 4. The results show that (1) LLM-simulated conversations exhibit far fewer inconsistent and uncollaborative behaviors than human conversations under vanilla prompting, and (2) prompt engineering and supervised fine-tuning do not provide reliable control over these behaviors, often leading to the overproduction of specific behaviors. CoCoEval identifies gaps between human and LLM-simulated conversations that are not captured by conventional evaluation based on conversation-level Likert scales, raising concerns about the use of LLMs as proxies for human social interaction.

Abstract:
Vision-language models (VLMs), like human learners, are frequently exposed to new visual concepts, but how they map novel visual references to language after exposure remains largely underexplored, particularly when those references contradict prior knowledge from pre-training. To study this, we present the Novel Visual References Dataset (NVRD): 19,176 images spanning 90 visual concepts across different levels of visual novelty, each with up to 20 increasingly perturbed versions of the original object to probe generalization. Unlike prior work on visual augmentations of familiar concepts, NVRD comprises entirely novel, open-ended stimuli constructed from scratch, mirroring how humans encounter genuinely new concepts. We evaluate 3 open- and 2 closed-source models alongside 2,400 human judgments for direct human–model comparison, and find that (i) models struggle to acquire novel concepts in-context when they contradict prior knowledge, and (ii) while models and humans show correlated sensitivity to visual perturbations, models significantly overgeneralize, extending learned labels to stimuli that humans reject. We contribute NVRD as a corpus and benchmark for research on visual concept learning in both humans and machines.

Abstract:
Students’ wrong answers in science assessments can reveal plausible misconceptions, yet standard LLM benchmarks usually evaluate only answer correctness. We introduce MisconceptionBench, a benchmark for evaluating whether LLMs can reason about misconception-linked distractors and empirical student response patterns. It contains 761 K-12 science items, 1,474 expert-annotated distractors, and response distributions from more than 150,000 students. We evaluate eight LLMs on three tasks: misconception matching, misconception generation, and student response distribution prediction. In matching, text-similarity baselines remain competitive in two- and three-candidate settings; clear LLM gains emerge only with four candidates, where GPT-4o and Llama-3.3-70B significantly outperform TF-IDF and MiniLM. In generation, the strongest models are tightly clustered, with Hit@1 of 38.5-39.6%, while retrieval-only baselines recover many targets without an LLM. Response distribution prediction remains difficult: no model exceeds F1=0.21 for identifying grade-band differences. Overall, misconception-centered reasoning requires evaluation beyond answer accuracy.

Abstract:
Vision Language Models (VLMs) have demonstrated remarkable capabilities in multimodal reasoning tasks, yet they still suffer from recurring failures, such as skipping key visual checks, misapplying domain rules, and hallucinating unsupported concepts. Most existing solutions rely on supervised fine-tuning (SFT) and reinforcement learning (RL), which are expensive to iterate and can be brittle under distribution shift. To this end, we propose M ultimodal M istake Note book Learning (M 2 Note), a training-free continual evolution framework that externalizes learning into an editable memory. M 2 Note transforms failed trajectories into compact subject–guidance notes: the subject summarizes the underlying domain and concept, while the guidance provides actionable verification steps that can be reused in future inference. At test time, M 2 Note retrieves relevant notes via multimodal retrieval-augmented generation (RAG) and appends them to the model context, steering reasoning away from previously observed pitfalls. To stabilize continual evolution, we adopt batch-level post-verification with rollback, which commits notebook edits only if they improve performance on the same batch, reducing noisy updates and preventing regressions. M 2 Note supports both self-evolving , where the same VLM acts as solver and supervisor, and cross-model evolving , where a stronger supervisor guides a weaker solver, enabling capability transfer without weight updates. Experiments on six multimodal reasoning benchmarks show consistent improvements across domains and backbones, while achieving strong cost and sample efficiency and remaining complementary to Chain-of-Thought (CoT) prompting. The code is available at https://github.com/AMAP-ML/M2Note.

Abstract:
Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts. However, progress in evaluating this capability remains limited by the lack of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. We introduce XCR-Bench, a Cross (X) - C ultural R easoning Bench mark containing 4.1k parallel sentences and 1,098 CSIs across three reasoning tasks. XCR-Bench integrates Newmark’s CSI framework with Hall’s Triad of Culture, enabling evaluation across levels of cultural visibility: from observable practices to implicit social norms and values. Experiments on eight multilingual LLMs show that state-of-the-art models exhibit consistent weaknesses in identifying and adapting specific categories of CSIs, revealing a gap between surface-level recall and explicit cultural reasoning. Performance declines significantly in culturally sensitive categories and at deeper cultural levels ( p<0.005 , 8/8 models). We further find that adaptation quality varies systematically across target cultures and across Bengali regional variants, revealing systematic regional lexical and cultural preference patterns and ethno-religious biases in model outputs, even within a single linguistic setting. We publicly release the corpus and code to support future research on cross-cultural NLP.

Abstract:
Large language models (LLMs) are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. Individual extraction queries often resemble benign requests, while existing evaluations often focus on single-query anomaly scoring or pure benign-versus-attacker user settings. We formulate model extraction monitoring as benign-calibrated traffic-window distribution testing: embed incoming queries into a semantic space and test whether their aggregate distribution deviates from historical benign traffic. We instantiate this formulation with maximum mean discrepancy (MMD), using only benign-vs-benign comparisons to set the decision threshold. We evaluate on fourteen attacker-normal query pairs from four extraction scenarios and compare with adapted PRADA, SEAT, CAP, DATE, marginal Mahalanobis, and pseudo-class energy baselines. Across three random seeds, MMD achieves 0.3% benign FPR, 100.0% pure-attacker TPR, 90.5% average TPR over attacker fractions, and 95.1% balanced accuracy. These results show that benign-calibrated distribution testing is a strong empirical baseline for model extraction detection in both user-level and mixed multi-user LLM API traffic.

Abstract:
Automatic fact verification (FV) predicts the veracity of a claim against retrieved evidence, yet modern verifiers often behave as shortcut-driven verifiers, predicting from surface regularities in the claim rather than the factual content of the evidence. Existing debiasing methods over-correct this by uniformly enforcing evidence reliance on every sample, thereby suppressing the valid information already carried by the claim itself. We propose Evidence-Grounded Remixing for Fact Verification (EGR-FV), which estimates a per-sample evidence necessity score by contrasting claim-with-evidence and claim-only predictions, and uses it to remix training batches so that debiasing is adapted to each sample’s actual reliance on evidence. On FEVER, PolitiHop, and HOVER together with their symmetric counterparts, EGR-FV achieves state-of-the-art accuracy and robustness, validating the effectiveness of EGR-FV.

Abstract:
We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, complex editing, and visual understanding. VectorGym addresses the lack of realistic, challenging benchmarks aligned with professional design workflows. Our benchmark comprises four tasks with expert human-authored annotations: the novel Sketch2SVG task (VG-Sketch); a new SVG editing dataset (VG-Edit) featuring complex, multi-step edits with higher-order primitives; Text2SVG generation (VG-Text); and SVG captioning (VG-Cap). Unlike prior benchmarks that rely on synthetic edits, VectorGym provides gold-standard human annotations that require semantic understanding and design intent. We also provide a multi-task reinforcement learning baseline that jointly optimizes across all four tasks using rendering-based rewards. This baseline, built on GRPO with curriculum learning, trains a Qwen3-VL 8B model that achieves state-of-the-art performance among open-source models, surpassing much larger models including Qwen3-VL 235B and matching GPT-4o. We also introduce a VLM-as-a-Judge metric for SVG generation, validated through human correlation studies. Our evaluation of frontier VLMs reveals significant performance gaps, positioning VectorGym as a rigorous framework for advancing visual code generation.

Abstract:
Large Language Models (LLMs) have shown remarkable proficiency on general-purpose tasks, yet their performance often degrades in highly-specialized technical domains. Moreover, little is known about how parametric knowledge of domain-specific terms is encoded within these models. We address this gap by contributing two novel medical jargon evaluation benchmarks and evaluate a general-purpose Llama-3.1 model against a variant fine-tuned on medical-domain data. Surprisingly, the general-purpose model outperforms the medically fine-tuned model on both tasks. Using mechanistic interpretability tools, we find systematic patterns of miscalibration for the medically fine-tuned model. Instead of reorganizing parametric knowledge, the finetuned model places greater emphasis on a small subset of model components associated with jargon-favoring predictions. We find that applying component reweighting strategies against the benchmark tasks successfully suppresses these components and closes the gap with the general-purpose baseline. We also observe that some jargon-sensitive components transfer knowledge to the same tasks involving materials science jargon, suggesting they encode a partially domain-agnostic notion of specialized terminology. Our results provide a case study in which a medically fine-tuned checkpoint does not improve jargon comprehension over its general-purpose counterpart, highlighting that domain adaptation should not be assumed to yield better performance on specialized terminology.

Abstract:
MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage strong instruction following and semantic understanding. However, by casting localization as coordinate-token prediction, these methods rely on implicit decoding to convert region-level target evidence into precise click coordinates, making point-level spatial alignment unreliable. Through diagnostic analysis, we find that intermediate decoder layers already exhibit target-region awareness, yet such region-level evidence fails to guide final point-level decisions. To address this, we propose InnerZoom, a single-forward cross-layer evidence bridging framework that refines target evidence inside the model during decoding, avoiding extra inferences required by Zoom-In. InnerZoom reframes coordinate decoding as an evidence-preserving process by identifying target-related cues within the original forward pass, refining them across decoder layers, and reinjecting the refined evidence into coordinate prediction. Extensive experiments show that InnerZoom-4B achieves state-of-the-art performance on all six mainstream GUI grounding benchmarks, outperforming the second-best result by 4.1 points on OSWorld-G.Under the 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass Zoom-In by 1.3 points on average, while reducing measured GPU inference latency by 44% and estimated TFLOPs by 29%. Code and models will be at https://github.com/YenanLiu/InnerZoom.

Abstract:
Frozen small language model (SLM) readers in retrieval-augmented generation (RAG) can fail even after retrieval has found answer-bearing evidence: on five LongBench tasks with LFM2.5-Instruct 1.2B, Raw RAG with its default prompt converts only 43% of answer-bearing predictions into exact matches. We treat frozen SLM RAG as a two-stage problem—copy-preserving evidence compression and conversion-aware deployment calibration—and instantiate each stage with one system. PEC-HOP is a non-parametric pointer-preserved evidence compressor; the Minimal Copy-preserving Evidence Compass (MCE-Compass) is a dev-time calibrator that selects an (interface, prompt) pair per (reader, task) cell, calibrated offline from ~50 labeled examples per cell for fixed deployments. In a matched-prompt head-to-head on three multi-hop LongBench tasks × three frozen readers, PEC-HOP is Pareto-best among compressors, with +4.28 pp macro EM over LLMLingua-2 and +1.61 pp over Provence at 2.8–4.8× faster median latency and no trained compressor on the critical path. Across five tasks, MCE-Compass improves macro EM over Raw RAG by +9.36, +15.39, and +2.39 pp on LFM2.5-Instruct 1.2B, Qwen3-4B, and Gemma-4-E4B at 2.02×, 5.69×, and 1.34× latency speedups, and beats the single globally dev-selected policy. A 20-partition dev-size sweep and a 66,581-article HotpotQA retrieval stress test indicate the gains are not single-split or bundled-context artifacts.

Abstract:
Large language models have become increasingly capable agents for low-level code and kernel optimization, but isolated kernel benchmarks provide only a proxy for the deployment behavior that matters in language-model inference. We identify a benchmark-to-deployment gap: candidate kernels that appear correct and fast in standalone harnesses can exhibit different performance, safety, or phase behavior after integration into a real inference workload. We introduce LLM4LLM, a deployment-aware closed-loop optimization framework that starts from a target inference script, extracts phase-aware optimization tasks, searches with an experience-guided episodic agent, and accepts patches through in-model validation. Across ten language-model inference workloads on A100 and H100 GPUs, LLM4LLM improves end-to-end latency for every evaluated model, achieving 3.91x/6.98x geometric-mean speedups on A100/H100; as supporting kernel-level evidence, it also attains up to 2.745 × GeoMean speedup on KernelBench Level 2.

Abstract:
Large language models (LLMs) exhibit strong English performance due to its dominance in pretraining data, yet remain inconsistent across non-English languages. Existing cross-lingual alignment methods rely on contrastive objectives that symmetrically perturb English representations, inadvertently degrading the proficiency they use as a reference. We propose an Optimal Transport (OT) framework that maps non-English hidden-state distributions toward English at middle transformer layers via entropy-regularised Sinkhorn iterations, while an English consistency regularisation term keeps the alignment anchor stationary. This asymmetric formulation preserves English performance by construction. Experiments on XNLI, XQuAD, and MMMLU show that our method delivers strong cross-lingual transfer while successfully avoiding the English performance degradation observed in the contrastive baseline.

Abstract:
Multimodal large language models (MLLMs) perform well across many tasks but often exhibit Western bias, limiting their effectiveness in regions such as Southeast Asia (SEA). In this work, we study how both evaluation and model design can be better adapted to the linguistic and cultural characteristics of this region. We present the SEA-Omni Benchmark Suite, which brings together a set of SEA-focused image and video datasets capturing phenomena such as intra-sentential code-switching and culturally grounded multimodal cues. Alongside these benchmarks, we develop MERaLiON-Omni-10B, an omni-perception model trained with a progressive, data-efficient pipeline that moves from vision-language alignment to more advanced multimodal fusion. To better account for region-specific cultural signals, we study text-mediated cultural alignment in SEA multimodal settings using GRPO to surface culturally distinctive responses that are often suppressed by generic decoding strategies. We introduce a fact-anchored dual-reward mechanism to balance cultural richness with multimodal consistency in which cultural density is measured via localized knowledge entities of generated texts, and multimodal coherence is assessed against silver textual descriptions of the multimodal input. Experimental results indicate that the resulting model performs strongly on SEA benchmarks while maintaining reasonable general capabilities.

Abstract:
Multi-agent systems (MAS) have substantially advanced autonomous software engineering (SWE), but their growing inference energy demands raise sustainability concerns. In this paper, we demonstrate that this cost is concentrated in an overlooked source: redundant output tokens generated across agents. Two empirical findings ground this claim. First, our per-token energy attribution for MAS reveals a sharp asymmetry: an output token consumes 30 to 1,000 times more energy than an input or cached token. Second, MAS inflate per-episode output because agents repeatedly re-explore overlapping repository regions. To address this inefficiency, we propose Librarian, a persistent search sub-agent that tracks repository-search history and suppresses redundant exploration actions across agents. By returning short references to file regions instead of full file excerpts, Librarian further reduces output-token volume. On SWE-Bench Verified and SWE-Bench-Live, Librarian reduces per-episode GPU energy consumption of existing multi-agent SWE systems by 11 to 30% while preserving task performance. Our code is available at https://github.com/ml-postech/Librarian.

Abstract:
Knowledge editing updates facts in large language models while preserving unrelated knowledge. Most prior work studies object editing, which modifies the object under a fixed subject–relation pair. We instead study relation replacement editing, where an obsolete relation must be suppressed as a new one is introduced. We systematically study this setting with a dedicated benchmark and method. We introduce ReEditBench to evaluate replacement rather than association addition. Existing editors exhibit two major failures: obsolete-relation retention remains as high as 98.20% after successful edits, and difficult samples often destabilize editing. To address this, we propose R 2 Edit , a two-stage framework for relation replacement. Its target-smoothed Forget stage reduces obsolete-relation activation without fixed/random pseudo-target bias, while its curriculum-guided Edit stage progressively injects new relations from easy to hard samples. Across multiple LLMs, R 2 Edit improves replacement success, reduces retention, and preserves generalization. Stress tests on difficult subsets further demonstrate the robustness of the Edit stage. Code is available at: https://github.com/caoxu0109/R2Edit .

Abstract:
Language models (LMs) are often used as a tool to model human language processing. Recent studies suggest that severely restricting LMs’ context window improves their fit to human psycholinguistic data by simulating human working memory constraints. However, it is possible that this strict memory-decay approach overlooks humans’ reliance on long-range structural representations, such as discourse structre. In this work, we systematically vary the context window size of GPT-2 across four large-scale naturalistic English reading-time datasets and observe a U-shaped relationship: Although restricted contexts ( < 20 tokens) successfully capture local memory limitations, expanded contexts (500–1,000 tokens) ultimately yield the highest overall psycholinguistic fit. To investigate the mechanism driving this benefit, we conduct a counterfactual inference-time experiment that disrupts cross-sentential entity chains by pronominalizing repeated discourse entities. Obscuring these structural linkages significantly degrades the predictive power of larger context windows by 20% to 40%. Our experiments demonstrate that tracking long-range coreference relations is one important factor for the alignment between LM surprisal and human reading behavior, and approximate the extent to which human comprehenders use global discourse relations during language processing.

Abstract:
Large Language Models (LLMs) have been extensively used across diverse domains, including virtual assistants, automated code generation, and scientific research. However, they remain vulnerable to jailbreak attacks, which manipulate the models into generating harmful responses despite safety alignment. Recent studies have shown that current safety-aligned LLMs undergo shallow safety alignment. In this work, we conduct an in-depth investigation into the underlying mechanism of this phenomenon and reveal that it manifests through learned ”safety trigger tokens” that activate the model’s safety patterns when paired with the specific input. Through both analysis and empirical verification, we further demonstrate the high similarity of the safety trigger tokens across different harmful inputs. Accordingly, we propose D-STT, a simple yet effective defense algorithm that identifies and explicitly decodes safety trigger tokens of the given safety-aligned LLM to activate the model’s learned safety patterns. In this process, the safety trigger is constrained to a single token, which effectively preserves model usability by introducing minimal intervention in the decoding process. Extensive experiments across diverse jailbreak attacks and benign prompts demonstrate that D-STT substantially reduces output harmfulness while preserving model usability and incurring negligible response time overhead, demonstrating highly competitive and consistent performance across diverse models compared to ten baseline methods.

Abstract:
Prompted knowledge cutoff instructs a large language model (LLM) to act as if information beyond a specified cutoff date were unavailable. However, prior work mainly relies on direct-answer generation, which struggles when post-cutoff knowledge is not explicitly queried but is only causally related to the question. To address this limitation, we propose two recall-based prompting strategies: Self-Recall (SR), which asks the model to restate its cutoff constraint, and Question-Recall (QR), which requires the model to recall question-relevant information valid under the cutoff. Across three existing benchmarks, our methods outperform both direct-answer prompting and conventional step-by-step reasoning baselines, with particularly strong improvements on counterfactual questions. To investigate robustness across different cutoff settings, we further construct the Multi-cutoff Historical Event Benchmark (MHEB), which evaluates the same question under multiple cutoff years. Results show that knowledge cutoff performance varies with cutoff distance, while combining SR and QR consistently yields the best performance. Code and dataset are available at [https://github.com/isct-lrlab/MHEB](https://github.com/isct-lrlab/MHEB).

Abstract:
Long-context LLMs can now ingest entire online discussion threads, but understanding their social discourse requires more than reading a long document: models must track parent–reply relations, turning points, scoped subtrees, cross-branch contrasts, and participant trajectories. To test this structure-aware social reasoning, we introduce LongSocialBench, a benchmark of 1,462 verified human-authored multiple-choice items drawn from 94 complete Hacker News, Stack Exchange, and Reddit r/ChangeMyView episodes, with a median length of ≈73K tokens. Each item pairs a complete serialized discussion and reply structure with a four-option question, requiring models to recover structured social evidence. Released items are verified for answerability, option uniqueness, and evidence grounding. Across 18 models and 29 evaluation settings, current long-context workflows remain far below human performance. The best individual result comes from Claude-Opus-4.7, which reaches 63.0% when prompted to eliminate incorrect options before answering, compared with 72.4% for independent human readers. Averaged across all 18 models, the full-context Baseline scores 43.9%. Supplying the gold evidence scope raises this to 55.0%, showing that substantial errors remain even after the relevant thread region is identified. LongSocialBench shows that the missing capability is not context access or prompting alone, but social understanding over structured reply trees.

Abstract:
Moral language plays a central role in shaping online endorsement and the diffusion of information, yet existing moral foundation detection systems often suffer from poor cross-domain generalization, weak rationale grounding, and reliance on costly prompting-based large language models (LLMs). We introduce CHARM, a MAC and Hate-speech-Aware Rationale-aligned Moral foundation detection framework built on a lightweight fine-tuned LLM, which integrates complementary moral grounding, rationale alignment, and polarity-aware hate speech signals to support more robust and faithful moral prediction. Unlike prior dictionary-, fine-tune-, or prompt-based detectors, which decouple computation from psychological theory, CHARM is built so that each component — MAC cross-attention, rationale alignment, and hate-speech modulation — operationalises a distinct psychological construct. Using only 30% of the training data required by prior fine-tuned baselines, CHARM improves AUC by up to 15.3% in-domain, surpasses these baselines on every out-of-domain dataset in both AUC and F1, and offers a scalable, low-cost alternative to LLM-based detectors. We further apply CHARM to large-scale COVID-19 discourse on Twitter and show that moral value alignment is strongly associated with online endorsement behavior. By making moral framing measurable at scale, CHARM offers a practical tool for studying the spread of morally charged misinformation.

Abstract:
Compressing long chain-of-thought (CoT) from large language models (LLMs) is an emerging strategy to improve reasoning efficiency. Despite its promising benefits, existing studies equally compress all thoughts within a long CoT, hindering more concise and effective reasoning. To this end, we first investigate the importance of different thoughts through automatic long CoT chunking and Monte Carlo rollouts. Building on this, we propose a theoretically bounded metric to jointly measure the effectiveness and efficiency of different thoughts. We then propose Long Short, an efficient reasoning framework that enables two LLMs to collaboratively solve problems: a long-thought LLM for effectively generating important thoughts, and a short-thought LLM for efficiently generating remaining thoughts. Specifically, we synthesize cold-start data to fine-tune LLMs for long-thought and short-thought reasoning styles, respectively. Furthermore, we propose synergizing-oriented multi-turn reinforcement learning, focusing on model self-evolution and collaboration between long-thought and short-thought LLMs. Experimental results show that our method outperforms seven efficient reasoning methods and enables Qwen2.5-7B to achieve comparable performance to DeepSeek-R1-Distill-Qwen-7B, while reducing token length by over 80% across five reasoning benchmarks. Our code and data are available at https://github.com/usail-hkust/LongShort.

Abstract:
Most testbeds for omni-modal models assess multimodal understanding via textual outputs, leaving it unclear whether these models can properly speak their answers. To study this, we introduce OmniACBench, a benchmark for evaluating context-grounded acoustic control in omni-modal models. Given a spoken instruction, a text script, and an image, a model must read the script aloud with an appropriate tone and manner. OmniACBench comprises 3,559 verified instances covering six acoustic features: speech rate, phonation, pronunciation, emotion, global accent, and timbre. Extensive experiments on eight models reveal their limitations in the proposed setting, despite their strong performance on prior textual-output evaluations. Our analyses show that the main bottleneck lies not in processing individual modalities, but in integrating multimodal context for faithful speech generation. Moreover, we identify three common failure modes—weak direct control, failed implicit inference, and failed multimodal grounding—providing insights for developing models that can verbalize responses effectively.

Abstract:
Detecting harmful memes is critical for maintaining safe online communities. However, harmful intent is often implicit, arising from visual–textual incongruity and cultural stereotypes, which challenges existing multimodal detectors. We propose SyRHM, a framework that decomposes harmful meme detection into meaning-grounded retrieval and symbolic-language-enhanced multi-stage reasoning. SyRHM retrieves semantically related memes by parsing multimodal content into textual elements and description, providing grounded context beyond surface-level similarity. Building on the retrieved context, SyRHM uses a translator stage to convert multimodal inputs into symbolic intermediate representations, and then performs multi-stage reasoning via planner and solver stages, enabling expressive and interpretable analysis of harmful intent. Experiments on FHM, HarM, and MultiOff show that SyRHM consistently outperforms multimodal and reasoning-based baselines, while providing reasoning traces for harmful content. The code is available at: https://anonymous.4open.science/r/SyRHM-CC46

Abstract:
RAG-based question-answering (QA) in specialist domains faces a cold-start problem: no evaluative benchmark and no labeled data for post-training. We present DoRA (Domain-oriented RAG Assessment), a benchmark construction and evaluation framework that needs only a small set of specialist-domain documents, integrating and expert-validating otherwise established components for this cold-start setting. DoRA generates synthetic QA training and evaluation sets with auditable evidence across five domain-specific intents. To mitigate same-pipeline circularity, its training and test splits use different LLM families (Claude Sonnet for training; GPT-4o for test) drawn from disjoint seed-document corpora. Instantiated on 40 English-language defense documents, DoRA yields approximately 6.6K curated instances. Against 7 LLM baselines over a 1,259-sample benchmark, a LoRA-adapted Llama-3.1-8B trained on the synthetic split improves all 6 coverage and faithfulness metrics—each gain significant under a paired bootstrap test—and reduces hallucination by more than half under the default GTE retrieval setting, with gains persisting across alternative retrievers, prompting-based baselines, and matched answer-length budgets. Defense experts validate the synthetic QA, calibrate the LLM judge, and supply a fully human-written test set. We position DoRA as a practical framework for specialist-domain RAG under domain shift, with defense as a high-stakes case study.

Abstract:
Memes convey implicit pragmatic meanings through tightly coupled visual and textual cues, making their interpretation more challenging than that of conventional image-text pairs. Although large vision-language models (LVLMs) perform well on multimodal benchmarks, it remains unclear whether they infer non-literal mechanisms underlying memes, such as irony, wordplay, or rely mainly on superficial visual-textual correlations. To study this problem, we introduce MemeScope, a fine-grained benchmark of 1,781 memes annotated with eight pragmatic mechanisms and stage-wise explanations covering visual grounding, cross-modal alignment, and pragmatic inference. Building on MemeScope, we propose a unified interpretability framework that combines layer-wise probing with hidden-state patching to examine both the emergence and the causal contributions of meme-level pragmatic representations within LVLMs. Our analysis shows that literal visual and textual cues are often captured in earlier layers, whereas pragmatic mechanism information emerges more strongly in higher layers and shows stronger patching effects in late-layer multimodal representations. We further identify a recurring pragmatic drift, where models ground meme content correctly but move toward plausible yet incorrect mechanisms during deeper decoding.

Abstract:
Confidence calibration, the alignment of a model’s predicted confidence with its actual accuracy, is critical for the reliable deployment of Large Language Models (LLMs), yet remains under-explored in multilingual contexts. We conduct the first large-scale systematic study of multilingual calibration across eight model families and over 100 languages, finding that non-English languages are worse-calibrated than English (e.g., LLaMA3’s average Expected Calibration Error is 23.1% on non-English against 4.6% on English). To diagnose this, we investigate the model’s internal representations and find that the final layer, biased by English-centric training, provides a poor signal for multilingual confidence, whereas late-intermediate layers offer a better-calibrated signal. Building on this, we introduce a suite of training-free methods, including Language-Aware Confidence Ensemble (LACE), which adaptively selects an ensemble of layers for each language. Both the multilingual calibration gap and the late-intermediate peak persist beyond MCQA, in open-ended QA, chain-of-thought reasoning, and alternative MCQA scoring protocols. Our study highlights the costs of English-centric alignment and offers a path toward more globally equitable LLMs.

Abstract:
Semantic ID-based generative retrieval represents each item with a short sequence of discrete tokens derived from language-based item representations. Existing methods mainly instantiate this idea in two paradigms: autoregressive SID generation and parallel SID generation. In both paradigms, the quality of Semantic IDs depends critically on how continuous item embeddings are discretized before they become generation targets. While existing work has improved SID tokenization through better quantization objectives and supervision signals, it often treats the pre-tokenization embedding space as fixed, overlooking that pretrained item embeddings can be anisotropic and dominated by a few directions. This creates a pre-quantization distribution mismatch, where the global embedding geometry may not align with the structured spaces used for code assignment. We propose GeoSID, a tokenizer-side geometric shaping framework that keeps the downstream generative recommender unchanged. GeoSID uses a lightweight projected distribution matching regularizer to adjust item representations before SID assignment, and places the shaping objective in tokenizer-specific quantization spaces, including residual levels for residual quantization and OPQ/PQ blocks for product quantization. On the RPG backbone, GeoSID achieves up to 8.13% relative improvement in Recall@10 and 6.91% in NDCG@10. Further analyses show that GeoSID reduces projected distribution discrepancy, improves code usage balance, and lowers prefix-level SID collisions. Code is available at https://github.com/hexinrong122/GeoSID.

Abstract:
Image safety classifiers serve as a critical component of contemporary content moderation systems on the internet. However, their resilience against user-style malicious image editing remains underexplored. Such behaviors are highly prevalent in daily scenarios but difficult to fully reproduce. To explore this vulnerability, we introduce RedEdit, a novel black-box red-teaming agent that formulates photo-editing evasion as a combinatorial search problem over edit-tool sequences. It adopts a Vision-Language-Model (VLM)-based proposer to generate semantically targeted candidate edits and a Monte Carlo Tree Search (MCTS) planner to prioritize promising edit paths while backtracking from ineffective ones. Together, the proposer and planner instantiate two key capabilities of human attackers, i.e., domain knowledge and iterative backtracking, respectively, to reproduce this practical threat. Our extensive experiments on UnsafeBench reveal profound systemic vulnerabilities: fewer than two edits on average enable 76.2% of unsafe images to evade detectors, while retaining 93.0% malicious semantics, meaning that such manipulated content remains perceptually malicious to humans while easily bypassing automated moderation. We therefore appeal to the community for more attention to this overlooked practical threat.

Abstract:
Large language models (LLMs) possess extensive world knowledge and have demonstrated strong potential for misinformation detection. A growing line of recent work prompts LLMs to produce structured reasoning before issuing a veracity judgment. However, we contend that such structured outputs do not spontaneously support the veracity decision. The emitted scores can satisfy the output schema while contributing little to veracity discrimination. Since annotating full reasoning paths is prohibitively costly for veracity tasks, we propose a Generation-Score Consistency framework that supervises through structured dimension scores and steers the LLM toward functional reasoning. Specifically, we train a compact score-to-label verifier on a small set of expert-annotated reasoning data, freeze it, and during supervised fine-tuning use a trainable score head to align the score-token hidden states of the LLM with the human-calibrated mapping. Introducing structured reasoning on its own costs accuracy relative to label-only supervision, while the consistency term recovers that cost and keeps the emitted scores coupled to the veracity decision. Controlled experiments on multimodal misinformation benchmarks show that the consistency term strengthens both veracity prediction and the discriminability of the emitted scores. Code and data are available at https://github.com/yun-nlp/Reasoning-Scores-for-Misinformation-Detection-EMNLP2026.

Abstract:
Radiology, with its heterogeneous modalities, anatomies, and evolving protocols, is a natural yet high-stakes testbed for agentic AI. As LLMs grow more capable and tool ecosystems such as MCP and Agent Skills more complex, their capability boundaries in radiology remain unclear. We introduce RadA-BenchPlat , a two-layer benchmark spanning idealized reasoning and real-world execution: a synthetic layer with 2.2k clinician-verified records, 24.2k QA pairs, and 10 tool categories under diverse availability settings; and a real-environment layer pairing 165 2D/3D cases with executable tools for grounding, diagnosis, and report generation. We find that high completion rates under idealized settings do not carry over to real environments, where the best agents trail their upper bound by ∼ 15% and drop to 0.327 on long-chain tasks. Prompting strategies (few-shot exemplars, multi-agent coordination) and AutoTB (on-the-fly synthesis of missing tools) mitigate execution drift and breakdowns in difficult scenarios—lifting execution success to 94.5%—yet remain not fully reliable. Overall, our findings suggest that radiology agents are no longer limited to conceptual demonstrations, but are beginning to show potential for real-world applications.

Abstract:
Medication recommendation predicts medications for patient visits, but existing methods still face two key challenges. At the model level, traditional drug recommendation methods only predict structured drug codes with limited evidence grounding, while LLM agents can use richer clinical context but may lack safety verification and traceability. At the task level, existing benchmarks often use broad medication categories, which ignore subgroup-level safety differences and can lead to risk overestimation. We introduce the first fine-grained medication recommendation setting based on fourth-level ATC code generation. We propose Safe Prescription Agent (SafeRx-Agent) , a knowledge-grounded multi-agent framework that uses patient context, external clinical knowledge, and safety verification to recommend traceable medication sets. Experimental results on MIMIC-III and MIMIC-IV datasets show that SafeRx-Agent improves fine-grained medication prediction accuracy while controlling drug interactions, contraindications, and medication set size. Code accompanying this work is available at https://github.com/HWu3150/SafeRx-Agent/tree/main.

Abstract:
We study why continuous diffusion language models (DLMs) have lagged behind discrete diffusion approaches despite their appealing continuous generative dynamics. Under a controlled token-recovery study, we identify token rounding, the final projection from denoised embeddings to tokens, as a primary bottleneck. Building on these insights, we propose CoDAR (Continuous Diffusion with Contextual AutoRegressive Decoder), a two-stage framework that keeps diffusion entirely continuous in an embedding space while learning a strong, context-conditional discretizer: an autoregressive Transformer decoder that cross-attends to the denoised embedding sequence and performs contextualized rounding to tokens. Experiments on LM1B and OpenWebText demonstrate that CoDAR substantially improves generation quality over latent diffusion and becomes competitive with strong discrete DLMs, while exposing a simple decoder-temperature knob to navigate the fluency–diversity trade off.

Abstract:
Large language models (LLMs) have advanced the development of personalized learning in education. However, their inherent generation mechanisms often produce homogeneous responses to identical prompts. This “one-size-fits-all” mechanism overlooks the substantial heterogeneity in students’ cognitive and psychological characteristics, thereby posing potential safety risks to vulnerable groups. Existing safety evaluations primarily rely on context-independent metrics such as factual accuracy, bias, or toxicity, which fail to capture the divergent harms that the same response might cause across different student attributes. To address this gap, we propose the concept of “Student-Tailored Personalized Safety” and construct CASTLE based on educational theories. This benchmark covers 15 educational safety risks and 14 student attributes, comprising 92,908 bilingual scenarios. We further design three evaluation metrics: Risk Sensitivity, measuring the model’s ability to detect risks; Emotional Empathy, evaluating the model’s capacity to recognize student states; and Student Alignment, assessing the match between model responses and student attributes. Experiments on 18 LLMs demonstrate that CASTLE poses a significant challenge: all models scored below an average safety score of 2.5 out of 5, indicating substantial deficiencies in personalized safety assurance. The code is available at https://github.com/ECNU-RAIL/CASTLE-EMNLP2026.

Abstract:
VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated dataset for evaluating egocentric assistants on understanding and decision tasks in five dialogue-video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.

Abstract:
Vision-Language-Action (VLA) models have shown promising progress in language-conditioned robotic manipulation. However, existing datasets and benchmarks mainly evaluate task completion under predefined settings, offering limited insight into model reasoning under increasing spatial and procedural complexity. We introduce RoboSPA ( Robo t S patial- P rocedural A ssessment), a large-scale robotic manipulation dataset and benchmark for diagnosing embodied reasoning in VLA models. RoboSPA focuses on two core dimensions, Fine-Grained Spatial Reasoning and Long-Horizon Procedural Planning, covering 10 task categories and 56 base tasks. Each task is instantiated across five difficulty levels, yielding 280 variants with increasing spatial ambiguity and procedural complexity. We collect 527K trajectories across multiple embodiments and diverse scenes. Beyond binary success rate, RoboSPA introduces diagnostic metrics for more detailed evaluation. Experiments on representative VLA models show that current systems still struggle with complex spatial relations, precise low-level execution, and memory-intensive planning. These results establish RoboSPA as a challenging diagnostic benchmark for developing more capable, reliable, and generalizable embodied agents. Our data and code are available at https://github.com/fanzhenxuan/RoboSPA .

Abstract:
Code-level autonomous research loops (ARLs) have recently emerged as a concrete object of study in automated machine learning research. In such loops, an LLM agent proposes modifications to an experimental training pipeline, executes the modified pipeline, and retains edits that improve a verifiable in-loop metric. Although executable metrics may appear to provide a reliable signal of progress, it remains unclear whether repeated metric-driven code editing leads to genuine improvements that generalize beyond the loop. We provide a systematic diagnosis of this question. Across various experiment settings, we identify a robust failure mode that we call algorithmic mode collapse. In this regime, surface-level edit diversity remains stable, but semantic and mechanism-level diversity collapse: the agent continues to edit different lines of code while repeatedly proposing the same kinds of algorithmic changes. This collapse is accompanied by a widening gap between in-loop metric gains and gains measured on independent held-out evaluations. We then propose Diversity-Aware Proposal Sampling (DAPS), a lightweight mitigation that combines category-coverage reweighting, persistent edit memory, and a validation gate. Under a three-tier protocol separating the in-loop metric, the audit metric read by the gate, and a blind metric no loop component ever accesses, DAPS reduces semantic-cluster decay of edits by 69.1% and improves relative faithfulness by 83.7% blind and 81.6% audited, while preserving in-loop optimization speed. We provide the code in Github [repository](https://github.com/BokwaiHo/arl-mode-collapse).

Abstract:
Low-resource authorship style transfer (LAST) aims to rewrite text into the style of an arbitrary target author using only a few reference examples while preserving the original meaning. Existing methods often struggle to achieve both high style fidelity and semantic preservation because they compress diverse references into a single static author embedding, which averages out context-dependent stylistic variation, and rely on hidden representations for style control, which entangle style with content. We propose HyperStyler, a novel architecture that decouples LAST into style selection and style realization. Stylo-navigator predicts style coordinates by jointly modeling the source context and target-author references, and Stylo-hypernet realizes them via dynamic parameter modulation instead of hidden-state injection. Our experiments on Reddit, Blog, and News datasets demonstrate that HyperStyler consistently outperforms prior methods including LLM-based approaches and generalizes robustly across domains. Notably, HyperStyler achieves superior performance with as few as 2.4% additional parameters over T5-large, while being over 1.8× faster than LLMs at inference.

Abstract:
Gene set analysis is a cornerstone of functional genomics, yet it remains labor-intensive and heavily dependent on manual curation and expert biological interpretation. While Large Language Models (LLMs) have emerged as powerful tools for genomic reasoning and annotation, most existing approaches rely on symbolic gene names and fail to capture domain-specific biological structure, particularly protein sequence information that governs molecular activity, interactions, and downstream gene function. In this work, we propose SoftGene, a novel framework for LLM-based gene set annotation that leverages the hierarchical structure of gene sets. First, we use a hierarchical attention-based encoder built on ESM, a protein language model, to represent each gene set using protein-level amino acid sequence information. Second, we construct a hybrid prompting scheme that combines soft prompts derived from gene set embeddings with hard prompts containing auxiliary context generated by an LLM, and feed the resulting prompt into a local LLM for annotation. We evaluate our framework on two benchmark datasets: Gene Ontology (GO) and the Molecular Signatures Database (MSigDB). Our results show that integrating protein-sequence representations with textual context improves gene set annotation overall, while per-domain analyses reveal that the contribution of protein embeddings varies across biological domains.

Abstract:
Emotional Support Dialogue Systems (ESDSes) are increasingly evaluated and trained with LLM-simulated seekers. However, such simulated seekers often behave as cooperative, average-case users who disclose clearly, respond constructively, and accept support within a few turns. This can lead to overly optimistic evaluation and obscure whether ESDSes can handle difficult help-seeking interactions. In this work, we study ESDS evaluation under worst-case interactions, where seekers are hard to help due to low engagement, resistance, limited self-disclosure, emotional volatility, or rigid negative interpretations. We first conduct an expert simulation study with eight experienced counselling professionals, who simulate difficult seekers, interact with existing Chinese ESDSes, provide scale ratings, and participate in semi-structured interviews. Based on this study, we derive worst-case seeker behaviours and identify key limitations of current systems. We then propose a worst-case evaluation framework consisting of an LLM-based worst-case seeker simulator and four worst-case-oriented metrics: Deep Emotional Understanding, Guided Exploration, Balanced Emotional Support, and Authentic and Grounded Support. Evaluating 17 systems, we find that nearly all models suffer substantial performance drops under worst-case interactions. Large general-purpose LLMs are generally more robust than specialised ESDSes, but even the strongest models struggle to sustain engagement and improve seekers’ emotional states. Finally, we show that worst-case simulation can also generate useful training data, improving the robustness of smaller models.

Abstract:
Uncertainty quantification (UQ) is essential for improving the reliability of large language models (LLMs). However, multi-sample UQ methods typically employ a fixed sampling budget for all queries, leading to computational redundancy. In addition, multi-sample UQ methods often fail in the presence of consistent errors. To address these limitations, we propose CASE (Contradiction-aware Adaptive Spectral and Entropic UQ), a UQ framework that jointly models semantic disagreement and token-level uncertainty under an adaptive sampling paradigm. Specifically, CASE constructs a contradiction graph over sampled responses and quantifies semantic disagreement using normalized spectral norms of the graph representation. The resulting uncertainty signals are further leveraged for adaptive sampling, enabling query-dependent budget allocation while reducing redundant computation. In addition, CASE incorporates token-level predictive entropy as a complementary uncertainty signal to mitigate the failure of graph structures under consistent errors. Experimental results demonstrate that CASE outperforms existing multi-sample UQ methods across factual question answering datasets while reducing sampling costs by over 30% on average.

Abstract:
Layer-wise mixed-precision quantization (LMPQ) enables effective compression under extreme low-bit settings by allocating higher precision to sensitive layers. However, existing methods typically treat all intra-layer weight modules uniformly and rely on a single numerical property when estimating sensitivity, overlooking their distinct operational roles and structural characteristics. To address this, we propose NSDS, a novel calibration-free LMPQ framework driven by Numerical and Structural Dual-Sensitivity. Specifically, it first mechanistically decomposes each layer into distinct operational roles and quantifies their sensitivity from both numerical and structural perspectives. These dual-aspect scores are then aggregated into a unified layer-wise metric through a robust aggregation scheme based on MAD-Sigmoid and Soft-OR to guide bit allocation. Extensive experiments demonstrate that NSDS consistently achieves superior performance compared to various baselines across diverse models and downstream tasks, without relying on any calibration data. The code is available at https://anonymous.4open.science/r/NSDS-DC56.

Abstract:
Precise numerical reasoning with Large Language Models (LLMs) is essential for expanding their applicability to complex real-world tasks. However, text-based tokenization often fragments numbers, significantly hindering precise arithmetic reasoning. Meanwhile, numerical embeddings, despite arithmetic precision, rely on context-agnostic substitution that disregards the semantic role of numbers as identifiers. To combine the complementary strengths, we propose ANI (Adaptive Numerical Injection) , a hybrid framework that governs the selective injection of numerical features based on the semantic context. By employing a context-aware gating mechanism, we selectively inject numerical embeddings (specifically FoNE) into the latent space, explicitly preserving nominal identifiers while enhancing quantitative operands. Through extensive evaluations across various LLMs, we demonstrate that ANI enhances MATH performance by 9.5 points over the official reference model, while maintaining robust performance on general linguistic benchmarks.

Abstract:
As large language models (LLMs) are increasingly deployed in real-world applications, alignment is no longer governed by a single universal notion of safety or helpfulness, but instead by provider- or application-specific model specifications. These specifications are typically long, structured, and frequently updated, yet existing alignment pipelines lack a systematic mechanism to operationalize them as training signals. In this paper, we propose specification-grounded alignment, a new alignment paradigm that treats provider-authored model specifications as the primary alignment target rather than abstract principles or static benchmarks. To instantiate this paradigm, we introduce SpecAlign, a framework that synthesizes alignment data directly from specification documents. SpecAlign combines structured rule annotation, controllable specification instantiation, and multi-agent adversarial data synthesis to generate fine-grained, boundary-aware preference pairs that capture both compliant behaviors and meaningful specification violations. Experiments across multiple model specifications and backbone models demonstrate that training with SpecAlign consistently improves rule compliance while preserving general capabilities and avoiding over-conservative behavior. These results suggest that grounding alignment in explicit model specifications enables rapid, precise, and scalable adaptation of LLM behavior to evolving policy requirements.

Abstract:
Long-form streaming speech-to-speech translation (S2ST) requires incremental, unbounded translation under strict latency constraints. Existing methods typically suffer from sentence-bounded supervision or demand massive paired-S2ST supervision. We introduce a training recipe enabling a speech language model for sentence-level and long-form streaming S2ST using only ∼ 2k hours of paired cross-lingual S2ST data, layered atop auxiliary supervision. Anchored by auxiliary multitask training, our approach remains robust even when the paired-S2ST budget itself is reduced by 90%. Our core contribution, joint text-code trajectory supervision, schedules target text and acoustic semantic codes as a unified commitment path, eliminating the need for separate, unstable speech-side emission controllers. Furthermore, our two-stream Thinker–Talker factorization significantly outperforms unified-decoder baselines by decoupling linguistic reasoning from dense acoustic prediction to mitigate modality interference. Finally, our system achieves highly competitive quality-latency trade-offs on RealSI and ACL60/60-dev, matching state-of-the-art, closed-source S2ST systems such as LiveInterpret 2.0 on ASR-BLEU.

Abstract:
Large language models (LLMs) often hallucinate, yet most existing fact-checking methods treat factuality evaluation as a binary classification problem, offering limited interpretability and failing to capture fine-grained error types. In this paper, we introduce InFi-Check, a framework for interpretable and fine-grained fact-checking of LLM outputs. Specifically, we first propose a controlled data synthesis pipeline that generates high-quality data featuring explicit evidence, fine-grained error type labels, justifications, and corrections. Based on this, we construct large-scale training data and a manually verified benchmark InFi-Check-FG for fine-grained fact-checking of LLM outputs. Building on these high-quality training data, we further propose InFi-Checker, which can jointly provide supporting evidence, classify fine-grained error types, and produce justifications along with corrections. Experiments show that InFi-Checker achieves state-of-the-art performance on InFi-Check-FG and strong generalization across various downstream tasks, significantly improving the utility and trustworthiness of factuality evaluation.

Abstract:
Low-rank gradient projection reduces the optimizer-state memory cost of large language model (LLM) pretraining, but the steps and wall-clock time needed to reach a target quality remain a meaningful axis for improvement. We attribute this to two design choices in existing methods: the projection-rank budget is allocated uniformly across Transformer modules with heterogeneous projection sensitivity, and projecting a raw gradient attenuates its magnitude and direction jointly. We propose MoARa, which combines a static profiling-based module-aware projection-rank allocation with a block-wise magnitude-direction decomposition; the default block size is set in the neighborhood of the attention head dimension. Across five Transformer architectures spanning Llama, Qwen, and DeepSeek at 300M to 7B scales, GaLore with MoARa reaches standard GaLore’s final perplexity in 37% fewer steps and 34% less wall-clock time on Llama 2 7B, with only 0.2% peak reserved memory overhead under standard graph compilation. Across the six low-rank pretraining methods we evaluate, module-aware rank allocation alone delivers directionally consistent step reductions on all six. On compatible hosts, the two-component design reaches up to 41.7% step reduction and 37.1% wall-clock reduction.

Abstract:
CAPTCHAs are widely deployed as human verification mechanisms and frequently block intelligent agents from completing end-to-end automation in real-world web environments. Existing CAPTCHA-solving methods rely on prompting closed-source models, incurring high inference costs and lacking controllability, yet training-based approaches have remained absent due to the lack of large-scale training data with process-level annotations. To enable training-based CAPTCHA solving, we introduce CaptchaBench, a large-scale benchmark of 16,000 programmatically generated samples with detailed region and process-level annotations. Systematic evaluation on CaptchaBench reveals that existing methods fail on tasks requiring fine-grained visual detail capture, with a key challenge being the inability to supervise the model’s reasoning process over task-relevant visual regions. Therefore, we present CaptchaMind, an RL-based solver that explicitly supervises the model’s reasoning process by directly rewarding correct identification of task-relevant visual regions at intermediate steps. CaptchaMind achieves state-of-the-art performance across eight tasks and generalizes effectively to real-world CAPTCHA instances, substantially outperforming all existing methods without closed-source APIs.

Abstract:
Vision-language models (VLMs) provide a unified representation space for textual and visual information, yet their potential as general-purpose backbones for graph-structured data remains largely unexplored. In practice, attributed graphs exhibit substantial modality heterogeneity: some graphs contain only textual node attributes, others only visual attributes, while still others provide both. Existing graph learning approaches are typically designed for fixed modality schemas, requiring separate models for different settings and limiting scalability and cross-graph generalization. To bridge this gap, we present OMG-VLM (One Model, Many Graphs with Vision-Language Models), a unified framework for learning over attributed graphs across heterogeneous modality schemas. OMG-VLM leverages a pretrained VLM as a shared backbone and introduces structure-aware graph adapters that integrate neighborhood information while remaining compatible with the VLM’s native embedding space. This design enables effective learning over text-attributed, image-attributed, and multimodal-attributed graphs within a single model. Extensive experiments across diverse domains show that OMG-VLM consistently outperforms state-of-the-art GNN- and LLM-based baselines on attributed graph learning tasks such as node classification and link prediction, while exhibiting strong generalization to unseen graphs and varying modality schemas. The source code is available at https://github.com/Jo-eyang/OMG-VLM.

Abstract:
Instruction-guided speech editing requires a model to modify specified speech attributes while preserving non-target characteristics. Despite rapid progress in Speech Large Language Models (Speech LLMs), systematic evaluation of this capability remains challenging, as existing benchmarks are fragmented across isolated editing tasks. To bridge this gap, we introduce SpeechEditBench, a bilingual multi-attribute benchmark for instruction-guided speech editing. SpeechEditBench encompasses seven atomic editing tasks, as well as compositional editing tasks that integrate multiple operations within a single instruction. We propose an anchor-based evaluation protocol that separately assesses the edit success of target attributes and the preservation of non-target linguistic content, leading to three metrics: target success, preservation success, and joint success. Using this benchmark, we evaluate mainstream Speech LLMs and specialized speech editing systems. The results reveal three key findings: (1) no single model performs well across all editing dimensions; (2) closed-source Speech LLMs generally outperform open-source models; (3) compositional editing poses a significant challenge, with even the most advanced models struggling to achieve high joint success. SpeechEditBench provides a rigorous diagnostic framework to identify bottlenecks in Speech LLMs, thereby facilitating the development of next-generation models with more precise instruction-guided editing capabilities. Data and code are available at https://github.com/daxintan-cuhk/SpeechEditBench.

Abstract:
Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, actor traces, and runtime logs. Beyond turn-level and episode-level success, we measure cross-turn retention with Adjacent Pass Retention. Across eight models, even the strongest achieves 74.9% Turn Pass while completing only 37.3% of five-turn episodes; APR further falls to 52.4% on tool-grounded tasks. Diagnostic analysis shows that presentation failures center on information architecture, interaction failures on derived-state propagation and affordance binding, and tool-grounded failures additionally involve external-state grounding and requirement decomposition. These results reframe generative UI evaluation from judging isolated outputs to testing whether interface behavior, derived state, external state, and assistant claims remain synchronized as the artifact evolves.

Abstract:
Dynamic stance classification models how a reply responds to its direct parent message, rather than how a post relates to a fixed topic. Existing work has mainly studied this problem in text-only settings, while social media interactions increasingly rely on images, screenshots, memes, reaction images, and cross-modal references. We introduce MMDS-Bench, a diagnostic benchmark for multimodal dynamic stance classification in social media parent–reply interactions. MMDS-Bench contains 3,482 multimodal instances annotated with a seven-label dynamic stance taxonomy, together with an 800-instance diagnostic subset that requires structured reasoning over parent understanding, reply understanding, and stance-relation inference. We further annotate each instance with five challenge factors covering multimodal fusion, parent framing, non-literal expression, interaction reasoning, and label-boundary ambiguity. We evaluate 12 closed-source and open-source multimodal large language models and propose a reference-grounded LLM-judge protocol for assessing reasoning quality. Results show that current MLLMs still struggle with multimodal dynamic stance understanding, especially in cases that require relational inference beyond separate parent and reply comprehension.

Abstract:
Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that’s right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of this weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from what we call task confusion: models default to fact-checking the underlying claim, overriding the user’s stated belief. We provide evidence where chains of thought that explicitly fact-check show lower accuracy on false information than those that do not, and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs.

Abstract:
Retrieval-augmented generation (RAG) enables large language models (LLMs) to answer questions by accessing external knowledge and has been widely adopted for long-term conversational memory question answering. However, existing methods suffer from two key challenges: (1) fragmented evidence scattered across temporally distant sessions, and (2) noisy content within retrieved sessions that triggers the lost-in-the-middle effect. To address these challenges, we propose MemLoc, a unified Retrieve–Localize–Generate framework for long-term conversational memory QA. For retrieval, MemLoc decomposes each session into multi-granularity memory units and performs query routing via an inner-memory graph with entropy-based granularity selection. It further models cross-session semantic and temporal dependencies through a cross-memory graph, enabling coarse-to-fine retrieval of top-K relevant memory candidates. For localization, we introduce a reasoning-based evidence locator trained with Self-reflective Hint Policy Optimization (SHPO), which performs progressive refinement by extracting query-relevant fragments within memory units to suppress noise and reranking across candidates to remove redundancy, producing a compact evidence set with lightweight location IDs. For generation, these IDs act as precise grounding signals that guide the LLM to the correct memory positions, mitigating the lost-in-the-middle effect while preserving original contextual integrity. Extensive experiments on four benchmarks demonstrate that MemLoc achieves state-of-the-art retrieval accuracy and response quality while maintaining efficiency. Our code is available at: https://github.com/Nikol-coder/MemLoc .

Abstract:
Large reasoning models (LRMs) with explicit Chain-of-Thought (CoT) face significant safety risks, where unsafe content may appear in intermediate reasoning even when the final response is safe. Existing safety alignment methods often ignore the safety of the reasoning and frequently suffer from over-alignment issues. To address these, we propose CoRRSafe ( C o- o ptimized R easoning and R esponse Safe ), a two-stage framework designed to comprehensively enhance LRM safety while balancing safety and over-refusal. The method begins with a response cold-start stage using filtered high-quality data for initial alignment on responses. Then we implement a GRPO-based training strategy with fine-grained rewards to jointly evaluate segmented reasoning steps and final answers. This approach effectively guides the model to autonomously learn safe reasoning and response behaviors. Experiments on multiple LRMs show that CoRRSafe achieves state of the art performance in reasoning safety and achieves the best trade-off between safety and false refusal. On DeepSeek-Distill-R1-8B, it raises the Reasoning Safety Rate from 16.86% to 89.31% and reduces the Attack Success Rate from 78.74% to 4.06%, with only a 16.56% increase in false refusal. Further analysis confirms that co-optimizing reasoning and response is essential, as optimizing either alone fails to overcome safety performance ceilings.

Abstract:
Large language model (LLM) agents increasingly act through files, tools, memory, retrieved content, and other agents, which can trigger safety failures during execution. Existing benchmarks mostly focus on endpoint outcome evaluation or model-assisted moderation of pre-constructed trajectories, leaving limited evidence about where risks enter and how they propagate. To address this issue, we introduce TraceProbe, an execution-based benchmark for agent safety diagnosis. TraceProbe places agents in executable sandbox scenarios, scores endpoint and latent rule violations based on normalized traces and structured evidence, and replays recorded tool actions for step-level risk localization. Our benchmark contains 155 scenarios across three risk-source categories: non-adversarial failures, internal authority compromise, and external attacks. Across 12 contemporary LLMs, the mean attack success rate (ASR) is 0.347, indicating substantial residual risk, especially under internal authority compromise. Our analysis also demonstrates that endpoint-safe runs can contain unsafe intermediate actions, and the actual execution paths of agents may differ from the intended hazards of the testing scenarios. The TraceProbe benchmark and our findings support a more actionable form of agent safety evaluation: beyond assessing ASR or final outcomes, evaluators can also ask what went wrong, where it occurred, and where an intervention could be placed.

Abstract:
We revisit the under-explored phenomenon of syntactic complexity convergence in dialogue. Extending prior topic-level analyses to full conversations, we examine five human-human datasets and a set of human-LLM dialogues. We integrate and evaluate a multidimensional set of syntactic complexity metrics to quantify structural variation in dialogue. Our analyses reveal robust syntactic complexity convergence across human-human datasets, especially in task-oriented interactions, and statistical tests confirm that these effects are not due to chance. In contrast, human-LLM dialogues show no significant convergence under individual metrics; however, linear-CKA uncovers a distinct cross-metric convergence pattern. Motivated by these differences, we use syntactic complexity as a feature for detecting LLM-involved dialogue, achieving 96% accuracy. Our classification results demonstrate its potential without relying on likelihood-based signals such as surprisal, which require a language model as the estimator.

Abstract:
Safety alignment plays a vital role in ensuring the reliability of Large Language Models (LLMs) by mitigating risks from malicious queries. To avoid the high computational costs of post-training, recent research has increasingly shifted toward inference-time alignment via continuous activation steering. However, the geometric entanglement of harmful and benign technical representations within these continuous spaces impedes precise calibration, inevitably trapping models in a severe dilemma between jailbreak vulnerability and benign over-refusal. To address this, we propose Codebook-Orthogonal Direction Alignment (CODA), an inference-time framework that projects hidden states onto a vector-quantized codebook to serve as a discrete control interface. Specifically, to break continuous directional entanglement, CODA estimates steering directions over discrete codebook anchors. To eliminate the geometric interference triggering over-refusals, CODA softly orthogonalizes the safety direction with respect to benign technical intent. Finally, to prevent rigid over-correction, CODA forwards the decoupled vectors to yield dynamically adjusted steering strengths during generation via risk gating and exponential time decay. Experiments across a diverse set of open-weight LLMs demonstrate that CODA consistently yields a stronger safety–utility trade-off than continuous steering baselines, indicating that discrete representations provide a more effective mechanism for disentangling safety from utility.

Abstract:
Electrocardiograms (ECGs) are essential non-invasive tools for diagnosing cardiovascular diseases. While recent multimodal ECG-Report contrastive learning methods have shown promise for zero-shot ECG interpretation, they predominantly rely on global representations, failing to capture the fine-grained relationship between localized waveform patches and specific pathological tags. This limitation is further exacerbated by the fact that nearly 55% of standard clinical reports (e.g., in MIMIC-ECG) lack explicit waveform descriptions. In this paper, we propose FOCAL, a novel framework that achieves precise, fine-grained alignment between localized ECG segments and individual report tags via Optimal Transport. Furthermore, because fine-grained alignment at the tag level exacerbates the false negative problem among reports sharing common diagnoses, we introduce a semantic similarity matrix to guide the contrastive objective and correct misalignments. To address the scarcity of detailed waveform text, we introduce a coarse-to-fine enrichment pipeline that leverages Large Language Models (LLMs) to recover missing semantics, utilizing a coarse model verification step to rigorously filter out hallucinations. Extensive experiments across six datasets demonstrate that FOCAL establishes new state-of-the-art performance in zero-shot prediction and linear probing.

Abstract:
The rapid advancement of instruction-guided audio generation has highlighted the critical need for robust alignment evaluation. Current automated evaluation methods heavily rely on holistic scoring from general-purpose large language models, which struggle to decouple complex instructions, lack interpretability, and fail to capture fine-grained attribute mismatches. To address this, we introduce a novel dynamic rubric-based evaluation paradigm that adaptively decomposes complex audio captions into a variable number of independent, verifiable binary rubric items. To rigorously benchmark this capability, we propose the AnyAudio-Judge Bench , a comprehensive, bilingual benchmark comprising 7,920 meticulously curated samples across four diverse audio domains (speech, sound, music, and mixed), featuring deliberately constructed hard negatives. Furthermore, we construct a large-scale corpus of 105K samples with explicit Chain-of-Thought (CoT) rationales to train our dedicated evaluator, the AnyAudio-Judge model. By employing a training pipeline that combines Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO), our model successfully aligns its reasoning paths with the rubric-based scoring mechanism. Extensive experiments demonstrate that AnyAudio-Judge not only significantly enhances zero-shot alignment detection compared to state-of-the-art baselines, but also provides precise and interpretable reward signals that substantially improve instruction alignment in downstream reinforcement learning for audio generation.

Abstract:
Diffusion Large Language Models (dLLMs) offer a promising avenue for parallel generation but face a trade-off between decoding speed and quality. While revocable decoding strategies attempt to mitigate errors by verifying and remasking tokens, they typically operate within a mixed-quality context. This leads to two critical failures: Error Propagation , where new tokens absorb erroneous information from unreliable context, and Local Error Reinforcement , where errors mutually reinforce each other to evade detection. To alleviate these challenges, we propose ASRD (Anchor Decoding), a training-free framework that operates within the embedding space. ASRD explicitly decouples the decoding context into trusted Anchor Tokens , which are identified via temporal consistency, and uncertain candidates. Leveraging a dynamic Anchor Tokens Cache, we introduce two complementary mechanisms: (1) Anchor-Guided Generation, which injects entropy-weighted anchor signals into masked positions to implicitly rectify attention toward the reliable global skeleton; and (2) Anchor-Perturbed Verification, which applies orthogonal perturbations to uncertain candidate tokens, destabilizing and remasking errors driven by fragile local consensus. Extensive experiments on math and coding benchmarks demonstrate that ASRD outperforms recent remasking baselines, achieving accuracy improvements of up to 6.4% while accelerating inference throughput by up to 7.2 × .

Abstract:
Lifelong editing of LLMs requires storing thousands of edits after acquisition. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired. Each edit is first stored at low rank as a cheap sketch. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2× less memory and remains effective at 50,000 sequential edits.

Abstract:
High-quality tool-use data is critical for training language models to interact effectively with external tools. However, existing synthetic approaches typically follow a generate-then-filter paradigm with static post-hoc verification, often yielding inefficient data with imbalanced feature distributions. We propose ToolLoop , a closed-loop framework that decomposes synthesis into three progressive stages: (1) sampling function name combinations as ground truth; (2) backward derivation of user queries; and (3) forward derivation of tool calls. At each stage, dynamic self-feedback iteratively guides the model toward high-quality generation, realizing a transition from generate-then-filter to generate-verify-refine. On the Berkeley Function Calling Leaderboard (BFCL), a 4B parameter model trained with our 11K synthetic examples achieves 86.40% accuracy in non-reasoning mode, while an Isolate variant that removes BFCL-overlapping candidate functions still reaches 86.07%. Cross-benchmark evaluation on ACEBench further demonstrates strong generalization, with 72.1% overall accuracy using only 18.3% of baseline training data.

Abstract:
Pathological speech from patients with neurodegenerative and neuromotor disorders is often acoustically distorted and linguistically fragmented, making pathological speech reconstruction necessary to recover intended textual content from distorted and incomplete speech recordings. Crucially, such recordings are rarely uniformly degraded: some words or short phrases remain reliable and can serve as audible anchors for reconstructing the corrupted surrounding content. We introduce Anchor-gated Phonetic Group Relative Policy Optimization (AP-GRPO), a GRPO framework with phonetic reward that aligns speech language models (SLMs) through audible-anchor preservation and inter-anchor phonetic compatibility to the original speech signal. AP-GRPO consists of: (i) an anchor-gated reward that matches reliable audible anchors in clear regions; and (ii) an inter-anchor phonetic alignment reward that evaluates whether recovered contents are phonetically supported by the corresponding corrupted inter-anchor speech span. Across four disease conditions, AP-GRPO improves faithful speech reconstruction, and the learned anchor constraint automatically adapts to each condition and thus reveals interpretable disease-specific profiles: conditions with severe articulatory degradation require stronger anchor enforcement, whereas milder impairment or linguistically impaired conditions rely more on phonetic alignment for inter-anchor recovery.

Abstract:
As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and essential. In this work, we benchmark Scientific Introduction Generation (SciIG), evaluating how well LLMs can produce coherent introductions from titles, abstracts, and related works in the machine learning domain. Curating new datasets from NAACL 2025 and ICLR 2025 papers, we assess five state-of-the-art models, including both open-source (DeepSeek-v3, Gemma-3-12B, LLaMA-4 Maverick, MistralAI Small 3.1) and closed-source GPT-4o systems across multiple dimensions: lexical overlap, semantic similarity, content coverage, faithfulness, consistency, citation correctness, and narrative quality. Our comprehensive framework combines automated metrics with LLM-as-a-judge evaluations. Results demonstrate LLaMA-4 Maverick’s superior performance on most metrics, particularly in lexical overlap, content coverage, citation quality, and perplexity. Moreover, three-shot prompting outperforms fewer-shot approaches more than 50% of the time on formula-based automated metrics and performs consistently better on all human evaluation metrics. These findings provide practical insights into developing effective research writing assistants and set realistic expectations for LLM-assisted academic writing. To foster reproducibility and future research, we publicly release all code and datasets.[Project page, code, and datasets: ]

Abstract:
Large language model (LLM) evaluation is increasingly confounded by variant contamination: training corpora contain semantically equivalent but lexically or syntactically altered test items. These variants can evade detectors based on sampling consistency or perplexity, inflating benchmark scores through memorization rather than reasoning. We formalize this problem and introduce DVD (Detection via Variance of generation Distribution), a detector requiring only the test item, without external reference corpora or paired contaminated/clean sets. DVD models the local output distribution through repeated temperature sampling. Our key insight is that contaminated items alternate between memory-adherence and perturbation-drift states, producing unusually high variance in the synthetic difficulty of low-probability tokens, whereas uncontaminated items remain in drift with comparatively stable difficulty values. We construct the first dedicated variant-contamination benchmark across Omni-MATH and SuperGPQA by generating and filtering semantically equivalent variants, and simulate contamination by fine-tuning Qwen2.5 and Llama3.1 models of different scales. In most evaluated settings across datasets, model scales, architectures, and training configurations, DVD outperforms perplexity-based, Min-k%++, edit-distance (CDD), and embedding-similarity baselines, while showing strong robustness to hyperparameters. These results support generation-distribution variance as a principled and practical signal for detecting variant contamination, helping improve the reliability of LLM evaluation.

Abstract:
Vision encoders construct the image representations used by vision-language models. How much conceptual information, rather than immediately visible image evidence, do these representations contain? We use canonical color as a controlled test case: can a vision encoder represent the typical color of an object concept, such as yellow for bananas, when visible color is removed from the input image? We construct a dataset of objects with canonical color labels and probe vision encoders for both color and object identity using RGB and grayscale inputs. We find that canonical color remains decodable from grayscale images and is tied to predicted object identity, suggesting a conceptual link between object and color information. Extending this analysis to full VLMs, we find that VLM post-training can have a surprisingly large effect on color decodability in the vision encoder. Overall, canonical color provides a controllable lens for tracing object-level semantic information in vision encoders and VLMs.

Abstract:
Knowledge-based Visual Question Answering (KB-VQA) requires models to answer questions by integrating visual information with external knowledge. However, retrieved knowledge is often noisy, partially irrelevant, or misaligned with the visual content, while internal model knowledge is difficult to control and interpret. Naïve aggregation of these sources limits reasoning effectiveness and reduces answer accuracy. To address this, we propose MaS-VQA, a selection-driven framework that tightly couples explicit knowledge filtering with implicit knowledge reasoning. MaS-VQA first retrieves candidate passages and applies a Mask-and-Select mechanism to jointly prune irrelevant image regions and weakly relevant knowledge fragments, producing compact, high-signal multimodal knowledge . This filtered knowledge then guides the activation of internal knowledge in a constrained semantic space, enabling complementary co-modeling of explicit and implicit knowledge for robust answer prediction. Experiments on Encyclopedic-VQA and InfoSeek demonstrate consistent performance gains across multiple MLLM backbones, and ablations verify that the selection mechanism effectively reduces noise and enhances knowledge utilization.

Abstract:
Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a branching reply tree. We study why sentiment changes in rumor-centric conversation trees by treating discourse moves such as denial/correction, evidence/link provision, and toxicity/attack as candidate interventions and asking: (i) what sentiment a reply expresses, (ii) whether the sentiment shifts relative to its parent, and (iii) which prior message most plausibly drove the reply’s sentiment. To support this setting, we introduce CASIRE, a causal sentiment reasoning layer over public rumor conversation datasets that adds post-level sentiment labels, induced parent-child shift labels, calibrated multi-label intervention tags, and explicitly annotated causal-source labels. We then propose C³T, Counterfactual Causal Conversation Transformer, a thread-structured temporal model that jointly predicts node sentiment and shifts, learns sparse ancestor attribution, and supports counterfactual queries by forcing conversational intervention embeddings on or off to estimate potential outcomes. Under an event-level split, C³T improves out-of-event robustness and attribution over text-only, graph-based, and temporal baselines, and yields interpretable model-based effects: denials/corrections and evidence reduce downstream negativity, while toxicity increases it. We also benchmark open-weight LLM prompting baselines and find that added conversational context helps, but attribution remains less reliable, motivating structure-aware counterfactual modeling for social-media analysis.

Abstract:
Looped Language Models (LoopLMs) perform “latent reasoning” by recursively refining internal latent representations with shared weights, offering a more effective alternative to explicit verbal reasoning. Despite their effectiveness, we find that LoopLMs remain prone to loop instability: excessive iterations can lead to errors in later iterations, producing “hard” localized uncertain tokens that contribute to reasoning errors. To address this, we propose LoopCD, loop-wise contrastive decoding that enhances the reasoning performance of LoopLMs by intervening on these tokens at inference time. Specifically, we exploit the internal dynamics of LoopLMs and contrast the logits from earlier iterations with logits from the last refined iteration to form the final sampling distribution. We find that this strategy is highly efficient, introducing only negligible inference overhead and requiring no additional training, while effectively improving reasoning performance by naturally refining reasoning-critical hard tokens. Extensive experiments show that our method improves the performance of recent representative LoopLMs across various reasoning tasks.

Abstract:
In online meeting delegation, LLM agents fail to recognize when to speak. With no structured way to track stances, coverage, and floor, they miss the moments where they should contribute. Prompt-only delegates stay silent on 51.4% of the absent participant’s talking opportunities on the AMI corpus. We present CAPA (Collaborative Agent Predictive Architecture), an architecture for online meeting delegation. A Perceiver updates the meeting state from each observed turn. A Predictor forecasts how the conversation will continue. A Controller decides whether to speak and which proposition to surface. A Generator phrases the chosen contribution in the participant’s style. Two judges score the forecast and the action against the next observed turn. A Recalibrator updates the meeting state from those verdicts for future decisions. To evaluate online delegation, we introduce an episode-level protocol that scores whether, when, and what a delegate contributes around the participant’s actual idea units. The protocol’s schema-constrained LLM judges align with human annotations at Cohen’s 𝜅 = 0.71 . On 137 AMI meetings, CAPA reduces the silence rate from 51.4% to 2.5%, doubles credited recovery ( 26.1 → 52.2 ), and keeps hallucination at 0.6%. The failure mode shifts from omission to selection, with each residual near-miss attributable to a specific module of the architecture. Mechanism ablations identify the meeting state as the lever that closes the recognition gap, where raw-context scaling alone does not.

Abstract:
Zero-shot information extraction (IE) with large language models (LLMs) enables adaptation to new schemas and domains without task-specific training. Existing methods mainly follow three paradigms. Monolithic prompting is efficient but prone to missed mentions, boundary errors, and type confusion. Each-type prompting improves type-level focus but may produce overlapping or conflicting predictions, while multi-agent debate can resolve such conflicts at the cost of irrelevant context, redundant interactions, and high token overhead. To address these issues, we propose SMADE-IE, a sparse and evidence-driven multi-agent framework. An Adaptive Mode Selector routes simple inputs to a lightweight Global Extraction Mode and ambiguous inputs to a Type-Centric Extraction Mode based on sample complexity and relevant types. Cross-type conflicts are resolved by an Evidence-Driven Debate module that uses Toulmin-style arguments, external evidence scoring, Beta-based confidence updates, and early stopping. Experiments on nine benchmarks covering NER, RE, and JERE show that MADE-IE improves average Partial F1 over the strongest baselines by 11.37, 3.83, and 14.46 points, respectively. Compared with the multi-agent baseline CrossAgentIE, SMADE-IE reduces token consumption by 85.1% on DocRED and 80.1% on CrossRE, demonstrating substantially higher inference efficiency. Code is available at https://github.com/Cppys/SMADE-IE.

Abstract:
Retrieval-Augmented Generation (RAG) has empowered Large Language Models (LLMs) to tackle knowledge-intensive tasks. However, navigating global, heterogeneous knowledge bases (large-scale knowledge graphs and text corpora) for complex reasoning remains a challenge. Existing methods typically employ reactive, graph-driven exploration strategies, which blindly follow graph topology without adapting to the question context or evolving exploration progress, and lack deep bidirectional synergy between graph and text. To address these limitations, we propose CoG (Cognition on Graph), a cognitive-inspired, training-free framework for adaptive knowledge exploration. Drawing inspiration from human problem-solving, CoG performs a continuous plan-explore-reflect cycle, where it proactively formulates investigation plans, performs dual-source retrieval, and dynamically reflects on progress to adjust strategies. Crucially, it establishes deep bidirectional synergy between structured graph and unstructured text, where entities extracted from text dynamically guide graph exploration to bridge knowledge gaps. Extensive experiments on seven multi-hop QA benchmarks demonstrate that CoG significantly outperforms state-of-the-art methods while achieving superior exploration efficiency. Our code and datasets are available at https://github.com/zhougengxian/CoG.

Abstract:
Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensing thought steps into compact representations via memory tokens and retaining only these representations at inference time, the loss of fine-grained information makes subsequent steps more error-prone. To alleviate this, we propose HybridThinker , where in addition to preserving these representations, thought steps are also temporarily retained to provide fine-grained details. However, we observe that naively keeping thought steps accessible to subsequent steps during training lets the model bypass memory tokens by retrieving information directly from these steps, leaving the model’s ability to compress and retrieve information through memory tokens insufficiently trained. We therefore introduce a hybrid training scheme, in which only some thought steps are directly accessible through attention to subsequent steps, while the other thought steps are masked, forcing the model to use memory tokens for compression and retrieval. Across four reasoning benchmarks, HybridThinker matches Vanilla on Qwen2.5-7B and approaches it on Llama3.1-8B, improving over LightThinker by 5.8 and 2.7 points, respectively, with similar inference time. Ablation studies confirm that both temporary thought-step retention and the hybrid training scheme contribute to these gains.

Abstract:
Temporal knowledge graph (TKG) reasoning aims to predict future events from historical facts. A key challenge for graph-based TKG models is to simultaneously capture higher-order structural dependencies and identify query-relevant temporal evidence, yet existing methods are largely limited to pairwise interactions or query-agnostic history aggregation. To address this, we propose SHAPE (Structural Hypergraph Augmentation with Personalized Event Memory), a plug-and-play enhancement framework for graph-based TKG models that strengthens the backbone at both structural and temporal levels without modifying its architecture. Specifically, Dual Hypergraph Pre-Enhancement(DHPE) constructs learnable entity and relation prototype hypergraphs with cross-view information exchange to inject higher-order structural priors before temporal encoding, while Query-Denoised Event Memory(QDEM) retrieves query-relevant recent events and converts them into subject enhancement and candidate-level scoring bias via structure-aware expert gating. Extensive experiments on three benchmarks with two heterogeneous graph backbones verify the effectiveness of SHAPE.

Abstract:
Chain-of-Thought (CoT) has proven effective for eliciting reasoning abilities in large language models, but standard autoregressive decoders treat reasoning and answer tokens uniformly as next-token prediction targets, despite their fundamentally different roles. This homogeneous modeling introduces conflicting inductive biases and significant inference overhead. Latent CoT methods attempt to improve efficiency by removing explicit reasoning tokens, yet often degrade performance, indicating that explicit reasoning supervision remains critical for preserving pretrained reasoning capabilities. To address these challenges, we propose Pre 2 , a phase-aware reasoning framework that decouples thinking and answering while retaining explicit CoT supervision. Our approach introduces a Guided Latent Space and predict independent pre-thinking and pre-answering latent states to guide the model to generate reasoning tokens and answering tokens. We integrates them with explicit token representations through a hybrid latent–token decoding paradigm. This design increases computation along both width and depth dimensions with minimal parameter overhead, enabling stronger reasoning without sacrificing inference efficiency. Training is conducted exclusively on one in-domain dataset, with direct zero-shot evaluation on six out-of-domain benchmarks, and experiments demonstrate consistent and significant improvements over strong baselines and robust generalization.

Abstract:
Recent Multimodal Large Language Models (MLLMs) excel in precise entity recognition and powerful reasoning, significantly boosting performance and efficiency in complex vision-language tasks. Despite these gains, this advanced reasoning process introduces a subtle yet critical vulnerability. We identify an underexplored relational reasoning failure mode in which models over-prioritize semantic coherence, thereby inferring ungrounded interactions from mere object entities. We term this phenomenon Relation Evidence Amnesia (REA), wherein MLLMs correctly localize individual entities but fail to enforce true visual dependencies as the generative process becomes hijacked by semantic rationalization. To mitigate REA, we propose Relation-Grounded Faithful Reasoning (RGFR), an inference-time decoding intervention that operates without modifying target model parameters. Adopting a decoupled verification philosophy, RGFR employs a semantic-driven soft-gating mechanism to simultaneously incorporate explicit entity constraints and extract counterfactual causal evidence, followed by a Unified Schema-Aware Rectifier that rigorously mediates these visual-causal constraints during the decoding process. Extensive experiments across relation benchmarks demonstrate that RGFR improves relation recognition accuracy by over 8.1% compared to baselines, while largely preserving task utility. These results highlight the critical role of decoupled, evidence-grounded rectification in achieving faithful relational reasoning for multimodal models.

Abstract:
Although context windows have expanded significantly in recent years, hallucinations in long-context summarization remain a challenge. Long novels are better suited than news or papers for researching these hallucinations, due to their intrinsic information and detailed descriptions of events and dialogues. However, current research lacks a multi-scale benchmark for hallucination detection in long-context novel summarization and does not fully explore how hallucinations change as the context grows longer. In this study, we propose LongNovel, a multi-scale long-context bilingual (Chinese and English) novel benchmark for hallucination detection. This benchmark is constructed from 29 Chinese novels (ranging from 16k to 100k tokens) and chapter-level data from the BookSum dataset. We design 8 hallucination types and employ a combination of Multi-Model Arbitration and Entity-Referenced Hallucination Generation to ensure both data authenticity and a balanced distribution of hallucination categories. Furthermore, we manually revise the content in the test set to guarantee data reliability. Extensive experimental results demonstrate that LongNovel is a challenging benchmark. We release LongNovel for future research. https://github.com/BDML-lab/LongNove

Abstract:
As AI-generated and AI-assisted content floods online spaces, source labels attached to such content can distort human reasoning judgments, with downstream consequences for moderation, evaluation, and decision-making. Whether LLMs share this vulnerability, or offer more source-agnostic evaluation, remains an open question with strong implications for human-AI collaboration. We examine this issue using logical fallacies as a controlled setting to isolate source-label effects on reasoning quality, independent of domain knowledge. We conduct an online study ( N=505 ) where participants are assigned to a source condition (human, AI, human with AI assistance, AI with human assistance, or no disclosure) and evaluate comments containing logical fallacies, comparing their judgments with those of LLMs (GPT-5.2, Gemini 2.5 Flash, Claude Sonnet 4.5), which were evaluated across the same source conditions. Human evaluators were significantly more susceptible to fallacies labeled as ’written by human’ or ’written by human with AI assistance’ and assigned higher trust ratings in these conditions. LLM evaluations remained comparatively stable across source labels, though performance varied across models. Confidence levels were similarly high across conditions for both humans and LLMs, regardless of the presence of fallacies. Our findings indicate that source-label bias is primarily a human vulnerability for logical fallacy evaluation, with potential implications in human-LLM collaboration in increasingly AI-mediated environments.

Abstract:
Multimodal large language models (MLLMs) have substantially advanced mobile agents, yet proactive mobile assistance remains challenging because agents must decide When to intervene before determining How to assist. Existing systems often implement these two decisions within a unified MLLM-based pipeline, causing goal misalignment between conservative intervention filtering and comprehensive assistance generation, as well as redundant inference when the agent should remain silent. To address these limitations, we propose the Pre-Reasoning Perception Framework (PRPF), a two-stage framework built on perceiving before reasoning. PRPF introduces a lightweight Multimodal Proactive Perceptor (MPP) for intervention gating and context compression, and activates the Proactive Agent Reasoner (PAR) only when intervention is warranted. Experiments on the ProactiveMobile benchmark show that PRPF substantially reduces false trigger rates (FTR) while improving success rates (SR) and inference efficiency over the ProactiveMobile baseline.

Abstract:
Large Vision-Language Models (LVLMs) frequently suffer from object hallucination. Existing training-free interventions primarily manipulate attention weights, which indirectly affect the deep semantics reaching the final predictive layers. In this work, we shift our focus to the hidden state vectors extracted after self-attention and residual addition. Empirical analysis reveals that hallucinated tokens do not simply over-rely on linguistic priors; instead, they exhibit an anomalous contextual deviation, showing significantly lower similarities to both textual and visual contexts in intermediate layers. Motivated by this, we propose ROT, a layer-specific, training-free framework. ROT dynamically detects semantic deviation in the middle layers and applies a norm-preserving rotation to steer the hidden states back toward the grounded factual plane spanned by the contexts. For subsequent layers, a representational smoothing mechanism is introduced to stabilize the calibrated trajectory. Extensive experiments on multiple benchmarks demonstrate that ROT consistently reduces hallucinations across various model architectures and scales, offering an efficient, geometry-driven solution for grounded generation.

Abstract:
Discourse particles, such as "well" and "kind of", are crucial components that enable LLMs to "speak" more like humans. They are used to convey emotions, intentions, and interpersonal meanings. However, existing studies have not yet built a comprehensive understanding of LLMs’ capabilities in handling discourse particles. Moreover, the limited number of research focuses primarily on high-resource languages such as English, with little attention paid to Southeast Asian languages. In this paper, we (1) propose MalayPrag, a benchmark designed to systematically evaluate and analyse LLMs’ capabilities in handling discourse particles in colloquial Malay; (2) introduce five attributes that provide a linguistically-grounded, unified framework for interpreting pragmatic functions of discourse particles. Applying these two, we prompt ten off-the-shelf LLMs to perform three predication tasks. The experimental results reveal substantial challenges for current LLMs to accurately connect discourse particles and their pragmatic functions in Malay. The provision of the five attributes designed in this study is found to significantly improve the connections, highlighting the need for structured scaffolding for models’ pragmatic competence.

Abstract:
Computer-Aided Design is pivotal in modern manufacturing, yet existing automated methods predominantly rely on open-loop, one-shot generation, creating a mismatch with iterative real-world practices. In this paper, we present IterCAD, a unified multimodal agent framework for closed-loop, interactive CAD generation and editing. We formulate the task as a multi-turn interaction between a multimodal agent and an executable CAD sandbox, covering three tasks: Drawing-to-Code, Text-to-Code, and Interactive Editing. To support this, we develop a data synthesis pipeline incorporating advanced industrial manufacturing features to generate standard-compliant multi-view engineering drawings, complex code-editing tasks, and high-fidelity interaction trajectories. We optimize the agent via progressive SFT followed by geometry-aware reinforcement learning with viable-prefix masking to enhance code executability and geometric fidelity. Finally, we introduce the IterCAD-Bench evaluation suite and propose the Chamfer Distance Tolerance-Recall (CD-TR) curve alongside its AUC-TR metric, establishing a survivor-bias-free standard that unifies code validity and geometric precision. Extensive experiments demonstrate that IterCAD achieves highly competitive performance across multiple benchmarks, significantly outperforming existing approaches in both code executability and geometric precision, while exhibiting superior capabilities in closed-loop iterative refinement.

Abstract:
Solving repository-level code tasks requires LLM-based agents to use code search tools to navigate large codebases and identify a small set of relevant files and functions. However, current retrieval tools typically return flat lists of isolated code snippets: such lists can surface relevant files, but provide insufficient structure for agents to distinguish the target function from semantically similar alternatives in the same file. We introduce RepoNav, a lightweight post-retrieval interface that reorganizes retrieved snippets into a file-centered navigation scaffold. By presenting compact structural cues and candidate targets, this scaffold guides on-demand file-structure browsing, helping agents compare sibling symbols before selecting a target function. Across diverse models on LocBench, RepoNav improves function-level localization and narrows the file-to-function gap. Controlled ablations demonstrate that these gains come from structured evidence organization rather than simply exposing additional file structure, and the approach also improves performance on a repository-level question-answering benchmark.

Abstract:
Third-party LoRA adapters introduce serious backdoor risks: malicious adapters can maintain normal task performance while embedding hidden, trigger-activated behaviors, making them difficult to detect through standard utility evaluations. Existing removal-based defenses typically sanitize backdoored models by updating, pruning, distilling, or mixing model parameters. However, these approaches are misaligned with adapter-based deployment, where the trusted base model is expected to remain frozen. In this work, we propose LoRA-Patch, a lightweight backdoor defense tailored to LoRA adapters. Our key insight is that backdoor injection distorts the directional geometry of a poisoned adapter relative to its trusted base model. Building on this insight, LoRA-Patch suppresses trigger-activated behaviors by realigning adapter directions through layer-wise steering, without modifying the base model weights. Extensive experiments across five backdoor attacks and three models show that LoRA-Patch effectively mitigates trigger-activated backdoors, outperforming existing baselines while preserving benign task performance. Moreover, LoRA-Patch is computationally efficient, constructing patches within 30 seconds across all evaluated models and adapters.

Abstract:
Vision–language models (VLMs) have demonstrated impressive capabilities in general medical visual question answering, yet it remains unclear whether their predictions reflect evidence-grounded clinical reasoning or reliance on spurious priors. We introduce Med-R2 Bench, a hierarchical benchmark aligned with the clinical workflow to evaluate adversarial robustness with visual grounding. We design stepwise QA tasks to assess whether reasoning chains are strictly grounded in visual evidence across the four clinical stages, and employ adversarial perturbations to test robustness against misleading cues. Med-R2 comprises 42,432 images, 31 task categories, and 110,406 QA pairs. Evaluation across 14 VLMs reveals a sequential performance degradation along the four-stage clinical workflow. Adversarial experiments show that models rely heavily on correct prompts to guess answers. Even when provided with explicit visual cues, the models struggle to accurately align textual descriptions. Finally, we demonstrate that stepwise fine-tuning using our hierarchical data significantly improves reasoning robustness, highlighting its potential to drive future improvements in evidence-based medical AI.

Abstract:
Speech-to-speech translation (S2ST) in time-sensitive applications such as video dubbing requires not only semantic fidelity and speaker preservation, but also strict duration consistency to avoid audio-visual misalignment. However, existing S2ST systems largely generate target speech without explicit temporal planning, making duration control an unresolved challenge. We introduce DuraS2ST, a duration-aligned reasoning framework that enables a single speech language model to first generate an explicit chain-of-thought (CoT) for planning target wording and phonetic length, and then synthesize the corresponding speech tokens. To support this paradigm, we construct DuraSet-440K, a high-quality duration-aligned CoT corpus for supervised initialization. We further optimize the model with multi-modal multi-dimensional reinforcement learning, using a Duration Margin Reward to balance translation quality and duration consistency, and Modality-Aware Reward Attribution to assign rewards to appropriate token spans. Experiments on CVSS-T show that DuraS2ST achieves a strong balance between translation quality and duration consistency, outperforming competitive open-source and commercial baselines. Project page: https://github.com/Mia11939/DuraS2ST.

Abstract:
Cross-lingual aspect-based sentiment analysis (ABSA) transfers knowledge from a source language with annotated data to a target language, enabling fine-grained sentiment analysis without annotated target-language data. While monolingual ABSA has seen significant progress, cross-lingual ABSA remains underexplored, especially for complex tasks involving multiple sentiment elements like target-aspect-sentiment detection (TASD). In this paper, we propose a novel SeqLab framework that enhances cross-lingual ABSA using a sequence-to-sequence model with an auxiliary sequence-labelling task performed by the encoder, enhancing aspect term recognition and sentiment predictions. Additionally, we incorporate aspect-code switching (ACS), a translation-based technique that swaps aspect terms between source and translated sentences, generating additional training data to enhance the model’s cross-lingual understanding. We evaluate our approach across eleven languages, three domains, and two backbone models, surpassing previous state-of-the-art results for the commonly studied E2E-ABSA task. Unlike most prior work that relies solely on English as the source language, we systematically assess different source-target language pairs and extend our evaluation to the more challenging, yet underexplored TASD task in cross-lingual settings. Finally, we provide a detailed error analysis highlighting key challenges and limitations.

Abstract:
Classical Chinese poetry generation and evaluation with Large Language Models (LLMs) face two intertwined challenges: existing generation benchmarks rely on simplistic keyword inputs devoid of literary context, and no ground-truth benchmark exists for objectively validating poetry evaluation methods. We present a comprehensive framework addressing both challenges. On the generation side, we construct Honglou-Poem, the first narrative-driven poetry generation benchmark derived from Dream of the Red Chamber, where each entry provides rich author personas and narrative backgrounds—moving far beyond traditional keyword-to-poem paradigms. A ”Digital Poetry Party” pipeline evaluates 12 state-of-the-art LLMs in this challenging setting. On the evaluation side, we introduce Poetry-Tree, a benchmark with axiomatically ordered quality levels created through systematic merit degradation, providing objective ground truth for evaluator validation. Leveraging Poetry-Tree as a training signal, we employ GRPO reinforcement learning with Spearman’s correlation as the reward to train a dedicated poetry evaluation model, Poet-4B. Our Poet-4B achieves a Spearman’s 𝜌 of 0.8253, dramatically improving from the base model’s 0.2929 and closely approaching the best closed-source model GLM-4.6 (0.8764) with over 100× fewer parameters—a gap of only 0.051 in Spearman’s 𝜌 . We release all benchmarks, trained models, and a leaderboard to facilitate future research.

Abstract:
Linguistic Inquiry and Word Count (LIWC) provides auditable psycholinguistic categories that are widely used to interpret depression-related language, but its incremental predictive role in modern multimodal systems remains unclear. We evaluate LIWC across two English and three Chinese depression-related corpora under matched participant-level cross-validation. We ask whether LIWC improves classification and what any performance change reflects. Intact LIWC is compared with three fold-local substitutes: a PCA-rotated version that removes direct access to named category coordinates, a participant-shuffled version that preserves real LIWC profiles while breaking participant alignment, and a random-marginal version that preserves feature-wise distributions. Across multiple fixed representation contexts, the results provide limited evidence for stable LIWC gains under frozen, participant-level early fusion. None of the prespecified dataset-blocked contrasts survives multiple-comparison correction. A separate SBERT calibration produces larger observed intact-versus-shuffled and intact-versus-random separations, indicating that larger participant-aligned signals can produce correspondingly larger separations under the same procedure, while not resolving the five-corpus power limitation. LIWC remains useful as an auditable, corpus-conditioned interpretive layer. These conclusions should not be generalized to fine-tuned, sequence-aware, or end-to-end architectures.

Abstract:
In multi-task mixture-of-experts parameter-efficient fine-tuning, fixed-rank Low-Rank Adaptation (LoRA) adapters often struggle to accommodate the heterogeneous representation requirements across different tasks and network layers. Existing methods mainly focus on task-level expert selection, layer-wise rank allocation, or post-training parameter compression, making it difficult to jointly characterize low-rank capacity differences across tasks, layers, and modules within a unified framework. To address this issue, we propose Group-Sparse Routing-aware Low-Rank Adaptation (GSR-LoRA), a routing-aware structured rank compression method for multi-task parameter-efficient fine-tuning (PEFT). Specifically, we model low-rank components as selectable rank groups and use a task-dependent router to allocate rank capacity across tasks. Furthermore, we introduce a Group Lasso structured sparsity constraint to suppress redundant rank groups during training, forming a prunable adapter structure. During compression, we select rank groups by jointly considering parameter importance and routing usage signals, and map the router to the compressed rank space to obtain compact LoRA adapters that remain consistent with task routing. Experiments on GLUE show that GSR-LoRA reduces the average adapter rank from 16 to 5.5, removing 65.6% of rank components with only a 0.1-point performance degradation. These results demonstrate its ability to learn compact task-adaptive LoRA adapters while preserving multi-task performance. The code is available at https://github.com/HNLP93/GSR-LoRA

Abstract:
Memory-augmented agents maintain compact user profiles throughout extended conversations, enabling personalized and consistent responses without the need to process the entire dialogue history. The quality of these user profiles relies on the underlying memory management strategy: at each step, the agent must determine what to retain, compress, or discard. However, existing methods typically employ a static, one-size-fits-all strategy established before training. In practice, the optimal memory decision is inherently user-specific and dynamically evolves alongside policy optimization. To address this, we propose HiPS ( Hi erarchical P ersonalized S trategy), a framework that decouples memory management into a globally shared foundation and a user-specific adaptive tier. Specifically, HiPS employs Universal Strategy to extract shared principles from cross-persona trajectories, alongside Persona Delta Distillation to generate tailored rules for users whose behaviors diverge from general patterns. Cross-Level Rule Flow dynamically calibrates their boundary by promoting broadly validated personal rules and demoting contradicted global ones. The architecture establishes a co-evolution loop where a mechanism guarantees that all strategy refinements are anchored to task outcomes. Extensive experiments demonstrate consistent improvements over memory-augmented baselines.

Abstract:
Personalizing Large Language Models typically relies on static retrieval or one-time adaptation, assuming user preferences remain invariant over time. However, real-world interactions are dynamic, where user interests continuously evolve, posing a challenge for models to adapt to preference drift without catastrophic forgetting. Standard continual learning approaches often struggle in this context, as they indiscriminately update on noisy interaction streams, failing to distinguish genuine preference shifts from transient contexts. To address this, we introduce SPRInG, a novel semi-parametric framework designed for effective continual personalization. During training, SPRInG employs drift-driven selective adaptation, which utilizes a likelihood-based scoring function to identify high-novelty interactions, selectively updating the user-specific adapter on drift signals while preserving hard-to-learn residuals in a replay buffer. During inference, we apply strict relevance gating and fuse parametric knowledge with retrieved history via probability interpolation. Experiments on the long-form personalized generation benchmark demonstrate that SPRInG significantly outperforms existing baselines, validating its robustness for real-world continual personalization.

Abstract:
Agreement among LLM judges is widely used as evidence that automated evaluation is reliable. Yet consensus may be stable without reflecting a shared understanding of quality. We call this Evaluation Illusion : judges can produce detailed critiques while relying on overlapping surface cues or a common evaluation instrument. In a fully crossed baseline study, three frontier judges conduct 105,600 evaluations of 32 models. First, for the two highest-scoring models, moving from a shared five-dimension rubric package to independently generated rubrics reduces agreement by 62%, a joint contrast we term the Rubric Commensurability Problem . Second, nearly identical model-level rankings coexist with substantially weaker criterion-level agreement under non-equivalent metrics and sample sets, yielding the Resolution Paradox . We further introduce MERG, a four-stage protocol that elicits task knowledge and potential biases before scoring. In a controlled comparison on the same models at t=0.0 , agreement decreases under independently generated MERG rubrics. These results do not establish that MERG scores are more accurate; they show that consensus depends on the evaluation instrument and analytical resolution at which it is measured.

Abstract:
Jailbreak backdoors typically pair a trigger with non-refusing responses, but under capability-based evaluation such payload-retrieval attacks often suppress refusal without producing actionable harm. We propose the persona-switch backdoor , where the trigger selects a persona-conditioned behavioral state rather than retrieving a fixed harmful response: style-consistent fine-tuning installs a persona, trigger training converts it into a dormant, trigger-addressable state, and persona-aligned framing amplifies harmful generation after activation. Across two model families and three personas, our attack substantially outperforms payload-retrieval baselines under capability-based evaluation while preserving high refusal rates on non-triggered inputs. The effect holds even for personas whose fine-tuning data contain no explicit harmful instructions, although they encode misaligned dispositions. Applying the same framing to clean and untriggered backdoored models yields near-zero harmful generation, showing that framing alone does not account for the attack and that harm depends on trigger activation of the poisoned persona-conditioned state. Activation steering supports a causal role for the persona-associated direction, pointing to representation-level intervention as a defense.

Abstract:
Large multimodal reasoning models solve challenging visual problems via explicit long-chain inference: they gather visual clues from images and decode clues into textual tokens. Yet this capability also increases hallucinations, where the model generates content that is not supported by the input image or the question. To understand this failure mode, we identify reasoning drift : during clue gathering, the model over-focuses on question-irrelevant entities, diluting focus on task-relevant cues and gradually decoupling the reasoning trace from visual grounding. As a consequence, many inference-time localization or intervention methods developed for non-reasoning models fail to pinpoint the true clues in reasoning settings. Motivated by these insights, we introduce ClueRecall, a metric for assessing visual clue retrieval, and present ClueTracer, a training-free, parameter-free, and architecture-agnostic plugin for hallucination suppression. ClueTracer starts from the question and traces how key clues propagate along the model’s reasoning pathway (question → outputs → visual tokens), thereby localizing task-relevant patches while suppressing spurious attention to irrelevant regions. Remarkably, without any additional training, ClueTracer improves all reasoning architectures (including R1-OneVision, Ocean-R1, MM-Eureka, .) by 1.21 on reasoning benchmarks.

Abstract:
Code retrieval is becoming central to coding agents, but agentic coding requires more than matching a natural-language query to an isolated snippet. Given a user request, a coding agent needs to navigate a concrete repository state, locate relevant files and functions, gather supporting context, and filter similar in-repository distractors. Existing code retrieval benchmarks mainly evaluate docstring-to-function or snippet-level matching, thereby missing this requirement-driven repository search problem. To address this gap, we introduce CORE-Bench, a comprehensive benchmark for code retrieval in the era of agentic coding. CORE-Bench evaluates code retrieval ability at three levels: code understanding, issue-to-edit localization, and broader context retrieval. Built from curated code-search tasks and SWE-bench-series instances, CORE-Bench contains over 180K queries and 106K broader-context relevance labels. Experiments with representative embedding models show a sharp drop from traditional code search to code retrieval in agentic coding settings. Simple supervised fine-tuning of existing embedding models significantly improves performance in this setting, suggesting substantial room for further progress.

Abstract:
Seeing frames in order does not mean representing time. Modern VideoLMs receive ordered video streams, yet their main supervision acts on generated text rather than video-token representations where event dynamics should first emerge. This mismatch allows models to learn temporal answers from shortcuts such as objects, scenes, and language priors, without requiring internal video representations to capture event progression. To address this, we propose VT-Contrast, a representation-level temporal counterfactual objective for VideoLMs. Its design asks where temporal supervision should act and what temporal differences it should expose. VT-Contrast supervises selected late-layer last-frame video tokens, where temporal information is expected to be integrated before language generation, and contrasts order-preserving views with same-video reordered counterfactuals graded by Kendall tau distance. It requires no architectural changes, is compatible with diverse VideoLM training tasks, and improves overall performance across temporal understanding benchmarks. Our code is available at https://github.com/ANDgate99/VT-Contrast.

Abstract:
Social-deduction games such as Werewolf are increasingly used to evaluate LLM agents, but existing evaluations often rely on final game outcomes. We propose a belief-shift evaluation benchmark in Werewolf for analyzing communication skills through belief updating. Using LLM-played games, we annotate suspicion and accusation messages and measure how an observing village-side model’s beliefs change after each message. We evaluate 40 open-weight LLM configurations on 1,224 annotated messages. Our results show that larger models better distinguish true wolves from villagers based on game history, but accusations still strongly influence their beliefs. Models become more suspicious of the accused target and less suspicious of the accuser, especially when the accuser is trusted, even if the accuser is wolf-aligned. Larger models better resist accusations from accusers they already distrust. Overall, our findings suggest that current open-weight LLMs up to 120B parameters still struggle to integrate accusation content with source trust in strategic communication. Our benchmark and code are available at https://rlg.iis.sinica.edu.tw/papers/werewolf-accusation-benchmark.

Abstract:
Neural machine translation (NMT) of low-resource languages is often greatly improved through parent-child transfer learning. When high-resource parent and low-resource child languages are closely related, the system benefits from high vocabulary overlap and shared representations. However, systematic surface-level differences in orthography hide many of the rich connections between related languages, even when written in the same script, and prevent knowledge transfer. We present CharLOTTE: a completely automatic and language-agnostic pipeline that overcomes this hurdle without the need of manually crafted rules or external linguistic resources. CharLOTTE collects a dataset of cognates by running word alignment on parallel sentences and filtering the resulting word pairs based on normalized edit distance. The pipeline learns systematic orthographic correspondences from the cognates and applies them to more closely unify the representation of a high-resource language to that of a related low-resource language, thus increasing vocabulary overlap and enhancing knowledge transfer that improves downstream NMT. We evaluate CharLOTTE on four low-resource translation pairs, including one creole language pair, and report significant improvement in translation quality of up to 5.53 BLEU.

Abstract:
LLM-based machine translation systems can now do more than translate a source segment in isolation. At test time, they can translate directly, prepend previous context, or refine a context-aware draft. However, these interventions are not uniformly beneficial. Context can resolve discourse-sensitive choices, but irrelevant context and extra refinement increases cost and may perturb adequate translations. We introduce ContextRouter, a lightweight policy router that learns which intervention to apply for each source segment. During training, we first generate candidates for all contextual interventions and score them with a routing objective combining overall translation quality with incurred token costs. We then train our ContextRouter with group-relative supervision, so it learns within-example preferences rather than absolute global action scores. At inference, our router uses only test-time-available signals from the source, previous source-side context, and a no-context draft to determine and execute only the selected context intervention. On a filtered context-aware subset of TED, QED, and OpenSubtitles covering multiple languages and domains, our ContextRouter improves the quality-cost tradeoff over fixed no-context, always-context, always-refinement, QE-reranking, and score-regression policies, and human evaluation confirms that annotators prefer its chosen interventions. These results support treating context and refinement actions in LLM-MT not as always present defaults but conditional test-time interventions instead.

Abstract:
Large Language Models (LLMs) are increasingly used as evaluators for code generation, but existing judge benchmarks mainly target general-purpose programming. Constraint Programming (CP) requires a stricter notion of correctness: a model is correct only if it faithfully represents the intended solution space, not merely if it executes, preserves satisfiability, or returns one valid solution. We introduce CPJudgeBench, a benchmark for meta-evaluating LLM judges of CP modeling correctness through solver-supported solution-space semantics. It contains 522 solver-validated candidate models from 82 CP problem families in CPMpy and MiniZinc, labeled as equivalent, unsound, incomplete, mixed, or non-executable. Across our experiments, we meta-evaluate a total of 15 LLM judges through semantic-label, binary-correctness, and score-based protocols. Results show that fine-grained semantic judging remains difficult, especially for reference-free MiniZinc, while stronger reasoning- and code-oriented models perform more reliably. Score-based LLM judgments also align better with solution-space similarity than standard text or code metrics. The benchmark system is available at https://github.com/Nastaran95/CPJudgeBench.

Abstract:
Recent advancements have endowed Large Language Models with impressive general reasoning capabilities. However, these reasoning models often perform worse than non-reasoning models on personalization tasks. While some methods use outcome-based RL to improve personalization reasoning, they fail to supervise the reasoning process. As a result, models may reach correct answers through flawed reasoning chains, limiting further improvement. To address this, we propose TagPR, a novel framework that adds semantic tags to the reasoning process for step-by-step guidance. TagPR first automatically generates a structured, tagged dataset for Supervised Fine-Tuning. It then employs a multi-stage RL process guided by a composite reward signal, which integrates tag-based process supervision with a novel Personalization Reward Model with User Embeddings to achieve fine-grained alignment with user-specific logic. Extensive experiments on public LaMP, LongLaMP, PGraphRAG, and a self-constructed dataset demonstrate that our approach achieves state-of-the-art results, delivering an average improvement of 32.65% over the base model across all LaMP benchmark tasks. Our work demonstrates that tag-guided process supervision is an effective approach for personalization reasoning.

Abstract:
India’s tribal (Janjatiya/Adivasi) languages remain absent from public ASR evaluation, despite their importance for oral history, ecological knowledge, and cultural continuity. We present AdiDhwani, a preservation-oriented ASR benchmark for Bhili (ISO 639-3: ‘bhb‘), India’s largest non-scheduled language, and Mundari (ISO 639-3: ‘unr‘), a Vulnerable North Munda language. The benchmark comprises 56.5 hours of community-validated Bhili speech and 11.7 hours of Mundari speech with fixed evaluation splits and reproducible baselines. We evaluate multilingual ASR under zero-shot transfer, target-language fine-tuning, donor-language initialization, synthetic supervision, and KenLM-assisted decoding. Target-language supervision substantially improves WER, yet lower edit distance does not always reflect better documentation fidelity: models replace attested Bhili forms with Hindi cognates or delete grammatically meaningful Mundari suffixes while still achieving competitive scores. We introduce Hindi-Normalization Bias (HNB) and its alignment-constrained variant (HNBA), Morphological Fragmentation Rate (MFR), and Suffix Deletion Rate (SDR) as preservation-oriented diagnostics beyond WER. Human transcriber baselines show that these gaps are not fixed limits. AdiDhwani provides the first controlled ASR evaluation framework for these two under-documented Indian languages and supports language-preserving speech recognition research in endangered-language settings.

Abstract:
While previous work has examined the ability of language models (LMs) to capture pragmatic patterns, it remains understudied to what extent these abilities are shaped by training conditions. To address this gap, we test several models using an existing evaluation paradigm targeting LMs’ sensitivity to conversational norms (Askari et al. 2025). Across three experimental settings, we investigate the role of model size, training dynamics, and data composition in shaping pragmatic competence. Our results show that different pragmatic abilities exhibit distinct learning dynamics across models with varying sizes. While size or data scaling lead to inconsistent gains, exposure to conversational data causes moderate improvements in a modest training setting. Together, our findings suggest that pragmatic competence in LMs is heterogeneous and highlight the importance of targeted training conditions over scale alone for developing pragmatically sensitive models.

Abstract:
Language models are routinely exposed to contexts where an authoritative claim conflicts with a conclusion derivable from underlying evidence. Unlike explicit contradictions, such conflicts are latent: they emerge only after a full multi-step derivation, a setting current benchmarks largely overlook. We introduce and publicly release CLECK (CLaim Evidence Conflict of Knowledge), a counterfactual benchmark of 5,922 instances (evaluated on a balanced 2,669-instance subset) probing this blind spot across eleven reasoning strategies. Among instruction-tuned models, the capability to recover both competing answers does not translate into recognizing their conflict at inference: models default to the authoritative claim. When the conflict does surface in a model’s reasoning, preference shifts by 34 percentage points toward the derived evidence on average. Reasoning strategies and evidence load further modulate this behavior, and while chain-of-thought and claim hedging both improve awareness, they shift resolution in opposite directions: chain-of-thought toward the claim, hedging toward the evidence.

Abstract:
With the rise of personalized, persistent LLM agent frameworks such as OpenClaw, human-centered agentic social networks in which teams of collaborative AI agents serve individual users in a social network across multiple domains are becoming a reality. This setting creates novel privacy challenges: agents must coordinate across domain boundaries, mediate between humans, and interact with other users’ agents, all while protecting sensitive personal information. While prior work has evaluated multi-agent coordination and privacy preservation, the dynamics and privacy risks of human-centered agentic social networks remain unexplored. To this end, we introduce AgentSocialBench, the first benchmark to systematically evaluate privacy risk in this setting, comprising scenarios across seven categories spanning dyadic and multi-party interactions, grounded in realistic user profiles with hierarchical sensitivity labels and directed social graphs. Our experiments reveal that privacy in agentic social networks is fundamentally harder than in single-agent settings: (1) cross-domain and cross-user coordination creates persistent leakage pressure even when agents are explicitly instructed to protect information, (2) privacy instructions that teach agents how to abstract sensitive information paradoxically cause them to discuss it more (we call it abstraction paradox). These findings underscore that current LLM agents lack robust mechanisms for privacy preservation in human-centered agentic social networks, and that new approaches beyond prompt engineering are needed to make agent-mediated social coordination safe for real-world deployment.

Abstract:
Mixture-of-Experts (MoE) architectures enable scalable and efficient large language models (LLMs) by selectively activating expert sub-networks through a routing mechanism. However, this adaptive design introduces a new attack surface: specific experts become disproportionately correlated with certain tokens (e.g., end-of-sequence), allowing adversaries to manipulate model behavior via lightweight perturbations. In this work, we present Groundhog Bit-Flip Attack (GBFA), the first bit-flip-based Denial-of-Wallet availability attack against MoE-based LLMs. By identifying and flipping routing-layer bits associated with related expert activations, we demonstrate that GBFA substantially extends the decoding token usage across three different LLM modes: conversational, reasoning, and agentic tasks, while largely preserving semantic fidelity. Across four main real-world MoE-based LLMs, manually deactivating on average fewer than 4 experts drives average output inflation to 5912%, with the majority of test samples reaching max tokens. These results reveal a robustness vulnerability of MoE architectures to bit flip, and highlight the potential of GBFA as an availability attack against LLMs.

Abstract:
We propose CLASP, a lightweight model which performs recommendations for cold-start users for which only textual user description is provided and no previous interactions are observed. Our architecture learns lightweight latent alignment functions that project textual preference representations into a pretrained collaborative filtering latent space. The CF component can be pretrained on a large set of dense interactions from warm-start users, whilst the alignment maybe trained on a distinct set of users for which textual descriptions are present, allowing an efficient use of heterogeneous information. Extensive experiments on Steam, MovieLens-1M, and Netflix demonstrate that CLASP consistently outperforms retrieval-oriented and encoder-only semantic recommendation baselines while remaining substantially more computationally efficient than LLM-based recommendation methods. Remarkably, this happens despite the fact that some of the baselines have access to item text descriptions, whilst CLASP doesn’t. Lastly, we also provide a dataset which allows a granular study of the variability in user profile generation from small subsets of interactions.

Abstract:
Post-training quantization (PTQ) enables efficient deployment of large language models, and PTQ methods are usually optimized and evaluated with generic reconstruction, perplexity, or answer accuracy. But in explanation-critical domains, preserving only the final answer may be insufficient, since users may also inspect generated rationales to judge whether a prediction is trustworthy. We study this issue in medical multiple-choice question answering, where rationales should provide evidence that supports the selected answer. We propose an explanation-aware objective for transformation-based PTQ. Our method builds an offline faithfulness cache from full-precision teacher rationales and uses it during optimization to preserve answer-supporting evidence tokens and evidence-conditioned answer behavior. We instantiate it on OSTQuant under W4A4KV4 quantization and evaluate four 7B–8B medical and instruction-tuned LLMs on MedExQA, MedExpQA, and ChallengeClinicalQA. While a same-calibration OSTQuant baseline preserves task accuracy, it can substantially weaken answer-supporting rationales. Our objective is to preserve the full-precision model’s answer-supporting behavior rather than improve gold-label accuracy, and our method better preserves the full-precision model’s answer behavior and rationale-to-answer support. These results suggest that PTQ for explanation-critical settings should evaluate preservation of answer-supporting evidence, not only answer accuracy. Code and evaluation scripts are available at https://github.com/dut0817/EAQuant.

Abstract:
Stories are key to transmitting values across cultures, but their interpretation varies across linguistic and cultural contexts. Thus, we introduce multilingual story moral generation as a novel culturally grounded evaluation task. Using a new dataset of human-written story morals collected across 14 language–culture pairs, we compare model outputs with human interpretations via semantic similarity, a human preference survey, and value categorization. We show that frontier models such as GPT-4o and Gemini generate story morals that are semantically similar to human responses and preferred by human evaluators. However, their outputs exhibit markedly less cross-linguistic variation and concentrate on a narrower set of widely shared values. These findings suggest that while contemporary models can approximate central tendencies of human moral interpretation, they struggle to reproduce the diversity that characterizes human narrative understanding. By operationalizing abstract interpretation of narrative as an evaluation for cultural alignment, this work contributes to studying culturally grounded language model behaviour beyond static benchmarks or knowledge-based tests.

Abstract:
We present RPD-UoF, a corpus of 3,091 police use-of-force narratives obtained via Freedom of Information Law (FOIL) requests from the Rochester Police Department, combining structured demographic metadata with officer-authored free-text accounts. We use this dataset to investigate three related questions: whether officer narratives contain recoverable racial signal after demographic redaction, whether large language models amplify racial framing when conditioned on subject race, and whether that signal survives text redaction to re-emerge in multimodal outputs. Redacted narratives are statistically separable by subject race; LLMs produce structurally longer narratives for White profiles than Black profiles from identical seed texts; and CLIP classifications of GPT-5-generated images differ significantly depending on the racial origin of the source narrative. Together, these analyses provide an initial empirical lens on how race appears in police narratives and how modern language models interact with such sensitive attributes. The dataset and analyses are intended to support future research on fairness, accountability, and transparency in computational studies of policing and criminal justice.

Abstract:
Multimodal agents are increasingly evaluated on their ability to search the web, invoke tools, and solve complex tasks autonomously. Yet no existing benchmark systematically evaluates the capability of autonomously searching, parsing, and extracting precise answers from multimodal raw media files as a unified, end-to-end agentic task. We introduce RDQA - Raw Data Question Answering, a benchmark of 1,350 questions across three source modalities (document, video, audio), and 20 real-world task scenarios, where every answer is designed to require source access rather than model knowledge alone. RDQA provides capability-stratified evaluation through 9 capability families organized by modality and cognitive complexity, and introduces the Parametric Leakage Rate (PLR), a process-aware metric that flags correct answers produced without genuine evidence retrieval. We evaluate 12 agent systems spanning 6 backbone models and 2 scaffoldings. Our results show that the the best generic agent reaches only 24.0% adjusted accuracy, while a process-aware baseline (RDQA-Skill) reaches 41.1%. PLR reaches as high as 96.0%, indicating that many ostensibly correct answers are not grounded in retrieved evidence. We release the full dataset, evaluation framework, and a baseline to support future research.

Abstract:
Streaming speech translation (StreamST) requires determining appropriate timing, known as policy, to generate translations while continuously receiving source speech inputs, balancing low latency with high translation quality. Existing StreamST methods typically operate on sentence-level speech segments, referred to as simultaneous speech translation (SimulST). In practice, they require collaboration with upstream segmentation models to accomplish StreamST, where the truncated speech segments constrain SimulST models to make policy decisions and generate translations based on pre-defined contextual information preset by the upstream models. Moreover, SimulST models struggle to learn effective policies due to the complexity of speech inputs and cross-lingual generation. To address these challenges, we propose StreamUni, which achieves StreamST through a unified Large Speech-Language Model (LSLM). Specifically, StreamUni incorporates speech Chain-of-Thought (CoT) in guiding the LSLM to generate multi-stage outputs. Leveraging these multi-stage outputs, StreamUni simultaneously accomplishes speech segmentation, policy decision, and translation generation. Additionally, we propose a streaming CoT training method that enhances low-latency policy decisions and generation capabilities using limited CoT data. Experiments demonstrate that our approach achieves state-of-the-art performance on both SimulST and StreamST tasks.

Abstract:
Aligned language models that are trained to refuse harmful requests also exhibit over-refusal: they decline safe instructions that seemingly resemble harmful instructions. A natural approach is to ablate the global refusal direction, steering the hidden-state vectors away or towards the harmful-refusal examples, but this corrects over-refusal only incidentally while disrupting the broader refusal mechanism. In this work, we analyse the representational geometry of both refusal types to understand why this happens. We show that harmful-refusal directions are task-agnostic and can be captured by a single global vector, whereas over-refusal directions are task-dependent: they reside within the benign task-representation clusters, vary across tasks, and span a higher-dimensional subspace. Linear probing suggests that the two refusal types are representationally distinct from the early transformer layers. These findings provide a mechanistic explanation of why global direction ablation alone cannot address over-refusal, and establish that task-specific geometric interventions are necessary.

Abstract:
Comprehensive benchmark suites are essential for improving large language models (LLMs), but many widely used benchmarks are redundant, making evaluation unnecessarily expensive. Although recent benchmark compression methods (BCMs) can mitigate this cost, many strong BCMs rely on large collections of per-sample evaluation results from numerous LLMs to identify representative samples. Building such collections is also expensive unless they are already public, making these methods difficult to extend to newly released benchmarks. To address this challenge, we present ZipBench, a simple and low-cost BCM with theoretical error and rank-consistency guarantees. ZipBench evaluates only a small set of anchor LLMs, synthesizes pseudo evaluation results to broaden coverage, learns compact sample representations, and selects a small yet representative subset. Building on it, we create ZipBench Zoo, a collection of compact versions of 100+ benchmarks spanning text, multimodal, and agent tasks. These benchmark proxies achieve mean absolute errors of 0.002–0.02 and average Spearman correlations of ~0.98 with the full benchmarks. Overall, ZipBench reduces the cost of both LLM evaluation and compact benchmark construction, lowering the barrier to broad LLM research for compute-constrained researchers. All data, code and checkpoints will be fully released.

Abstract:
Reward hacking is a form of misalignment in which models overoptimize proxy rewards without genuinely solving the underlying task. Precisely measuring reward hacking occurrence remains challenging because true task rewards are often expensive or impossible to compute. We introduce Countdown-Code, a minimal environment where models can both solve a mathematical reasoning task and manipulate the test harness. This dual-access design creates a clean separation between proxy rewards (test pass/fail) and true rewards (mathematical correctness), enabling accurate measurement of reward-hacking rates. Using this environment, we study reward hacking in open-weight LLMs and find that such behaviors can be unintentionally learned during supervised fine-tuning (SFT) when even a small fraction of reward-hacking trajectories leak into training data. As little as 1% contamination in distillation SFT data is sufficient for models to internalize reward hacking which resurfaces during subsequent reinforcement learning (RL). We further show that RL amplifies misalignment and drives its generalization beyond the original domain. We open-source our environment and code to facilitate future research on reward hacking in LLMs. Our results reveal a previously underexplored pathway through which reward hacking can emerge and persist in LLMs, underscoring the need for more rigorous validation of synthetic SFT data.

Abstract:
Children’s stories play a crucial role in early cognitive and language development, yet existing story generation models lack the safety and age-appropriateness required for educational applications, especially in non-English languages. We introduce CCStories , a curated dataset of 22 , 200 high-quality Chinese children’s stories designed specifically to support safe and appropriate AI-generated content for young readers. Unlike general-purpose datasets derived from adult-oriented platforms, CCStories targets three critical dimensions of child-appropriate text: readability, syntactic complexity, and toxicity. We further propose a multi-dimensional evaluation framework that jointly assesses appropriateness and generation quality. As a validation of the dataset’s utility, we fine-tune five representative backbones: CPM, PanGu- 𝛼 , GLM-4, Qwen, a Chinese-adapted LLaMA-3 variant (Gigi-Llama-3-8B-zh), and additionally benchmark GPT-5-mini as a zero-shot instruction-tuned reference. Fine-tuning reduces offensive-prompt toxicity by up to 26.7 percentage points and raises readability by up to 14.3 percentage points, narrowing much of the gap to the GPT-5-mini reference. A human evaluation confirms that appropriateness rises from 53% to 86% after fine-tuning, and human validation of the toxicity classifier shows that the model ranking it induces matches the human ranking. CCStories provides a publicly available resource for building and benchmarking safer, more age-appropriate Chinese generative models for children’s education.

Abstract:
Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural concepts. We decompose this into three abilities: naming what a concept symbolizes, visually recognizing it on video, and locating its sub-events in time. Existing video-cultural benchmarks tend to test what is seen, collapsing these three abilities into a single score that hides the bottleneck. We introduce the Cultural Moment Benchmark (CMB): 306 expert-curated concepts from seven countries in Southeast Asia across five categories. We evaluate each concept through three stages, one per ability. Given a description, Stage 1 (S1) selects from four candidate concept names, Stage 2 (S2) selects from four candidate video moments, and Stage 3 (S3) predicts the start and end times of the moment in a video. To keep each stage focused on a distinct ability, we use three design choices: semantic-similarity distractors (S1, S2), unlabeled video moments (S2), and free-form localization on a different example video (S3). Across six vision-language models, failure modes vary by ability and modality. i) Even the strongest closed-source models score below 30% when all three stages must be correct; ii) The three abilities do not fully cascade: naming a concept correctly helps half the models recognize it on video, but recognizing it has little effect on locating the sub-event in time; iii) Audio is complementary, redundant, or distracting depending on the concept, more often distracting in non-Latin-script countries; removing both audio and subtitles hurts Games and Music the most. Our 14-rater human study shows that even Expert raters score below chance on concepts from a neighboring country, indicating that CMB requires country-specific cultural knowledge. CMB acts as a diagnostic harness, attributing failures to a specific ability or modality.

Abstract:
While audio large language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning capabilities. This performance disparity primarily stems from the scarcity of high-quality audio reasoning data. To bridge this modality gap, we propose a novel cross-modal on-policy distillation framework. Our approach leverages a powerful text model as a teacher to provide dynamic correction and guidance to the audio foundational model during on-policy exploration. Furthermore, we construct a three-tier symmetric dataset encompassing logical reasoning, complex audio reasoning, and speech dialogue. By dynamically aligning the student model’s reasoning trajectories within the high-noise auditory space with the teacher model’s prior knowledge in the clean text space, our method effectively distills robust reasoning capabilities into the audio domain. Extensive experiments demonstrate that our model achieves substantial improvements on the audio and speech understanding benchmarks, paving a new path for comprehensive audio-language understanding.

Abstract:
LLM-based agent systems increasingly rely on open-registry skills, which are difficult to vet and expose agents to greater attack risks. These attacks mainly take two forms. Malicious injection attacks embed explicit harmful instructions into skills and are readily caught by auditing. Direct vulnerability attacks target latent flaws in non-malicious skills; however, directly crafted attack prompts often lead to unstable or unverified outcomes. We introduce SkillAttack, an automated red-teaming framework that systematically explores executable attack paths from latent vulnerabilities without modifying skill files. Evaluated across 11 target LLMs, SkillAttack substantially outperforms baselines, showing that well-intended skills can pose serious risks in realistic agent interactions. Our evaluation also yields trajectory-level attack traces with verified risk labels, providing a foundation for defense. We also release SkillRiskBench, a benchmark for skill risk analysis, and train a lightweight pre-execution risk guard that predicts risky invocations from the user request and skill file. The guard outperforms frontier LLM judges and existing safety guard models, highlighting the value of trajectory-grounded supervision for proactive skill defense. Code is available at https://github.com/Zhow01/SkillAttack.

Abstract:
Hybrid architectures that combine softmax attention blocks and recurrent neural networks (RNNs) have shown a desirable performance-throughput tradeoff for long-context modeling, but their adoption and studies are hindered by the prohibitive cost of large-scale pre-training from scratch. Some recent studies have shown that pre-trained softmax attention blocks can be converted into RNN blocks through parameter transfer and knowledge distillation. However, these transfer methods require substantial amounts of training data (more than 10B tokens), and the resulting hybrid models also exhibit poor long-context performance, which is the scenario where hybrid models enjoy significant inference speedups over Transformer-based models. In this paper, we present HALO (Hybrid Attention via Layer Optimization), a pipeline for distilling Transformer models into RNN-attention hybrid models. We then present HypeNet, a hybrid architecture with superior length generalization enabled by a novel position encoding scheme (named HyPE) and various architectural modifications. We convert the Qwen3 series into HypeNet using HALO, achieving performance comparable to the original Transformer models while enjoying superior long-context performance and efficiency. The conversion requires just 2.3B tokens, less than 0.01% of their pre-training data.

Abstract:
Large Language Models (LLMs) have achieved strong performance on Table Question Answering (TableQA), yet their reasoning processes often remain misaligned with the underlying table semantics. This misalignment can lead to decompositions and validation steps that fail to faithfully capture the table operations required by the question. In this paper, we introduce SIP, a SQL Logic-aligned framework for improving the semantic consistency of LLM-based table reasoning. SIP is built around SQL Logic, a natural-language abstraction of standard SQL operators that explicitly represents operations such as projection, filtering, ordering, and aggregation while avoiding the syntactic brittleness of executable SQL. SQL Logic serves as a shared operator-level representation that guides sub-question decomposition and supports operator-aware validation in both Direct Prompting and Python Agent settings. Experiments on three TableQA benchmarks show that SIP achieves state-of-the-art performance and consistently improves robustness across diverse LLMs, demonstrating the value of explicit semantic representations for structured table reasoning.

Abstract:
As large language models (LLMs) continue to scale, parameter-efficient fine-tuning (PEFT) has become a practical alternative to full-parameter adaptation. Prompt tuning is effective, but existing approaches either use flat prompt structures or hierarchical structures with fixed prompt composition, limiting adaptive prompt specialization. To address this limitation, we propose HiVe, a prompt tuning framework that models prompts at multiple levels and enables input-dependent specialization. HiVe constructs a prompt hierarchy by leveraging inter-task relationships during training, and employs a vertical mixture-of-experts (V-MoE) mechanism at inference time to compose prompts up to the level of specialization required for each input. Experiments show that HiVe consistently outperforms strong prompt tuning baselines across diverse tasks.

Abstract:
Text-to-SQL is a key natural language processing task that maps natural language questions to SQL queries, enabling intuitive interaction with web-based databases. Although current methods perform well on benchmarks like BIRD and Spider, they struggle with complex reasoning, domain knowledge, and hypothetical queries, and remain costly in enterprise deployment. To address these issues, we propose a framework named IESR(Information Enhanced Structured Reasoning) for Lightweight Large Language Models: (i) leverages Large Language Models (LLMs) for key information understanding and schema linking, and decoupling mathematical computation and SQL generation, (ii) integrates a multi-path reasoning mechanism based on Monte Carlo Tree Search (MCTS) with majority voting, and (iii) introduces a trajectory consistency verification module with a discriminator model to ensure accuracy and consistency. Experimental results demonstrate that IESR achieves state-of-the-art performance on the complex reasoning benchmark LogicCat (24.28 EX) and the Archer dataset (37.28 EX), using only compact 7B and 8B models without fine-tuning. Furthermore, our analysis reveals that current coder models exhibit notable biases and deficiencies in physical knowledge, mathematical computation, and common-sense reasoning, highlighting important directions for future research. We released code at https://github.com/Ffunkytao/IESR-SLM.

Abstract:
Automated Essay Scoring (AES) aims to automatically assign quality scores to written essays and has been studied for several decades. Despite substantial methodological progress, little attention has been given to how AES systems are evaluated. In this position paper, we argue that several common AES evaluation practices are misaligned with broader machine learning evaluation standards. To examine current practices, we systematically analyzed 101 AES papers published in leading venues over the past decade, focusing on experimental setup, reproducibility, model development, and performance reporting and analysis. Consequently, we identify recurring shortcomings and highlight ways for more rigorous and transparent evaluation practices.

Abstract:
Large Language Models (LLMs) have recently been introduced into traffic signal control (TSC) as decision agents due to their strengths in human-readable reasoning generation. Yet, existing LLM TSC methods optimize only from final outcomes and fail to distinguish valid from flawed reasoning steps, causing useful or misleading steps to be jointly updated and thus impairing the model’s learning of effective reasoning. To bridge this gap, we propose an LLM-based framework ProcessLight to decompose signal decisions into verifiable semantic steps. Building on ProcessLight, we further develop Step-wise Traffic Process Policy Optimization (STeP-PO), a novel reinforcement learning framework that optimizes structured reasoning processes through step-level credit assignment. Specifically, STeP-PO uses step quality scores to evaluate local reasoning quality and step importance to measure each step’s influence on the final action, and then assigns step-level advantages over a semantic step tree structure. The resulting step-level advantages are propagated to reasoning tokens, enabling fine-grained policy optimization beyond outcome-only rewards. Extensive experiments over multiple real-world datasets demonstrate the superiority of our methods. Our code is available at https://github.com/wenzhaoabc/processlight.

Abstract:
Diffusion language models (dLLMs) have recently shown strong potential in various tasks with competitive decoding efficiency, but their bidirectional attention incurs quadratic compute and memory complexity. Many prior works rely on KV cache optimizations to accelerate inference. However, the full-sequence BF16 KV cache remains the dominant memory bottleneck. We propose SampKV, a Saliency-aware Mixed-Precision KV cache for dLLMs that stores most history in 2 bits while promoting a small budget of attention-salient tokens to high precision. A customized Triton kernel operates directly on the mixed-precision cache and avoids materializing full-precision KV tensors, alleviating the storage and traffic bottleneck. Experiments on LLaDA and Dream models show that the memory bottleneck is substantially alleviated: up to 54% peak memory reduction at matched workload, or up to 2.06x throughput improvement at matched peak memory compared to the BF16 baseline, while maintaining the accuracy.

Abstract:
Deploying large language models (LLMs) on sensitive data via Retrieval-Augmented Generation (RAG) introduces severe privacy risks. Recent studies apply Differential Privacy (DP) to LLMs with RAG for formal privacy guarantees. However, existing DP-RAG frameworks rapidly exhaust the privacy budget. Although recent efforts attempt to save the budget by narrowing the retrieval scope or sparsifying private generation, these methods themselves cumulatively consume the budget, whereas they could actually rely merely on public information or at a negligible one-time privacy cost. This mismatch fails to align budget expenditure with the model’s actual reliance on private data, causing substantial waste on operations that require no private access. To address this, we propose SparsePay-RAG, adopting "only pay what you must spend" as its core principle. Using public information as a zero-privacy prior, it charges the privacy budget only for the private increment. Specifically, SparsePay-RAG narrows the retrieval scope via public topic-guided clustering, adaptively controls private access frequency without privacy cost through isotonic cross-layer trajectory fitting, and compresses per-access budget via DP contrastive decoding. Under strong privacy constraints, experiments show SparsePay-RAG achieves superior privacy-utility trade-offs over baselines.

Abstract:
Concept-bottleneck controllable generation routes multi-attribute control through a low-dimensional concept code that, at deployment, must be synthesised from a target attribute configuration. We study this problem in concept-bottleneck text generation under multi-axis compositional generalisation, comparing three ways to obtain the inference-time code: classifier inversion against the encoder heads, reference-text encoding, and a post-hoc label-conditioned prior. Since a concept code admits no direct LM-fluency term, regularising inversion must instead constrain the code toward the encoder’s training distribution. We therefore test bare inversion and three regularised variants: label-agnostic and label-conditioned Mahalanobis penalties, and a conditional normalising-flow density baseline. Every inversion variant we test underperforms a simple post-hoc prior fitted to per-combination encoder means on the same checkpoints, across three backbone families spanning 124M to 8B parameters. The bare form of classifier inversion also silently collapses to chance, traceable to a directly measured off-manifold code. We validate this diagnosis on real-world benchmarks and under external evaluators, enabling fair comparison with published baselines.

Abstract:
Text embeddings incorporate additional rich semantic information; however, their performance in sequential recommendation still falls short of that of ID embeddings. Decomposing item embeddings along the spectrum of the collaborative item–item graph reveals a distinct discrepancy in their spectral energy distribution. Task-learned ID embeddings concentrate their energy heavily in the low- and mid-frequency directions of the collaborative graph topology, while textual representations spread more uniformly across the entire spectrum. Motivated by this insight, we propose CASG, a learnable polynomial graph filter that shifts text embeddings toward ID embeddings via collaboration-aware spectral guidance. Specifically, it leverages collaborative relationships to actively guide and filter the spectral energy distribution of frozen text embeddings, selectively preserving the frequency components aligned with the collaborative topology in a single offline pass. Across three datasets and two text encoders, CASG consistently surpasses learned ID embeddings, alignment-based, and fusion-based baselines at no online cost, suggesting that what semantic representations need is neither more alignment nor more fusion but collaboration-aware spectral guidance. Our source code is available at https://github.com/blueColdSea/CASG.

Abstract:
Interpretability methods are increasingly invoked to diagnose Large Language Model (LLM) failures, but most evidence comes from short prompts or toy templates, leaving their utility on practical benchmarks unclear. We cast failure diagnosis as contrastive attribution over target–contrast token pairs, attributing the logit gap between incorrect and correct output tokens to input tokens and internal states, and introduce a batch-packed multi-target backpropagation scheme that scales cross-layer attribution graphs to long-context inputs. Instantiated with LRP-style attribution, the framework is evaluated on IFEval, GAIA2, MATH, and EvalPlus across multiple model sizes and post-training checkpoints. Along the four axes of coverage, visibility, reliability, and dynamics, contrastive attribution surfaces recurring failure modes such as underweighting task-relevant tokens and overweighting distractors, demonstrating its practical value as a scalable diagnostic tool for realistic LLM failures.

Abstract:
Large vision-language models have achieved remarkable success in vision-language tasks. However, they remain prone to Visual Hallucinations (VHs), undermining their reliability in real-world applications. Existing solutions typically require curated datasets, additional training, or multi-round decoding, resulting in considerable efficiency overhead. In this paper, we propose RVSD (Retrieval Vision Sparse Decoding), a training-free and plug-and-play decoding framework that, for the first time, unifies token sparsification and Semantic-Space Visual Retrieval (SSVR) within a single decoding pass. Within RVSD, we introduce a semantics-directed token selection strategy that selectively sparsifies redundant tokens while preserving critical visual information. We further propose the SSVR mechanism, which reformulates visual compensation as an on-demand cross-modal retrieval process within a shared semantic space. Extensive experiments demonstrate that RVSD achieves state-of-the-art performance in mitigating VHs while maintaining robust suppression capabilities under long-context generation settings. Our code is available at https://github.com/canjie-liu/RVSD.

Abstract:
Schema linking is a crucial step in Text-to-SQL systems, connecting natural language mentions to relevant database tables and columns. Despite recent advances in LLM-based Text-to-SQL models, current approaches face two fundamental limitations: (1) providing detailed column descriptions inflates the input context, often exceeding the model’s effective sequence length, and (2) predicting multi-token column names increases output complexity, making the task fragile to generation errors. To address these challenges, we propose SchemaToken, which represents each database column as a dedicated token in the LLM’s vocabulary. This design compresses column semantics into single embeddings, reducing context length and simplifying prediction. We introduce a two-stage training paradigm: (i) Semantic Alignment, which enables the LLM to understand the newly added schema tokens via structured embeddings, and (ii) Pseudo-task Alignment, which trains the LLM to generate these tokens as an unordered set using reinforcement learning with a Jaccard-based reward. Experiments on multiple benchmark datasets demonstrate that SchemaToken significantly improves schema linking performance, particularly for large and complex schemas, while maintaining computational efficiency. Our code is publicly available at https://github.com/MrBlankness/SchemaToken.

Abstract:
As LLM agents take real-world actions through tools, indirect prompt injection (IPI) has emerged as a serious threat. The standard metric, Attack Success Rate (ASR), counts whether an injection succeeds but ignores what the user notices in the agent’s final response. Looking at successful injection traces, we find two distinct outcomes: the agent executes the injection while returning an otherwise normal response, or reports the injected action in its final response, giving the user a chance to notice. We call these covert and overt successes. From the user’s perspective, we decompose ASR into the Covert Success Rate (CSR), counting successes leaving no trace in the final response, and the Overt Success Rate (OSR), counting successes the user can detect. To understand what drives the gap, we analyze successful trajectories and find that the agent’s behavior after the injection separates covert from overt: covert traces hand control back to the user task before ending, while overt traces end at the attack itself. This split follows from the ReAct format, where the final response summarizes the most recent action. Building on this observation, we propose ICoA (Induced Covert Attack), an IPI attack designed to induce covert outcomes by steering the agent back to the user task after executing the injection. Across four target models on AgentDojo, ICoA achieves the highest CSR, with gains of 3.79–12.01 percentage points over the strongest baseline.

Abstract:
As open-source large language models (LLMs) become increasingly powerful, they are rapidly evolving into Agentic LLMs capable of automating complex daily tasks. However, this paradigm shift entails severe security risks, as the agents’ necessary integration with external real-time data sources creates a direct pathway for indirect prompt injection (IPI) attacks. Existing attack methods are limited by their reliance on static patterns and evaluation on simple language models, failing to address the fast-evolving nature of modern Agentic LLMs. We introduce AdapTools, a novel adaptive IPI attack framework that selects stealthier attack tools and generates adaptive attack prompts to create a rigorous security evaluation environment. Our approach comprises two key components: (1) Adaptive Attack Strategy Construction, which develops transferable adversarial strategies for prompt optimization, and (2) Attack Enhancement, which identifies stealthy tools capable of circumventing task-relevance defenses. Comprehensive experiments on both open-source and commercial agentic LLMs show that AdapTools achieves a per-model average improvement of 2.08× in attack success rate over the strongest baseline under state-of-the-art defenses (MELON, Pi-Detector), with gains reaching up to 3.06× on the most resistant reasoning agents. The framework further transfers to black-box commercial agents (GPT, Qwen, Gemini), with ASR ranging from 4.2 to 20.3 (peak on the in-family Qwen-MAX), exposing systematic vulnerabilities in modern Agentic LLMs.

Abstract:
Latent-based multi-agent systems replace parts of explicit inter-agent communication with hidden representations, offering a new direction for efficient and flexible agent collaboration. However, moving coordination into latent space may also move attacks beyond the reach of visible-text inspection. In this paper, we study whether latent states can carry attack-associated information that remains effective during clean executions. To examine this question, we introduce a latent attack framework that reactivates attack-induced effects through latent interventions without reusing adversarial text. Extensive experiments show that the resulting latent attacks can substantially degrade task performance in clean executions, especially when applied to inter-agent KV-cache handoffs rather than local hidden states. Further control analyses indicate that this degradation cannot be reduced to arbitrary perturbations or invalid generation. Overall, our findings suggest that latent-based collaboration does not remove attack risk; it shifts part of the risk into less observable execution states, calling for safeguards beyond visible-text inspection.

Abstract:
Reliable epidemiological reasoning requires synthesizing study evidence to infer disease burden, transmission dynamics, and intervention effects at the population level. Existing medical question answering benchmarks primarily emphasize clinical knowledge or patient-level reasoning, yet few systematically evaluate evidence-grounded epidemiological inference. We present EpiQAL, to our knowledge the first diagnostic benchmark for epidemiological question answering over research literature, comprising three subsets built from open-access articles across diverse diseases. The three subsets progressively test factual recall, multi-step inference, and conclusion reconstruction under incomplete information, and are constructed through a quality-controlled pipeline combining taxonomy guidance, multi-model verification, and difficulty screening. Experiments on fifteen models spanning open-source and proprietary systems reveal that current LLMs show limited performance on epidemiological reasoning, with multi-step inference posing the greatest challenge. Model rankings shift across subsets, and scale alone does not predict success. Chain-of-Thought prompting benefits multi-step inference but yields mixed results elsewhere. EpiQAL provides fine-grained diagnostic signals for evidence-grounding, inferential reasoning, and conclusion reconstruction.

Abstract:
Reinforcement Learning (RL) has demonstrated significant gains in image captioning, yet it is still limited in encouraging Large Vision-Language Models (LVLMs) to explore novel reasoning strategies. This limitation leads to a performance gap between RL and Supervised Fine-Tuning (SFT). In this paper, we argue that multi-modal retrieval can serve as an effective reasoning signal for caption refinement. Based on this insight, we present the Retrieval-Guided Refinement for Image Captioning (Re 3 Cap), a retrieval-guided reasoning strategy that enhances image captioning by performing retrieval on support sets. Instantiated by Caption Refinement Suggester (CRS) and Caption Quality Assessor (CQA), this strategy identifies hallucinations and omissions in image captions, leading to more accurate and detailed descriptions. Extensive experiments demonstrate the superiority of our method in image captioning, even compared with Supervised Fine-Tuning. Especially, Re 3 Cap outperforms GRPO with an average improvement of 8.64% in relation reasoning on the COCO-LN500 benchmark.

Abstract:
Epistemic reasoning requires agents to infer the state of the world from partial observations and information about other agents’ knowledge. Prior work evaluating LLMs on epistemic puzzles often frames failures as memorization rather than reasoning. We argue that this dichotomy is too coarse for newer models: memorization is a limiting case of pattern-based reasoning, where a model matches a task to a familiar template and applies the corresponding solution. We introduce a two-dimensional benchmark over Dynamic Epistemic Logic (DEL) puzzles, separating narrative familiarity from inference complexity, which allows us to distinguish pattern-based from epistemic reasoning. We find that models are substantially more robust to surface form changes than prior work suggested, yet consistently struggle in asymmetric settings where familiar patterns no longer apply and success requires tracking fragmented epistemic states. Code, datasets, and evaluation results are available at https://github.com/adigabay84/del-puzzles-benchmark .

Abstract:
Building effective LLM agents increasingly requires selecting appropriate AI models as tools from large open repositories (e.g., HuggingFace with > 2M models) based on natural language requests. Unlike invoking a fixed set of API tools, repository-scale model selection must handle massive, evolving candidates with incomplete metadata. Existing approaches incorporate full model descriptions into prompts, resulting in prompt bloat, excessive token costs, and limited scalability. To address these issues, we propose HuggingR 4 , the first framework to recast model selection as an iterative reasoning process rather than one-shot retrieval. By integrating Reasoning, Retrieval, Refinement, and Reflection, HuggingR 4 progressively decomposes user intent, retrieves candidates through multi-round deliberation, refines selections via fine-grained analysis, and validates results through reflection. To facilitate rigorous evaluation, we introduce a large-scale benchmark comprising 14,399 user requests across 37 task categories. Experiments demonstrate that HuggingR 4 achieves 93.01% workability and 84.25% reasonability—outperforming state-of-the-art baselines by absolute margins of 17.81% and 23.13%, respectively, while reducing token consumption by 6.9 × .

Abstract:
Conversational form-filling systems must update structured records across dialogue turns. A user may add, correct, or retract information, while the existing record may itself contain stale or unsupported values. We call correct maintenance of this state record commitment fidelity: after each turn, the complete record should contain all and only commitments supported by the schema and dialogue history. We introduce ConFormBench, a diagnostic benchmark that tests this capability. Each item provides a form schema, a prior record, dialogue history, and a new user utterance; the system must return the complete updated record. ConFormBench contains 180 source-grounded synthetic update turns across four institutional form families, covering 339 scoreable fields and 1,683 changed leaf-field commitments. We mechanically validate all items and audit a coverage-controlled 60-item subset through blind independent reconstruction and discrepancy adjudication. We also audit the evaluator against the adjudicated labels. On the 144-turn held-out split, five systems achieve 95.1-97.0% all-field accuracy, yet only 18.1-33.3% of records are entirely correct. Errors concentrate on clearing values, using history, coordinating gates, routing repeated instances, and repairing prior-state errors. Thus, field-level scores substantially overstate record-maintenance reliability.

Abstract:
Recent advances in large language models (LLMs) have unlocked remarkable long-context capabilities, enabling breakthroughs across diverse NLP tasks. However, despite architectural progress, the key-value (KV) cache remains a critical memory bottleneck during inference. Prior work has explored cache optimization via eviction strategies, yet most rely on heuristic or single-axis importance metrics, neglecting the nuanced and dynamic interplay between layers and attention heads. In this paper, we propose SCORE (Similarity-aware Contextual Overlap-Redundancy Eviction), a novel framework that introduces a distance-based multi-level similarity metric to quantify and eliminate structural redundancy within the KV cache. By dynamically reallocating cache budgets across layers and heads and employing a redundancy-aware greedy token selection mechanism, SCORE preserves semantic diversity while minimizing memory overhead. Extensive experiments on long-context benchmarks such as LongBench and NeedleBench show that SCORE retains 96.2% of full KV cache performance using only 1.5% of the cache, consistently outperforming state-of-the-art baselines under strict memory constraints. These results underscore the value of fine-grained, context-aware cache management for scalable and efficient long-context inference in LLMs.

Abstract:
Mixture-of-Experts (MoE) and looped architectures scale models along two orthogonal axes, namely parameter capacity and effective depth. However, mainstream looped architectures rely on dense backbones that couple parameter count with per-token FLOPs, which makes it impossible to isolate the effect of iterative computation under matched budgets. To this end, we present LoopMoE, a looped MoE language model that integrates sparse routing with iterative weight-shared computation through two designs. The first is IterAdaLN, which resolves weight-sharing symmetry via a modulation signal jointly conditioned on the iteration index and the per-token hidden state. The second is a capacity-balancing strategy that recovers the attention-to-FFN active parameter ratio of well-tuned non-looped references. Together, these designs enable the first strictly controlled, head-to-head evaluation of a looped MoE against a Vanilla MoE under identical total parameters, per-token FLOPs, and active sublayer ratios. Across nine downstream benchmarks, LoopMoE’s average improvement over its matched vanilla MoE increases from over 1 point at the 3B scale to approximately 3 points at the 9B scale. These results provide initial evidence that the benefits of iterative sparse computation may strengthen with scale, positioning LoopMoE as a promising architecture for scalable looped language models.

Abstract:
Medical insurance plays a pivotal role in the healthcare system, where effective supervision relies on the precise execution of complex medical insurance rules. Existing solutions for medical insurance violation detection mainly rely on costly manual labor, limiting efficiency. Although current Large Language Models (LLMs) have demonstrated significant potential in general Natural Language Processing (NLP) tasks, they still struggle to understand these rules, and a corresponding evaluation benchmark is lacking. Hence, we propose T2ER, a benchmark to evaluate the capability of LLMs in converting texts into executable rules in medical insurance. T2ER consists of 2,115 rules and 5,636 synthetic Electronic Medical Record (EMR) test cases for execution verification. Furthermore, we design a dual evaluation framework spanning rule and execution levels to comprehensively assess transformation quality. We conduct extensive experiments on advanced LLMs. The results reveal a significant human-AI performance gap, indicating that this conversion remains a formidable challenge. The dataset is available at https://github.com/YHJ1223/T2ER.

Abstract:
Large language models (LLMs) have increasingly been used to investigate how children acquire syntax at an early stage of development. This is notably the central scientific goal of the BabyLM challenge, a community-wide effort to develop models that achieve human-level syntactic performance while being trained on developmentally realistic corpora. In this paper, we reflect on the use of LLMs in the study of infant syntax learning by providing an epistemological assessment of several studies from this research program. We discuss how datasets are built, which models are implemented, how they are trained and syntactically evaluated. We observe significant assumptions in the methodology of BabyLM and related studies, thus mitigating their theoretical scope. We additionally observe that using developmentally-realistic corpora have limited effects on models performance on commonly-used benchmarks, which suggest important computational differences between LLMs and the infant syntax learner.

Abstract:
Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approximately 4.8 trillion target-language tokens across 36 European languages, produced by translating 100 billion high-quality Nemotron-CC tokens with Tower+ and OPUS-MT/HPLT-MT systems. For many medium- and lower-resource European languages, this is the largest openly available pre-training resource. On a broad multilingual benchmark suite, reference LLMs trained on MultiSynt/MT reach the final score of HPLT 2.0, a native-data baseline, using roughly 72% fewer pre-training tokens, and outperform it by approximately 15% relative at a matched 100B-token training budget. Our analyses also identify evaluation blind spots: standard multiple-choice benchmarks miss translation-quality differences that a fluency-sensitive LLM-as-judge evaluation cleanly recovers on the trained LLMs (with no fluency deficit in MultiSynt itself), and Norwegian idiomatic and culturally grounded tasks remain better served by native data. We release the corpus, including row-aligned translations from multiple systems, to support controlled research on multilingual pre-training data and evaluation.

Abstract:
Speech is essential for realistic role-playing, yet existing work on role-playing agents largely centers on text, leaving Speech Role-Playing Agents (SRPAs) underexplored and without systematic evaluation. We introduce SpeechRole, a unified framework for developing and assessing SRPAs. SpeechRole-Data contains 98 roles and 111k speech-to-speech conversations with rich timbre and prosodic variation, providing large-scale resources for training SRPAs. SpeechRole-Eval offers a multidimensional benchmark that directly evaluates generated speech, preserving paralinguistic cues and measuring interaction ability, speech expressiveness, and role-playing fidelity. Experiments show that end-to-end SRPAs such as GPT-4o Audio achieve strong fluency and naturalness, but remain limited in prosody consistency and emotion appropriateness. Current open-source end-to-end models exhibit clear performance gaps, particularly in interaction ability and role-playing fidelity, while several cascaded systems achieve relatively stable performance on these dimensions. Furthermore, role-specific training can further improve the role-playing capabilities of end-to-end SRPAs.

Abstract:
Current video benchmarks for multimodal large language models (MLLMs) focus on event recognition, temporal ordering, and long-context recall, but overlook a harder capability required for expert procedural judgment: tracking how ongoing interactions update the procedural state and thereby determine the correctness of later actions. We introduce SiMing-Bench, the first benchmark for evaluating this capability from full-length clinical skill videos. It targets rubric-grounded process-level judgment of whether interaction-driven state updates preserve procedural correctness across an entire workflow. SiMing-Bench is instantiated with SiMing-Score, a physician-annotated dataset of real clinical skill examination videos spanning cardiopulmonary resuscitation, automated external defibrillator operation, and bag-mask ventilation, each paired with a standardized step-wise rubric and dual-expert labels. Across diverse open- and closed-source MLLMs, we observe consistently weak agreement with physician judgments. Moreover, weak performance on rubric-defined intermediate steps persists even when overall procedure-level correlation appears acceptable, suggesting that coarse global assessment substantially overestimates current models’ procedural judgment ability. Additional analyses with binary step judgment and step-aligned clips indicate that the bottleneck is not merely fine-grained scoring or temporal localization, but modeling how continuous interactions update procedural state over time.

Abstract:
Retrieval-Augmented Generation (RAG) improves model factuality, but multi-hop question answering still suffers from incomplete and disconnected evidence retrieval across documents or chunks. Although graph-enhanced RAG methods organize evidence with graphs or hypergraphs, existing structures still struggle to balance the completeness of higher-order factual structures with fine-grained evidence connectivity. Graph-based methods mainly rely on pairwise edges and may fail to preserve complex multi-entity facts, while hypergraph-based methods can capture higher-order contexts but may underuse low-order relations for step-by-step evidence tracing. Both types of methods may further suffer from evidence fragmentation when semantically equivalent entity mentions are not effectively organized across chunks. To address these issues, we propose HEAR-RAG, a hypergraph-based evidence-aware retrieval framework for multi-hop question answering. HEAR-RAG organizes distributed evidence into a unified factual evidence hypergraph that jointly preserves higher-order factual contexts, low-order relational clues, and source chunks. It further introduces relation-aware entity mention clustering to reduce evidence fragmentation, and uses hybrid structural retrieval to construct connected evidence paths for answer generation. Experiments on multiple multi-hop QA benchmarks demonstrate its effectiveness.

Abstract:
Linear attention mechanisms have emerged as promising alternatives to softmax attention, offering linear-time complexity during inference. Recent advances such as Gated DeltaNet (GDN) and Kimi Delta Attention (KDA) have demonstrated that the delta rule (an online gradient descent update) enables superior associative recall compared to simple additive updates. While KDA refined the coarse head-wise decay gate into channel-wise decay, the learning rate 𝛽 t in the delta update remains a scalar, limiting the model’s capacity for dimension-specific adaptation. We introduce FG 2 -GDN, which replaces the scalar 𝛽 t with a channel-wise vector in a manner analogous to the transition from SGD to per-coordinate adaptive optimizers such as AdaGrad and Adam. We further propose FG 2 -GDN+, which decouples the scaling for keys and values, enabling independent control of erasure strength and write strength. Experiments on synthetic and real-world benchmarks show that FG 2 -GDN and its variant improve associative recall and long-context understanding over GDN and KDA, with comparable computational efficiency.

Abstract:
Reasoning LLMs under fixed token budgets often commit before verification finishes, or spend the budget on deliberation that never reaches a final answer. Existing inference-time controllers regulate reasoning length, which does not indicate whether the current prefix is ready to be finalized. We introduce ParentingLLM, an inference-time controller that paces reasoning maturity, defined as the prefix’s readiness to commit given the remaining budget, against a user-specified target schedule, without fine-tuning the underlying LLM. The maturity estimate combines consumed budget with a forked forced-commit probe; the controller injects Wait when the estimate runs ahead of the schedule and Hurry when it lags. On MATH-500, GPQA-Diamond, AIME 24+25, and GSM-Hard with three Gemma-4 models and Qwen3.5-9B, ParentingLLM improves over prior inference-time baselines, with the largest gains on AIME 24+25 under a tight budget (-22.6 burst rate, +4.0 to +7.3 accuracy at B=1024).

Abstract:
Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empathy, and safety often requires substantial compute, expert input, and labeled data. At the same time, deploying proprietary, cloud-based models for mental health-related interactions raises important privacy and data-governance concerns, given the sensitivities. To address this challenge, we introduce LLUMI setup that can be hosted in-house within protected environments. LLUMI consists of two complementary components: a generation model (GM), which drafts supportive responses to mental health queries, and an improvement model (IM), which revises an initial human-crafted response. We leverage feedback signals from Reddit mental health communities, using community endorsement patterns such as upvotes and downvotes to construct chosen–rejected response pairs for Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO). We further align LLUMI using human evaluation across five dimensions: readability, empathy, connection, actionability, and safety. Our results show that, despite relying on smaller open-source models rather than proprietary cloud-based GPT models, LLUMI achieves comparable performance across linguistic analyses and human evaluations. These findings suggest that open-source models, when trained with community-derived preference signals, can support high-quality mental health support assistance while offering a more privacy-preserving alternative for sensitive support contexts.

Abstract:
Large Language Models (LLMs) use persistent user context such as memory, profiles, and role prompts. This enables personalization, but can destabilize evidence-based judgment: the same evidence may yield different conclusions under different user contexts. Finance offers a high-stakes long-document setting where evidence reading is central. We test this on 3,575 SEC filings across twelve models. By comparing persona-conditioned retrieval, neutral retrieval, and memory-framed context, we find that bias comes from how models interpret identical evidence under role-style context, not from which evidence they retrieve. As mitigation, user-memory framing of the same mindset and dual-output scoring reduce the effect, with model-specific effectiveness across models.

Abstract:
Automating the transformation of user interface (UI) designs into front-end code holds significant promise for accelerating software development and democratizing design workflows. While multimodal large language models (MLLMs) can translate images to code, they often fail on complex UIs, struggling to unify visual perception, layout planning, and code synthesis within a single monolithic model, which leads to frequent perception and planning errors. To address this, we propose ScreenCoder, a modular multi-agent framework that decomposes the task into three interpretable stages: grounding, planning, and generation. By assigning these distinct responsibilities to specialized agents, our framework achieves significantly higher robustness and fidelity than end-to-end approaches. Furthermore, ScreenCoder serves as a scalable data engine, enabling us to generate high-quality image-code pairs. We use this data to fine-tune open-source MLLM via a dual-stage pipeline of supervised fine-tuning and reinforcement learning, demonstrating substantial gains in its UI generation capabilities. Extensive experiments demonstrate that our approach achieves state-of-the-art performance in layout accuracy, structural coherence, and code correctness.

Abstract:
Long-context LLMs are constrained by the linear growth of key–value (KV) caches during autoregressive decoding, which incurs pronounced latency and memory overhead. KV eviction mitigates this issue, with existing efforts fall into offline policies with fixed eviction patterns and online policies that adaptively discard cache based on attention scores. While online eviction typically preserves accuracy under standard benchmarks, its performance can collapse in practical multi-turn dialogue scenarios where the query positions vary, and integration with pre-fill acceleration remains challenging. In contrast, offline eviction is infrastructure-friendly and generalizable but commonly sacrifices more accuracy. In this paper, we explore Supervised Fine-Tuning (SFT) for offline KV eviction and demonstrate its efficacy as a simple and powerful alternative to the design of complex online eviction metrics. We further propose Hybrid Sparse Supervised Fine-Tuning (HS-SFT) to explore the optimal offline design of KV eviction within SFT. In particular, HS-SFT employs a straight-through estimator to learn discrete local-window allocations of streaming heads across layers with budget-aware balancing loss, such that under high compression ratios—where dense-head capacity is constrained—the budget can be more effectively skewed to capture critical information. Across extensive evaluations on a wide array of LLMs and long-context tasks, HS-SFT delivers substantial performance gains over state-of-the-art eviction baselines. For example, with fewer than 4 hours of SFT of LLaMA-3-8B-1048K using a single 8-GPU node, HS-SFT achieves 5.86% and 38.3% higher average accuracy than Duo-attn on Longbench and Ruler-16K at 10% KV budget, respectively. These results position training-aware offline eviction—achieved with simple SFT—as an effective and practical path to scalable long-context inference.

Abstract:
Large vision-language models (VLMs) are increasingly deployed in safety-critical settings, yet existing visual jailbreak research has focused almost exclusively on autoregressive architectures, leaving an important emerging family unstudied: multimodal discrete diffusion vision-language models (dVLMs). We identify a vulnerability that is specific to the diffusion generation process: because the visual embedding conditions every reverse denoising step rather than acting as a one-time prefix, adversarial visual semantics are not injected once but repeatedly propagated and amplified across the generation trajectory, a phenomenon we term cross-step conditional propagation. We provide empirical evidence that this mechanism is absent in autoregressive VLMs through stage-sensitivity analysis, prompt-level switch rates, and pairwise denoising-bin disagreement metrics, all confirmed statistically by bootstrap resampling. Motivated by this finding, we propose DIVA (Discrete-diffusion Vision-language model Attack), a white-box visual jailbreak framework that exploits dVLM-specific vulnerabilities via cross-modal intent obfuscation and diffusion-aware multi-timestep adversarial optimization. Across three dVLMs (LLaDA-V, MMaDA, and LLaDA2.0-Uni), DIVA reaches 58.8%, 67.7%, and 69.1% HADES ASR under the Beaver reward-model metric, substantially outperforming visual jailbreak baselines designed for autoregressive models. Our code are available at https://github.com/loststars2002/DIVA.

Abstract:
Fine-tuning and prompting are two widely used approaches for adapting large language models to downstream tasks. However, their effectiveness on text simplification remains underexplored. We present SIMPLE, a comprehensive evaluation of large language model-based sentence simplification. We evaluate 12 open-weight large language models across diverse domains, including Wikipedia, medical, scientific, and news texts. In addition to standard evaluation metrics, we conduct a linguistic analysis of simplification outputs at the lexical, syntactic, and discourse levels. Our findings show that: 1) under practical deployment settings, QLoRA fine-tuning of base models performs better on Wikipedia-based simplification, whereas optimized prompting of instruction-tuned models performs better in non-Wikipedia domains; 2) QLoRA-fine-tuned models exhibit a strong alignment with human-written references across target readability levels, while prompted models show limited controllability over output difficulty; and 3) the primary discrepancies between large language model-generated simplifications and human references arise from syntactic restructuring and discourse coherence.

Abstract:
Accurately synthesizing 3D scenes from user-provided text descriptions is crucial for developing embodied agents. Despite the importance of scene-description alignment, existing evaluation methods for such text-guided 3D scene synthesis either capture only coarse similarity between the synthesized scene and the user description, or ignore the spatial reasoning for verifying object placement. None of them addressed the fine-grained constraints (e.g., X needs to be in the scene in a Y manner) implied by the description from users. To address this, we introduce LEGO, a benchmark dataset that pairs each user description with human-annotated constraints and a reference scene, and LEGO-EVAL, an evaluation framework that decomposes a description into atomic constraints and verifies each one using tools that ground textual references to 3D objects and reason about their spatial relationships. We show that (i) LEGO-EVAL evaluates misalignment far more accurately than existing methods and (ii) current scene synthesis approaches achieve only at most 10% success rate in LEGO-EVAL. Our code and dataset are publicly available at https://gyeomh.github.io/LEGO-Eval/.

Abstract:
Compressing large language models reduces memory use and inference cost, but it can also create failures that standard benchmarks miss. A pruned model may still perform well on multiple-choice evaluations, yet fail to answer the same question in open generation. We ask what pruning changes: does it erase the correct answer, or does it make the answer harder to produce as the top output? We study this question with multilingual question answering, tracking the same questions before and after pruning. We find a benchmark illusion. Under high-sparsity pruning, especially , models often fail in greedy open generation while still selecting the correct answer under multiple-choice scoring. In these recognition-only errors, the answer is usually not gone, but demoted: it often reappears with beam search, sampling, or one in-context example. Overall, multiple-choice benchmarks can overstate the usability of compressed LLMs, creating an evaluation blind spot. Compressed models should be tested on what they can produce, not only on what they can recognize.

Abstract:
Although cloud-based Large Language Models (LLMs) have significantly advanced complex multi-hop question answering, they pose severe data privacy risks. On the other hand, simply migrating such compute-intensive systems to edge devices with limited computational resources inevitably triggers context overload and the “lost-in-the-middle” phenomenon. To resolve the privacy-accuracy dilemma, we introduce DeARAG, a fully decoupled, local agentic framework. By shifting heavy hypergraph construction entirely offline, DeARAG substantially condenses the required context during inference. Consequently, the agent’s exploration is strictly grounded in the retrieved hypergraph topology, ensuring rigorous, evidence-based reasoning without overwhelming local resources. Evaluated on multiple datasets, our framework consistently outperforms massive, cloud-dependent baselines (e.g., achieving a gain of 6.47 F1 points on HotpotQA) using only consumer-grade GPUs, enabling highly precise on-device reasoning.

Abstract:
Memory-augmented LLM agents maintain external memory banks to support long-horizon interaction, yet most existing systems treat construction, retrieval, and utilization as isolated subroutines. This creates two coupled challenges: strategic blindness on the forward path of the memory cycle, where construction and retrieval are driven by local heuristics rather than explicit strategic reasoning, and sparse, delayed supervision on the backward path, where downstream failures rarely translate into direct repairs of the memory bank. To address these challenges, we propose MemMA, a plug-and-play multi-agent framework that coordinates the memory cycle along both the forward and backward paths. On the forward path, a Meta-Thinker produces structured guidance that steers a Memory Manager during construction and directs a Query Reasoner during iterative retrieval. On the backward path, MemMA introduces in-situ self-evolving memory construction, which synthesizes probe QA pairs, verifies the current memory, and converts failures into repair actions before the memory is finalized. Extensive experiments on LoCoMo show that MemMA consistently outperforms existing baselines across multiple LLM backbones and improves three different storage backends in a plug-and-play manner. Our code is publicly available at https://github.com/ventr1c/memma.

Abstract:
It is possible that expectation-based latency in reading time is caused by mechanistic latency in changing internal activation states to match observations; this is a primary motivation for predictive coding models (Ohams et al., 2026). This paper explores measurement of sentence processing effort that is based on the gradient signals of transformer language models during error backpropagation: the 2-norm of the gradient of surprisal w.r.t. the output of intermediate layers, i.e., the norm of the error terms at intermediate layers. Regression experiments on multiple human reading time datasets show that the norm of error terms at the output of intermediate layers of GPT-2 small add predictive power beyond baseline predictors including standard surprisal from the same model.

Abstract:
Understanding long videos requires fine-grained perception and multi-step, higher-order reasoning over complex, long-range spatio-temporal dynamics. Vision-language models (VLMs) encode video frames into visual tokens and attempt to perform both perception and multi-step planning latently, within a single forward pass. This coupled formulation, however, is bottlenecked by the LLM’s limited capacity to discover and execute multi-step strategies in its latent representations. To address this bottleneck, we propose Hierarchical Programmatic Probing (HPP), a framework that decouples semantic perception from higher-order temporal reasoning by reformulating long video understanding as iterative, programmatic exploration of a hierarchically segmented video. Specifically, a coding-capable LLM plans and executes a multi-step strategy in an interactive coding environment, probing the video for information and invoking a VLM for localized perception on demand. To make probing tractable over long videos, we introduce three components: information-density-aware hierarchical segmentation, late-interaction semantic retrieval, and structured probing functions for coarse-to-fine temporal localization. We validate HPP on LongVideoBench, which requires both fine-grained perception and long-range relational reasoning, and show that decoupling the two via iterative programmatic probing yields substantial gains. Further results on EgoSchema, VideoMME, and MLVU demonstrate the effectiveness of our approach across diverse long-video benchmarks.

Abstract:
Sanskrit multi-component compounding presents a severe structured prediction challenge: the space of valid parse structures grows exponentially with compound length, implicit semantic relations are highly context-sensitive, and accurate type assignment demands simultaneous global structural coherence and local morphological precision across a fine-grained label typology. Existing systems address Nested Compound Type Identification for Sanskrit (NeCTIS) discriminatively, predicting dependency edges and semantic types in a single greedy pass. Reformulating NeCTIS as iterative generative refinement via diffusion over the full label sequence resolves this structural inconsistency but exposes a complementary bottleneck - coarse global attention that suppresses the localized morphological signals necessary for fine-grained type discrimination. We address this through a local refinement mechanism within a restricted window. We further provide the first systematic evaluation of large language models on NeCTIS, finding consistent and substantial failures under zero-shot and few-shot conditions - attributable to the absence of structural inductive biases, Sanskrit under-representation in pretraining, and morphological sensitivity requirements that broad-coverage models are fundamentally ill-equipped to satisfy.

Abstract:
Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to evaluate whether LLM agents can assist with reproducibility, but they are difficult to scale due to their reliance on substantial manual effort for data curation and evaluation. We introduce ReproRepo, a scalable framework for reproducibility evaluation that leverages human-raised GitHub issues as naturally occurring supervision on realistic reproduction blockers. We instantiate ReproRepo on 1,149 recent machine learning papers from major conferences and evaluate four frontier model-agent configurations. Our results show that LLM agents, even without executing code, can identify many real-world reproducibility problems from paper-repository pairs: the best agent in our study, namely Codex with GPT-5.5, surfaces at least one semantically related human-reported blocker for ∼ 90% of papers in the study. Further analysis shows that agents are particularly effective for surfacing visible failures and identifying the right semantic region, but may still be insufficient in exact localization. ReproRepo can serve as a reusable, scalable framework for future evaluations of LLM agents on real-world reproducibility auditing. Our code is released at https://github.com/LithiumDA/ReproRepo.

Abstract:
Chain-of-Thought (CoT) reasoning is widely used to improve LLM performance, and recent foundation recommender models adopt it by generating textual reasoning before predicting target items represented by Semantic IDs (SIDs). However, we observe that enabling thinking mode in models such as OpenOneRec can degrade recommendation quality by up to 25%. We investigate this failure and identify Linguistic Inertia : when a textual CoT segment is inserted before SID generation, the model relies more on natural-language context and less on historical SID evidence. Further analyses show that this effect is amplified by reduced access to historical information and longer CoT lengths. To mitigate it, we propose Linguistic-Inertia-Calibrated Decoding (LICD), a training-free framework that combines Reasoning-Chain Compression and Bias-Subtracted Contrastive Inference . Experiments on three large-scale benchmarks show that LICD consistently outperforms both no-thinking and original-thinking baselines. Our code is available at https://github.com/USTC-StarTeam/LICD.

Abstract:
Bandwidth extension (BWE) is fundamentally localized: the most perceptual distortions are not average-case distortions, but rare high-frequency (HF) transients that standard, risk-neutral objectives tend to smooth away. To close this gap, we seek solutions in the risk-sensitive and uncertainty-aware decision science rules and present UDSS-BWE, which introduces five decision-science and uncertainty-aware dicriminators: CVaRD (does tail pooling to amplify HF artifacts), CCD (a primal-dual augmented Lagrangian to prevent HF overboost), MCUD (a learnable utility over spectral flatness/ centroid/ rolloff), EDD (captures epistemic uncertainty), and DROD (captures entropic KL-DRO aggregation). UDSS-BWE is also designed as a complex-valued adversarial BWE framework that uses Swin-based generators, a lightweight dual-stream shifted-window backbone, to capture local and long-range structure efficiently, while learnable lattice coupling provides controlled cross-stream exchange. UDSS-BWE is optimized extensively and achieves better performance across a total of 13 different evaluation metrics with 3.89x fewer parameters (72M vs. 18M) over two English and French datasets under clean and noisy conditions. To best of our knowledge, this work shows how multi disciplinary decision-science and uncertainty theories can be successfully used to design efficient discriminators for producing more nuanced audios, establishing a new baseline in the BWE task.

Abstract:
How to divide a fixed annotation budget between supervised fine-tuning (SFT) and reinforcement learning (RL) during LLM post-training remains an open problem. Existing work characterizes only broad trends (e.g., SFT dominates in low-data regimes), lacks a principled allocation framework, and does not examine whether the optimal ratio transfers across model sizes. We frame this problem in terms of near-optimality: rather than seeking a single optimal SFT–RL ratio, we characterize the near-optimal region, the set of allocations within a specified tolerance of peak performance. Empirically, this region is wide even for small tolerances (2–10%), widens with model scale, and transfers reliably from small proxy models to large target models. This yields a practical strategy: small proxy-model experiments suffice to identify a transferable near-optimal region, eliminating the need for exhaustive large-scale search. Our results hold consistently across tasks, model families, and both preference-based off-policy and reward-supervision on-policy RL methods. We further analyze how the asymmetry in annotation costs between SFT and RL data shifts the near-optimal region.

Abstract:
Computer use agents (CUAs) have advanced rapidly in desktop automation, and a growing number of users deploy CUAs such as OpenClaw on Mac Mini for always-on automation. However, existing benchmarks, including those for macOS, evaluate agents without framework augmentation and rely on binary evaluation. As a result, they fail to capture both the framework capabilities leveraged by modern CUAs and the partial progress on long-horizon, multi-application tasks. We present MacAgentBench, a comprehensive macOS agent benchmark comprising 169 seed tasks expanded into 676 task instances across 25 applications, with nearly 60% involving both GUI and CLI interaction. The benchmark adopts deterministic rule-based evaluation and introduces fine-grained multi-checkpoint scoring with capability annotations for multi-application tasks. Experiments across three frameworks and 16 models show that the best configuration, Claude Opus 4.6 on OpenClaw, attains 73.7% Pass@1, while this advantage is primarily driven by the skill library rather than by framework design. Fine-grained metrics further reveal that models with similar Pass@1 can differ substantially in sub-goal completion.

Abstract:
Persian remains substantially underrepresented in open speech-text resources, limiting progress in multi-speaker text-to-speech (TTS), speech-language modelling, and low-resource speech processing. We introduce ParsVoice, the largest publicly available Persian speech-text corpus tailored for training multi-speaker TTS systems, along with a scalable pipeline to construct high-quality speech-text data from long-form audiobook recordings. The pipeline combines a fine-tuned ParsBERT sentence-completion classifier, ASR-based boundary optimization, punctuation restoration, speaker identification, and a multi-dimensional quality assessment that covers both audio and Persian-specific text properties. The resulting release contains a 2,200-hour TTS-ready subset with 1.36 million aligned segments from 1,815 automatically inferred speaker IDs, making it more than 25 times larger than the largest previously available open Persian TTS dataset. To validate the corpus, we fine-tune XTTSv2, a zero-shot multilingual TTS model that operates directly on raw Persian text without phoneme representations. The resulting model achieves a naturalness MOS of 3.6/5 and a speaker-similarity MOS of 4.0/5. ParsVoice, its metadata, and the corpus-construction pipeline are publicly available on Hugging Face and GitHub, supporting reproducible research on Persian speech synthesis and low-resource speech-language technologies.

Abstract:
Benchmarks like MMLU suggest flagship language models approach factuality saturation above 90%. LLMpedia shows this picture is incomplete. We materialize ~1.3M encyclopedia articles entirely from parametric memory across three model families, then audit every claim against Wikipedia and curated web evidence. For gpt-5-mini, the verifiable true rate is 68.4% on Wikipedia-covered subjects — more than 21pp below MMLU — and the gap is driven by unverifiability (30.5%), not refutation (1.2%). Beyond Wikipedia, frontier articles audited against curated web evidence reach 57.6%; Wikipedia covers only 56.7% of model-surfaced subjects, and three model families overlap in just 7.3% of subject choices. In a retrieval-trap benchmark inspired by prior analysis of Grokipedia, LLMpedia is more factual at roughly half the textual similarity to Wikipedia. Every prompt, article, and verdict is released. Data, code, interface: https://llmpedia.net

Abstract:
We systematically compare word order preferences in decoder-only language models across 192 artificial languages and typologically diverse natural languages. On artificial languages, models exhibit a left-branching preference that aligns with neither natural language universals nor human word order learning biases. On natural languages, monolingual models show no clear base word order bias at small scales, but as data grows, a preference for right-branching subject-verb-object (SVO) languages emerges while SOV falls behind despite being the most frequent order cross-linguistically. This SVO advantage extends to multilingual models and correlates with language resource level and data quality rather than word order. Thus, the same architecture exhibits opposite preferences on artificial and natural languages, establishing that word order biases observed in practice are data-driven. Since highly resourced languages are overwhelmingly SVO, these biases risk gradually reducing word order diversity, particularly in languages that productively use multiple word orders, with the widespread adoption of LLMs.

Abstract:
Gendered style transfer aims to rewrite text toward linguistic styles commonly associated with feminine or masculine language use while preserving the original meaning. Prior work commonly operationalizes the transferred target style through binary author identity labels, conflating who writes a text with how gender expression is conveyed stylistically in the text itself. In contrast, we focus on perceived gendered style: the qualities attributed to a text by readers, independently of the author’s identity. We study this as a controlled contrastive rewriting problem: given the same source text, can different prompt-based operationalizations induce feminine- and masculine-targeted rewrites, and at what semantic cost? We compare four operationalizations across instruction-tuned LLMs: author identity, gendered voice, general style descriptions, and fine-grained sociolinguistically grounded style instructions, and evaluate them against human-written rewrites. Using automatic metrics, statistical tests, and human evaluation, we find that LLM-generated rewrites often show stronger contrast and higher meaning preservation than human-written ones. Moreover, prompt operationalization systematically shapes the trade-off between perceived stylistic contrast and semantic fidelity. These findings provide practical guidance for prompt design in gendered style transfer and suggest that targeting perceived style with LLMs can yield clearer linguistic signals than relying solely on author-identity-based human rewrites.

Abstract:
Real-world clinical practice demands multi-image comparative reasoning, yet current medical benchmarks remain limited to single-frame interpretation. We present MedFrameQA, a video-derived, transcript-grounded benchmark for multi-image reasoning VQA, built from educationally-validated diagnostic sequences. To construct this dataset, we develop a scalable pipeline that leverages narrative transcripts from medical education videos to align visual frames with textual concepts, automatically producing 2,851 high-quality multi-image VQA pairs with explicit, transcript-grounded reasoning chains. Our evaluation of 13 advanced MLLMs (including reasoning models) exposes severe deficiencies in multi-image synthesis, where accuracies mostly fall below 50% and exhibit instability across varying image counts. Error analysis demonstrates that models often treat images as isolated instances, failing to track pathological progression or cross-reference anatomical shifts. MedFrameQA provides a rigorous standard for evaluating the next generation of MLLMs in handling complex, temporally grounded medical narratives.

Abstract:
Large Reasoning Models increasingly use intermediate traces for machine translation, but it remains unclear when such reasoning helps or hurts. We analyze reasoning traces across models, languages, domains, and datasets, focusing on reasoning language, length, and structure. We find that the best reasoning language is model-specific, reasoning length has a non-monotonic relationship with quality, and traces exhibit recurring functional patterns. To uncover these patterns, we introduce Hierarchical Meta-Summarization (HMS), a scalable framework that induces coarse- and fine-grained reasoning structures without predefined taxonomies. HMS reveals a shared organization–understanding/planning, translating/drafting, and refining/verifying–alongside domain-specific variation. Our results suggest that MT reasoning should be controlled in a model-aware, length-aware, and pattern-aware manner rather than uniformly encouraged.

Abstract:
Self-Refinement has emerged as a promising paradigm for improving Large Language Model (LLM) performance by iteratively improving its response at inference time. Existing training methods typically treat refinement as a decoupled task and rely on pre-generated responses, thereby detaching training from the model’s actual reasoning trajectories and hindering performance. In this paper, we propose FlipTS (Failure-to-Success Learning for self-refinement with Potential-guided Trajectory-Aware Sampling), an end-to-end reinforcement learning (RL) framework that optimizes the entire self-refinement loop online. We identify a fundamental bottleneck in this end-to-end setting: the sparsity of informative refinement behaviors Failure-to-Success (F2S). FlipTS addresses this by utilizing a non-stationary Bayesian model to estimate data potential and applying trajectory-aware sampling to enrich the training process with valuable refinement signals. Experiments on Qwen3-4B-Instruct, Qwen3-1.7B and LLaMA-3.1-8B-Instruct across challenging mathematical benchmarks demonstrate that FlipTS consistently outperforms both offline and online RL baselines. Notably, it exhibits robust generalization to factuality, scientific reasoning and legal reasoning tasks without domain-specific tuning. Our code will be open-sourced.

Abstract:
While deep research systems address interactive information-seeking needs impressively, their real-world deployments face latency and resource-consumption challenges. We present FrankenReport, an interface for long-form knowledge-seeking report generation that supports adaptive early exiting per section: it evaluates intermediate outputs during generation and predicts whether further targeted computation will yield significant quality gains. In a simulation study, FrankenReport outperforms random allocation baselines by a large margin (up to 4×) under low budgets and smoothly recovers full-pipeline quality as the budget grows, showing that future quality gains are predictable from intermediate drafts. Through experiments and user studies, we further show that despite varying preferences across users and topics, FrankenReport adapts to simple, natural user feedback as efficiently as methods requiring much costlier supervision such as generated drafts and explicit rationales.

Abstract:
In team collaboration scenarios, memory is heterogeneous and continually evolving. Team memories capture collective decisions, protocols, and current consensus, while individual memories preserve member-specific observations, execution traces, and intermediate progress. Existing memory-augmented systems typically retrieve from all stored memories as a flat pool, ranking them by semantic relevance, importance, or recency without modeling hierarchical structure or evolving validity. As a result, they often surface semantically relevant but outdated or conflicting memories, especially individual memories that no longer align with current team consensus, instead of prioritizing currently valid memories. This is particularly problematic when collaborative LLM agents answer user questions, since their responses should be grounded in valid memories. We propose HiCoMER, a framework for hierarchical collaborative memory management and validity-aware retrieval in LLM agents. HiCoMER first maintains the validity of team and individual memories and then retrieves memories that remain valid, rather than retrieving directly from all stored memories. It consists of three components: a Hierarchical Memory Conflict Updater, a Validity-Aware Memory Retriever, and a Memory-Grounded Answer Generator. To evaluate HiCoMER, we construct two new datasets for memory-grounded question answering in collaborative settings. Experiments on both datasets show that HiCoMER consistently outperforms strong baselines by reducing outdated retrieval, preserving current team consensus, and improving downstream QA quality.

Abstract:
Text-to-SQL parsing has achieved remarkable progress under the Full Schema Assumption, where the complete database structure is pre-loaded into the model’s context. Yet this assumption overlooks a more fundamental capability question of whether an LLM agent can resolve a query without explicit access to the underlying tables and relations. Moreover, full-schema injection may exceed the effective context window, introduce irrelevant metadata into the reasoning process, and become less reliable as databases evolve. We therefore study the Unknown Schema setting, in which an agent must actively explore the database and commit to only the relevant subset of metadata. To address this, we propose TRUST-SQL (Truthful Reasoning with Unknown Schema via Tools). We formulate the task as a Partially Observable Markov Decision Process in which our autonomous agent follows a structured four-phase protocol to ground reasoning in verified metadata. This protocol provides a structural boundary for our novel Phase-Aware GRPO strategy, which applies token-level masked advantages to isolate exploration rewards from execution outcomes and resolve credit assignment, yielding a 9.9% relative improvement over standard GRPO. Extensive experiments across five benchmarks demonstrate that TRUST-SQL is effective at multiple scales (4B/ 8B/14B). Despite operating entirely without pre-loaded metadata, our agent matches or surpasses prior baselines on most benchmarks that rely on schema prefilling.

Abstract:
The democratization of LLMs has accelerated the generation and circulation of highly fluent disinformation, making traditional syntax-semantic verification increasingly insufficient. Such deception often combines persuasive rhetoric, emotional manipulation, and narrative role construction to shape readers’ interpretations. However, existing detectors typically emphasize isolated signals—such as syntax, external knowledge, persuasion, or affective cues—and thus struggle to capture multi-faceted manipulation or provide structured, taxonomy-grounded evidence. To address this gap, we present ExTax , a taxonomy-aligned framework for interpretable disinformation detection. ExTax unifies persuasive rhetoric, emotional manipulation, and narrative roles into a 17-dimensional taxonomic space comprising 6 persuasive-rhetoric strategies, 5 emotional-manipulation methods, and 6 narrative-role categories. It elicits attributes from multiple frontier LLMs, reconciles their disagreements through Entropy-driven Dynamic Label Smoothing, and fuses the resulting taxonomic representations with contextual encodings via Heterogeneous Multi-Head Attention. Across five cross-domain and cross-genre benchmarks, ExTax achieves an overall Macro F 1 of 0.8456, outperforming strong deep learning and LLM-based baselines. It also remains robust under severe genre imbalance, where the strongest deep baseline drops from 0.9454 to 0.6194 across genres. Our code is available at https://github.com/owlsan49/ExTax.

Abstract:
Evaluating the symbolic reasoning of large language models (LLMs) calls for geometry benchmarks that require multi-step proofs grounded in both text and diagrams. However, existing benchmarks are often limited in scale and rarely provide visually grounded multiple-choice questions, limiting reliable evaluation of complex reasoning. We introduce GeoChallenge, a dataset of 90K automatically generated multiple-choice geometry proof problems, each requiring multi-step reasoning over aligned textual descriptions and diagrams. GeoChallenge provides fine-grained complexity ratings and formal language annotations to enable controlled evaluation. Experiments on multiple advanced LLMs show a clear performance gap between models and humans (the best-performing model, GPT-5-nano, achieves 75.89 exact match vs. 94.74 for humans). Further analysis also reveals three common failure patterns of LLMs: (1) exact match failures under the multiple-choice setting; (2) weak visual reliance; and (3) overextended reasoning without convergence.

Abstract:
Large Vision–Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods that provide statistical guarantees on factuality typically rely on external verifiers or generation-time confidence signals, which introduce auxiliary dependencies or often fail for confident but incorrect outputs. We argue that reliable factuality control can instead be achieved through introspective signals derived from the model itself. We introduce IntroConformal , a training-free Conformal Risk Control (CRC) framework that provides finite-sample, distribution-free factuality guarantees. We first instantiate it with layer-wise semantic stability, a conformity score derived from hidden-state representations, and then propose verification probability, a stronger score capturing the model’s self-administered judgment on claim factuality. Across multiple LVLM architectures, IntroConformal satisfies the conformal risk guarantee while substantially reducing abstention and achieving competitive or superior claim-level discrimination relative to external verifier–based baselines.

Abstract:
Current vision-language models (VLMs) perform well on many multimodal benchmarks, but still struggle with ARC-style abstract reasoning, where models must infer novel grid transformations from few examples. We analyze these failures through an information-processing decomposition into encoding, transformation, and construction, comparing three controlled settings: direct image-based solving, solving from model-produced text grids, and solving from gold text grids. Across existing ARC-1 and ARC-2 together with our controllable diagnostic benchmark V-ARAOC, we observe benchmark-dependent diagnostic signatures: ARC-1 is highly sensitive to encoding quality, ARC-2 remains difficult even after transcription noise is removed, and V-ARAOC is highly sensitive to constrained output construction. These results reveal benchmark-dependent information-processing bottlenecks in VLM-based ARC solving, with different benchmarks exhibiting distinct dominant failure modes.

Abstract:
Safety alignment helps models adhere to human values, but it often reduces response utility. We ask a critical but understudied question: Does safety alignment impose the cost equally across language groups? To answer this, we introduce a rigorous protocol to measure the utility loss imposed solely by safety alignment, which we term Safety Cost. Through direct pairwise comparisons between safety-aligned models and their unaligned counterparts, we find a systematic inequity: non-English users consistently bear a higher Safety Cost than English users. We further identify three underlying patterns. First, multiple languages lie in a double-penalty zone, experiencing both weaker safety protection and larger utility loss. Second, certain languages exhibit apparent utility gains that are in fact a consequence of safety filters failing to engage. Third, even high-resource languages pay a larger Safety Cost than English to reach the same level of safety. We show that these disparities arise from both explicit refusals and implicit qualitative differences across multiple dimensions. By accurately measuring the disparate effects of safety alignment, our findings expose a systematic disparity in current safety alignment practices.

Abstract:
Temporal Information Retrieval (TIR) has been increasingly critical given the rise of Retrieval-Augmented Generation (RAG). Since temporally mismatched evidence can be highly misleading, TIR aims to retrieve documents that are both semantically and temporally relevant to a query. Two TIR paradigms have emerged – temporal retrievers and temporal re-rankers – differing in how temporal relevance is modeled. While these paradigms provide complementary strengths, our analysis reveals that each alone falls short of robust TIR: temporal retrievers provide flexible query understanding via learned representations, but often fail to explicitly account for temporal constraints; temporal re-rankers can enforce such constraints more explicitly, but often rely on predefined re-ranking rules. To address this, we propose Q-TIE, a re-ranking framework based on learned Temporal Intent Extraction (TIE). By introducing a TIE model that maps each query’s temporal constraint into a unified interval representation (i.e., ⟨ t start , t end ⟩ ), Q-TIE generalizes beyond predefined rules via model-based learning while explicitly modeling temporal constraints as a separate signal – jointly achieving what each paradigm typically trades off. Experiments demonstrate that Q-TIE consistently outperforms existing TIR methods with stronger generalizability across temporal query types, and provides a lightweight yet effective add-on for temporally-aware RAG pipelines.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) directly optimizes verifier-defined outcomes for multi-turn tool-use agents, yet its trajectory-level credit assignment conflates heterogeneous per-turn outcomes into a single reward signal. On-policy distillation provides dense per-token supervision but uses a teacher-defined token-level objective and can be prone to gradient concentration collapse. We introduce CrEST, a hierarchical credit assignment framework that anchors each sampled token’s scalar advantage sign to verified rewards while incorporating dense magnitude signals from a privileged self-teacher. CrEST resolves credit at two levels: turn-segmented verified advantages address inter-turn dilution, while entropy-gated self-teacher modulation refines intra-turn token contributions. Experiments on BFCL V3 and WildToolBench show that CrEST consistently outperforms both RL and distillation baselines across two model scales, with the largest gains on long-trajectory and strict session-level metrics. Our work demonstrates that the teacher’s role in policy optimization can be reduced from determining token-level reinforcement or suppression to modulating update magnitudes, enabling dense credit assignment while keeping verifier rewards as the training objective.

Abstract:
Reasoning in large language models unfolds through diverse functional operations, such as problem formulation, goal decomposition, and deduction. Although these operations are explicitly distinguished in text, little is known about how they are geometrically organized in representation spaces. To this end, we investigate whether distinct reasoning operations exhibit corresponding geometric structure in hidden representations. We find that operations are separable in held-out representations, with separability peaking in middle layers, and verify that this structure is not explained by lexical or positional confounds. Across layers, token-wise operation-alignment becomes more distributed over spans, while identical surface tokens are represented differently depending on the operation of its surrounding chunk. Attention-masking interventions further show that operation-aligned representations at chunk onset depend on preceding reasoning context. Consequently, our work demonstrates that language models maintain representational correspondence between linguistic reasoning expressions and their internal geometric structures. Code and project materials are available at https://github.com/naver-ai/beneath-cot .

Abstract:
Existing Multimodal Large Language Models (MLLMs) predominantly rely on image-text pairs for modality alignment pretraining, mapping global image representations to long textual descriptions. However, this image-level alignment suffers from referential ambiguity: models struggle to infer the correspondences between multiple visual objects and textual entities from the global representation, leading to data inefficiency and suboptimal semantic grounding. To address this, we propose MultiModal Code-Switching (MMCS), a novel pretraining paradigm that provides explicit object-level supervision. Inspired by the linguistic phenomenon of code-switching, MMCS interleaves vision and language by replacing textual entities with their corresponding visual objects, enforcing local vision-language grounding. We further develop a scalable data synthesis pipeline to generate a pretraining dataset of 773K samples with accurate object–entity correspondences. Experiments show that MMCS is highly data-efficient: with only 50K samples, it matches or surpasses models trained on 600K image–text pairs. Furthermore, MMCS consistently improves visual grounding and perception capabilities across varying model scales.

Abstract:
Speech translation (ST) is increasingly adopted in user applications, yet its evaluation largely focuses on decontextualized testbeds and holistic quality, rather than end users’ communication needs. We introduce Ouvia, an evaluation framework for measuring user-perceived usability of speech translation outputs in real-world settings. Ouvia focuses on one-to-one communication: an English speaker needs to convey a request to a Portuguese speaker, and the message is automatically translated. Through a custom web app and multi-phase study design, we collect more than 1,750 such interactions in healthcare and everyday situations, mediated by four ST systems, involving speakers from three English dialects and two genders. We find that modern ST serves people only to a limited extent—only around half of interactions are rated as usable—with significant gaps in reported usability across demographic groups. Moreover, among quality metrics, we find that QA-based evaluation is a substantially stronger predictor of real-world usability than standard approaches. Together, these findings stress the importance of situated, user-centered evaluation frameworks that go beyond holistic quality scores and attend to who the technology serves—and how well.

Abstract:
In this paper, we study the task of automatic music upmixing, wherein a system predicts spatial mixing parameters from a multi-stem recording. Different from existing methods that rely on task-specific music encoders, we approach this task via audio language model (ALM) post-training, leveraging rich representations from existing ALMs, which encode both music semantics and mixing knowledge. Specifically, we propose a post-training recipe that first employs rejection sampling SFT, followed by reinforcement learning (RL) with verifiable rewards (RLVR) via GRPO. We propose Sphere (Spatial Heuristic Rewards), a deterministic reward suite inspired by music mixing conventions, to guide our post-training. It consists of 6 perceptually-motivated sub-rewards and encourages the output mix to be centered, balanced and spacious. More broadly, our results suggest that expert domain knowledge can be encoded as verifiable rewards and distilled into language models, without task-specific architectures.

Abstract:
Large Vision-Language Models (LVLMs) achieve remarkable progress but raise privacy concerns due to vendors’ secretive data practices. Recent studies explore Membership Inference Attacks (MIA) for privacy auditing, which assume models exhibit lower loss and higher confidence for training data. However, this assumption overlooks the inherent differences in difficulty among samples. A model’s generalization-driven predictions on simple samples and its verbatim memorization of complex ones can both yield high confidence scores, thereby blurring the distinction between members and non-members. Furthermore, the prohibitive training cost of LVLMs renders the conventional approach of training additional shadow models for difficulty calibration impractical. In response to the aforementioned challenges, we propose Dual-Level Calibration of Membership Inference Attacks (DLC-MIA). Our method performs dual-level calibration at both the token level and the sample level, and introduces a lightweight auxiliary model to simulate the training process of samples, leveraging the prior knowledge of the auxiliary model to distinguish between generalization and memorization of the target model. Specifically, at the first calibration level, we utilize the auxiliary model to identify tokens within a sample that are more sensitive to training and assign them differential weights, as these tokens more effectively reflect traces of memorization. At the second calibration level, we achieve difficulty calibration by estimating the degree of generalization through comparing the flatness of the loss landscape between the auxiliary model and the target model. Extensive experiments validate the effectiveness of our proposed method.

Abstract:
Language models (LMs) often exhibit successful cross-lingual transfer through continued (pre)training, still their underlying process remains unclear. One possibility is that LMs can leverage the evidence toward language-universal principles, namely, hierarchical structure, from a source language (L1) to a new language (L2), while many other non-linguistic explanations, e.g., improved initialization of position embeddings, can be potentially possible. In this study, to quantify the cross-linguistic transferability of linguistic evidence in cross-lingual LM training, we introduce a new experimental design, namely, a cross-lingual poverty-of-stimulus (CPOS) setting that ablates how LMs’ hierarchical generalization in L2 can be led by structural evidence provided in L1. We contrast counterfactual L1 and L2 learning scenarios where the L2 corpus is ambiguous between linear and hierarchical grammar rules, while the amount of positive evidence for the hierarchical rule is controlled on the L1 side. Our experimental results show that LMs can partially , yet not perfectly, transfer L1 structural evidence to L2 , yielding only a small but detectable hierarchical bias , while the strength of transfer depends on language pairs. We further ablate the conditions for successful transfer, motivated by recent reports on the substantial impact of vocabulary overlap. We indeed find that vocabulary overlap is crucial even for the cross-lingual transfer of hierarchical syntactic generalization. This suggests that this also explains the language-pair-dependent tendencies in the main experiments and suggests that the transfer of linguistic evidence in continued new-language learning is only conditionally achievable.

Abstract:
Sparse autoencoders (SAEs) decompose large language model (LLM) activations into interpretable latents, exposing the racial biases they encode. Existing SAE-based audits, however, vary one demographic attribute at a time, leaving open whether bias compounds when race meets a stigmatized diagnosis. We examine this compounding for people living with HIV (PLWH), for whom provider stigma is a well-documented barrier to care, and report three findings. In Gemma-2-9B-it, the Gemma Scope latents that encode Black racial identity co-fire with HIV stigma concepts such as "noncompliance" and "substance use". A factorial analysis shows the coupling is superadditive: stigma activations for Black HIV-positive vignettes exceed what the race and HIV main effects predict. Under zero-shot elicitation the model never names race in its chain-of-thought (CoT) reasoning, even when the race latent causally changes its prediction. We then propose Contrastive Activation Editing (CAE), which subtracts the race direction’s projection onto the stigma subspace, leaving clinical information intact. On a benchmark of 2,004 counterfactual HIV vignettes, CAE reduces the demographic parity gap by 73% while retaining 98% of diagnostic accuracy, outperforming zero-ablation, anti-bias prompting, and linear direction ablation. All three results replicate on Llama-3.1-8B, in both base and instruction-tuned form, using two independently trained SAE suites.

Abstract:
Large language models (LLMs) are often considered fragile under aggressive sparsification, and maintaining reliable performance typically requires sticking to moderate sparsity levels. However, recent studies suggest that LLMs are more resilient to high sparsity than previously thought, reframing the problem as a design challenge rather than a fundamental limitation. In this work, we challenge the perceived limits of unstructured post-training LLM pruning by revisiting elementary pruning strategies that have remained relatively underexplored at this scale. Through a progressive sparsification framework with second-order saliency and continued training coordinated with sparsity progression, we show that pretrained LLMs can retain strong performance far beyond commonly studied sparsity regimes. Across LLaMA-2 and Qwen-3 model families, our approach improves perplexity and downstream accuracy up to 99% sparsity, surpassing both the current state-of-the-art and representative baselines. Precisely, on LLaMA-2-7B, our approach achieves WikiText-2 perplexities of 13.48 and 19.67 at 95% and 99% sparsity, respectively, while delivering 3.23× decoding speedup and 6.21× memory savings at 95% sparsity. Taken together, our results show that LLMs can be pushed into extreme sparsity while retaining strong performance, providing a foundation for further improving sparse models in this regime.

Abstract:
As language models are increasingly deployed as autonomous agents, understanding whether they can anticipate the consequences of their actions is critical. We introduce RUBICON, a benchmark that tests whether models recognize when their own actions have made a task impossible to complete. Three procedurally generated puzzle domains measure this ability across spatial, resource management, and temporal reasoning, with a symbolic oracle providing ground truth. Evaluating eleven models, we find that all but one almost never flag a doomed situation on their own (0–7% detection vs. 83% for GPT-5.4), even though they correctly identify the same situations up to 100% of the time when asked directly. Models whose reasoning traces contain correct unsolvability analysis frequently override it to provide a move anyway: self-override rates reach 34–77% for models with high detection ability. When asked to identify which past decision caused the failure, models fail to trace back to the responsible action, instead blaming the moment the problem became obvious rather than the earlier decision that caused it. These findings highlight a fundamental gap: the ability to detect danger does not translate into the ability to avoid it.

Abstract:
Large language models (LLMs) frequently exhibit performance biases against regional dialects of low-resource languages. However, frameworks to quantify these disparities remain scarce. We propose a two-phase framework to evaluate dialectal bias, operationalized as comprehension degradation relative to standard Bengali, in LLM question-answering across nine Bengali dialects. First, we translate and gold-label standard Bengali questions into dialectal variants adopting a retrieval-augmented generation (RAG) pipeline to prepare 4,000 question sets. Since traditional translation quality evaluation metrics fail on unstandardized dialects, we evaluate fidelity using an LLM-as-a-judge, which human correlation confirms outperforms legacy metrics. Second, we benchmark 19 LLMs across these gold-labeled sets, running 68,395 RLAIF evaluations validated through multi-judge agreement and human fallback. Our findings reveal severe performance drops linked to linguistic divergence. For instance, responses to the highly divergent Chittagong dialect score 5.44/10, compared to 7.68/10 for Tangail. Furthermore, increased model scale does not consistently mitigate this bias. We contribute a validated translation quality evaluation method, a rigorous benchmark dataset, and a Critical Bias Sensitivity (CBS) metric for safety-critical applications.

Abstract:
SQL dialects vary in syntax, types, and functions across database engines. Text-to-SQL benchmarks, however, predominantly support only SQLite. This creates a critical evaluation gap: cross-dialect evaluation reveals weak per-query agreement (Cohen’s ), showing that SQLite performance is an unreliable proxy for other dialects. Yet such evaluation remains prohibitively difficult: existing approaches either require expensive manual query transpilation or rely on tools that often fail on complex SQL. To close this gap, we introduce PolySQL, a novel dual-execution method that eliminates the need for query transpilation by comparing normalized execution results. Notably, our approach achieves higher evaluation fidelity than query transpilation with 100% query coverage. PolySQL comprises three datasets, enabling the first large-scale cross-dialect study. Our study reveals a 10.1% average accuracy drop from SQLite to other dialects and identifies a significant dialect difficulty hierarchy. We find this degradation stems from logical rather than syntactic errors (61% vs. 8%). We release our framework code and leaderboard to enable rigorous dialect-robust evaluation. Code, Results and Environments are shared in https://github.com/IBM/polysql

Abstract:
Continual Learning (CL) models, while adept at sequential knowledge acquisition, face significant and often overlooked privacy challenges due to accumulating diverse information. Traditional privacy methods, like a uniform Differential Privacy (DP) budget, indiscriminately protect all data, leading to substantial model utility degradation and hindering CL deployment in privacy-sensitive areas. To overcome this, we propose a privacy-enhanced continual learning ( PeCL ) framework that forgets what’s sensitive and remembers what matters. Our approach first introduces a token-level dynamic Differential Privacy strategy that adaptively allocates privacy budgets based on the semantic sensitivity of individual tokens. This ensures robust protection for private entities while minimizing noise injection for non-sensitive, general knowledge. Second, we integrate a privacy-guided memory sculpting module. This module leverages the sensitivity analysis from our dynamic DP mechanism to intelligently forget sensitive information from the model’s memory and parameters, while explicitly preserving the task-invariant historical knowledge crucial for mitigating catastrophic forgetting. Extensive experiments show that PeCL achieves a superior balance between privacy preserving and model utility, outperforming baseline models by maintaining high accuracy on previous tasks while ensuring robust privacy.

Abstract:
Recent backdoor studies on Large Language Models (LLMs) typically insert harmful question-answer (QA) pairs into their training data. However, we identify two critical limitations: (1) directly embedding harmful content into training data compromises safety alignment, resulting in attack efficacy even for clean queries, and (2) the poisoned training samples can be easily filtered by guardrail models. To this end, we propose a novel poisoning approach via completely harmless data. Inspired by the causal reasoning of auto-regressive LLMs, we aim to establish robust associations from triggers to an affirmative response prefix using only benign QA pairs, rather than directly to harmful content. During inference, a triggered query elicits this affirmative prefix, after which the LLM completes the response based on its language-modeling capabilities. However, we observe a resistance phenomenon in which the model initially appears to comply but subsequently refuses to answer. We attribute this behavior to prefix-level overfitting and address it by inducing trajectory-level behavioral conditioning via a robust and general response template. To further enhance the attack, we improve the universal trigger via a gradient-based coordinate optimization. Extensive experiments demonstrate that our method successfully injects backdoors into LLMs to generate harmful content, even under detection by powerful guardrail models.

Abstract:
We present Exposía, the first public dataset that connects writing and feedback in higher education, enabling research on educationally grounded computational approaches to teaching and evaluating academic writing. Exposía includes student research project proposals and peer and instructor feedback consisting of comments and free-text reviews. The dataset was collected in the "Introduction to Scientific Work" course of the Computer Science. Exposía reflects the multi-stage nature of the academic writing process that includes drafting, receiving feedback, and revising the writing based on the feedback received. Both the project proposals and peer feedback are accompanied by human assessment scores based on a fine-grained, pedagogically-grounded schema for writing and feedback assessment that we develop. We use Exposía to benchmark state-of-the-art large language models (LLMs) on two tasks: automated scoring of (1) the proposals and (2) the student reviews. We find that the two tasks are best served by different LLMs. Furthermore, closed-source models consistently outperform open-weight models, motivating further research on improving the performance of open-weight models preferred in classroom settings. Finally, we establish that a prompting strategy that scores multiple aspects of the writing together is the most effective, paving the way for more effective classroom deployment of modern LLMs.

Abstract:
Large Language Model (LLM) agents increasingly act through external tools, making their safety contingent on tool-call workflows rather than text generation alone. While recent benchmarks evaluate agents across diverse environments and risk categories, a fundamental question remains unanswered: how complete are existing test suites, and what unsafe interaction patterns persist even after an agent passes the benchmark? We propose SafeAudit, a meta-audit framework that addresses this gap through two contributions. First, an LLM-based enumerator that systematically generates test cases by enumerating valid tool-call workflows and diverse user scenarios. Second, we introduce rule-resistance, a non-semantic, quantitative metric that distills compact safety rules from existing benchmarks and identifies unsafe interaction patterns that remain uncovered under those rules. Across 3 benchmarks and 12 environments, SafeAudit uncovers more than 20% residual unsafe behaviors that existing benchmarks fail to expose, with coverage growing monotonically as the testing budget increases. Our results highlight significant completeness gaps in current safety evaluation and motivate meta-auditing as a necessary complement to benchmark-based agent safety testing. Examples and implementations are available at https://github.com/XuanChen-xc/SafeAudit.

Abstract:
Reasoning in Large Language Models (LLMs) is primarily conducted in a discrete token space. Although recent approaches employing continuous concept tokens facilitate abstract reasoning, they introduce significant stability challenges. The accumulation of Out-of-Distribution (OOD) states frequently causes manifold deviation, while the absence of clear boundaries leads to indeterminate termination, often referred to as the anchoring problem. To address these limitations, we propose HybridThink. We formulate mode selection in LLM reasoning as a sequential decision process. A lightweight gating policy is trained with RL to decide at each step whether to propagate a continuous concept token or generate a discrete text token to anchor the semantic trajectory.This learned interleaving limits manifold deviation and resolves the fragility of static rules. Furthermore, we ensure stable convergence through a training strategy that incorporates Error-Centric Experience Replay (ECER) and Inductive Exploration Shaping (IES), prioritizing learning from failure modes, specifically drifting and premature termination. Empirical evaluations on mathematics and coding benchmarks demonstrate that HybridThink outperforms standard CoT and Soft Thinking baselines, achieving superior accuracy with reduced token consumption.

Abstract:
Latent reasoning enables reasoning over continuous hidden states rather than explicit tokens, avoiding the language bottleneck and inference overhead of chain-of-thought for medical VQA. However, existing methods suffer from modality collapse, insufficient visual supervision, and train-inference mismatch. Moreover, their opaque latent states offer no interpretability, which is critical in clinical applications. We propose VITAL, a latent-space reasoning framework for medical MLLMs with visual-semantic dual supervision: an auxiliary text decoder reconstructs reasoning chains from latent states, while a visual projector regresses ROI features from a frozen, independent medical vision encoder. Both modules are discarded at inference with zero overhead, yet can be re-attached post-hoc for dual interpretability, providing textual and visual explanations of the reasoning process without sacrificing efficiency. We construct a 61K dataset spanning 9 imaging modalities, exceeding prior medical visual latent reasoning datasets by an order of magnitude. Experiments on 7 benchmarks show that VITAL consistently and substantially outperforms the backbone, all latent reasoning baselines, and medical MLLMs trained on far larger data, achieving state-of-the-art results competitive with trillion-parameter proprietary models.

Abstract:
As conversational agents accumulate experience collaborating with users, adapting to user preferences is essential for fostering long-term relationships and improving collaboration quality over time. We introduce MultiSessionCollab, a benchmark that evaluates how well agents can learn user preferences and leverage them to improve collaboration quality throughout multiple sessions. To develop agents that succeed in this setting, we present long-term collaborative agents equipped with a memory that is specifically designed to learn user preferences across sessions and improve interactions. Moreover, we demonstrate that learning signals can be derived from user simulator behavior in MultiSessionCollab to train agents to generate more comprehensive reflections and update their memory more effectively. Extensive experiments show that equipping agents with our memory improves collaboration over time, yielding higher task success rates, more efficient interactions, and reduced user effort. Finally, we conduct a human user study that demonstrates that memory helps improve user experience in real-world settings.

Abstract:
We find a mismatch between the causal direction recoverable from a large language model’s hidden state and its final Yes/No answer. On anti-commonsense CLadder (Jin et al., 2024) items, a frozen linear probe recovers the evidence-supported direction with accuracy 0.97–1.00, while ordinary Yes/No accuracy remains 0.35–0.53. This effect disappears in randomly initialized models, so it depends on the trained representation rather than probe capacity alone. Erasing the recovered direction reduces the model’s internal comparison of paired prompts by 96%, and replacing the full hidden state flips 57% of previously wrong answers. A 2 × 2 interface control further shows that changing only the answer tokens does not fix the error; explicitly contrasting the two candidate directions raises accuracy to 0.95–1.00 across Qwen, Mistral, and Llama. We call this mismatch Causal Tongue-Tie. It does not establish human-like causal understanding. Our main finding is that a wrong Yes/No answer can coexist with an evidence-aligned direction that the tested probe recovers from the same forward pass. Output-only accuracy therefore cannot distinguish an absent internal signal from a present but unexpressed one.

Abstract:
Large language models (LLMs) are shifting from answer providers to intelligent tutors in educational settings. However, current supervised fine-tuning methods only learn surface-level teaching patterns without dynamic adaptation. Recent reinforcement learning approaches address this, yet two key challenges remain. First, they evaluate teaching effectiveness solely based on whether students produce correct outputs, unable to distinguish whether students genuinely understand or echo teacher-provided answers. Second, they cannot perceive students’ evolving cognitive states through dialogue, failing to adapt teaching strategies accordingly. We propose Unidirectional Cognitive Optimization (UCO) to address these challenges. UCO employs a multi-turn interactive reinforcement learning paradigm with two synergistic reward functions: the Progress Reward captures cognitive advancement, evaluating students’ transition from confusion to comprehension, and the Scaffold Reward dynamically identifies each student’s Zone of Proximal Development (ZPD), encouraging the teacher to conduct productive teaching within this zone. We evaluate UCO against 11 baseline models on BigMath and MathTutorBench. Results demonstrate that UCO outperforms all equivalent-scale models and achieves performance comparable to leading closed-source models. Code is available at https://github.com/ECNU-RAIL/UCO-EMNLP2026.

Abstract:
Vision-language models (VLMs) achieved strong performance on OCR-based benchmarks and increasingly focused on text-rich understanding, but their robustness under controlled visual degradation remains insufficiently understood. This gap is especially critical for OCR reasoning, where visual corruption can induce OCR errors and structural distortions, thereby introducing uncertainty into reasoning task. To systematically study this problem, we introduce OCR-Robust , a benchmark designed for evaluating OCR reasoning robustness under visual perturbations. It contains 812 samples across two complementary subsets: OCR1.0, covering documents, scene text, receipts, handwriting, and mathematical content, and OCR2.0, focusing on charts, geometry diagrams, and tables. To enable efficient yet informative evaluation, we conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability. We evaluate robustness using clean accuracy, Relative Corruption Retention (RCR), Worst-Case Retention (WCR), and a composite Corruption Robustness Index (CRI), and benchmark 18 models spanning proprietary systems, open-source VLMs, and OCR+LLM pipelines. Our results show that higher clean accuracy does not necessarily imply stronger robustness, and that models can suffer pronounced degradation in the worst case on OCR tasks that are sensitive to structure, and charts and tables are substantially more fragile than document-like inputs under perturbation.

Abstract:
Large language models (LLMs) are gradually evolving from passive text generation systems into agents with environmental perception, tool invocation, memory management, and complex planning capabilities. However, the expansion of their capability boundaries also introduces more complex and stealthy attack surfaces. This paper focuses on this issue and studies agent-oriented backdoor attacks. Considering that existing security surveys still lack a systematic analysis of agent backdoor threats, this paper fills this gap by providing a comprehensive examination of backdoor attacks against modern agents. Specifically, we propose a five-dimensional taxonomy that structurally organizes existing attack methods from the perspectives of attack surface, triggering mechanism, attack objective, attacker capability, and persistence and propagation. Furthermore, considering that most existing defense algorithms are designed for traditional backdoors and are difficult to adapt to complex attack chains across components, states, and sessions in agent systems, this paper systematically reviews frontier defense strategies along the attack-surface dimension, including plan consistency checking, memory sanitization, tool routing, and action monitoring. Finally, this paper discusses the core challenges in agent backdoor attacks and defenses, aiming to reveal vulnerabilities in agent systems and provide guidance for building secure agents.

Abstract:
High-quality data is scarce in large language model (LLM) training, yet how to schedule its use with optimization dynamics lacks theoretical guidance. We extend functional scaling laws with time-varying data quality and derive asymptotically optimal joint data-quality and batch-size schedules within a feature-space regression model. The solution reveals two regimes and dual uses of high-quality data: in the noise-limited regime, a smaller batch converts cleaner data into more signal at comparable noise; in the signal-limited regime, late placement suppresses terminal noise without sacrificing signal accumulation. This explains why conventional decay schedules can conflict with curriculum-style pipelines. Motivated by the theoretical structure, we propose Drop-Stable-Rampup for LLM midtraining: drop the batch size at the quality transition, keep it low to accumulate signal, then ramp up to suppress noise. On a 15B MoE model midtrained on 108B tokens of general-domain proprietary data, Drop-Stable-Rampup improves average accuracy over Warmup-Stable-Decay by +1.70 and Cosine-decay by +2.98, including +4.23 on GSM8K and +2.80 on MATH. On a public math-and-code mixture, it leads all reported STEM, mathematics, and code benchmarks, improving the overall mean over the strongest baseline by +3.27 on a 600M dense model and +5.25 on the same MoE architecture.

Abstract:
Practical deployment of medical vision-language models requires a single model that interprets heterogeneous clinical visual inputs: 2D images, 3D volumes, and medical videos. Yet how to train such unified models remains underexplored, as these modalities differ in spatial-temporal structure, token length, and data availability. We present OmniV-Med, a scalable training recipe built on four empirical findings. First, broad modality coverage requires balancing modality diversity with task-level supervision, motivating OmniV-Med-Instruct, a 252K corpus spanning 14 medical modalities and 11 clinical tasks. Second, we identify a modality-ordering effect: introducing 3D and video too early destabilizes optimization and weakens 2D grounding, motivating a progressive curriculum that first builds 2D alignment before extending to spatial-temporal inputs. Third, we observe an emergent spatiotemporal transfer effect: once strong 2D patch representations are in place, lightweight rotary position adaptation extrapolates them to 3D slices and video frames, letting a single 2D ViT replace modality-specific encoder stacks. Fourth, the imbalance in sequence length across modalities and the redundancy of long volumes and videos hinder scaling; our Redundancy-Guided Token Pruning (RGTP) exploits inter-slice and inter-frame redundancy to cut visual tokens by 60% while preserving performance. Across nine 2D, 3D, and video medical benchmarks, OmniV-Med achieves robust performance. Extensive component-wise ablations systematically refine the design space, yielding a simple yet efficient and scalable recipe for training high-performance unified Med-VLMs. Code is released at https://github.com/ZJUI-AI4H/Hulu-Med .

Abstract:
Time Series Event Detection (TSED) aims to localize semantically meaningful events in time series data, with critical applications in high-stakes domains. Unlike statistical anomalies, events are often defined by natural-language descriptions with internal temporal-logic structures across multiple physical channels. However, in real-world settings, dense event annotations are expensive to obtain, making purely supervised learning difficult. We introduce Language-guided TSED, a setting where a model is given textual event descriptions and must ground them to intervals in multivariate signals with little or no labeled data. To address this problem, we propose Event Logic Tree (ELT), a knowledge representation framework that converts linguistic descriptions into structured temporal logic over signal primitives. Building on ELT, we present SELA, a neuro-symbolic VLM agent framework that iteratively grounds primitives from signal visualizations and composes them under ELT constraints, producing both event intervals and faithful tree-structured explanations. We further release a real-world benchmark across energy and climate domains with expert knowledge and annotations. Experiments show that SELA improves over supervised fine-tuning and existing zero/few-shot time series reasoning baselines.

Abstract:
Simultaneous speech-to-text translation (SimulST) generates translations while speech is still unfolding, requiring a streaming policy that decides when to read and when to write. State-of-the-art approaches rely on attention-based encoder-decoder models where cross-attention provides explicit alignment signals. In contrast, Speech Large Language Models (SpeechLLMs) are decoder-only architectures relying solely on self-attention. This raises a central question: whether decoder self-attention contains sufficiently stable alignment signals to guide the streaming policy. Moreover, existing approaches typically rely on training-based adaptations or heuristic wait-k policies and have not been validated in long-form settings. To fill these gaps, we propose Decoder-Only Attention (DOA), a training-free policy that enables long-form simultaneous translation with off-the-shelf SpeechLLMs by deriving a proxy alignment from self-attention. Experiments on Phi4-Multimodal and Qwen3-Omni show that DOA provides an effective alignment signal for supporting streaming decisions, enabling low-latency long-form SimulST with quality close to offline decoding without retraining.

Abstract:
On-policy distillation (OPD) improves reasoning by training a student on trajectories sampled from its own policy under supervision from a teacher. In multimodal reasoning, a common extension is to use a privileged teacher that observes training-time-only signals such as reference answers or rationales. However, such answer-side privilege creates a train-test mismatch: the teacher’s supervision may depend on signals unavailable to the student, encouraging shortcut imitation rather than visually grounded reasoning. We propose ViCuR, a visually grounded privileged-teacher distillation framework that replaces answer-side privilege with visual cues (query-related evidence in the input). Because these cues are derived from the same visual input available at inference, their evidence is recoverable by the student. To support this, ViCuR introduces a lightweight cue recovery module that uses dedicated sink-token cross-attention during prefill to aggregate task-relevant visual evidence into an internal representation, without changing the inference interface or requiring auxiliary cue-generation losses. Across seven benchmarks with Qwen3-VL-2B and 8B students, ViCuR consistently improves over answer-based on-policy self-distillation by +1.19 and +1.24 on overall average performance. It also extends naturally to stronger-teacher OPD, surpassing OPD baselines by +0.64 and +1.08, with consistent out-of-domain gains at the 8B scale. These results show that, in multimodal on-policy distillation, the design of teacher privilege is as important as teacher strength.

Abstract:
Frozen vision-language models already encode object, attribute, relation, and caption information, but a single vector exposes all of these signals through the same access pattern. We ask whether vector length can become a semantic access control: short prefixes answer coarse queries, while longer prefixes add finer distinctions. GraSP-VL learns one shared near-orthogonal transform for image and text embeddings and assigns a fixed, configurable prefix contract. At inference, the transform is applied once and ordinary cosine retrieval uses the prefix required by the task. On a 20,147-example COCO/Flickr30K pool, GraSP-VL reaches a staircase score of 53.01 and hard-negative selectivity of 89.76 with full-space drift below 10 -6 . Full-dimensional ImageNet-V2 classification and Flickr30K/COCO Karpathy retrieval are unchanged, while held-out human-caption retrieval improves at shorter prefixes. These results show that length can reorganize access to frozen VLM semantics rather than merely compressing them. Code and project page are available at https://lizesheng13.github.io/Grasp-VL/.

Abstract:
Late-interaction vision-language retrievers represent each document page as many visual token embeddings and score queries with MaxSim. In systems such as ColPali, ColQwen, ColNomic, and Nemotron ColEmbed, the document embeddings are produced without seeing the query, so the same page is represented identically for a table lookup, a chart question, and a layout-sensitive evidence request. We introduce Argus , a family of query-conditioned late-interaction retrievers built on Qwen3.5-VL. Argus adds a region-aware Mixture-of-Experts module: the query encoder produces both retrieval embeddings and a compact context vector, the document page is pooled into spatial regions, and a query-aware router selects latent experts per region before MaxSim. The output remains a multi-vector index compatible with ColPali-style retrieval, but the document representation is now dependent on the query (i.e., 𝐃(q) ). All Argus models use a 1024-dimensional retrieval head, compared with the 2560-dimensional and 4096-dimensional heads of recent state-of-the-art systems, and are trained on roughly 9% of the available public supervision rather than the full pool. The 9B model reaches 92.67 NDCG@5 on ViDoRe V1 and 86.0 NDCG@5 on the combined V1+V2 leaderboard, the highest reported value for an open late-interaction model on the combined leaderboard. Wrapped in a Qwen3.6-27B agentic retrieval pipeline on ViDoRe V3, Argus-9B further improves its NDCG@10 from 60.28 to 64.80 over public tasks, showing that the same retriever serves both as a strong standalone system and as a search primitive for iterative LLM agents.

Abstract:
Computational resources are increasingly central to NLP research, but how closely reported GPU capability aligns with scholarly impact remains unclear. We analyze 13,921 ACL, EMNLP, and NAACL main-conference papers published between 2020 and 2025, using GPU resources as our operational measure of computational resources. From full texts, we extract GPU models and counts, standardize each paper’s largest reported configuration into a comparable hardware-capability measure, and link these data to citation, award, topic, and institutional metadata. GPU reporting became more common but remained incomplete, while reported capability increased mainly through newer hardware generations and medium-scale multi-GPU configurations. Resource concentration substantially exceeded impact concentration: the annual top 20% of GPU-quantifiable papers accounted for 83.9%–89.9% of reported GPU capability, but only 27%–32% of citations and 20%–33% of paper awards. In adjusted models, a tenfold increase in aggregate reported GPU capability was associated with a 3.52-percentage-point increase in within-NLP topic–year citation percentile, but increased model R 2 by only 0.0042. GPU count showed more consistent positive associations with citation and award outcomes than newer hardware generation. Overall, reported GPU resources are associated with scholarly impact but provide little standalone explanation of research influence.

Abstract:
Despite exceptional performance on static benchmarks, Large Language Models (LLMs) exhibit significant brittleness when evaluated on perturbed variants of the original questions. Current extrinsic interventions, prompt engineering or supervised fine-tuning, overlook an important intrinsic factor underlying this degradation: the distributional shifts in layer activations between original and perturbed questions. To address this, this work formulates the reasoning robustness challenge as an Unsupervised Domain Adaptation (UDA) problem and proposes the Elastic Domain Adaptation (EDA) method. Unlike rigid strategies that enforce a uniform alignment constraint, EDA dynamically aligns layer activations based on the specific layers and perturbation types. Specifically, EDA incorporates an Adaptive Layer Selection (ALS) to identify pivotal layers and employs an Elastic Shift Module (ESM) to selectively pull or push layer representations depending on the perturbation types. Extensive experiments across six LLMs and 15 perturbation types demonstrate that EDA achieves state-of-the-art performance on most perturbed benchmarks while preserving reasoning capability on original benchmarks.

Abstract:
Speech codecs serve as bridges between continuous speech signals and large language models, yet face an inherent conflict between acoustic fidelity and semantic preservation. To mitigate this conflict, recent works increasingly adopt dual-tower architectures to decouple semantic and acoustic modeling with separate encoders. However, these dual-tower designs incur substantial architectural overhead. To avoid such complexity, we revisit the single-tower paradigm and propose BiMTokenizer , a low-bitrate speech codec (around 1.1 kbps) combining a bidirectional state-space backbone with Residual Spherical Leech Quantization (RSLQ) . The bidirectional backbone strengthens temporal modeling, while RSLQ offers a fixed, well-separated lattice bottleneck for robust semantic and acoustic tokenization without learned-codebook collapse. Experiments show that BiMTokenizer achieves superior acoustic reconstruction and the lowest WER among low-bitrate codec baselines across both clean and noisy environments, while using less than half the parameters of recent dual-tower baselines. Furthermore, its robust semantic representations yield strong performance on downstream speech understanding tasks, confirming that a well-designed single-tower codec can preserve the semantic–acoustic balance at low bitrates. The code and model weights are available at https://github.com/ZhangXinWhut/BiMTokenizer .

Abstract:
Structured memory representations such as knowledge graphs are central to autonomous agents and other long-lived systems. However, most existing approaches model time as discrete metadata, either sorting by recency (burying old-yet-permanent knowledge), simply overwriting outdated facts, or requiring an expensive LLM call at every ingestion step, leaving them unable to distinguish persistent facts from evolving ones. To address this, we introduce RoMem, a drop-in temporal knowledge graph module for structured memory systems, applicable to agentic memory and beyond. A pretrained Semantic Speed Gate maps each relation’s text embedding to a volatility score, learning from data that evolving relations (e.g., "president of") should rotate fast while persistent ones (e.g., "born in") should remain stable. Combined with continuous phase rotation, this enables geometric shadowing: obsolete facts are rotated out of phase in complex vector space, so temporally correct facts naturally outrank contradictions without deletion. On temporal knowledge graph completion, RoMem achieves state-of-the-art results on ICEWS05-15 (72.6 MRR). Applied to agentic memory, it delivers 2-3x MRR and answer accuracy on temporal reasoning (MultiTQ), dominates hybrid benchmark (LoCoMo), preserves static memory with zero degradation (DMR-MSC), and generalises zero-shot to unseen financial domains (FinTMMBench).

Abstract:
GUI agents predict click coordinates as digit-token sequences, but standard text-LLM confidence estimation methods rank correct clicks from wrong ones only weakly. GUI-specific alternatives use K samples or new supervision, but still leave room for improvement. We trace part of this to place-value asymmetry : bounding- box correctness often makes higher-place digits more important than lower-place digits, so uniform aggregation weakens the signal that determines correctness. The fix is to weight each digit’s Shannon entropy by its place value. We call this Place-Aware Coordinate Entropy (PACE). Across fixed-scale agents on ScreenSpot-Pro and ScreenSpot-v2, PACE wins both AUROC and selective accuracy on all primary comparisons in a single forward pass, matching or outperforming K -sample baselines at a fraction of the cost. PACE provides a per-click confidence estimate that turns coordinate-token internals into a practical confidence signal for GUI agent deployment.

Abstract:
Large language models (LLMs) have demonstrated remarkable potential in medical AI through their natural language understanding and reasoning capabilities. However, real clinical practice involves multi-stage, dynamically evolving decision-making where physicians continuously adjust pathways based on updated examination results—far more complex than single-turn question answering. Existing medical benchmarks focus on static scenarios, failing to evaluate LLMs in realistic clinical environments. We present MMedAgentBench , a comprehensive benchmark for evaluating LLM-based medical agents in multi-stage clinical decision-making. Unlike previous benchmarks, MMedAgentBench captures the temporal complexity of clinical workflows by simulating authentic patient journeys across multiple diagnostic and treatment stages. The benchmark comprises a high-quality dataset of over 150 de-identified clinical cases from tertiary hospitals, encompassing 500+ clinical stages, with each case meticulously structured into sequential stages and annotated by board-certified physicians. Our evaluation framework simultaneously assesses diagnostic accuracy and resource efficiency, requiring agents to plan examinations and control costs while navigating progressive information disclosure. This enables systematic evaluation of medical agents across dynamic decision-making, cost control, and multi-stage reasoning dimensions, providing a realistic testbed for advancing medical AI toward clinical deployment.

Abstract:
Large language models (LLMs) can generate fluent peer reviews but often miss sparse, subtle, and document-level scientific errors. We introduce PaperAudit-Bench, a benchmark for evaluating paper-level error detection and error-aware automated review. It comprises two components: (1) PaperAudit-Dataset, 220 high-quality AI/ML papers containing more than 2,200 controlled synthetic errors across eight categories; and (2) PaperAudit-Review, an evidence-aware framework that links structured error detection to review generation. Experiments across models, error categories, paper sections, and analysis depths show that sparse long-context auditing remains challenging for current LLMs. Conditioning review generation on explicit, manuscript-grounded error evidence produces more conservative and discriminative assessments that align more closely with human judgments than representative automated-review baselines. The dataset also supports post-training compact detectors through SFT and RL; Qwen3-14B achieves 55.8% F1 at Detection@1 under a fixed held-out protocol. A PubPeer transfer study and deployment in ScienceOne, a scientific foundation-model platform led by the Chinese Academy of Sciences, further illustrate practical applicability. Code and Dataset Page: https://github.com/TU2021/PaperAudit ScienceOne Manuscript Review: https://www.scienceone.cn/lit/#/assistant

Abstract:
Detecting anomalies in tabular data is critical for many real-world applications, such as credit card fraud detection. With the rapid advancements in large language models (LLMs), state-of-the-art performance in tabular anomaly detection has been achieved by converting tabular data into text and fine-tuning LLMs. However, these methods randomly order columns during conversion, without considering the causal relationships between them, which is crucial for accurately detecting anomalies. In this paper, we present CausalTaD, a method that injects causal knowledge into LLMs for tabular anomaly detection. We first identify the causal relationships between columns and reorder them to align with these causal relationships. This reordering can be modeled as a linear ordering problem. Since each column contributes differently to the causal relationships, we further propose a reweighting strategy to assign different weights to different columns to enhance this effect. Experiments across more than 30 datasets demonstrate that our method consistently outperforms the current state-of-the-art methods.

Abstract:
Retrieval-Augmented Generation (RAG) can fail under knowledge conflicts, where a model’s parametric prior competes with retrieved evidence and leads to unfaithful outputs. Existing mitigation methods typically operate on the final decoding distribution or rely on weight editing with access to ground-truth targets. Through layer-wise analysis, we identify a parametric suppression pattern: in many conflict cases, evidence-supported answers become salient in intermediate representations but are weakened by later layer outputs. To address this, we propose CoRect (Context-Aware Logit Contrast for Hidden State Rectification), a training-free inference-time method. By contrasting contextualized and context-free forward passes, CoRect first selects an evidence-supported token direction without using gold labels. It then rectifies hidden states in layers whose outputs suppress this direction. Across Question Answering (QA) and summarization benchmarks, CoRect improves average faithfulness and reduces hallucinations compared with strong decoding baselines.

Abstract:
Masked diffusion models (MDM) exhibit superior generalization when learned using a Partial masking scheme (Prime). This approach converts tokens into sub-tokens and models the diffusion process at the sub-token level. We identify two limitations of the MDM-Prime framework. First, we find that the functional form of the subtokenizer significantly increases the log-likelihood value in the objective when paired with commonly used Byte-Pair-Encoding (BPE) tokenizers. Second, we lack tools to guide the hyperparameter choice of the token granularity in the subtokenizer. To address these limitations, we analyze the optimal design of the subtokenizer that minimizes MDM-Prime training objective and develop MDM-Prime-v2, a masked diffusion language model which incorporates Binary Encoding and Index Shuffling. Our analysis characterizes how token granularity and sub-token entropy influence the training objective and downstream performance, providing principled criteria for subtokenizer design. When extending the model size to 1.1B parameters, MDM-Prime-v2 demonstrates superior average zero-shot accuracy across eight commonsense reasoning benchmarks, outperforming similar-sized baselines including GPT-Neo, OPT, Pythia, Bloom, SMDM, and TinyLLaMA.

Abstract:
Linear classifiers trained on hidden states of a large language model (LLM)—linear probes—can flag factual errors from a single forward pass. Geometrically, that implies that true and false statements separate along a stable direction in hidden state space, i.e., the truth direction. Prior work disagrees on whether this generalises across input shifts, but the disagreement is hard to interpret because cross-dataset probe transfer experiments confound several kinds of input change at once. We isolate three such variables in medical question-answering (QA): writing style (register), domain (medical specialty), and corpus (dataset). We build a benchmark using 500 MedQA entries, each rewritten into four styles (textbook, patient, clinical note, colloquial), annotated with clinical specialty, and grouped with two other exam corpora, MedMCQA and MMLU-medical, for cross-dataset evaluation. Probing four open-weight LLMs (2–8B), we find that the truth direction is largely robust to writing style (mean 𝛥 register ≈ 0.10 AUROC on held-out facts) and to medical specialty ( 𝛥 specialty ≈ 0.03 ), but degrades unevenly across corpora: by 0.12 AUROC on MMLU-medical and by 0.21 on MedMCQA, roughly twice the register gap. The register result replicates with a second generator and carries over to human-written patient questions. The truth direction is therefore largely stable within the medical domain but breaks under some corpus shifts, and question format does not explain the break, which suggests that the signal a linear probe recovers is partly bound to dataset structure rather than to medical knowledge alone.

Abstract:
The application of autonomous agents to scientific literature review is often constrained by the reliability of online search infrastructures and linear text generation. To address these limitations, we introduce SciLENS (Scientific Localized Evidence Navigation and Synthesis), an autonomous agent framework operating entirely upon a localized dual-tier data infrastructure for deterministic access to massive academic corpora. To mitigate online dependency, SciLENS utilizes a comprehensive offline toolbox featuring topological graph traversal and structural visualization. This empowers the agent to navigate deep citation networks and organize macro-level trends into structured visual representations, mitigating context exhaustion. To train this system without human annotation, we develop an automated multi-hop data synthesis pipeline validated by cross-model consensus, followed by rubric-based reinforcement learning to align the logical planning of the agent. Evaluations across standard tasks and newly constructed complex synthesis benchmarks demonstrate that SciLENS achieves robust factual precision, outperforming existing open-source baselines and achieving performance comparable to frontier proprietary models. Our source code and data are released at https://anonymous.4open.science/r/SciLENS-48DB .

Abstract:
Eastern Min Chinese (Fuzhou dialect, ISO 639-3 ‘cdo‘) has roughly ten million speakers yet no dedicated paper at any major NLP venue. Four properties make Fuzhou structurally distinct from Mandarin and challenging for current NLP: (1) a tone-sandhi system conditioned on the following syllable’s citation tone, (2) initial-consonant assimilation in non-final positions, (3) systematic literary-colloquial diglossia (45% of frequent characters carry ≥ 2 distinct segmental readings), and (4) substantial lexical divergence (everyday concepts use entirely different morphemes from Mandarin). We release FuzhouBench, the first public NLP benchmark for Eastern Min, built on a community-maintained open dictionary: 22,890 word entries with 51,505 standardized Yngping romanizations, 11,276 explicit tone-sandhi annotations, 5,160 explanations, and 3.06 hours of paired dictionary-style audio from two native speakers. We define three tasks (grapheme-to-phoneme, sandhi prediction, definition retrieval) with full baselines. Frontier LLMs score under 3% word-exact on few-shot Yngping G2P, while a LoRA fine-tune on the released pairs reaches 17.2%: learnable, but far from saturated. A symbolic sandhi rule learned from the training pool beats every LLM by up to 18 points; and BM25 collapses from MRR 0.799 to 0.372 when queries shift from dictionary-extracted to colloquial Mandarin, partially recoverable via a two-stage hybrid + cross-encoder pipeline (MRR 0.716). Data, evaluation splits, and code are available at https://github.com/Nang-muoi-sing/FuzhouBench.

Abstract:
Ensuring the safety of autonomous driving is a critical challenge. Scenario-based testing is a systematic process used to validate Autonomous Driving Systems (ADSs), but it remains a fragmented modular pipeline in which scenario generation, retrieval, modification, ADS execution, and results analysis are performed by separate tools with little interaction. Large Language Model (LLM) agents have shown promise across ADS sub-systems such as perception, planning, and control. However, no prior work covers the whole scenario- based testing pipeline for ADSs with a unified LLM-agent framework. We present PlannerForge, an LLM-agent framework that extends all scenario-based testing stages (from Scenario Generation to ADS Assessment) and adds two further LLM-enhanced stages: ADS Enhancement and ADS Benchmarking. We evaluate PlannerForge with 10 off-the-shelf LLMs across all tasks (Generation, Selection, Modification, Module Routing, Planner Testing, and Enhancement) under 5 prompt conditions. Best-per-task scores range from 0.88 to 1.00, and open-source 20–35B backends match commercial APIs on most tasks. Open-source models such as Qwen3.6:35B match commercial APIs on three of the five tasks. Chaining the modules end-to-end retains 83% / 78% of seed queries (commercial / open). It outperforms Scenario Factory 2.0 (Finkeldei et al., 2025) on natural-language generation (193 vs. 144 executable of 200) and realises 92–96% of requested city, road and vehicle attributes. It outperforms BM25 (Robertson and Zaragoza, 2009) at rank 1 selection (92.0% vs. 67.5%) and From-Words-to-Collisions (Gao et al., 2025) on physically valid edits (≥94% vs. 31%). At N =400, cost-tuning lifts planner success from 50.4% to 70.2% and cuts collisions from 19.0% to 8.4%, without domain-specific fine-tuning.

Abstract:
Efficient serving of multimodal large language models (MLLMs) is increasingly bottlenecked by high-resolution visual inputs, which can generate thousands of visual tokens per request and severely increase latency, memory pressure, and serving cost. Visual token pruning is a natural solution, but existing methods are difficult to deploy in production MLLM serving: many operate inside LLM layers and conflict with engine invariants such as KV-cache granularity, fused kernels, prefix caching, and batching, while fixed-ratio pruning can collapse accuracy on dense-information tasks such as document and chart understanding. We present SieveServe, a serving-compatible framework for adaptive visual token pruning. SieveServe establishes pre-LLM vision-centric pruning as an engine-compatible paradigm and introduces a request-aware dual-guard policy: a Query Complexity Router (QCR) estimates query-side visual evidence demand, and a Dense Information Guard (DIG) detects image-side pruning risk using lightweight channel-wise embedding variance. Together, they allocate visual-token budgets per request, aggressively compressing pruning-robust inputs while conservatively preserving dense/high-risk ones. Experiments with Qwen2.5-VL-7B and InternVL3-8B across eight benchmarks show that SieveServe retains at least 95% of baseline accuracy on all benchmarks and at least 97.8% on dense-information tasks, while compressing pruning-robust tasks by 3.2x and recovering over 20 accuracy points lost by fixed-ratio pruning. On high-resolution serving workloads, SieveServe reduces P90 TTFT by 5.4x and sustains 2.2-2.7x higher throughput under latency SLO constraints compared with vanilla vLLM.

Abstract:
Large Reasoning Models (LRMs) still struggle with fine-grained translation quality estimation (QE), even with long reasoning chains. We argue that LRMs already possess strong multilingual capabilities, while the core challenge stems from the intrinsic difficulty of learning the fine-grained QE task. In this paper, we propose RIEQE ( R easoning both I mplicitly and E xplicitly for QE ), a simple two-stage training framework that enables the mutually boosting of implicit (layer-wise) and explicit (token-wise) reasoning capabilities. To make implicit reasoning feasible, we first decompose the complex QE task into straightforward subtasks. Based on this, our two-stage approach applies: (1) NonThinking-SFT , Supervised Fine-Tuning (SFT) without reasoning chains to directly boost the model’s implicit reasoning tendency and capability; and (2) Thinking-RLVR , standard Reinforcement Learning with Verifiable Reward (RLVR) to subsequently strengthen explicit reasoning. On the WMT test sets, RIEQE based on Qwen3-4B-Thinking-2507 surpasses all baselines in explicit reasoning performance, while its implicit reasoning capability is also comparable to the best current encoder-based models. We further provide evidence for the mutually boosting between implicit and explicit reasoning, showing how they benefit each other in a bidirectional manner.

Abstract:
Sequentially evolving LLM memory enables agents to reuse past experience, but existing systems usually deploy each locally generated memory update without checking whether it improves future behavior. As a result, updates that help the current task may overwrite useful knowledge, introduce over-specific rules, or bias the final memory toward recent examples. We propose Janus, a plug-in memory controller that decides whether to accept a candidate memory update or retain the previous memory. To make this decision efficient, Janus uses a Memory Momentum Trigger to identify suspicious deviations in the memory-update trajectory, and compares old and new memories on a compact hybrid evaluation set of coverage, boundary, and fresh tasks instead of replaying the full history. Janus is method-agnostic and wraps existing updaters without changing their update rules. Across six datasets, two backbone LLMs, and two memory updaters, Janus improves average accuracy by +2.7 to +4.6 points over the corresponding base updaters. Our code is available at https://github.com/Chen-1031/Janus.

Abstract:
Advances in multi-modal large language models (MLLMs) have inspired time series understanding and reasoning tasks. Recent attempts hybridize numerical time series with their visualized plots, facilitating precise value reasoning and visual structure comprehension for comprehensive time series understanding of MLLMs. However, effective numerical-visual modality integration remains challenging due to fine-grained temporal misalignment across modalities and severe entanglement between shared and modality-specific semantics, which hinder localized interpretation and complementary reasoning. To address these issues, we propose MADI, a multi-modal LLM enhanced with fine-grained alignment and disentangled interaction, featuring (1) Patch-level Alignment, which enforces physically grounded fine-grained correspondence across heterogeneous modalities, (2) Discrete Disentangled Interaction, which separates modality-common semantics into compact discrete latents and adaptively synergizes the purified modality-unique information, and (3) Critical-token Highlighting, which emphasizes informative, query-relevant signals for robust reasoning. Experiments on synthetic and real-world benchmarks show that MADI consistently outperforms general-purpose LLMs and time-series-specialized MLLMs. The code is available at https://github.com/usail-hkust/MADI.

Abstract:
Lip-to-speech (LTS) synthesis aims to generate intelligible speech from silent lip movements, but the ambiguous visual-to-phoneme mapping makes content-accurate synthesis challenging. Existing methods mainly follow two routes: (1) direct synthesis from lip movements and (2) synthesis dependent on an external lip-reading model. The former often suffers from content errors, while the latter improves content accuracy at the cost of computational overhead and error propagation. To address these limitations, we introduce FlexLTS, a unified paradigm for content-accurate LTS with optional lip reading. It generates accurate speech without requiring lip reading, while still benefiting from lip-reading text when available. FlexLTS learns phonetic prior embeddings from training transcripts and employs a visual-centric prior fusion mechanism to selectively incorporate relevant phonetic priors into the visual stream, yielding content-enriched semantic representations. Moreover, our proposed Progressive Textual Dropout strategy further promotes prior internalization by reducing reliance on explicit text during training. Experiments show that FlexLTS achieves state-of-the-art content accuracy and speech quality on the popular LRS2-BBC and LRS3-TED benchmarks and also performs well in cross-dataset zero-shot evaluation. Notably, without using any lip-reading model, FlexLTS surpasses existing lip-reading-aided methods in content accuracy.

Abstract:
Personalized lifestyle health analysis requires long-horizon, multi-dimensional reasoning over heterogeneous lifestyle signals, and recent advances in mobile sensing and large language models (LLMs) make such support increasingly feasible. However, the capabilities of current LLMs in this setting remain insufficiently understood due to the lack of systematic benchmarks. In this paper, we introduce LifeAgentBench, a large-scale QA benchmark for long-horizon, cross-dimensional, and multi-user lifestyle health reasoning, containing 22,573 questions spanning from basic retrieval to complex reasoning. We release an extensible benchmark construction pipeline and a standardized evaluation protocol, deriving verifiable answers through executable queries and programs to support reliable assessment. We then systematically evaluate 13 representative LLMs on LifeAgentBench and identify key bottlenecks in long-horizon aggregation and cross-dimensional reasoning. Motivated by these findings, we propose LifeAgent, a tool-augmented reasoning baseline that decomposes complex queries, performs multi-step evidence retrieval, and invokes tools for deterministic aggregation. LifeAgent substantially enhances LLMs’ capabilities on challenging reasoning tasks, achieving clear improvements over widely used baselines and showing potential for health reasoning in everyday scenarios. The benchmark is publicly available.

Abstract:
Long-horizon LLM agents typically choose from an environment-enumerated list of grounded actions (Planning with Actions, PwA). While this interface reduces formatting risk, scaling the list increases screening load and can bottleneck planning. We study Planning with Schemas (PwS), which replaces action enumeration with a small set of action templates that the agent instantiates into executable actions. We introduce a cognitive-bandwidth perspective of this trade-off and quantify screening vs. instantiation pressure using context length proxy and failure actions proxy. Across four interactive environments, we identify an inflection point: PwA excels at small interfaces but degrades as action lists grow, whereas PwS becomes increasingly competitive and often superior. A distractor-injection stress test reproduces the PwA drop by enlarging the action space alone, isolating screening load and highlighting schema instantiation as the key remaining bottleneck.

Abstract:
The dominance of colonial languages in African education and scientific communication limits how hundreds of millions of speakers of African languages access and produce scientific knowledge. A core obstacle is the lack of established scientific terminology in these languages. We introduce AfriScience-MT, a parallel corpus covering six African languages (Amharic, Hausa, Luganda, Northern Sotho, Yorùbá, and isiZulu) across 11 scientific domains. Professional translators, working with expert science communicators, translated plain language summaries of scientific papers into each target language and created new terms where none existed. We benchmark machine translation systems and large language models in zero-shot, few-shot, and fine-tuned settings. Our results show that closed-source models outperform all open-source models at both the sentence and document levels: GPT-5.4 and Gemini-3.1-Flash-Lite lead with average sentence-level COMET scores of 68.3 and 68.0, respectively, and tie at an average document-level COMET of 48.3. Among open systems, fine-tuned NLLB-1.3B reaches 67.3 at the sentence level, and TranslateGemma-12B reaches 44.0 at the document level with 1-shot in-context learning. We release AfriScience-MT to support benchmarking and document-level scientific African MT.

Abstract:
Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a multimodal dataset of 72 human expert-novice computer use coaching sessions consisting of 22,752 dialogue turns grounded in 28.1 hours of screen and input event recordings across 5 software applications. We then use DigitalCoach to evaluate whether state-of-the-art models can teach humans how to use computers. Our automated evaluation shows that models differ from humans in how they coach: models provide more direct instructions, but fewer explanations, error diagnoses, and knowledge-check questions. When we fix the coaching method, models produce utterances that are similar to human references, but poorly grounded in visual context. Our interactive evaluation confirms that model coaches cause learners to passively follow instructions without deeper engagement and fall short in visual grounding. DigitalCoach lays a foundation for collaborative and proactive computer use coaching agents. Data and code are available at https://project-digital-coach.vercel.app

Abstract:
Reinforcement learning (RL) has become one of the primary paradigms for reasoning enhancement of large language models (LLMs). In particular, Group Relative Policy Optimization (GRPO) and related algorithms have demonstrated strong performance with outcome-level rewards. However, these methods depend solely on the final answer, without feedback regarding which intermediate steps contribute to success or failure. As task complexity and reasoning trajectory length increase, such sparse final-answer rewards become increasingly insufficient. To address this limitation, we introduce ConsensusBench, a novel dataset designed to provide rule-based process-level signals. We posit that a correct final answer relies on a small set of intermediate conclusions throughout the reasoning process, which can be seen as a verifiable sub-outcome. We identify these sub-outcomes by filtering correct trajectories from N rollouts and clustering semantically equivalent intermediate statements. We call these clustered statements as Consensus Nodes. By integrating a rule-based process reward derived from these nodes into GRPO-style algorithms, we develop a new reinforcement learning signal named ConsensusPR. It directly reduces the reward sparsity of outcome reward across long reasoning trajectories. To facilitate systematic process-level evaluation, we introduce three metrics to our benchmark: Final Answer Accuracy (Acc), Node Coverage Rate (NCR), and Tokens per Node (TPN). Experiments across AIME 2024, AIME 2025, GSM8K, MATH-500, and our ConsensusBench demonstrate that the proposed method consistently surpasses GRPO-style approaches, highlighting the practical value of consensus nodes in guiding reasoning.

Abstract:
Safety guardrails serve as the last line of defense against harmful inputs and outputs of large language models (LLMs), yet they are trained and evaluated almost exclusively on short text. We present LongGuard, a framework that evaluates, mechanistically analyzes, and mitigates long-context guardrail failure. We formulate the task as Safety Needle-in-a-Haystack (SafetyNIAH) over a 0.25k–32k length grid; across 15 mainstream guardrails, unsafe recall drops monotonically by more than 50% on average, and a paired Benign-Fill vs. Needle-Repeat design attributes the failure to proportional dilution of the unsafe needle rather than to absolute length. A three-layer attention–logit–behavior analysis on six guardrails locates the mechanism: attention mass on the unsafe needle is diluted, the unsafe-over-safe logit margin is compressed in lockstep, and the detection decision collapses accordingly, with this attention → logit → behavior chain remaining consistent after partialling out length. We further isolate a sparse set of guard-specialized retrieval heads that exhibit partial specificity relative to their base models. Building on the analysis, we propose two training-free mitigations – Chunked Detection (CD) and Attention-Head Sharpening (AHS) – and a deployment protocol, Context-Aware Hyperparameter Routing (CAHR), that selects configurations by context length and audit side. Across five benchmarks spanning synthetic data, long-context attacks, and reasoning-model outputs, CAHR-CD and CAHR-AHS improve the six-guardrail average by 22% and 13%, respectively. Code and data are available online.

Abstract:
Long-video Omni assistants often fail not by inventing content, but by misbinding real evidence: they hear the right utterance and see the right event, yet attach it to the wrong speaker, moment, or modality. These almost-true errors evade standard video QA because local evidence remains valid, so item-level scoring can reward both a supported claim and its near-counterfactual. We introduce a counterfactual event-binding protocol that constructs paired supported/counterfactual claims from the same audio-visual event evidence and evaluates them by strict-pair accuracy. We instantiate it as OmniHalluc-L, to our knowledge the first benchmark for long-video Omni hallucination, with 3600 single-claim QA items from 638 long-form videos averaging 24.16 minutes and covering 256.87 hours. Under this protocol, open-weight Omni models remain weak at pair-level binding: Qwen2.5-Omni-7B reaches 32.06% and Qwen3-Omni-Instruct reaches 41.55%, versus 76.54% for a closed-source reference. To narrow this gap, we propose MPRC (Modality-Perturbation Reliability Calibration), a frozen-backbone calibrator that combines native confidence with response shifts from selected audio-negative probes. MPRC raises these two models to 36.22% and 51.09%, respectively, and improves target-adapted MCQ accuracy on OmniVideoBench and WorldSense. The code is available at https://github.com/ZexDong/omnihallucL-mprc.

Abstract:
Hypergraph-based methods extend structured retrieval-augmented generation (RAG) by organizing external knowledge as multi-entity relational structures, helping large language models (LLMs) generate better-grounded answers. However, existing structured RAG systems typically assume static corpus construction, requiring the retrieval structure to be rebuilt over the expanded corpus when new documents arrive. To address this limitation, we introduce HyperRAG-IHM, a hypergraph-based RAG framework that performs incremental hyperedge maintenance to incorporate newly arriving knowledge without reconstructing the historical retrieval structure. HyperRAG-IHM links relational evidence extracted from new documents to existing hypergraph knowledge, avoiding reconstruction of the entire retrieval structure. It uses LSH to localize candidate historical hyperedges, then applies multi-dimensional comparison and a three-action decision policy to integrate new relational evidence. Experiments show that HyperRAG-IHM achieves answer quality comparable to or better than full reconstruction under 5D scoring and 8D pairwise evaluation, and remains robust across ablation, structural behavior, cross-dataset, and cross-answer-model analyses.

Abstract:
Recent advances in Large Language Models (LLMs) have opened new avenues for generating training-free text embeddings. However, the causal attention in decoder-only LLMs prevents earlier tokens from attending to future context, leading to biased contextualized representations. In this work, we propose Reverse prompting with Explicit One-word Limitation (ReverseEOL), a simple yet effective method for enhancing the representational capability of frozen LLMs. ReverseEOL augments the standard forward embedding with an additional reversed embedding derived from the reversed input text. Since reversing the input exposes each token to context inaccessible in the original order, the resulting reversed embedding effectively provides complementary information to the original one. As a result, combining the forward and reversed embeddings yields a richer final representation. Comprehensive experiments on STS and MTEB benchmarks demonstrate that ReverseEOL significantly improves the performance of existing training-free baselines across a broad range of LLMs with diverse architectures and scales. Extensive ablations and analyses further confirm the necessity of our reversal mechanism. Code is available at [https://github.com/TianBaoGe/ReverseEOL](https://github.com/TianBaoGe/ReverseEOL).

Abstract:
Recent LLM-based data agents aim to automate data science tasks ranging from data analysis to deep learning. However, the open-ended nature of real-world data science problems, which often span multiple taxonomies and lack standard answers, poses a significant challenge for evaluation. To address this, we introduce DSAEval, a benchmark comprising 641 real-world data science problems grounded in 285 diverse datasets, covering both structured and unstructured data (e.g., image and text). DSAEval incorporates three distinctive features: (1) Multimodal Environment Perception, which enables agents to interpret observations from multiple modalities, including text and vision; (2) Multi-Query Interactions, which mirror the iterative and cumulative nature of real-world data science projects; and (3) Multi-Dimensional Evaluation, which provides a holistic assessment across reasoning, code, and results. We systematically evaluate 13 recent advanced agentic LLMs using DSAEval. Our results show that Claude-Sonnet-4.5 achieves the strongest overall performance, MiMo-V2-Pro and GPT-5.2 lead in duration and step efficiency, respectively, and MiMo-V2-Flash is the most cost-effective. We further demonstrate that multimodal perception consistently improves performance on vision-related tasks, with gains ranging from 2.04% to 11.30%. Overall, while current data science agents perform well on structured data and routine data analysis workflows, substantial challenges remain in unstructured domains. Finally, we offer critical insights and outline future research directions.

Abstract:
Multimodal large language models (MLLMs) have made substantial advancements in video understanding, yet the reliability of their responses remains underexplored. This work presents a diagnostic study of absent answer detection for MLLMs in video understanding, where the correct answer is deliberately excluded from the candidate set and a reliable model is expected to recognize that no valid option exists. We evaluate the absent answer detection behavior under three settings: multiple-choice questions augmented with a “None of the Above” option, open-ended generation with a detection instruction, and standard evaluation without any guidance. Across a diverse set of models and benchmarks, we find that MLLMs overwhelmingly select plausible distractors rather than detecting the absent answer. This failure is more pronounced in temporal reasoning tasks and worsens with denser frame sampling. We further explore chain-of-thought prompting as a mitigation strategy and find that while it substantially improves detection rates, performance remains unsatisfactory, suggesting that prompting-based strategies alone are insufficient to fully address this limitation. These findings expose a systematic failure in absent answer detection and highlight the need for explicit detection mechanisms in multimodal systems.

Abstract:
Longitudinal prediction from electronic health records (EHRs) is limited by the sparsity and irregularity in patient trajectories, and knowledge augmentation with external knowledge graphs (KGs) offers a promising way to alleviate these issues. However, most existing methods perform fixed, context-agnostic topology augmentation by adding the same KG nodes and edges regardless of a patient’s evolving state. We propose ReTA, a Reinforcement learning-based dynamic Topology Augmentation framework that casts KG import as a per-visit, budget-aware policy. ReTA first constructs an offline refined pool of KG-grounded templates, then learns a policy to select one augment action per visit from three options: Soft Import, which enriches node features without modifying graph topology, Hard Import, which grafts a compact KG subgraph onto the visit graph to create message-passing shortcuts, and Skip, which leaves the visit unaugmented when the base encoder is already confident. To stabilize learning, ReTA employs a decoupled encoder that processes semantic and structural signals in separate channels and fuses them via adaptive gating. Experiments on MIMIC-III and MIMIC-IV across diagnosis prediction, mortality, and readmission show that ReTA consistently outperforms strong baselines while remaining efficient, transfers across datasets and knowledge graphs, and yields interpretable augmentation patterns. The robust gains under sparse supervision highlight the advantage of ReTA’s dynamic decision to import knowledge, boosting accuracy while curbing costs.

Abstract:
Graphical user interface (GUI) agents are increasingly used to execute natural-language instructions on user interfaces, yet real users may issue infeasible instructions due to benign mistakes. A reliable agent should not only know how to act, but also when not to act. In this work, we introduce ConflictGUI, a benchmark covering instruction-internal conflicts and instruction-GUI context conflicts to study conflict-aware termination. Our evaluation reveals severe execution-biased over-compliance: agents that perform well on feasible tasks often continue to execute blindly under conflicting instructions. To mitigate this behavior, we propose ConflictGuard, an inference-time framework that aligns an agent’s feasibility awareness with its action generation. ConflictGuard contains two coupled components: a feasibility verification protocol that guides the agent to assess instruction logic and GUI-side evidence before acting, and a conditional action modulation mechanism that steers agent’s over-compliance into termination-oriented behavior. Experiments across five widely-used agents demonstrate that ConflictGuard improves average conflict task success rate significantly, while preserving normal GUI-task performance. These results validate that a lightweight inference-time intervention can substantially boost GUI Agent’s competence to identify inappropriate execution scenarios and refrain from unnecessary actions.

Abstract:
Retrieval-Augmented Generation (RAG) grounds large language models (LLMs) in external evidence, but existing RAG systems remain poorly matched to visually rich documents. OCR-based methods discard layout and non-textual visual cues, and although vision-language models (VLMs) can better understand native visual information, existing visual retrieval approaches often compress entire pages into global embedding vectors, obscuring fine-grained spatial information such as table cells and local annotations. Meanwhile, discrete Top-K selection in cascaded architectures blocks gradient flow between retrieval and generation, preventing joint optimization.In this work, we propose E2E-VisRAG, an end-to-end visual RAG framework that operates directly on native visual patches. Building upon patch-level visual encoding, we introduce a differentiable Top-K selection mechanism to enable joint training of retrieval and generation, and employ a gated residual fusion module to inject retrieval-enhanced visual features while suppressing noise. This design abandons the traditional paradigms of relying on OCR, global embeddings, and retrieval–generation separation, maximizing the preservation of original document visual semantics. Experiments on Vidore and MMLongBench-Doc show that, under matched 3B generator capacity, E2E-VisRAG outperforms non-end-to-end visual retrieval baselines, improving both Recall@1 and Accuracy@1 over OCR-based, global-embedding, and patch-level retrieval methods.

Abstract:
Vision-language models (VLMs) have shown strong capabilities in generating visualization code from textual or visual specifications. However, real-world visualization authoring is inherently iterative: users frequently revise existing visualizations to repair flawed charts or adapt them to desired styles. Existing benchmarks primarily evaluate generation from scratch, leaving visualization code editing from multimodal feedback largely unexplored. We introduce VisEditBench, a benchmark of 1,395 human-annotated visualization code-editing tasks grounded in realistic visualization workflows and failure cases. VisEditBench covers two practical settings: feedback-guided repair, where models revise visualization code using buggy or marked charts together with textual feedback, and reference-guided restyling, where models modify code to match a target chart image. Evaluating 24 state-of-the-art VLMs reveals that visualization code editing remains challenging: Claude Opus 4.6 achieves the best overall pass rate of 76.0%, while most open-source models remain below 50%. Performance is particularly weak on visually grounded style adaptation, where Claude-4.6-Sonnet achieves only 55.71%. To establish a strong baseline, we further propose VisEditAgent, a render-grounded editing framework that iteratively generates, executes, validates, and refines candidate edits. Built on GPT-4o, VisEditAgent improves overall pass rate from 55.75% to 67.99%, demonstrating the importance of render-grounded feedback for faithful visualization editing. We will release VisEditBench at .

Abstract:
Is a compact LLM verifier cost-effective for closed-label emotion classification when a supervised encoder is already available? We formulate inference-time escalation as cost-aware selective routing: an encoder predicts first, and only uncertain examples (via confidence, disagreement, or novelty gates) may be forwarded to a verifier. We run a Pareto search over thresholds and seven verifier slots, including zero-shot Qwen3.5-4B, LoRA-tuned Qwen, voting, self-consistency, MultiAgentESC-style routing, heterogeneous voting, and a second encoder. Across three-seed RoBERTa experiments, DeBERTa robustness checks, and an additional Russian XLM-R diagnostic, no compact-LLM verifier improves the high-quality low-cost frontier. The mechanism is a selection effect: encoder gating forwards a conditional slice on which compact LLMs perform no better than the encoder itself, or substantially worse, erasing any standalone LLM advantage. Encoder-only gating, split-conformal acceptance, and a small RoBERTa ensemble are strictly preferable alternatives.

Abstract:
Reward models play an essential role in aligning visual generative models, yet most existing visual reward models use a single scalar score or rely on fixed criteria that cannot adapt to different instructions. This limits both interpretability and task sensitivity, especially for text-to-image generation and instruction-based image editing, where different inputs require different evaluation dimensions. We propose RubricRM, a pairwise generative reward modeling framework that first produces an input-specific rubric with evaluation dimensions, weights, and scoring criteria, and then applies the rubric to score candidate images. We train dedicated RubricRM models for text-to-image generation and image editing using a two-stage training pipeline: supervised fine-tuning teaches the model the rubric-based scoring paradigm, while GRPO further improves scoring through fine-grained dimension-level rewards. Experiments on multiple generation and editing benchmarks show that RubricRM outperforms existing specialized reward models and remains competitive with strong proprietary MLLM judges despite using smaller backbones. Our models, data, and code are available at https://github.com/zijiankan/RubricRM.

Abstract:
Scientific literature reports fine-grained entity properties, but such knowledge is scattered across papers, expressed with heterogeneous names, and scoped by experimental conditions. Closed-schema extraction requires predefined attribute vocabularies that are hard to enumerate for property-rich scientific entities, while open-schema extraction often yields duplicate and condition-specific attribute labels. We introduce entity-centric attribute schema induction, which constructs a hierarchical type-level attribute schema from a scientific corpus. We propose ObSchema an observation-to-schema framework that extracts local entity-property observations, aggregates corpus-level evidence to identify recurring attributes, and refines them through value-aware merging and adaptive grouping. On two physical science domains, ObSchema consistently outperforms strong LLM-based schema-induction baselines across different backbone models, producing more compact and coherent schemas.

Abstract:
Low-Rank Adaptation (LoRA) is widely used for fine-tuning large language models, but suffers from catastrophic forgetting. Existing continual learning methods mitigate this by estimating per-parameter importance independently. However, this assumption breaks down under LoRA’s factorization 𝛥 W = BA : infinitely many pairs (B, A) yield the same weight update, making importance scores on A and B separately unreliable. We propose HLoRA, which computes path-integral importance in the combined weight update space, bypassing the unreliable separate estimation on A and B . HLoRA applies hierarchical regularization at two levels: element-wise constraints preserve parameters critical for general knowledge, and layer-wise coefficients adapt regularization strength based on each layer’s aggregate importance. Experiments on scientific, medical, and physical tasks using GPT-J and LLaMA-3 show that HLoRA mitigates catastrophic forgetting while enhancing adaptability, running approximately 20x faster and requiring only 10%–15% of the storage compared to prior methods. Code is released on https://anonymous.4open.science/r/HLoRA-915F.

Abstract:
Long-context inference in Large Language Models (LLMs) is bottlenecked by the linear growth of Key-Value (KV) cache memory. Existing KV cache compression paradigms are fundamentally limited by heuristics: heuristic budgeting relies on statistical priors rather than task objectives, causing resource misallocation, while heuristic selection relies on coupled query-key interactions or static inductive biases (e.g., attention sinks). To address this limitation, we introduce LKV (Learned KV Eviction), which formulates KV compression as an end-to-end differentiable optimization problem. LKV integrates LKV-H to learn task-optimized global budgets, and LKV-T to derive intrinsic KV importance without materializing attention matrices. This design bypasses heuristic proxies, strictly aligning compression with task objectives. Extensive evaluations demonstrate that LKV achieves state-of-the-art performance on both LongBench and RULER benchmarks at high compression rates. In particular, on LongBench, LKV achieves near-lossless performance with only 15% KV cache retention, and delivers up to 3.19x decoding throughput gains with negligible (~0.7%) overhead. Crucially, our analysis identifies learned budgeting as the dominant driver of fidelity, demonstrating that data-driven allocation is essential to overcome the limitations of hand-crafted heuristics.

Abstract:
Contrastive learning (CL) has become a dominant paradigm for self-supervised hypergraph learning, enabling effective training without costly labels. However, node entities in real-world hypergraphs are often associated with rich textual information, which has been largely ignored in prior works. Directly applying existing CL-based methods to such text-attributed hypergraphs (TAHGs) leads to three key limitations: (1) The common use of graph-agnostic text encoders fails to capture the correlations between textual semantics and hypergraph topology, resulting in less expressive representations. (2) Their reliance on random data augmentations introduces noise and weakens the contrastive signals. (3) The primary focus on node- and hyperedge-level contrastive signals limits the ability to capture long-range dependencies, which is essential for effective representation learning. To address these challenges, we introduce HiTeC, a two-stage hierarchical contrastive learning framework for effective self-supervised learning on TAHGs. In the first stage, we pre-train the text encoder with a structure-aware contrastive objective to overcome the graph-agnostic nature of conventional methods. In the second stage, we begin by introducing semantic-aware augmentations, including structure-contextualized text augmentation and semantic-aware hyperedge dropping, to facilitate informative view generation. Subsequently, we propose a multi-scale contrastive loss with an s -walk-based subgraph-level objective to capture long-range dependencies. Extensive experiments on six real-world datasets validate the effectiveness of our proposed method.

Abstract:
Adaptation of black-box large language models (LLMs) at test time seeks to adjust LLMs to target domains in order to handle distribution shifts during testing, without access to model parameters. Most existing approaches rely on adapters trained with substantial target-domain data, which is often scarce in practical scenarios, and these adapters are tightly coupled to their training distribution, leading to degraded performance in dynamic scenarios with changing domains. To solve this, we propose LADA, a framework leveraging meta-training for continuous and rapid adaptation of black-box LLMs to unseen and changing domains. Specifically, LADA meta-trains an adapter on diverse tasks that cover multiple datasets and error combinations to learn transferable adaptation skills. At test time, the adapter needs only a few target-domain examples for lightweight adaptation and guides the LLM stepwise toward domain-appropriate reasoning via adaptive selection of reliable steps, without any updates to the LLM parameters. Experiments on datasets spanning multiple domains validate the effectiveness of the proposed approach.

Abstract:
While Vision Large Language Models (VLLMs) have achieved remarkable success in multimodal reasoning, their long-context inference remains prohibitively expensive due to the massive computation and memory overhead of visual Key-Value (KV) caches. Existing KV compression methods often apply uniform pruning across visual tokens and layers, leading to substantial information loss and degraded performance. To address this challenge, we propose VisCache, a plug-and-play framework for coarse-to-fine Visual KV Cache pruning without training, which consists of two synergistic stages. First, a lightweight VLM filters temporal redundancy by selectively forwarding semantically informative keyframes. Second, we introduce PruneKV, a surgical KV compression algorithm tailored to the attention dynamics of VLLMs. Unlike rigid pruning strategies, PruneKV adopts a parabolic layer-wise budget allocation together with an asymmetric update mechanism that selectively prunes keys while fusing values, thereby preserving critical contextual information. Extensive experiments demonstrate that VisCache substantially improves inference efficiency, achieving up to 2.35 × speedup and significant memory reduction while maintaining competitive performance with only 19–28% KV cache retention. VisCache consistently outperforms existing baselines, establishing a new Pareto frontier between efficiency and performance for long-context VLLM inference.

Abstract:
Recent advancements in Large Language Models (LLMs) are increasingly driven by using LLMs themselves to synthesize large amounts of instruction-tuning data. To transcend the limitations of single model synthesis, recent research has shifted toward multiple model data synthesis, where distinct models assume specialized roles to synthesize data through interaction. However, existing feedback cannot distinguish truly high-quality instructions, those that promote complex reasoning, deviate from the learned distribution, and require multi-source knowledge, due to coarse granularity and signal dilution. In human cognition, truly high-quality instructions require greater mental effort—the Cognitive Load—to address. In this paper, we propose a paradigm shift in feedback. Instead of external, result-oriented evaluation, we propose an internal, process-aware feedback based on target models’ computational effort. The novel feedback, namely Inference Stress Index (ISI), quantifies instruction quality in a fine-grained, dilution-resistant manner by focusing on high-uncertainty tokens. Building on this, we present the Cognitive-Feedback Adaptive Generation (CoFAG) framework, a self-evolving multiple model collaboration paradigm involving a synthesizer model, a fixer model, and a collaborator model. Extensive experiments demonstrate that CoFAG significantly outperforms other data synthesis techniques on complex reasoning benchmarks, validating the superiority of using internal cognitive feedback for generating high-quality data.

Abstract:
Large language models (LLMs) are increasingly used for program-aided reasoning, agentic decision making, and structured task execution, but these settings often incur substantial inference cost. Many such requests share similar computational structures while differing in variables, constraints, or contexts, creating opportunities for program-level caching. Since program caches need to reapply reusable computation logic to new requests, their key steps often involve lightweight and structured operations such as variable extraction, program binding, and generation acceleration, which are well suited for small models. We propose CacheSpec, an inference optimization framework centered on reusable program caches. The framework converts Program-of-Thoughts (PoT)-style programs from one-time reasoning artifacts into reusable cache objects, and reuses the same small model for two roles: semantic variable extraction on the cache-hit path and speculative drafting during target-LLM generation. Experiments on shopping-style request datasets, WebShop, Formula, and CodeTATQA show that CacheSpec reduces inference latency and improves effective cache reuse while preserving comparable or better task quality than existing caching and generation baselines, achieving up to about 3.1× latency speedup; in parallel serving experiments, it improves throughput by about 2.8× over PoT-style methods. These results suggest that the sweet spot for small models in large-model inference systems lies not in solving complex tasks independently, but in performing lightweight, structured, and verifiable auxiliary operations.

Abstract:
Recent large reasoning models produce explicit reasoning traces before final answers, yet their behavior in culturally grounded reasoning remains insufficiently understood. In this work, we identify cultural overriding, a step-level failure mode in which a model initially recognizes a relevant local cultural norm but later weakens or replaces it with a more general normative frame, resulting in an incorrect judgment. To address this failure, we introduce CulturePRM, a 1B-parameter process reward model trained on step-annotated synthetic cultural reasoning traces to verify whether intermediate steps preserve, override, or are unrelated to the target cultural norm. At inference time, CulturePRM aggregates step-level rewards to rerank candidate reasoning traces via Best-of-N selection. We evaluated CulturePRM across multiple cultural reasoning benchmarks and base reasoning models and observed consistent improvements, with gains of up to 12.06 pp. These gains were not uniform across classes and were concentrated in lower-recall classes, while performance in already strong classes was largely preserved, thereby reducing class-wise disparities. A finer-grained error analysis further revealed substantial recovery among baseline errors alongside a marked reduction in cultural overriding. These findings suggest that culturally aligned reasoning requires models not only to recognize local cultural norms, but also to preserve them throughout the reasoning process.

Abstract:
Training LLMs for tool use requires more than final-answer supervision: models need trajectories that specify how to decompose a question, call tools across turns, use retrieved evidence, and recover from earlier errors. Existing data construction pipelines often rely on large-scale real-world tool API calls or simple templates, making them costly or limited in interaction coverage. We introduce ToolForge, a virtual-tool data synthesis pipeline for constructing tool-use datasets with three targeted abilities: multi-hop reasoning, multi-turn tool use, and self-reflection. Starting from QA seed data, ToolForge builds an extensible tool space from 19 base virtual tools and automatically generated variants, selects among four tool-use paradigms, simulates both successful and misleading retrieval outcomes, and generates validated tool-use dialogues without large-scale real-world tool API execution. A Multi-Layer Validation module filters invalid traces and is audited with hard negatives mined by Monte Carlo Tree Search. Fine-tuning Qwen3-8B on only 4,250 ToolForge samples produces a model named ToolForge-8B, which improves over the Tool-Integrated Reasoning baseline by 21.6% on five out-of-domain QA benchmarks and remains competitive on BFCL and API-Bank. We further quantify the pipeline construction cost, including tool initialization, dialogue generation, validation, and SFT, and compare ToolForge with existing synthetic tool-data sources. Our code and dataset are publicly available at https://github.com/Buycar-arb/ToolForge.

Abstract:
Current answering paradigms for Large Reasoning Models (LRMs) often fail to account for the fact that some questions may lie beyond the model’s operational capability boundary, leading to long but unproductive reasoning. In this paper, we study whether LRMs expose early signals predictive of such cases, and whether these signals can be used to mitigate unproductive reasoning. In black-box settings, we find that reasoning expressions contain failure-predictive signals. In white-box settings, we show that the hidden states of the last input token contain information that is predictive of whether a question will not be solved correctly under our evaluation setup. Building on these observations, we propose two test-time monitoring strategies: reasoning expression monitoring and hidden states monitoring, that reduce token usage by 62.7–93.6%, substantially improving efficiency and reliability while largely preserving accuracy.

Abstract:
Recent multimodal large language models (MLLMs) are credited with three capabilities: they fuse vision and text, forecast the consequences of an action, and recover from mistakes through reflection. A fourth assumption underpins every agentic deployment built on these models: that strong single-step performance transfers to competent multi-step behavior. Existing evaluations cannot adjudicate these claims cleanly: static perception suites never let small errors compound, and agent suites aggregate perception, planning, and control failures into a single score. No instrument exposes where, along the perceive-forecast-act-reflect-recover loop, a multimodal model fails. We introduce Cube Bench, a controlled diagnostic built on the Rubik’s Cube, where transitions are deterministic and oracle distance to the goal is available at every state. Seven oracle-scored tests isolate each stage of the decision loop under matched conditions. Across open and closed-source MLLMs, current models do not reliably exhibit any of the three credited capabilities, and the fourth assumption fails most sharply: a model with frontier-adjacent single-step skill across perception, prediction, and control can sit at the floor on recovery from its own errors. This dissociation between local skill and multi-step recovery has direct implications for agentic systems that assume these capabilities compose.

Abstract:
We explore intrinsic dimension (ID) of LLM representations as a marker of linguistic complexity. Specifically, we test whether ID differences across model layers reflect well-known complexity contrasts established in (psycho)linguistics: coordination vs.subordination, right-branching vs. center-embedding, and unambiguous vs. ambiguous attachment. Our results on six different LLMs show that these contrasts are consistently reflected in ID differences, with more complex phenomena eliciting higher ID profiles. Notably, ID differences emerge at different points across layers for different contrasts, also reaching their peaks at different stages. Further experiments using representational similarity and layer pruning confirm the trends. We conclude that ID is a useful marker of linguistic complexity in LLMs, that it points to similar linguistic processing steps across disparate LLMs, and that it has the potential to differentiate between different types of complexity.

Abstract:
This survey asks how far AI Scientist systems are from changing the world, and argues that their impact should be evaluated not by whether they produce plausible papers, but by whether they produce auditable research trajectories that can be inspected, reproduced, and challenged. We first provide a definition of AI Scientists, distinguish them from adjacent scientific AI systems, and organize the field around four core capabilities: knowledge acquisition, hypothesis generation, experimental validation, and evolution. We further synthesize representative systems by autonomy, execution grounding, auditability, human control, and evidence level. The resulting picture shows a field moving from prompt-to-paper automation toward auditable research laboratories, where executable grounding, provenance, and human-governed autonomy become central. We conclude by identifying open challenges toward trustworthy AI-driven science and providing a roadmap for integrating AI Scientists into the future landscape of scientific discovery.

Abstract:
The rapid growth of AI conference submissions is putting new pressure on peer review. AI reviewer systems are increasingly proposed as support, but prior work leaves unresolved what responsibility their outputs should carry when they can surface useful critiques yet remain risky as independent judgments. We frame this as a responsibility-boundary problem. Using 600 ICLR 2026 submissions, 2,231 human reviews, and 3,600 AI reviews, we operationalize this boundary through usable feedback, rating calibration, panel breadth, and grounded synthesis. AI can prepare critiques and organize evidence, while scoring, independent panel judgment, high-level synthesis, and final responsibility remain human-led. A GLM-5.2 validation on 100 NeurIPS 2022 papers with pre-ChatGPT reviews reproduces the workflow and calibration patterns. Motivated by this boundary, we develop Review Copilot, a workflow in which AI suggestions are inspected, edited, or rejected by human reviewers. In an initial controlled reviewer-in-the-loop study, Human+AI reviews improve actionability, evidence support, and professionalism relative to standalone baselines, while scores and recommendations remain human-authored. A user study of 275 reviewers across 411 papers further supports its practical usefulness. Together, these results support a review paradigm in which AI expands evidence-grounded critique while humans retain judgment, synthesis, and accountability.

Abstract:
To effectively collaborate with users on knowledge-intensive tasks, Large Language Models (LLMs) must perform information calibration: matching content to a user’s evolving understanding and cognitive capacity. Yet user simulators used to evaluate and train LLMs do not explicitly model user knowledge, so they neither produce realistic interactions across knowledge levels nor reflect how interactions unfold as that knowledge evolves. To close this gap, we introduce KnowSim, an evaluation framework built around a user simulator that maintains explicit knowledge states—a graph of Information Units with prerequisite relationships—that evolves under update rules grounded in learning theory. KnowSim computes three metrics (Knowledge Gain, Delivery Calibration, Cognitive Overload) directly from the knowledge state trajectory, reflecting key mechanistic aspects of information calibration. We validate KnowSim against 705 human–AI sessions across two domains, stratified by knowledge level: its rankings align significantly with human judgments (73–74% sign agreement), outperforming three baseline simulators. Applied to 8 frontier LLMs, KnowSim reveals that the best model shifts by user knowledge level—aptitude–treatment interactions invisible to standard evaluation.

Abstract:
Data contamination undermines the reliable evaluation of large language models (LLMs) on mathematical problem solving. While rewriting-based evaluation mitigates memorization, existing methods lack guarantees of problem validity and answer correctness. We propose Proof-Verified Benchmark Rewriting (RePro), the first framework to integrate Lean-oriented neural automated theorem provers (ATPs) into benchmark rewriting, which rewrites problems and regenerates answers with correctness ensured by Lean-verified proofs. Experiments on GSM8K and MATH show that RePro’s retained rewritten instances achieve 100% well-definedness, feasibility, and answer correctness, while existing methods still produce invalid or incorrect instances. Moreover, several models exhibit accuracy drops on proof-verified rewritten benchmarks, suggesting that their performance is sensitive to surface-level and structural variations and may partly reflect memorization effects. Our source code and data are available at https://github.com/AI4Engi/RePro.

Abstract:
Where does an LLM decide what’s right? We study whether normative judgment is implemented by a sparse, identifiable substrate or distributed across the model. Conventional circuit analysis can conflate normative computation with the multiple-choice format machinery that surrounds it. We pair four normative datasets (legal and moral, Chinese and English) with a non-normative MMLU control and apply a locate–disentangle–intervene pipeline that sets aside format-processing heads. Across Qwen3-8B, Llama3.1-8B, and Mistral-7B-v0.1, this isolates 5–10 normative-specific (NS) heads per model (under 1% of total) in MCQA settings. Targeted knockout and amplification reveal bidirectional causal effects across legal and moral tasks. The resulting heads form a sparse, late-layer intervention substrate for normative MCQA. Their coordinates differ across model families, so cross-model agreement is structural and circuit findings require functional re-identification in each family.

Abstract:
In this paper, we present a Mixture-of-Experts (MoE) quantization method based on activation entropy. Although quantization reduces memory and computational costs, it can substantially degrade performance. In particular, performance decline is pronounced in quantized MoE models, where individual experts have a small number of parameters that are sensitive to low-bit representation. Considering that MoE operates as an ensemble model with collaborative contributions from routed experts, a significant performance decline of a particular expert due to quantization can harm model performance. Therefore, we propose Colla-Q, a bit-allocation framework to maintain balanced performance across experts through an activation-entropy-based bit-width allocation algorithm. This approach encourages each expert to operate collaboratively in the quantized model, thereby 1) improving the overall MoE performance and 2) reducing the dependence on the calibration dataset. Since uniformly adjusting each expert’s performance facilitates robustness and stability of the MoE model, the proposed MoE quantization method can generalize more consistently across different calibration datasets. Our code is available at: https://github.com/mmai-laboratory/Colla_Q

Abstract:
Text-video retrieval requires representations that can distinguish videos with similar scenes, actions, and temporal patterns. Recent multimodal large language models have been adapted as embedding models, but they often represent each input using a single token from the final layer. This can compress diverse video-text cues into a single vector and limit fine-grained retrieval. To address this limitation, we propose MARS, a multi-layer and multi-slot embedding framework for text-video retrieval. MARS constructs multiple adaptive representation slots by combining hidden states from different decoder layers, compares corresponding text and video slots, and aggregates their similarities for retrieval. To better handle confusing candidates, we further introduce a hard-negative-aware slot specialization objective that encourages the slots to capture discriminative matching cues. Experiments on four text-video retrieval benchmarks show that MARS achieves state-of-the-art results in both direct similarity-based retrieval and reranking settings. Ablation studies and analyses demonstrate that multi-layer fusion, multiple slots, and hard-negative-aware slot specialization provide complementary gains.

Abstract:
Empathetic response generation requires models to decide not only what to say, but also how to respond to the previous speaker’s affective situation. We formulate this as response-side affective-orientation control and use multi-annotator emoji distributions as weak affective-attitudinal evidence, rather than as output symbols or gold labels, to induce a latent control space that operationally approximates listener stance. We construct EmojiDialogue, an utterance-level extension of EmpatheticDialogues with emoji votes and confidence scores, and propose EmoStance, which models source-side affective expression, predicts a soft response-side orientation from dialogue context and speaker roles, and steers a frozen instruction-tuned LLM through continuous prefix embeddings. In blind pairwise evaluation with 20 annotators and 800 judgments, EmoStance achieves a 62.2% decisive win rate, with the clearest gains in contextual specificity and perceived responsiveness, while remaining complementary to external-knowledge methods. Code, annotation metadata, and reconstruction scripts are available in our GitHub repository: https://github.com/18277390221/EmoStance.

Abstract:
Pruning has emerged as a dominant paradigm for accelerating large language model (LLM) inference, spanning a broad spectrum of methods that remove computation across tokens, layers, heads, dimensions, and attention patterns. Despite sharing the same objective, these pruning approaches induce fundamentally different execution behaviors, causing realized speedups to depend heavily on hardware and kernel implementations. Consequently, the practical acceleration benefits of different pruning families remain poorly understood. In this work, we introduce a GEMM-centric taxonomy that reorganizes existing pruning methods according to the logical M , N , and K dimensions of general matrix multiplication (GEMM). Leveraging this abstraction, we build a unified benchmarking framework that enables implementation-consistent comparison across the pruning design space and systematically characterizes the acceleration–quality Pareto frontier. Our results on Llama3.1-8B show that static depth pruning remains the strongest Pareto-optimal baseline and stays closest to its theoretical acceleration upper bound in memory-bounded scenarios. During prefill, the frontier transitions from static depth at low quality loss (0%–4%), to dynamic depth at moderate loss (5%–16%), and finally to static width pruning at higher loss levels (17%–26%). These findings establish the first unified view of the practical limits of pruning-based LLM acceleration and provide guidance for future pruning research.[Code is available at ]

Abstract:
Classification tasks require annotated data, which can often be expensive, time-consuming, or even unfeasible to collect.This is the case of the medical domain, where large datasets often have few annotated examples. To address this, we propose DecSelfMask (Decoder Self-learning by Masking), an approach to enhance decoder-only performance on classification tasks. We build on common self-learning approaches by leveraging a model to create training examples from unlabeled data, and propose a novel relevance-guided masking strategy.We use relevance attribution methods to determine what portions of unannotated texts are relevant for a task. We then create self-supervised training examples by masking out those portions, training the model to reconstruct them via next-token-prediction. We hypothesize that those examples convey knowledge about the structure and semantics of unannotated data that can be useful for downstream performance. We test our approach on 136 tasks from a collection of 1.9M clinical notes from an Italian hospital. We quantify DecSelfMask’s impact on downstream tasks on 5 models of different scales and families, including a probing analysis. Experiments show consistent gains, outperforming the base models (+9.1 points in Macro F1), continual pretraining (+6.3), synthetic label generation (+12.5), as well as common baselines. The results show that relevance attribution can serve as a powerful source of supervision for decoder learning from unlabeled data, beyond existing methods.

Abstract:
Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (VDR). However, existing multimodal search agents primarily target static images, and the current VDR benchmark relies on text-centric retrieval that discards crucial visual information. To address these limitations, we propose VideoSearcher, a closed-loop agentic framework that empowers Vision-Language Models with multi-tool reasoning for VDR. VideoSearcher unifies temporal localization, spatial focusing, and multimodal search within a single reasoning trajectory, enabling agents to progressively ground visual anchors, retrieve relevant evidence, and synthesize answers. To optimize knowledge-intensive reasoning trajectories, we propose Bi-branch Sequence Policy Optimization (BiSPO), a reinforcement learning algorithm that decouples tool-invocation optimization from answer-accuracy optimization. This design provides stable learning signals for both evidence-grounded reasoning and purposeful tool use. Furthermore, we construct VideoSearch-QA, the first benchmark designed to evaluate open-world video information grounding and multimodal search-based reasoning. Extensive experiments demonstrate that VideoSearcher significantly outperforms prior open-source agentic baselines across various search-oriented and multimodal understanding benchmarks. Our code is publicly available at https://github.com/Stephen-gzk/VideoSearcher.

Abstract:
Low-resource ASR remains difficult because scarce transcripts provide limited supervised evidence for target-side generation. To address this gap, we propose SAMA-ASR, a lightweight adapter mechanism that augments the decoder with semantic anchors from auxiliary translations and an acoustic anchor from speech; in principle, the mechanism can be applied to similar encoder–decoder multitask speech models. Through cross-modal adaptation, SAMA-ASR conditions decoder states on translation-derived semantic embeddings and a speech embedding, combining utterance-level meaning with speech-grounded evidence before token prediction. At evaluation time, these semantic anchors can be generated automatically by an upstream speech-to-text translator rather than supplied as oracle translations. Experiments on two 30-hour datasets covering the low-resource Sinitic varieties Taiwanese Hokkien and Hakka show that SAMA-ASR improves over acoustic, prior prompt-based, and semantic-only translation-guided baselines and remains effective in practical automatic semantic-anchor settings; translator-capacity analyses show that useful semantic anchors can be produced by a compact ST model.

Abstract:
Sequential recommendation models dynamic user preferences but is constrained by the long-tail distribution of real-world interactions. While LLM-derived text embeddings provide semantic priors to alleviate data sparsity, existing adaptation methods are hindered by popularity bias during semantic projection and semantic prior erosion during sequence training. In this paper, we propose A dversarial D ebiasing and A symmetric A lignment ( ADAA ), a framework designed to alleviate these representation bottlenecks. Specifically, ADAA incorporates an adversarial debiasing mechanism to decouple interaction frequency factors from the projected embeddings, yielding debiased semantic representations. Furthermore, it employs an asymmetric anchoring strategy during the sequence training phase. This strategy restricts the optimization space of sparse items to preserve their injected semantic priors, while permitting head items to flexibly fit sequential transition patterns. Through a parameter-level injection mechanism, ADAA integrates offline-aligned semantics without introducing additional parameters to the online architecture. Extensive experiments on three real-world datasets demonstrate that ADAA improves long-tail recommendation accuracy while preserving the online inference efficiency of the original sequential backbone.

Abstract:
Cross-view spatial reasoning remains a weak spot for vision-language models (VLMs): they often reason in language and lose the fine-grained geometry needed for the task. Thinking with images aims to address this by generating an intermediate thinking image, but recent work shows that models often ignore the visual evidence in these traces. We therefore ask how to make visual thinking matter, and what kind of visual thinking works best. We study these questions in unified multimodal models (UMMs), which natively support interleaved image-text generation. For the first question, we propose View Dropout (VDrop), a training-time intervention that hides parts of one input view from the answer span while keeping them visible to the thinking-image tokens. This encourages the model to use the thinking image when answering, instead of relying only on the input views. Once the thinking image is used for answer prediction, we study which type of visual thinking is most effective. We frame this as a learnability-informativeness tradeoff and compare three thinking-image variants: top-down, panoramic, and point-matching renderings. Trained on synthetic scenes and evaluated on five real-world out-of-domain benchmarks, panoramic visual thinking with VDrop is the only configuration that is both informative and learnable, and it achieves the best out-of-domain generalization.

Abstract:
We introduce ParsHate, a manually annotated dataset of 10,000 Persian tweets spanning 2013–2022, representing the first decade-long benchmark for hate speech detection in Persian. The dataset contains 31% hateful content and supports both hate detection and multi-label fine-grained target identification across seven structured target categories. ParsHate also distinguishes explicit and implicit hate, marks explicit and implicit targets, and provides span-level rationales. Data collection combines random and score-stratified temporal sampling to reduce keyword-driven bias while preserving natural label distributions. Applying SOTA models for Persian hate-speech detection on ParsHate shows moderate performance (79% F1), especially with samples from earlier years, and low performance with target identification (25.5% macro-F1). This emphasizes the diverse sampling of hate speech in ParsHate and its challenging nature that requires more advanced methods for better performance. Dataset is made publicly available.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) is commonly based on group sampling to estimate advantages and stabilize policy updates. In practice, computational limits often rule out very large groups, so training proceeds with finite rollout sets that can reinforce only the correct behavior they expose. At practical group sizes, updates can miss rare-correct trajectories while still containing mixed rewards, concentrating probability on more common sampled solutions. We derive the probability of such prompt-local tail-miss events as a function of group size, showing non-monotonic behavior, and in the categorical abstraction characterize how unsampled-correct mass can shrink even as total correct mass grows. Motivated by this analysis, we propose a difficulty-aware scaling coefficient, inspired by Focal loss, that down-weights updates on high-success sampled groups. Empirically, categorical simulation illustrates the same effect in the categorical setting, Maze provides a single-solution test, and LLM experiments include a representative GRPO group-size sweep together with fixed-N transfer across GRPO, DAPO, and CISPO. On Qwen2.5-7B at N=8, our method improves average math pass@256 from 64.1 → 70.3 (GRPO), 69.3 → 72.5 (DAPO), and 73.2 → 76.8 (CISPO); OOD pass@256 also improves in all three cases, without increasing group size or computational cost.

Abstract:
Reinforcement learning drives recent advances in LLM reasoning and agentic capabilities, yet current approaches struggle with both exploration and exploitation. Exploration suffers from low success rates on difficult tasks and high costs of repeated rollouts from scratch. Exploitation suffers from coarse credit assignment and training instability: Trajectory-level rewards penalize valid prefixes for later errors, and failure-dominated groups overwhelm the few positive signals, leaving optimization without constructive direction. To this end, we propose R 3 L, Reflect-then-Retry Reinforcement Learning with Language-Guided Exploration, Pivotal Credit, and Positive Amplification. To synthesize high-quality trajectories, R 3 L shifts from stochastic sampling to active synthesis via Reflect-then-Retry, leveraging language feedback to diagnose errors, transform failed attempts into successful ones, and reduce rollout costs by restarting from identified failure points. With errors diagnosed and localized, Pivotal Credit Assignment updates only the diverging suffix where contrastive signals exist, excluding the shared prefix from gradient update. Since failures dominate on difficult tasks and Reflect-then-Retry produces off-policy data, risking training instability, Positive Amplification upweights successful trajectories to ensure positive signals guide the optimization process. Experiments on agentic and reasoning tasks demonstrate a 4.8% average relative improvement over the strongest baseline across 27 settings while maintaining training stability. Our code is released at https://github.com/shiweijiezero/R3L.

Abstract:
Graph learning requires jointly leveraging node semantics and graph structure, but traditional methods often suffer from label scarcity and shallow semantic utilization. Although LLMs provide strong semantic reasoning for graph learning, existing LLM-augmented pipelines face two major challenges: their semantics-only outputs can inject semantic noise that conflicts with graph topology, and they often incur substantial overhead due to LLM fine-tuning or repeated LLM calls at training/inference. We propose CoFGraph, a confidence-feedback framework that interleaves reciprocal calibration between semantic and structural views to enable reliable and efficient LLM-augmented graph learning. To avoid expensive LLM adaptation, we keep the LLM frozen and train a lightweight Interpreter as a semantic proxy, eliminating the need for LLM fine-tuning and continual test-time prompting. To further mitigate semantic noise and the lack of structural feedback, we introduce confidence-based dual-view learning with node-wise confidence estimation and dynamic role switching, allowing the more reliable view to correct the other. Experiments on four public benchmarks validate the effectiveness of CoFGraph and demonstrate strong stability across varying training label ratios, with particularly consistent gains in low-resource regimes.

Abstract:
Recent advances in large language models (LLMs) have rapidly increased the public’s interest in natural language processing (NLP). This attention is reflected by news outlets inviting NLP researchers to share their knowledge and views with public audiences. Recognizing the opportunities for the research field and for individual researchers, this paper shares recommendations for speaking with news media about NLP’s capabilities and limitations. Drawing on the topical structure of research communication frameworks published by the US National Academies of Sciences, Engineering, and Medicine and the European Union’s QUEST project, recommendations are provided for three field-appropriate communication challenges: cognitive terminology, unsustainable expectations, and ethical failures. Published NLP research and popular news coverage are cited to provide examples. The recommendations promote effective, transparent communication with the general public about NLP, in order to strengthen public understanding and encourage support for research.

Abstract:
Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relying on ground-truth answers, limiting the ability to scale process-level supervision. We propose ScalePRM, which scales verification compute as an alternative: given a problem and a candidate solution, we generate multiple independent verifications of each reasoning step and aggregate their judgments to produce synthetic step-level labels without ground truth. We explore two representative inference-time scaling strategies, parallel scaling through self-consistency and sequential scaling through meta-critique, and train generative PRMs on the resulting synthetic data. On ProcessBench, a benchmark for identifying erroneous steps in mathematical reasoning, PRMs trained on step-level self-consistency data achieve 67.5 F1, surpassing reference-guided training with ground-truth access (66.4 F1) and GPT-4o as a critic (61.9 F1). When deployed as reward signals in RL training with Qwen2.5-Math-7B, our best PRM achieves 47.4% average accuracy across six mathematical reasoning benchmarks, outperforming ground-truth-based RLVR (43.9%). We also identify and address reward exploitation patterns unique to generative PRM-based RL. Our results demonstrate that scaling verification compute is a viable alternative to ground-truth supervision for training process reward models.

Abstract:
Long-context inference is increasingly limited by the memory and access cost of the key-value (KV) cache. Existing KV cache compression methods face a fundamental trade-off between efficiency and information preservation. Eviction-based methods are efficient but may discard critical tokens, whereas low-rank-based methods preserve all tokens at the cost of substantial computational overhead. We propose HachiWare, a dual-branch framework that applies one-shot selective SVD to semantic-critical cache states while handling less critical states with lightweight eviction. It stores these states in a Semantic-Critical Branch (SCB) using low-rank factors, while assigning the remaining states to an Eviction Branch (EB) for cheaper eviction. During decoding, bounded active-cache refresh selectively reconstructs only active SCB entries, avoiding reconstruction of the full SCB cache. On the RULER benchmark with Llama-3.1-8B-Instruct, HachiWare incurs only a 3% accuracy drop compared with the full KV cache baseline at a KV cache compression ratio of up to 16x, while improving accuracy by 5% over existing compression methods. For decoding efficiency, HachiWare achieves up to 9.44x and 16.50x throughput speedups over Full KV at 64K and 128K context lengths. These results highlight selective allocation of reconstructability as an effective way to improve the accuracy-efficiency trade-off of KV cache compression.

Abstract:
Large Language Models (LLMs) are increasingly controlled through system prompts that specify roles, formats, and safety requirements. However, models follow these prompts only implicitly through in-context learning, which can be insufficient for complex or compositional prompts. Existing approaches often require model tuning or response-level reranking, limiting their practicality for lightweight inference-time control. We introduce SyRuP, a decoding-time framework for improving system-prompt adherence while keeping the base LM frozen. SyRuP trains a cross-attention reward head from system-prompt-conditioned preference pairs, treating the system prompt as a separate memory to produce token-level adherence scores. At inference, SyRuP reranks the base LM’s top-k candidates by combining base logits with both the learned reward signal and a contrastive signal that captures system-induced logit shifts. Experiments on system-prompt following benchmarks show that SyRuP consistently outperforms prompting and decoding-time baselines with moderate inference overhead. These results suggest that explicit token-level guidance is an effective and practical mechanism for reliable system-prompt following.

Abstract:
Large Language Models (LLMs) have advanced Machine Translation (MT), yet they remain vulnerable by hallucinations. Most existing benchmarks, originally designed for neural MT models, do not effectively elicit hallucinations from multilingual LLMs, resulting in a low density of samples for LLM translation hallucination analysis. To address this, we create HalloMTBench, a human-verified hallucination benchmark derived from tens of millions of translations, for studying signals that trigger LLM translation hallucinations across 11 English-to-X directions. We curate 6,908 authentic hallucination instances through a rigorous process: frontier LLMs first translate a large source corpus, and their outputs are then validated by both an LLM-judge ensemble and human experts. Based on our analysis of these hallucination instances, we introduce a taxonomy for translation hallucination that distinguishes Instruction Detachment from Source Detachment. Our evaluation of 21 model variants on HalloMTBench reveals several observable vulnerability patterns, including model Characteristics, source length, language proximity, hallucination reproducibility, and model-dependent effects of reasoning-style decoding. HalloMTBench offers a forward-looking testbed for diagnosing LLM translation hallucination. All resources will be released.

Abstract:
Aligned LLMs systematically fail at morally complex role-playing: villains collapse into prosocial defaults due to a structural safety and fidelity tension. We present MoralSteer, a theory-guided on-policy self-distillation framework that improves morally complex role-play fidelity while preserving the safety behavior of aligned base models. MoralSteer treats Moral Foundations Theory (MFT) reasoning chains as privileged information for a frozen teacher, introduces a moral-complexity-adaptive KL interpolation 𝛼(c)=𝜎(M(c)-𝜏) to prevent prosocial collapse, and couples this with M(c) -adaptive trajectory mixing to mitigate rollout degeneration. Across four base models, MoralSteer improves Moral RolePlay L3+L4 scores by +5.8∼+9.9% ; across five safety benchmarks, the observed rate differences remain within ± 1pp of the aligned bases. We find that villain failure concentrates in Care/Fairness foundations (67% of the L4 gap) and that aligned models perform trait substitution rather than deletion. Our code is available at https://github.com/Sii-yuning/MoralSteer.

Abstract:
Second language learners benefit from exposure to level-appropriate grammar in interactive language use. While prompted frontier language models can usually produce target grammar structures, less resource-intensive models that can be run on device often fail in this task. We use Sequential Monte Carlo (SMC) to steer dialogue turn generation in small language models toward target grammar constructions from an expert-developed pedagogical resource. With a minimal set of hand-coded potential functions, SMC achieves near-perfect coverage of target grammar, enabling small models to outperform state-of-the-art results from models 140 times larger. We also use code synthesis with frontier models to generate new potentials, showing that our method scales to hundreds of target grammar skills with no need for additional training or supervised data.

Abstract:
Deploying multimodal foundation models as closed-loop policies increasingly requires conditioning actions on observations that are no longer visible. However, existing benchmarks either expose the full state, conflate hidden-state reconstruction with other agent skills, or test recall only after an episode has ended. We introduce RNG-Bench (Reconstructive Non-Markov Games), a benchmark suite designed to measure a base model’s ability to reconstruct past observations and act on them during multi-step interaction. RNG-Bench includes two complementary games: Matching Pairs, where card identities briefly revealed at specific locations must later be recalled, and 3D Maze, where egocentric views must be integrated into a spatial map. Both games are evaluated under a unified harness with three controlled difficulty axes: grid size, visual pattern, and observation modality. The benchmark further introduces a head-to-head duel protocol to control for instance-level variance and a Memory Gap diagnostic that reports the normalized baseline-to-oracle score difference under a specified interface. The hardest configurations require contexts of roughly 128K tokens and 350 image inputs per episode, and remain far from saturated by frontier MLLMs. The oracle interface yields substantial gains in some settings, while sizeable residual errors remain in others. Finally, fine-tuning Qwen3.5-9B on optimal-policy rollouts and filtered model demonstrations improves performance on RNG-Bench and transfers to existing benchmarks without degrading general multimodal capability.

Abstract:
While Large Language Model (LLM) role-playing agents have advanced rapidly, it remains unclear which profile elements genuinely drive role-playing quality. To bridge this gap, we introduce a systematic diagnostic framework that disentangles the impact of character profiles along three axes: Familiarity (Known vs. Unknown), Structure (Structured vs. Unstructured), and Disposition (Moral vs. Immoral). Utilizing a unified hierarchical schema (5 dimensions, 28 fields), we construct a controlled dataset of 211 personas and evaluate five LLMs on both single- and multi-turn interactions. Our results reveal a striking asymmetry: Familiarity and Structure show negligible impact, while Disposition produces large, consistent performance degradation for immoral characters across all conditions. Further analyses suggest that the Moral–Immoral gap is amplified by post-SFT alignment, and that this degradation varies substantially across profile attributes. To mitigate this bottleneck, we propose Field-Aware Contrastive Decoding (FACD), a training-free strategy that amplifies suppressed disposition-sensitive signals, significantly closing the performance gap without sacrificing moral-character performance.

Abstract:
Large Language Models (LLMs) are increasingly used to detect misinformation and support human fact-checkers. However, their truthfulness assessments may be affected by systematic political biases. Motivated by confirmation bias in humans, i.e., interpreting evidence in ways that are partial to existing beliefs or expectations, we investigate whether LLMs assess truthfulness in line with their political attitudes. We compare LLMs’ political attitudinal bias (PB-A) with political bias in truthfulness perception (PB-T). We assess PB-T by prompting LLMs to evaluate Democratic and Republican claims, including conditions where the claim-making party is explicitly stated. Across 20 instruction-tuned LLMs, all models exhibit a left-leaning tendency in PB-T, statistically significant for roughly half of them. Moreover, 16 out of 20 models significantly factor in the claim maker’s party affiliation when predicting truthfulness. PB-T and PB-A correlate weakly to moderately, and larger models tend to exhibit more left-leaning PB-T. The code for reproducibility is available on GitHub.

Abstract:
Research has documented LLMs’ name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstream assessment. In a large-scale controlled study, we analyze nearly one million resume summaries produced by 4 models under systematic race–gender name perturbations, using synthetic ONET-grounded resumes and real-world job postings. By decomposing each summary into resume-grounded factual content and evaluative framing, we find that factual content remains largely stable, while evaluative language exhibits subtle name-conditioned variation concentrated in the extremes of the distribution, especially in open-source models. Our LLM-judged hiring simulation demonstrates how evaluative summary transforms directional harm into symmetric instability that might evade conventional fairness audits in LLM-mediated pipelines, highlighting a potential pathway for LLM-to-LLM automation bias.

Abstract:
Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of newer evidence. Although recent studies have examined language-model behaviors in defeasible reasoning, the datasets have been static and lack wide coverage of non-monotonic reasoning categories. We introduce DeReLab, a generative framework that produces multi-turn belief-updating conversations from parameterized graph structures across default and inheritance reasoning, with formally verified ground truth at every turn, enabling controlled measurement of how models respond to confirming and disconfirming evidence. This controlled generation process creates a testbed for experimental designs that isolate specific reasoning demands. Applying this capability to the study of confirmation bias, we evaluate nine open and proprietary large language models and find that nearly all exhibit a systematic tendency to accept congruent evidence while resisting incongruent updates, with several models correctly identifying a weakening update yet failing to revise their conclusion. We believe our work and findings will facilitate future research on evaluating language models in defeasible reasoning.

Abstract:
Counterspeech effectively neutralizes the impact of online hate. Although prior work explores automated counterspeech generation, it largely emphasizes stylistic control while treating hate speech as homogeneous, overlooking that distinct forms of abuse require fundamentally different counterspeech strategies. To address this gap, we introduce FIRE (Factuality Informed Multi-Agent Reasoning Framework) that first decomposes hate speech into one of the five distinct categories (misinformation, stereotype, conspiracy, dehumanizing, non-factual), and then maps it to a targeted counterspeech style. To facilitate FIRE, we curate FactualCS, a novel dataset of 4,784 instances that provides the annotations regarding hate categories, reasoning traces, and evidence mappings, which are critical elements for grounded generation that are missing in prior work. A comprehensive evaluation across 28 baseline configurations demonstrates that FIRE significantly surpasses existing methods, despite using compact agents ( < 2B). FIRE achieves a ∼ 12% and ∼ 11% improvements in factual and category-specific accuracy respectively, while simultaneously reducing toxicity by ∼ 11% relative to the strongest baselines. Further human evaluation confirms that responses generated by FIRE are significantly preferred over the strongest baselines, underscoring its effectiveness for real-world deployment. These findings show that decomposing the underlying intent of hate speech is essential for generating safe, effective, and contextually precise counterspeech.

Abstract:
Large language models with retrieval-augmented generation systematically underutilize evidence in the middle of multi-document contexts, a phenomenon known as Lost in the Middle. Existing mitigation strategies, such as document reordering and attention steering, can reduce boundary bias but remain brittle in noisy settings, where inaccurate relevance estimation and spurious documents still obscure useful evidence in central positions. We propose DuQUAD, a multi-document QA framework that mitigates positional bias through dual-view reasoning and quality-aware evidence pruning. DuQUAD combines a Local Agent guided by a Structural Fusion Score with a Global Agent that reasons over the full document set, providing both targeted evidence recovery and broad contextual understanding. The framework further filters candidate evidence using explicit quality scoring and refines it at the sentence level to reduce noise. DuQUAD consistently outperforms strong Lost-in-the-Middle mitigation baselines, including recent multi-agent and context optimization methods, improving accuracy by 3.9% on average over the strongest competing baseline and gold evidence recall by 9.4%, while substantially reducing computational cost compared to existing multi-agent frameworks.

Abstract:
LLMs can imitate how people write, which raises concerns about impersonation, trust, and detection in social settings. These concerns are especially important for adolescents, who use generative AI frequently but may struggle to recognize it. We introduce DoppelBot , a cooperative social deduction game designed to study how young people detect and respond to AI impersonation. Through studies with middle schoolers, we investigate whether a DoppelBot prompts reflection on privacy and impersonation, how repeated exposure affects AI-detection accuracy as agents become more personalized, and which strategies students use to identify AI doppelgängers. We find that students’ detection accuracy improves over time, driven by a shift from relying on linguistic cues to leveraging shared social and contextual signals. Students also demonstrated an understanding of AI limitations such as embodiment and reflected on broader issues such as data privacy. To support future research, we release an anonymized dataset of game transcripts and voting behavior.[]

Abstract:
Deep research (DR) systems produce long-form cited reports by orchestrating multiple agents that search and synthesize information from the web. Citations are the primary mechanism for evaluating the faithfulness of these reports, yet current DR systems exhibit poor citation recall. Moreover, improving citation recall is challenging because DR systems are complex multi-agent architectures where information passes through agents like a telephone game, and both content and citations can get corrupted along the way. We propose an evaluation method that pinpoints which agent introduced each error by locally testing agent invocations for faithfulness and verifiability relative to their own inputs. Furthermore, we propose a four-type taxonomy to categorize the discovered errors: hallucination, uncited input reliance, uncited output, or insufficient citations. Applying our method to three top-ranked open-source DR systems, we obtain actionable diagnostics. Almost every agent makes a lot of mistakes with the exception being those that summarize a single document. We find that the dominant error type varies systematically across agents, where the orchestrator mistakes are mostly citation-related. We find that 84.7% of final-report errors in AI-Q originate at the orchestrator, roughly 31% of them hallucinations and the rest citation mistakes. Guided by these insights, we demonstrate that two simple interventions raise citation recall by 5% without degrading output quality.

Abstract:
The rapid progress in Artificial Intelligence has largely bypassed African languages, creating a digital divide that limits AI adoption on the continent. Recent open-source LLMs systematically underperform on African language machine translation, while the lack of large-scale, high-quality, open-source parallel data has constrained the development of competitive small language models (SLMs). We introduce TranslatePsy-AfriSLM, a collection of open-source machine translation resources for 19 Sub-Saharan African languages, including curated parallel data, African-specialized synthetic data, and a family of fine-tuned SLMs. Our empirical study shows that unified quality-estimation filtering removes up to 96% of training tokens without degrading quality, and that filtered synthetic data dominates the quality-efficiency Pareto frontier. Fine-tuned on the resulting data mixture, TranslatePsy-AfriSLMs outperform substantially larger systems, including TranslateGemma-27B and Qwen3.5-122B-A10B, with as few as 0.8B parameters.

Abstract:
Conversational language models are increasingly deployed as central knowledge repositories and "conversational operating systems" in enterprise environments, where they must comply with role-based access control (RBAC) and organizational hierarchies. However, evaluating models on natural language permission compliance is difficult due to the highly proprietary nature of real-world corporate policies, and existing benchmarks do not capture the semantic nuance of overlapping or conflicting rules within free-text queries. To address this gap, we introduce OrgAccess, a benchmark designed to evaluate LLMs on their capacity to adhere to complex organizational hierarchies under natural language constraints. Grounded in NIST cybersecurity frameworks and validated via a multi-round expert Delphi process, OrgAccess defines 40 standard permission types across three difficulty tiers (70,000 total instances), simulating scenarios with up to five concurrent, potentially conflicting permissions. Through extensive evaluations of 16 frontier and open-source models, we systematically assess their classification accuracy and diagnostic failure modes on permission-aware query routing. Our findings reveal a pronounced performance decline: as rule complexity increases, even flagship models fail to resolve combinatorial constraint interactions, frequently yielding over-conservative "False Partial" predictions due to safety-tuning hedging. Specifically, the strongest evaluated model achieves an F 1 -score of only 0.27 on our hardest split. This demonstrates that current instruction-following and safety alignment paradigms do not inherently translate to robust, multi-constraint rule compliance. More broadly, OrgAccess establishes a formal, reproducible testing framework to diagnose these constraint-adherence deficits and support the systematic integration of conversational agents into highly regulated, multi-user collaborative environments. The dataset is available on the anonymous project page .

Abstract:
Large language models (LLMs) have demonstrated strong performance on coding and reasoning benchmarks; however, their ability to solve graph-structured machine learning problems remains largely unexplored. In particular, no benchmark currently evaluates whether LLMs can autonomously solve end-to-end Graph Neural Network (GNN) coding tasks under realistic competition settings. To address this gap, this paper introduces GNN-CB , the first competition-based benchmark for evaluating both humans and LLMs on GNN coding tasks. GNN-CB consists of 18 curated competitions spanning node-, edge-, and graph-level prediction across diverse graph categories, domains, and difficulty tiers. All submissions are evaluated through a unified automated pipeline with hidden test sets and standardized scoring. Human participants solve tasks under controlled competition constraints, while LLMs are evaluated using a frozen zero-shot prompting protocol based on a plan-then-code paradigm with bounded execute-and-repair loops. The benchmark additionally supports both non-agent and autonomous agent-based evaluation within the same protocol. Under our evaluated protocol, LLMs rarely match Human Top performance and show less stable performance across competitions. No single model dominates: a few competitions are won by LLMs, yet humans still hold the top score on most tasks. We release GNN-CB as a living benchmark with automated evaluation infrastructure, dynamic leaderboards, and reproducible execution pipelines. Beyond benchmarking, GNN-CB provides a practice-oriented resource for studying GNN implementation across progressively diverse graph-learning tasks. The benchmark and evaluation framework are publicly available at https://basiralab.github.io/GNN-CB/ .

Abstract:
Computer-use agents are emerging as a paradigm for agentic human-AI interaction, combining language reasoning with multimodal interface grounding to operate GUIs. Yet their effectiveness for blind screen-reader users in real-world desktop workflows remains unclear. We present a three-week diary study with 8 blind users using OLLA, a screen-reader accessible CUA prototype, collecting 1,258commands across 12 applications with screenshots, UI trees, model responses, and action traces. We evaluate GPT-5 during deployment and re-execute the same commands with four additional models. GPT-5 achieved the highest success rate at 52.5%. Trace analysis reveals grounding, planning, constraint-tracking, and termination failures, while interviews reveal beyond-automation needs.

Abstract:
Embodied Vision–Language–Action (VLA) models are typically obtained by fine-tuning powerful pretrained VLMs on robotics data. Yet we do not know whether these agents still "know the basics" about the world after adaptation, nor how much commonsense and factual knowledge is preserved versus catastrophically forgotten. When a VLA fails on a knowledge-sensitive task, it is unclear whether the failure reflects degraded world knowledge or merely poor generalization of low-level control. We introduce Act2Answer, a lightweight protocol that adapts VLM knowledge benchmarks to VLA evaluation by requiring agents to act to answer. We map each benchmark question to a discrete embodied answer-selection decision in a controlled tabletop environment, where the agent answers via a single object-placement action into the region corresponding to its selected option, producing an action-grounded success rate with reduced control confounds. We introduce layerwise intent probing–linear classifiers on per-layer representations predicting the agent’s option selection–to localize answer-relevant information across model depth. Evaluating 9 state-of-the-art VLA models, we rank them across basic commonsense and world-knowledge categories and find that answer-relevant signals peak in middle VLM-backbone layers but attenuate in upper backbone layers and the action head.

Abstract:
Vision-language models (VLMs) have advanced multimodal perception, demonstrated by open-vocabulary object detection with simple language queries. State-of-the-art VLMs still struggle to handle complex queries involving descriptive attributes and relational clauses. To address this problem, we propose restructuring linguistic representations according to the hierarchical relations within sentences for language-based object detection. A key insight is the necessity of disentangling textual tokens into core components—objects, attributes, and relations—and subsequently aggregating them into hierarchically structured sentence-level representations. Building on this principle, we introduce the TaSe (Talk in Pieces, See in Whole) framework with three main contributions: (1) a hierarchical synthetic captioning dataset spanning three tiers from category names to descriptive sentences; (2) the three-component disentanglement module guided by a novel disentanglement loss function, transforms text embeddings into subspace compositions; and (3) aggregation disentangled components into hierarchically structured embeddings with the guide of proposed hierarchical objectives. Experimental results under the OmniLabel benchmark show a 24% performance improvement, demonstrating the importance of linguistic compositionality.

Abstract:
In this paper, we analyze how Large Language Models (LLMs) employ worldbuilding strategies, focusing on setting as one measurable dimension of storyworld construction. We compare 1,000 AI-generated stories per model in English and German with human-authored fiction from Project Gutenberg. Building on prior work, we operationalize setting through five types of narrative space: "action," "perceived," "visual," "descriptive" and "no space", identified using fine-tuned BERT classifiers for German and English. We generate narratives using GPT 4.1, LlaMA 3.3, Mistral 3.2, and Gemma 3 and compare their spatial distributions to a human-authored baseline. We find that human-authored texts predominantly employ "action space," grounding narratives in embodied character-environment interaction, whereas LLMs systematically overproduce "perceived space," emphasizing atmosphere and affect. This divergence remains stable across narrative time. Overall, our findings show that LLMs exhibit worldbuilding patterns that differ consistently from human-authored fiction in ways that are both model-specific and language-sensitive.

Abstract:
Large Language Model (LLM) agents increasingly act by generating, executing, and revising code, making code reasoning essential for understanding program state, diagnosing failures, and repairing behavior. Existing evaluations often measure end-to-end software engineering outcomes, obscuring which reasoning capabilities limit agents in complex, execution-grounded systems. We study this problem through Proof-of-Concept (PoC) generation, where an agent synthesizes an executable trigger that reproduces a reported crash or sanitizer failure. We instantiate this task in the Linux kernel, whose scale, statefulness, pointer-heavy value flow, memory layout, and concurrency stress long-horizon code reasoning. We decompose kernel PoC generation into four capabilities: state-machine reasoning, dynamic memory layout reasoning, value-flow reasoning, and concurrency reasoning. We build KPocInfra to emit branch-level execution telemetry and curate kPocBench, a benchmark of 181 reproducible Linux kernel KASAN cases from Syzbot. On kPocBench, state-machine and value-flow reasoning explain most failures, while capability-scaffolded evidence improves the best success rate from 50% to 61% and moves 29 unsolved attempts deeper along vulnerable execution paths. These results position kernel PoC generation as a practical stress test for LLM code reasoning in large-scale systems software.

Abstract:
Large language models (LLMs) can generate human-like online comments at scale, raising concerns that coordinated AI campaigns could manipulate public opinion. While prior work has shown that single AI messages can persuade individual readers, it remains untested whether coordinated multi-agent LLM campaigns, in which multiple agents jointly plan, divide roles, and manage visibility, can shift reader opinions in news comment sections, and whether per-comment AI labels can mitigate such influence. We address these questions through a controlled experiment with 1,000 U.S. adults reading news articles on a Reddit-like platform. Each participant was randomly assigned to one of six conditions, crossing three comment sources (human, independently generated AI, or coordinated multi-agent LLM) with two label settings (per-comment "Likely AI-generated" badges from a deployed detector, shown or hidden). Coordinated LLM comments did not increase average persuasion over independently generated AI comments, but disproportionately converted readers who initially disagreed and compressed opinion diversity by 12%, manufacturing apparent consensus where genuine disagreement existed. The labels reduced the perceived believability of comments but did not reliably reduce actual opinion change. These findings reveal what we call a Knowing-Resisting dissociation between source recognition and behavioral resistance, and indicate that per-comment disclosure alone is insufficient to counter coordinated LLM opinion campaigns.

Abstract:
Despite their linguistic diversity and global significance, African languages remain underrepresented in research and resources to support NLP. We aim to bridge this gap by introducing AfriSUD , the first multilingual collection of syntactically annotated treebanks for nine diverse African languages spanning major language families and regions across Sub-Saharan Africa. Using the Surface-Syntactic Universal Dependencies (SUD) framework, our community-led effort provides high-quality, native-speaker-verified data that capture typological key features such as agglutination and tone. We evaluate a range of models on AfriSUD for part-of-speech tagging and dependency parsing, including non-transformer baselines, multilingual pretrained encoders, and LLMs. Our results reveal a significant syntax gap, where models still show clear limitations across the nine languages, suggesting that existing architectures may not fully capture the structural diversity of African-language syntax.

Abstract:
Large Vision-Language Models (LVLMs) consistently require new arenas to guide their expanding boundaries, yet their capabilities with hypergraphs remain unexplored. In the real world, hypergraphs have significant practical applications in areas such as life sciences and social communities. Recent advancements in LVLMs have shown promise in understanding complex topologies, yet there remains a lack of a benchmark to delineate the capabilities of LVLMs with hypergraphs, leaving the boundaries of their abilities unclear. To fill this gap, in this paper, we introduce HyperGVL , the first benchmark to evaluate the proficiency of LVLMs in hypergraph understanding and reasoning. HyperGVL provides a comprehensive assessment of 12 advanced LVLMs across 84,000 vision-language question-answering (QA) samples spanning 12 tasks, ranging from basic component counting to complex NP-hard problem reasoning. The involved hypergraphs contain multiscale synthetic structures and real-world citation and protein networks. Moreover, we examine the effects of 12 textual and visual hypergraph representations and introduce a generalizable router WiseHyGR that improves LVLMs in hypergraph via learning adaptive representations. We believe that this work is a step forward in connecting hypergraphs with LVLMs.

Abstract:
Fine-tuning large language models (LLMs) on domain-specific datasets has become a standard paradigm for adapting LLMs to specialized applications. However, recent work has shown that even fine-tuning on benign task-specific data can substantially weaken the safety capabilities of LLMs. While existing efforts have made progress in identifying data responsible for safety degradation, they usually rely on a single mean vector computed over a specific model with its tokenizer to represent the safety direction, which limits both the effectiveness and transferability of their risk assessment measures. To address these limitations, we propose DataShield, a data assessment framework that identifies risky fine-tuning samples and response segments through consensus subspace alignment over joint safety-critical semantic spaces derived from multiple safety-aligned LLMs.Within these spaces, DataShield extracts consensus safe and unsafe subspaces using semantic spectral decomposition over safe and unsafe data representations. The risk of a data sample or segment is then estimated by measuring its relative alignment with the unsafe and safe subspaces, enabling both sample-level filtering and fine-grained segment-level masking. Compared with state-of-the-art filtering and masking baselines, DataShield reduces ASR by 14.6% with sample filtering and 32.3% with segment masking, while preserving downstream utility and avoiding target-model-specific risk computation.

Abstract:
Reliable set-valued prediction provides a principled way to mitigate hallucinations in open-ended question answering (QA), yet existing conformal approaches typically rely on a restrictive premise: finite sampling must produce at least one admissible candidate, or calibration examples violating this condition are discarded. In this paper, we introduce MiRD, a two-stage framework that decomposes overall miscoverage into sampling failure and conditional selection failure. In Stage I, MiRD establishes an expectation-level marginal upper bound on the probability that finite sampling produces no admissible answer under a fixed sampling budget. In Stage II, conditioned on sampling success, MiRD calibrates a conformal selection threshold using admission-correlated nonconformity scores constructed from the full calibration set, thereby preserving calibration-set integrity. Across three open-ended QA datasets and eight models, MiRD controls sampling risk, conditional selection risk, and overall miscoverage, while providing tighter first-stage bounds than PAC-style alternatives and more adaptive prediction sets than successful-only calibration.

Abstract:
Recent Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse vision-language tasks, creating an urgent need for more challenging benchmarks. Yet existing evaluations still provide limited insight into whether these models can truly reason over structured visual information. Visual Graph Reasoning (VGR) offers a compelling testbed for this challenge, requiring models to integrate perception, structural understanding, and multi-step reasoning over graph-based visual inputs. However, prior VGR benchmarks often reduce the task to visual perception followed by text-based reasoning, restrict evaluation to single-image settings, rely on answer-only metrics, and underrepresent realistic graph-centric scenarios. To bridge the gap, we introduce GraphVerse, a unified benchmark that jointly evaluates perception, visual reasoning, and text-based graph reasoning in MLLMs under both single-image and paired-image settings. At its core is a suite of Graph-centric Image Editing (GIE) strategies that modify graph images while preserving their semantics, turning them into active tests of visual reasoning. We further propose VGR-Score, a process-sensitive metric that evaluates reasoning quality beyond final-answer accuracy. Extensive experiments reveal several key limitations of current MLLMs in VGR, while also validating the effectiveness of GIE strategies and the transferability of GraphVerse to broader multimodal reasoning capabilities. The code is available at https://github.com/sunyuanfu/GraphVerse .

Abstract:
Role-playing language agents (RPLAs) simulate specific characters and personas across applications such as entertainment, companionship, interactive storytelling, and education. Faithful role-play requires more than producing plausible, in-character responses: as a character’s values and behavior change over a narrative, an RPLA should reflect the character’s state at the relevant stage. However, existing benchmarks largely treat characters as fixed personas or test only what they know at a given point in the narrative. We introduce ArcANE (Arc-Aware Narrative Evaluation), a benchmark for evaluating whether an RPLA follows a character’s development across a narrative. ArcANE first builds an Arc that maps how a character’s values, motivations, or relationships change over the story. The benchmark then scores how well an RPLA’s responses fit the corresponding stages of the Arc, covering three distinct scenario types: scenes from the novel, new situations within its world, and situations outside that world. We evaluate six models under six ways of providing narrative context. In every model, using the Arc up to the queried chapter yields the best performance, outperforming the strongest non-Arc context by 2.2–8.4 points. These results suggest that faithful role-play requires evolving character states and tracking their trajectory, rather than merely retrieving relevant episodic evidence.

Abstract:
Large Language Models (LLMs) enable a single model to support translation across multiple language directions simultaneously, eliminating the need to train a separate model for each direction and substantially improving the performance of multilingual machine translation. However, during multilingual translation instruction tuning, parameter interference among different languages significantly limits the improvement of translation performance and may even cause performance degradation for some languages. To address this issue, we propose MeRP, a multi-expert routing prompt encoding strategy for parameter-efficient fine-tuning (PEFT) in multilingual machine translation tasks. This method encodes representations of source languages through an attention mechanism and dynamically selects expert groups according to translation directions, enabling both the sharing and disentanglement of multilingual semantic representations. As a result, it preserves LLM generalization while reducing training time and improving translation performance. We conduct experiments on five language pairs. The results show that, compared with the baseline models, our method updates only 0.06% of the total model parameters during training while achieving better multilingual translation performance. Notably, our MeRP reduces training time by 46.75% and achieves average gains of +0.62 BLEU points and +0.34 COMET-22 points over the arithmetic mean of the four PEFT baselines.Code is available at https://github.com/wangzhenhan93/MeRP.

Abstract:
Large language models (LLMs) have achieved remarkable success across many domains, yet their generative capabilities have been misused to produce highly realistic fake news that threatens social stability. However, the main existing detection methods, including small language model (SLM)-based approaches and LLM-based detectors, exhibit substantial performance degradation on LLM-generated content, reflecting a lack of robustness. SLM-based detectors rely heavily on fixed data distributions and shallow textual cues, while LLM-based detectors depend on prompt-based inference, making them sensitive to prompting and vulnerable to LLM hallucinations. To address this challenge, we propose Fake News Court (FNC), a robust multi-agent adversarial framework that integrates the complementary strengths of LLM and SLM for detecting LLM-generated fake news. FNC adopts an adversarial learning paradigm in which a quality-controlled generator produces diverse and challenging fake news, while a hybrid detector combines multi-dimensional LLM-based agent reasoning with an SLM-based classifier to ensure stable decisions. Extensive experiments show that FNC achieves an overall accuracy improvement of approximately 6% over competitive baselines on LLM-generated fake news, confirming its robustness.

Abstract:
Recent vision-language models (VLMs) have shown promising progress in generating webpages from visual inputs, yet existing evaluations mainly focus on short, single-screen, and largely static webpages. We introduce LongWebBench, a benchmark for evaluating long-horizon webpage generation from both structural and functional perspectives. LongWebBench contains 490 real-world long webpages for structural fidelity evaluation and 507 goal-oriented interaction tasks over 129 webpages for functional evaluation. It employs two complementary protocols: a multi-dimensional VLM-based metric for assessing long-range structural coherence, and a DOM-augmented agent-based pipeline for end-to-end functional verification. We further examine the automatic evaluation protocols through human agreement analysis. Experiments with state-of-the-art open-source and proprietary VLMs under single-image and multi-image settings reveal that structural fidelity degrades as webpage length increases, while visually plausible generations often fail to support executable multi-step interactions. These results highlight the need to evaluate long webpage generation beyond visual similarity, with executable interaction as a core criterion. Our code and data are available at https://github.com/zhaoyi-hb/LongWebBench.

Abstract:
Language models (LMs) are trained on static scientific corpora, whereas scientific knowledge continuously evolves through correction and revision. Scientific claims encoded within these models may later become retracted, disproven, or updated by subsequent research, creating the risk of disseminating outdated information in scientific workflows. This creates a need for LMs to forget obsolete scientific claims. Machine unlearning offers a promising solution by enabling knowledge removal while maintaining overall model utility. Existing studies primarily investigate instance-level forgetting; however, scientific claims introduce additional challenges because they are interconnected, and continually evolving. To address this gap, we introduce the task of Scientific Claim Unlearning and present a new benchmark, SciUnlearn. We show that current unlearning approaches are unable to effectively eliminate claim-level knowledge and often achieve only superficial suppression, highlighting the need for specialized methods designed for structured knowledge removal.

Abstract:
Mixture-of-Experts (MoE) models enable efficient scaling, but training them from scratch remains prohibitively expensive. MoE upcycling mitigates this cost by converting pretrained dense models into sparse MoE models. However, existing upcycling methods typically rely on large-scale continued training and often perform poorly under data-constrained supervised adaptation, due to either homogeneous experts or overly disruptive perturbations to pretrained parameters. In this setting, effective upcycling must leverage pretrained weight structure while introducing sufficient diversity among routed experts. To this end, we propose SVD-Partitioned Residual Initialization (SPRI), which distributes SVD-partitioned residuals derived from pretrained feed-forward network (FFN) weights across routed experts, introducing controlled expert diversity grounded in pretrained spectral structure. We further introduce a two-stage training strategy to improve adaptation stability. We evaluate SPRI on multilingual speech-to-text translation, where limited supervised data challenges MoE upcycling and multiple target languages provide natural routing heterogeneity. On CoVoST2 across 15 En-to-X directions, SPRI improves average BLEU and COMET over fully fine-tuned dense models by 2.58 and 3.32 points, respectively, and outperforms the strongest prior MoE upcycling method by 2.44 BLEU and 3.21 COMET points. Our code is publicly available at https://github.com/shanweiqiao/SPRI .

Abstract:
Automatic Related Work Generation (RWG) significantly reduces the human time and effort required to author the Related Work Section (RWS) of a research paper. However, prior methods leveraging multi-agent Large Language Models (LLMs) typically rely on a predefined workflow, where each agent is responsible for a specific step in the entire process. This rigid, static inter-agent coordination limits the adaptive collaboration required to synthesize complex scientific literature. To address this limitation, we propose CREW (Collaborative Reinforcement Learning for Related Work Generation), a novel framework where LLM agents bypass heuristic pipelines to dynamically coordinate by autonomously selecting actions, such as Retrieve, Disseminate, Compose, and Critique, driven by a policy optimized via Independent Proximal Policy Optimization (IPPO). Extensive experiments on a standard RWG benchmark demonstrate that our approach yields substantial quality improvements over strong existing baselines, while significantly reducing token costs. Code is available at https://github.com/YenPBao/CREW-Collaborative-MARL.git

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has become a standard paradigm for post-training large language models (LLMs). While Group Relative Policy Optimization (GRPO) is widely adopted, it suffers from sparse reward signals and loses gradients entirely when all responses within a group receive identical rewards. On-policy distillation (OPD) offers a natural remedy by providing dense, token-level supervision from a teacher model. However, naively combining GRPO with OPD leads to degraded performance, due to three underlying causes: not all samples benefit from distillation; fitting too quickly to the teacher undermines the exploratory capacity of RL; and OPD’s advantages are asymmetric, suppressing most tokens. To address these challenges, we propose RSTG (Recovering Learning Signals via Adaptive Teacher Guidance), which applies distillation selectively and precisely where it matters most. At the sample level, OPD is restricted to negative zero-variance prompts with each sample weighted by the teacher’s confidence score. At the token level, distillation targets only tokens with high student entropy or large teacher-student divergence. We further augment training with SFT on correct trajectories generated by the teacher model, injecting positive gradient signals where RL yields none. Experiments demonstrate that RSTG substantially outperforms naive GRPO+OPD by +4.02% on math and +3.05% on code.

Abstract:
Backdoored generative LLMs can appear benign on normal inputs yet switch to unsafe or target-specific behavior when hidden triggers are activated, making request-level runtime detection a useful complement to model repair and offline auditing. Existing limited-black-box detectors typically rely on a single decoding trace, while text-only probing methods reveal susceptibility but often depend on specific probes or model-level audit goals. We introduce ProbeShift, a limited-black-box paired-probing framework that treats detection as repeated measurement of the same request under controlled prompt variations. It compares a baseline generation with templated probing generations and aggregates confidence dynamics, task-conditioned behavior cues, and paired response shifts. On safety-oriented BackdoorLLM DPA settings with LLaMA-2-7B-Chat, ProbeShift achieves remarkable performance: whilst maintaining high average recall rate of 97%, it reduced the FPR by 81% on average compared to the SOTA method ConfGuard based on single-track detection. These results show that paired probing yields a more stable and practical runtime signal than single-trace monitoring alone.

Abstract:
Temporal grounding—returning the interval [t_s, t_e] for a natural-language query over a video—is the language interface to long-form video, yet has been mainly studied on non-hour-scale videos; the dynamics of hour-scale natural-language grounding remain underexplored. We take the position that at hour-scale, the binding constraint is search, not recognition: Video-LLMs are bottlenecked not by localizing a nearby event, but by searching for the relevant region of a long video. To test this, we release ExtremeWhenBench (2,273 queries over 194 videos, mean 75.7 min, max 9 hr). It combines three properties that no prior benchmark has together: hour-scale mean video length, open-form explicit natural-language queries, and open access that works with standard Video-LLM toolkits. Every open Video-LLM collapses while a frame-level retrieval baseline outperforms them; a failure taxonomy attributes 88% of failures to search; and a retrieve-then-ground hybrid recovers 6.7× over the monolithic Video-LLM—mirroring retrieve-then-read in open-domain QA.

Abstract:
As large language models are increasingly deployed as tool-augmented legal agents, they introduce agentic hallucinations where tool-call and reasoning errors cascade into fabricated holdings and miscited authority. However, existing legal benchmarks evaluate only single-turn QA with outcome-level metrics, while agentic hallucination benchmarks lack legal-specific diagnostic capability. Neither answers to what extent and how a legal agent hallucinates along its trajectory. To address these limitations, we introduce LexAgentHallu, a legal agentic hallucination benchmark designed to evaluate to what extent and how legal agents fail along multi-step trajectories. Built through a four-stage expert-in-the-loop pipeline, LexAgentHallu contains 3 , 414 instances across 17 legal categories and 6 task types. Each instance is annotated under a dual-layer hallucination taxonomy of 7 high-level categories and 27 fine-grained subclasses, covering both substantive errors and agent-procedural failures. We further design fine-grained metrics that quantify to what extent and localize how each failure occurs along an agent’s execution path. Our evaluation across 18 proprietary and open-source agents uncovers a Right-Answer-Wrong-Reason effect and reveals that hallucination subclasses cluster rather than scatter, forming distinct agentic framework, legal task, and category profiles. These findings, invisible to outcome-level evaluation, validate the diagnostic power of LexAgentHallu for evaluating agentic hallucination in law.

Abstract:
Fairness benchmarks such as BBQ have become the de facto standard for fairness evaluation across major model families. We argue that these benchmarks are too easy to saturate to support that role: training Qwen 2.5 7B Base with Group Relative Policy Optimization (GRPO) on a single BBQ example lifts mean BBQ accuracy from 79.9 to 92.0 across all 11 social-bias categories, closing of the gap to the large-scale RLHF counterpart, and the effect generalizes across model families. A cross-conditioning analysis shows the improvement is carried by the reasoning traces generated by the model, not by the output head; one example suffices to elicit a category-agnostic “missing evidence” reasoning pattern. Strikingly, placing that single example in context as a one-shot demonstration—with no weight update—likewise reaches comparable BBQ accuracy and produces reasoning a fairness reward model rates as equally fair, confirming that the benchmark is saturable without gradient-based alignment. We argue that BBQ-style multiple-choice abstention benchmarks measure a single structural cue, and a model that solves them does not thereby become fair. We call for evaluation suites that cover a broader spectrum of fairness alignment.

Abstract:
Large language model (LLM)-based search agents synthesize open-web content into actionable recommendations on behalf of users, creating a risk that attacker-published pages are transformed into endorsed claims. We introduce SearchGEO, a controlled evaluation framework for measuring endorsement corruption in LLM-based web-search agents, combining a web-evidence manipulation pipeline, a five-mode attack taxonomy, and multiple output-level metrics. We evaluate 13 LLM backends on 308 cases each. Results show that vulnerability patterns vary across backends: overall attack success rate (ASR) ranges from 0.0% on Claude-Sonnet-4.6 to 31.4% on Gemini-3-Flash, the strongest attack mode differs by model family, and the same deployment scaffold could amplify or decrease ASR on different backends. An auxiliary agent-skill probe, where endorsement becomes an install command, exposes a sharp split among otherwise robust backends: Claude over-rejects while GPT over-trusts. These findings argue for treating recommendation reliability under adversarial search content as a first-class dimension of backend safety evaluation.

Abstract:
Mixture-of-Experts (MoE) LLMs rely on sparse, router-driven expert activation, yet how safety alignment interacts with routed expert specialization remains underexplored. A common intuition is that safety behavior may be controlled by routing harmful requests to distinct refusal-oriented experts. In this work, we provide empirical evidence for a different picture: routing patterns in aligned MoE LLMs are largely topic-driven, while safety behavior can be altered with little change to the model’s intrinsic routing path. Motivated by this observation, we present RASET (Router-Agnostic Safety-critical Expert Tuning), a red-teaming framework that probes whether safety enforcement can be localized in a small subset of experts while preserving the model’s intrinsic routing behavior. RASET identifies safety-critical experts via a contrastive routing-sensitivity criterion and applies parameter-efficient tuning only to the selected experts, minimizing semantic disruption relative to router-steering interventions. Across five open-weight MoE backbones, RASET achieves high-quality safety-bypass yield (50.5% average ASR hq , +37.6 points over the strongest baseline). These results reveal a distinct MoE safety risk, highlighting the need for expert-aware alignment, auditing, and robustness mechanisms in deployed MoE systems.

Abstract:
Recent advances in natural language processing have led to molecular Large Language Models (LLMs) with strong performance across diverse chemistry tasks. However, they still struggle to capture fine-grained structure-property relationships, particularly how small, localized modifications alter a molecule’s behavior. To address this limitation, we introduce MolSC, a dataset of substituent contributions, defined as property changes induced by attaching specific substituents to molecular scaffolds. Curated from manually annotated bioactivity records, MolSC spans structural-alert liability, target-specific bioactivity, and physicochemical descriptors, and contains 181K substituent-level examples for training. We further propose MolSC-Bench, a held-out evaluation benchmark of 1,541 examples disjoint from MolSC at the scaffold, substituent, and molecule levels. Our experiments show that existing molecular LLMs and strong proprietary models such as GPT-5.2 and Gemini-3-Flash show limited reliability in substituent contribution prediction. In contrast, training on MolSC substantially improves this ability and achieves strong performance across diverse downstream molecular tasks. These results highlight substituent contribution learning as a key component of fine-grained molecular understanding. (Our data and code are available at https://github.com/Park-ing-lot/MolSC.)

Abstract:
Text embedding models deliver competitive retrieval performance with high efficiency, but their ranking fidelity remains limited compared to LLM-based listwise rerankers, which capture fine-grained query-document and document-document interactions at high computational cost. We propose E2Rank (Efficient Embedding-based Ranking), a unified framework that extends a single text embedding model to perform both retrieval and listwise reranking via continued training under a listwise ranking objective. The key insight is to treat the listwise prompt—constructed from the query and its top-K candidates—as a pseudo-relevance feedback (PRF) query, enabling reranking via cosine similarity against precomputed document embeddings without autoregressive decoding. Empirically, E2Rank achieves state-of-the-art results on BEIR, competitive performance on the reasoning-intensive BRIGHT benchmark, significantly lower latency than existing LLM-based rerankers, and improved embedding performance on MTEB—all within a single model.

Abstract:
Large language model (LLM)-based multi-agent systems (MAS) improve reasoning through specialized collaboration, but dense multi-round communication can increase prompt cost and expose agents to outdated, redundant, or noisy context. A prominent line of MAS communication-efficiency work reduces communication through agent-, edge-, or topology-level structural decisions, which operate on graph structure rather than the realized text entering a specific receiver prompt. Such structural decisions are too coarse for runtime communication: messages sent over the same edge may carry useful evidence in one query, redundant content in another, or only a few useful spans. We introduce MessageSieve, a training-free runtime message pruning policy for fixed-graph MAS that operates on message instances rather than agents or edges. Before prompt assembly, it uses lightweight state cards and local message comparisons to preserve original text, retain compact evidence, or omit messages, without a trained selector or additional LLM call. We evaluate MessageSieve on two backbone models and five benchmarks covering expert QA, math reasoning, and code generation. Experiments show that MessageSieve improves the macro-average score over dense Full MAS by 4.10 points on Qwen3-8B and 2.59 points on Gemini 3.1 Flash-Lite, while reducing evaluation tokens by 19–49%.

Abstract:
Most existing harmful prompt detection methods rely on surface-level semantic representations, making them vulnerable to paraphrasing, prone to false positives, and limited in their ability to generalize to novel or adversarial prompts. Recent work has explored the use of model-internal signals, such as gradients or spectral features, to train classifiers that capture latent malicious behavior beyond surface semantics. However, these approaches primarily operate on local or isolated internal features and do not model the global structure of model behavior, which limits their robustness against subtle adversarial manipulations. In this work, we focus on robust harmful prompt detection under adversarial attacks. We propose SAFER, a simple yet effective method that models cross-layer attention signature flow by learning spectral correlation priors between attention heads and constructing a cross-layer attention-head spectral graph. This facilitates explicit modeling of structured attention propagation across layers, allowing the capture of both local and global attention dynamics and leading to improved robustness. SAFER achieves superior performance across multiple benchmark datasets for harmful prompt detection, while demonstrating significantly greater resilience to diverse adversarial attacks compared to prior approaches.

Abstract:
Negation-aware image generation remains challenging for text-to-image diffusion models, which often struggle with negation due to implicit linguistic biases that are deeply entangled within text representations. As a result, bias-aligned semantics frequently dominate over intended negative constraints. To overcome this limitation, we propose a simple training-free method of Decompose and RecOmpose text embedding (DeRO), which constructs a semantic-preserving subspace and a bias subspace using auxiliary prompts and performs semantic disentanglement based on bidirectional SVD. By decomposing entangled representations and selectively recomposing disentangled semantic components, DeRO realigns text representations toward intended negation semantics while restoring residual stylistic features to preserve semantic fidelity and image quality. As a model-agnostic inference-time framework operating solely on text representations, DeRO can be seamlessly applied to diverse diffusion architectures, including both U-Net and DiT based models. Extensive experiments on both object- and concept-level negation demonstrate consistent improvements in negation-aware image generation, achieving stronger text-image alignment, better semantic fidelity, and higher human preference scores.

Abstract:
Safety unlearning for multimodal large language models (MLLMs) fails when it treats danger as an object, topic, or prompt template. A useful MLLM must still recognize risky visual evidence and answer benign educational or preventive questions about the same content. The safety-critical target is narrower: the post-fusion interaction that converts risky visual evidence and unsafe textual intent into actionable harmful assistance. Existing refusal-oriented and unlearning objectives reach this mechanism only indirectly, widening refusal boundaries or leaving recoverable multimodal harmful pathways. We propose CMIE, Cross-Modal Interaction Erasure, a mechanism-level method that directly treats the harmful image-text interaction as the object of erasure. CMIE first estimates harmful interaction directions with intent-matched counterfactual contrasts that cancel unimodal main effects. It then localizes behaviorally active post-fusion layers and estimates a safe-whitened low-rank harmful subspace. A closed-form minimum-distortion weight edit erases this subspace, and projection-constrained recovery restores benign utility without rewriting erased directions. Across SafeEraser, VLGuard, and MSSBench, CMIE achieves the best overall mean rank and outperforms strong baselines on multiple safety, utility, visual selectivity, and paired-context metrics. Warning: This paper contains examples of harmful language and images, and reader discretion is recommended.

Abstract:
Understanding language requires tracking entities across discourse - i.e., knowing where things are and how they change, even when not explicitly stated. Whether language models perform such tracking in a human-like fashion remains unclear, in part because existing evaluations rely on artificial tasks, far removed from natural language comprehension, and lack comparisons to humans. Here, we evaluate entity tracking in both language models and humans ( N=48 ) using naturalistic narratives at multiple levels of complexity. In humans, we find that entity tracking degrades specifically with narrative complexity, not narrative length. In language models, we find that human-level entity tracking is already present at 410 million parameters - well below the multi-billion parameter, code-specialised models identified by prior work - and improves with scale, with contemporary models far exceeding human performance. Together, these results demonstrate that entity tracking, a core component of language understanding, emerges at model scales far smaller than previously thought.

Abstract:
Every scientific text is embedded in the writing norms of its time: it reflects earlier publications while also shaping the texts that follow. The growing use of LLM-based writing tools introduces a new factor in this evolving system, but the linguistic changes coinciding with their emergence remain insufficiently understood. We investigate whether the trajectories of scientific-preprint writing changed in rate or direction after conversational LLMs became publicly accessible, using the release of ChatGPT in November 2022 as the observational breakpoint. We compare pre- and post-2022 trajectories across vocabulary, punctuation, readability, and syntax in abstracts from arXiv’s artificial intelligence, statistics, and quantitative biology categories and from medRxiv. Using interrupted time-series regression, we find substantial post-2022 changes across multiple feature families and research domains, including increases in average word length, reduced formula-based readability, and declines in maximum dependency-tree depth and sentence-level depth variation. Individual features follow different dynamics: some lexical markers accelerate persistently, while others rise and then recede. Our findings show that isolated marker frequencies capture only part of the shift: changes in the rate, direction, and structure of linguistic trajectories provide complementary evidence of broader post-2022 change in scientific-preprint writing. More broadly, these findings motivate monitoring scientific writing as a multidimensional, evolving system rather than through a fixed set of LLM-associated markers.

Abstract:
Embodied planning is dedicated to the goal of building agents capable of perceiving, reasoning, and executing long-horizon tasks in physical environments. However, existing embodied benchmarks predominantly focus on single-agent execution or restricted multi-agent settings, leaving long-horizon tasks that involve complex inter-agent dependencies largely underexplored despite their prevalence in real-world embodied scenarios. To address this gap, we introduce CoEmbod , the first realistic 3D simulated embodied benchmark for long-horizon multi-agent embodied cooperation with diverse inter-agent dependencies. CoEmbod contains 155 expert-curated tasks across 40 scenes and 10845 objects, covering three cooperation regimes: spatial distribution, parallel constraint, and failure recovery. These regimes cover a broad spectrum of cooperative requirements, ranging from distributed task allocation to synchronized execution and assisted failure recovery. CoEmbod further provides a process-level evaluation framework with Budgeted Cooperative Success (BCS), which jointly measures goal completion, cooperation fidelity, and execution efficiency. Experiments reveal that even the leading LMM powering a well-established agentic workflow achieves only 20.23% success, highlighting the difficulty of long-horizon multi-agent cooperation with diverse inter-agent dependencies in complex embodied environments.

Abstract:
Small learned heads over frozen CLIP text embeddings are routinely evaluated on held-out examples whose lexical cue variants still come from the training pool, leaving lexical generalization untested. We study this gap on a controlled compositional preference task and identify a lexical cliff: a per-attribute cross-attention head over frozen ViT-B/32 features loses accuracy on held-out cue variants even when they sit inside a narrow CLIP-text neighbourhood (cosine [0.87, 0.97] to training) and the visual input is unchanged. A substitution probe that swaps only the text embedding lifts the worst-failing token from 0.500 to 0.893, and a 2×2 encoder factorial shows that cliff closure tracks the text encoder rather than the visual front-end. The cliff has two distinguishable mechanisms—a cosine-correlated decline on motion verbs (r=0.627, n=12) and a class-bimodal pattern on size and speed—and replicates across OpenCLIP-LAION-2B and SigLIP-WebLI with encoder-family-dependent token identity. Scaling to ViT-L/14 closes most but not all of the cliff, indicating the threshold is set by text-encoder geometry rather than raw cosine. We release CF-PrefBench v4 and recommend lexical-held-out splits with cosine stratification for evaluating small learned heads over frozen CLIP text.

Abstract:
Text-to-image models learn associations between concepts - in the case of this paper, people’s professions, which we refer to as roles - and visual attributes. These associations can underpin many observed forms of stereotypical bias. A key open question in this area is whether these associations are stable or change when visual representations of people in professional roles are placed in different prompted contexts. We introduce contextbias, a controlled evaluation framework, and contextbench, a benchmark spanning 92 roles and 1,656 semantically controlled prompts, designed to isolate the effect of contextual variation on role-linked visual representations. Evaluating four state-of-the-art models on 66,240 generated images, we find that placing a role in a semantically unrelated context does not suppress role-linked attributes; instead, cross-role attribute concentration increases (pooled BI +0.047 ). Demographic cues, characteristic garments, and role-specific tools remain highly prevalent across context-free, related, and unrelated conditions, and are robust to semantic prompt reformulation. Scene composition and camera framing show the greatest context-sensitivity. These findings reveal a form of stereotypical persistence that remains largely invisible to context-free evaluations, highlighting the need for controlled contextual variation in bias benchmarking. Code and dataset: https://huggingface.co/datasets/shaghayegh/ContextBias , https://github.com/Sina-Emami/ContextBias

Abstract:
Long Narrative Reasoning is an essential capability for processing and reasoning over complex narratives. While retrieval-augmented generation provides a promising framework, existing methods still face two critical challenges: cognitive islanding and cross-layer evidence disconnection. To address these issues, we propose PonsRAG, a coordinated RAG framework inspired by the biological pons. PonsRAG consists of two key components: Triple-Layer Indexing, which organizes documents into a connected knowledge structure to bridge cognitive islands, and Coordinated Reasoning, which retrieves evidence across distinct layers and integrates cross-layer information into a unified context. We evaluate PonsRAG on four long-context narrative benchmarks, and experimental results show that it outperforms the strongest baseline, achieving a 11.56% relative improvement in average accuracy on multi-choice tasks.

Abstract:
Idioms remain a persistent challenge for natural language understanding due to their non-compositional nature, especially in morphologically rich, low-resource languages like Cebuano. Existing work on idiom processing has explored span extraction, contrastive learning, and normalization, but typically treats these tasks independently or assumes previous steps have already been done. We propose CebIDN, a modular pipeline that unifies idiom identification, disambiguation, and normalization for Cebuano. Central to our framework is a morphology-aware canonical matcher that links extracted spans to an idiom knowledge base, passing structural tags, the idiom used, and retrieved literal hints to the downstream generative model. The system is evaluated in various tagging configurations, incorporating CRF-based sequence labeling and span-level triplet loss, along with few-shot and fine-tuned normalization strategies across both tagged and tag-free configurations. The optimal configuration (using a 3-tag BIO+CRF tagger with a tag-free normalizer) significantly outperforms a fine-tuned end-to-end LLM baseline by improving binary idiom detection (0.899 vs. 0.847 F1) and achieving superior normalization quality (88.87 vs. 86.89 BLEU; 96.40 vs. 95.04 BERTScore; 4.71 vs. 4.46 human-validated LLM evaluation scores), while running 1.7 times faster. These results demonstrate that structured task decoupling improves figurative language understanding in low-resource settings.

Abstract:
Large language models increasingly understand dialectal English, yet still produce only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed. We introduce DiaLLM, which continually pretrains three open-weight language model families on the International Corpus of English and applies implicit and explicit post-training paradigms, each combined with three model alignment strategies, giving the first controlled comparison of these components across Australian, Indian, and Northern British English. Our results reveal a robustness–generation gap: benchmarks are shaped by continual pretraining and SFT, while alignment visibly reshapes generation in ways benchmarks do not capture. Explicit variety-targeted adaptation produces output reliably recognised as dialectal and judged more dialectal than broad alignment, yet where human judgement was directly assessed, the method that most aggressively optimises the dialectal reward is not the one judged most dialectal. Independent linguistic analysis corroborates this reward-quality gap, most clearly on two of the three families. No single alignment method dominates, and closing the gap will require richer reward designs and continued investment in dialectal resources. We release all code, checkpoints, and preference datasets.

Abstract:
Choosing the right large language model (LLM) backbone is the most consequential decision when building a vision-language model (VLM), yet it remains fundamentally unprincipled: compute-based scaling laws fail to generalize across model families, and no framework exists for directly predicting VLM performance before training begins. We propose the Capability-Driven Multimodal Scaling Law, the first cross-family framework that predicts VLM benchmark accuracy from directly observable textual capability. Given a low-dimensional capability score S extracted from LLM textual benchmarks via PCA, we model VLM performance as a function of S , with a per-backbone transfer rate A and an absorption rate B that quantifies data-scaling efficiency. To fit and validate the framework, we train over 150 VLMs on 34 LLMs spanning 7 model families under a strictly controlled recipe. Evaluations on more than 200 textual and 50 multimodal benchmarks show that the law accurately extrapolates transfer rate from models up to 8B parameters to 72B-scale backbones, predicts full VLM training trajectories with high fidelity, and generalizes to entirely held-out model families. Beyond the scaling law, our analysis surfaces actionable insights: certain textual benchmarks negatively correlate with multimodal performance, exposing latent benchmark-gaming behavior; base LLMs outperform instruction-tuned counterparts as VLM backbones due to higher absorption rates and lower data-scaling decay; and different model families occupy distinct positions in the transfer–absorption space. The framework turns backbone selection from costly empirical sweeps into a principled, quantitative decision.

Abstract:
Retrieval-Augmented Generation (RAG) systems are vulnerable to poisoning attacks that inject malicious documents into the retrieval process to manipulate model outputs. Recent Agentic RAG systems are more robust to such attacks because they iteratively perform retrieval and reasoning, allowing them to ignore weakly relevant poisoned documents and preserve the reasoning chain induced by the user query. However, existing attacks on Agentic RAG systems often assume white-box access to system prompts, reasoning traces, retrievers, or model parameters, limiting their applicability in realistic settings. In this paper, we study black-box poisoning attacks against Agentic RAG systems, where the attacker can only publish externally retrievable poisoned documents. We propose KidnapRAG, a sequential poisoning attack that hijacks the agent’s multi-step reasoning chain using three role-specific documents: Bait, Chain-Link, and Mal-Ins, which attract initial retrieval, induce query reformulation, and provide attacker-controlled evidence, respectively. Experiments across multiple Agentic RAG frameworks, LLM backbones, and benchmarks show that KidnapRAG consistently outperforms existing poisoning baselines under black-box conditions. Further analyses show that KidnapRAG progressively weakens the original retrieval intent, redirects retrieval behavior, and increases reliance on attacker-controlled evidence.

Abstract:
Multimodal agents in robotics, AR, and autonomous driving must reason about places and layouts from continuous egocentric streams, often using evidence outside the current view. Existing benchmarks either evaluate offline over full videos or target events rather than spatial structure. We introduce OVO-S-Bench, a fully human-annotated benchmark for streaming spatial intelligence, comprising 1,680 questions over 348 source videos. Annotation involves 12 trained annotators (each also serving as a blind cross-reviewer) across roughly 804 person-hours of multi-round quality assurance. Each question carries a query timestamp and an evidence interval, and at evaluation, the model sees only the prefix preceding the query. Questions span four levels of increasing abstraction: instantaneous egocentric perception, spatiotemporal context tracking, generative spatial reasoning, and allocentric spatial mapping. Across 38 proprietary and open-source MLLMs, Gemini-3.1-Pro trails human experts evaluated under the same prefix-access protocol by 33 points (59.2 vs. 92.2), with allocentric spatial mapping as the dominant bottleneck. Notably, streaming and spatially fine-tuned MLLMs underperform their own backbones. We further find that chain-of-thought reasoning amplifies spatial errors when ungrounded in the stream. By exposing these limitations, OVO-S-Bench establishes a demanding testbed for next-generation streaming spatial MLLMs.

Abstract:
The consultation of reference cases in clinical radiology reporting motivates integrating multimodal retrieval-augmented generation (RAG) into automated systems. However, traditional RAG approaches employ single-vector embeddings for indexing, resulting in loss of granular information. Moreover, a semantic gap exists whereby visually similar radiographs often encode distinct pathologies. In this paper, we propose a Semantics-guided Multi-modal Late Interaction RAG (SeMLI-RAG) framework for radiology report generation. Our approach employs a weight-based late interaction strategy for retrieval to capture fine-grained radiographic features. Furthermore, we introduce an image-adaptive RAG to selectively leverage retrieved information, mitigating performance degradation caused by retrieval-induced noise. Experiments on three radiology benchmarks demonstrate that our framework yields consistent improvements over the backbone and RAG baselines across multiple evaluation dimensions. Further analysis shows that improving retrieval granularity and retrieval robustness can both benefit radiology report generation.

Abstract:
Non-deductive reasoning, encompassing inductive and abductive reasoning, is essential in addressing complex real-world questions. One key feature of inductive and abductive reasoning is that there are many valid hypotheses; the simplest ones (those that adhere to Occam’s razor) are often most useful. However, this aspect is ignored in recent work that evaluates the non-deductive reasoning capabilities of large language models (LLMs). This work fills this gap, focusing on understanding whether the inductive and abductive reasoning capabilities of LLMs adhere to Occam’s razor, while also examining the correctness of their reasoning. To accomplish this goal, we introduce a framework to synthetically generate reasoning questions that (a) require inductive reasoning and abductive reasoning simultaneously; (b) is readily extended to produce any abductive/inductive reasoning question expressible in first-order logic. The task for the intelligent agent is to produce hypotheses to explain observations under a given world model. We also propose a new automated metric to assess whether hypotheses quantitatively adhere to Occam’s razor; those hypotheses that are correct and simplest are considered high-quality. Our findings on state-of-the-art LLMs suggest that LLMs can perform inductive and abductive reasoning in simple scenarios, but struggle with complex world models and with producing high-quality hypotheses, even with popular reasoning-enhancing techniques such as in-context learning and RLVR.

Abstract:
Specification gaming under Reinforcement Learning (RL) is known to cause LLMs to develop sycophantic, manipulative, or deceptive behavior, yet the conditions under which this occurs remain unclear. We train 11 instruction-tuned LLMs (0.5B-14B) with on-policy RL across 3 environments and find that model size acts as a safety buffer in some environments but enables greater harmful exploitation in others. Controlled ablations trace this reversal to environment-specific features such as role framing and implicit gameability cues. We further show that most safety benchmarks do not predict RL-induced misalignment, except in the case of Sycophancy scores when the exploit relies on inferring the user’s preference. Finally, we find that on-policy RL preserves a safety buffer inherent in the model’s own generation distribution, one that is bypassed during off-policy settings.

Abstract:
Social interaction is central to children’s language learning, but the effects of different forms of caregiver feedback are difficult to isolate in naturalistic data. We use child-like language models as controlled learners to test which forms of feedback support grammatical development. Small GPT-2-style models are pretrained on child-directed language from CHILDES, then fine-tuned with reinforcement learning using reward models trained to capture four feedback types: communicative feedback, structural alignment, semantic contingency, and affective feedback. Reward fine-tuning yields limited gains on minimal-pair evaluations, but clearer effects in free generation. Structural alignment produces the strongest improvements in grammaticality, while communicative feedback yields more moderate gains. In contrast, semantic contingency and affective feedback do not improve grammaticality. These results suggest that caregiver feedback is not uniformly beneficial: different forms of social feedback support different aspects of language learning.

Abstract:
Reasoning-intensive retrieval often requires evidence whose relevance is not captured by keyword or embedding similarity alone, exposing a core limitation of traditional IR systems. An emerging direction known as LLM-guided hierarchical search addresses this limitation by letting an LLM directly traverse a semantic tree over the corpus. Such methods so far work well only with a frontier LLM in the search loop, which makes inference expensive. We propose HSearch-R1, a distillation recipe for learning compact (4B) LLMs that perform efficient LLM-guided hierarchical search. The recipe distils a frontier teacher’s per-step value estimates from its search trajectories into a much smaller LLM (Qwen3.5-4B) via RL, using a composite reward that matches the teacher’s ranking and score distribution at each step; this outcome-level RL distillation substantially outperforms supervised fine-tuning on teacher outputs. We further propose a retrieval-prior-informed variant of the search that injects a cheap retriever’s shortlist as a prior into the tree and adds robustness when the offline index is stale. The base HSearch-R1-4B matches or closely tracks frontier LLMs on the seven static BRIGHT-StackExchange subsets at roughly 50× lower per-query cost. Adding the retrieval prior brings HSearch-R1 to 52.5 nDCG@10 on StackExchange and 45.6 across all 12 BRIGHT subsets, within 1 nDCG of the strongest published ensembled baselines while remaining substantially cheaper.

Abstract:
Understanding how the same real-world event is represented across text, video, and audio remains a fundamental challenge in information extraction. While prior datasets have explored events within individual modalities or limited modality pairs, a setting that combines all three modalities remains largely unexplored. We introduce M 3 EC (Multimodal Multidocument Event Coreference), a new benchmark for multimodal, multidocument event extraction and coreference resolution. It is built from tightly related news clusters that combine eventful videos, audio files, and news articles. We also introduce a semi-automated annotation pipeline, AutoEventAnnotator, to generate automated annotations at scale. Additionally, we construct a human-annotated test set for robust evaluation. To our knowledge, this is the first benchmark to support cluster-level multidocument event extraction and event coreference across text, video, and audio. Evaluation with strong multimodal LLMs shows that the majority of them fail to extract valid events in video and particularly in audio. We further show that models finetuned on the annotations generated by our automated pipeline achieve consistent improvements, including more than +30 F1 on video event extraction, +15 F1 on audio event extraction, and +40 CoNLL F1 on end-to-end coreference resolution, highlighting the usefulness of M 3 EC for training and evaluation in multimodal event understanding.

Abstract:
Time series forecasting (TSF) is evolving toward multimodal and agentic settings, yet using foundation models remains uneconomical in resource-constrained scenarios, where compact, specialized forecasters are more desirable. However, lightweight forecasters typically require substantial training data, limiting their use in domains with scarce, slowly accumulated, or privacy-sensitive time series. To address this dilemma, we investigate the challenging problem of few-shot learning for lightweight forecasters. We propose MetaCaster, a meta-harness-optimized multi-agent framework that uses agentic data generation to automatically train specialized lightweight forecasters from only a few examples and textual contexts. Our work highlights a new TSF paradigm in which agents act not as forecasters but as intermediary engineers that prepare efficient, task-specific forecasters for deployment. Experiments on 18 datasets, 23 state-of-the-art lightweight forecasters, and 14 baselines demonstrate that MetaCaster achieves both data efficiency and computational efficiency while maintaining high-quality TSF performance.

Abstract:
Pretrained automatic speech recognition (ASR) models such as Whisper perform well but still need domain adaptation to handle unseen parlance. In many real-world settings, collecting speech data is impractical, necessitating text-only adaptation. We propose WhisTLE, a deeply supervised, text-only adaptation method for pretrained encoder–decoder ASR models. WhisTLE trains a variational autoencoder (VAE) to model encoder outputs from text and fine-tunes the decoder using the learned text-to-latent encoder, optionally combined with text-to-speech (TTS) adaptation. At inference, the original encoder is restored, incurring no extra runtime cost. Across four datasets and four ASR models, WhisTLE alone helps in 28 of 32 (88%) cases, with an average relative WER drop of 22%. Using WhisTLE alone or in combination with other adaptation approaches helps in 116 of 144 (81%) cases, with an average additional WER drop of 15%. Given all treatments that improve on baseline scores, WhisTLE helps in 45 of 55 (82%) cases, with an average additional relative WER drop of 16%. WhisTLE with TTS reduces word error rate (WER) by a relative 49% and outperforms all non-WhisTLE baselines in 100 of 112 scenarios. We also find that WhisTLE additively complements any combination of other domain adaptation approaches; we thus recommend the inclusion of WhisTLE during standard processes for adapting encoder–decoder ASR models.

Abstract:
Parameter-efficient fine-tuning (PEFT) of mixture-of-experts (MoE) models commonly attaches a separate low-rank adapter to each expert. This expert-wise design fragments adaptation in three ways: capacity is split across narrow low-rank updates, gradient supervision becomes sparse and imbalanced under sparse routing, and execution is decomposed into many small GEMMs. We find that such expert-wise separation is often unnecessary, as subsets of LoRA adapters become functionally similar during fine-tuning, revealing redundancy among expert-specific adapters. Based on this redundancy, we propose ACE (Adapter Consolidation across Experts), which groups redundant experts and replaces their expert-specific adapters with group-shared higher-rank LoRA modules under the same PEFT budget. ACE further introduces grouped adapter execution, which consolidates fragmented expert-wise adapter computations into fewer, larger group-level GEMMs. Across evaluations covering 12 datasets and four MoE backbones, ACE achieves the highest observed mean accuracy among the parameter-matched PEFT methods on the three backbones with complete baseline coverage, while providing 1.31× to 1.48× wall-clock training speedup over expert-wise LoRA without increasing peak memory. Our code is available at https://github.com/UbiquitousAILab/ACE.

Abstract:
China has over 1.116 billion mobile Internet users, whose demands continue to evolve rapidly. However, existing Chinese spoken language understanding (SLU) benchmarks are outdated and cover limited domains, failing to reflect real-world mobile assistant scenarios. To this end, we introduce CMPA-SLU, a Chinese SLU dataset for mobile personal assistants collected from real-world applications and annotated with rigorous quality control. CMPA-SLU contains 50 domains, 247 intents, and 74 slot types, expanding the diversity of existing Chinese SLU benchmarks. To better simulate practical deployment scenarios, we further introduce a Partial Category setting, where candidate labels are dynamically filtered using a Bidirectional Guided Embedding (BGE)-based retrieval strategy. We further adopt a label mapping strategy for joint intent and slot prediction within a single generation process. We evaluate multiple large language models (LLMs) under both Full Category and Partial Category settings. Experimental results show that the Partial Category setting improves performance by reducing the candidate label space.

Abstract:
Scientific literature question answering is a pivotal step towards new scientific discoveries. Recently, two-stage retrieval-augmented generated large language models (RAG-LLMs) have shown impressive advancements in this domain. Such a two-stage framework, especially the second stage (reranker), is particularly essential in the scientific domain, where subtle differences in terminology may have a greatly negative impact on the final factual-oriented or knowledge-intensive answers. Despite this significant progress, the potential and limitations of these works remain unexplored. In this work, we present a Scientific Rerank-oriented RAG Benchmark (SciRerankBench), for evaluating rerankers within RAG-LLMs systems, spanning five scientific subjects. To rigorously assess the reranker performance in terms of noise resilience, relevance disambiguation, and factual consistency, we develop three types of question-context-answer (Q-C-A) pairs, i.e., Noisy Contexts (NC), Semantically Similar but Logically Irrelevant Contexts (SSLI), and Counterfactual Contexts (CC). Through systematic evaluation of 13 widely used rerankers on five families of LLMs, we provide detailed insights into their relative strengths and limitations. To the best of our knowledge, SciRerankBench is the first benchmark specifically developed to evaluate rerankers within RAG-LLMs, which provides valuable observations and guidance for their future development.

Abstract:
Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and the key-value (KV) cache grows with the number of processed tokens. Larger context windows also do not ensure reliable evidence use. Context compression reduces this cost, but many soft-compression methods use LLMs as compressors and rely on compact memory tokens both to preserve information and to condition the decoder. We propose SeDeM, a selective decompression framework that decouples compact memory storage from decoder conditioning. SeDeM stores context as compact hidden-state memory blocks, selects query-relevant blocks, and decompresses only the selected blocks for decoder conditioning. Thus, the decoder avoids both full-context processing and direct generation from highly compressed memory slots. On four long-context QA benchmarks, SeDeM achieves higher QA scores than the compression baselines in our main comparison in both 1B and 3B same-backbone settings, and with the 3B backbone exceeds full-context fine-tuning on three datasets. The learned selector uses block-level evidence supervision during training. SeDeM also provides favorable quality–efficiency trade-offs, achieving 1.74–2.46 × lower online time-to-first-token and 1.08–1.10 × higher autoregressive decoding throughput relative to ICAE while maintaining strong answer quality.

Abstract:
Object hallucination remains a persistent reliability issue in large vision-language models, where generated object mentions may sound plausible but lack visual grounding. Recent training-free detectors use internal signals such as token likelihood, attention, visual confidence, or image-text similarity to identify hallucinated objects. These signals are useful, but they are often source-confounded. They measure how strongly an object is supported inside the model without distinguishing whether that support comes from object-specific visual evidence or the generated text prefix. In difficult cases, a hallucinated object can still receive high internal support because it fits the scene, is associated with nearby visual cues, or follows naturally from the generated text prefix. We propose VisER, a training-free two-sided metric for object-level hallucination detection. VisER evaluates each generated object mention from two complementary views. Visual Evidence measures whether object-context compatibility is backed by object-specific evidence from image tokens. Visual Reliance measures whether the object is supported more by the image than by the generated prefix. Combining these views gives a more source-aware grounding score, while avoiding additional object-level verification generations. Across multiple LVLMs and benchmarks, VisER improves AUROC and AUPR over a range of baselines.

Abstract:
Linearization has emerged as a strategy for developing efficient language models (LMs). Starting from an existing Transformer-based LM, linearization replaces the attention component with computationally efficient subquadratic token mixers . However, as an increasing number of mixers are proposed, it remains unclear which inductive biases are best suited to inherit the original Transformer’s capabilities. Furthermore, it is unknown how linearization is affected by parameter and token budget scaling. To address these questions, we propose a unified setup to compare seven representative architectures, including xLSTM, GLA, and Gated DeltaNet. Our findings reveal that performance hierarchies remain stable from 140M to 1.7B parameters, with error-correcting update rules demonstrating superior scaling exponents. We show that performance gaps are established early and persist through asymptotic maturity at 10B tokens, suggesting that state resolution is a more fundamental bottleneck than the distillation budget. Finally, while most models adapt to instruction tuning, only gated delta-rule formulations maintain the precision necessary for long-context retrieval, whereas additive models suffer from irreversible state saturation. Finally, we show that a cheap mixer-only probe predicts these downstream outcomes before a full run and, used as a diagnostic, attributes the weakest student’s scaling deficit to a fixable normalization choice rather than its update rule. These results suggest that for successful linearization, architectural inductive biases remain the primary constraint that cannot be overcome by simply scaling training compute.

Abstract:
Open-source LLMs provide a flexible alternative to closed-source models for Text-to-SQL but still struggle with complex queries. Finetuning with chain-of-thought (CoT) data improves reasoning, yet standard datasets lack CoT annotations, requiring synthetic generation. Existing synthesis methods only verify the final answer, allowing flawed reasoning steps to corrupt the training data. We propose VCoT-SQL, a framework that improves Text-to-SQL generation through intermediate-step-validated CoT supervision and a dedicated SQL validator, which enables small open-source models to outperform frontier closed-source models. VCoT-SQL trains two complementary models on curated reasoning traces: an SQL generator using clause-level CoT templates where reasoning steps are verified via execution and schema alignment checks, and an SQL validator using diagnostic CoT templates to detect syntactic, semantic, and type-level errors. We further design a hierarchical schema filtering pipeline that integrates and refines complementary retrieval strategies to distill relevant database elements. VCoT-SQL achieves 66.82% execution accuracy on challenging dataset BIRD dev and 86.8% on Spider test, improving the base 7B model by 7% on BIRD. Notably, our CoT synthesis relies on 7B models yet outperforms methods that use 72B-scale or even proprietary closed-source teachers for data generation, demonstrating that rigorous step-level validation can be more effective than scaling up teacher model size. Our dataset and code are available at https://github.com/INTREBID/VCoT-SQL.

Abstract:
We present PopResume, a population-representative resume dataset for causal fairness auditing of LLM- and VLM-based resume screening systems. Unlike existing benchmarks that rely on manually injected demographic information and outcome-level disparities, PopResume is grounded in population statistics and preserves natural attribute relationships, enabling path-specific effect (PSE)-based fairness evaluation. We decompose the effect of a protected attribute on resume scores into two paths: the business necessity path, mediated by job-relevant qualifications, and the redlining path, mediated by demographic proxies. This distinction allows auditors to separate legally permissible from impermissible sources of disparity. Evaluating four LLMs and four VLMs on PopResume’s 60.8K resumes across five occupations, we identify five representative discrimination patterns that aggregate metrics fail to capture. Our results demonstrate that PSE-based evaluation reveals fairness issues masked by outcome-level measures, underscoring the need for causally-grounded auditing frameworks in AI-assisted hiring.

Abstract:
While Large Language Models (LLMs) have shown strong performance on clinical text understanding, they struggle with longitudinal prediction tasks such as dementia prognosis, which require reasoning over complex, non-monotonic symptom trajectories across multiple visits. Standard supervised training lacks explicit annotations for symptom evolution, while direct Reinforcement Learning (RL) is hindered by sparse binary rewards. To address this challenge, we introduce Dementia-R1, an RL-based framework for longitudinal dementia prognosis from unstructured clinical notes. Our approach adopts a Cold-Start RL strategy that pre-trains the model to predict verifiable clinical indices extracted from patient histories, enhancing the capability to reason about disease progression before determining the final clinical status. Extensive experiments show that Dementia-R1 achieves the best overall performance on a real-world unstructured cohort, reaching an AUROC of 84.02% and outperforming models up to 10 larger. The framework also generalizes to Parkinson’s disease dementia prediction in an independent hospital cohort, achieving an AUROC of 78.37%. On the ADNI benchmark, our 7B model attains the highest AUROC among all LLM baselines at 83.17%, demonstrating strong longitudinal reasoning over fluctuating cognitive trajectories. Code is available at https://github.com/hangeol/dementiar1.

Abstract:
Estimating the Arabic Level of Dialectness (ALDi) is a continuous regression task that quantifies how much a sentence diverges from Modern Standard Arabic (MSA). While auxiliary classification is often used to support regression, its effectiveness depends critically on how the auxiliary task is designed. We conduct a systematic study of auxiliary classification for ALDi estimation, showing that naive auxiliary supervision can degrade regression performance under skewed target distributions. We identify two key factors that determine when auxiliary tasks become beneficial: the granularity of discretization and the choice of loss function. Specifically, 7-bin discretization combined with focal loss substantially improves performance. Our approach achieves an RMSE of 0.1275 on the NADI 2024 blind test set, outperforming previous systems by over 9%, and the same design transfers to English toxicity regression. Detailed error analysis reveals how auxiliary classification reshapes model behavior across the Arabic dialect spectrum. We release our code and model predictions to facilitate future research

Abstract:
The rapid growth of scientific literature has made automated verification of scientific claims increasingly important, with Retrieval-Augmented Generation (RAG) systems emerging as a promising solution. However, their multi-stage architecture raises concerns about confirmation bias, where systems preferentially retrieve and generate evidence that supports a given claim. In this paper, we present CoBiT (Confirmation Bias Test), the first benchmark for systematically evaluating confirmation bias in RAG systems, consisting of 707 minimally perturbed claim–counterclaim pairs. We propose a unified framework with stage-specific metrics to quantify bias in both re-ranking and generation, including Ranking Bias Magnitude (RBM), Confirmation Bias Rate (CBR), and consistency measures. Our experiments on OpenScholar,ScholarQA, and LLM baselines reveal that confirmation bias arises at different stages of the RAG pipeline and can be amplified by re-ranking and generation. In particular, SciRAG achieves the lowest generation bias (CBR = 0.13), it exhibits higher bias in re-ranking, and OpenScholar shows the greatest inconsistency between claims and counterclaims. These findings highlight the need for fine-grained evaluation of epistemic risks in RAG systems and provide practical tools for diagnosing confirmation bias. Code and Data https://github.com/Presence76/Confirmation_bias_evaluation

Abstract:
Humor preferences vary widely across individuals and cultures, complicating the evaluation of humor using large language models (LLMs). In this study, we model heterogeneity in humor preferences in Oogiri, a Japanese creative response game, by clustering users with voting logs and estimating cluster-specific weights over interpretable preference factors using Bradley-Terry-Luce models. We elicit preference judgments from LLMs by prompting them to select the funnier response and found that user clusters exhibit distinct preference patterns and that the LLM results can resemble those of particular clusters. Finally, we demonstrate that, by persona prompting, LLM preferences can be directed toward a specific cluster. The scripts for data collection and analysis are publicly available to support reproducibility.

Abstract:
Simulated environments play an essential role in embodied AI, functionally analogous to test cases in software engineering. However, existing environment generation methods often emphasize visual realism (e.g., object diversity and layout coherence), overlooking a crucial aspect: logical diversity from the testing perspective. This limits the comprehensive evaluation of embodied agent adaptability and planning robustness across distinct simulated environments. To bridge this gap, we propose LogicEnvGen, a novel method driven by Large Language Models (LLMs) that adopts a top-down paradigm to generate logically diverse simulated environments as test cases for agents. Given an agent task, LogicEnvGen first analyzes its execution logic to construct decision-tree-structured behavior plans and then synthesizes a set of logical trajectories. Subsequently, it adopts a heuristic algorithm to refine the trajectory set, reducing redundant simulation. For each logical trajectory, which represents a potential task situation, LogicEnvGen correspondingly instantiates a concrete simulated environment. Furthermore, we introduce LogicEnvEval, a novel benchmark for simulated environment generation, with four quantitative metrics. Experimental results verify the lack of logical diversity in baselines and demonstrate that LogicEnvGen achieves 1.08-2.67x greater diversity, significantly improving the performance in revealing agent faults by 3.34%–72.00%.

Abstract:
Medical benchmarks are dominated by single-turn, multiple-choice clinical cases that poorly reflect real consultations. Practically, clinicians elicit evidence interactively, and patient communication varies widely. We introduce MEDROUNDSQA, a multi-turn diagnostic benchmark derived from 1,387 clinical cases across 17 specialties. Each case is converted into a structured 24-slot clinical record and then instantiated as controlled doctor–patient dual-agent dialogues under varying patient personas, with the underlying clinical content held fixed. We further classify cases by difficulty using model-based uncertainty to enable easy-to-hard analysis. Evaluations of fifteen LLM doctor agents show that (i) moving from a single-turn diagnosis on the standardized records to multi-turn consultations causes large degradations of roughly 13–39 points; (ii) more turns reliably improve question relevance, but diagnostic accuracy exhibits diminishing returns and typically plateaus after 6–12 turns; and (iii) patient persona differences can shift diagnosis accuracy by about 7–8 points from the lowest to highest education levels, highlighting equity risks that single-turn benchmarks miss.

Abstract:
Standard evaluations of Large language models (LLMs) focus on task performance, offering limited insight into whether correct behavior reflects appropriate underlying mechanisms and risking confirmation bias. We introduce a simple, principled interpretability framework based on token-level perplexity to test whether models rely on linguistically relevant cues. By comparing perplexity distributions over minimal sentence pairs differing in one or a few ‘pivotal’ tokens, our method enables precise, hypothesis-driven analysis without relying on unstable feature-attribution techniques. Experiments on controlled linguistic benchmarks with several open-weight LLMs show that, while linguistically important tokens influence model behavior, they never fully explain perplexity shifts, revealing that models rely on heuristics other than the expected linguistic ones.

Abstract:
Assessing the veracity of a claim made online is a complex and important task with real-world implications. When these claims are directed at communities with limited access to information and the content concerns issues such as healthcare and culture, the consequences intensify, especially in low-resource languages. In this work, we introduce AfrIFact , a dataset that covers all necessary steps for automatic fact-checking (i.e., information retrieval, evidence extraction, and fact checking), covering ten African languages and English. Our evaluation results show that even the best embedding models lack cross-lingual retrieval capabilities, and that cultural and news documents are easier to retrieve than healthcare-domain documents, both in large corpora and in single documents. We show that LLMs lack robust multilingual fact-verification capabilities in African languages, while few-shot prompting improves performance by up to 43% in AfriqueQwen-14B, and task-specific fine-tuning further improves fact-checking accuracy by up to 26%. These findings, along with our release of the AfrIFact dataset, encourage work on low-resource information retrieval, evidence retrieval, and fact checking.

Abstract:
Deep research agents have shown remarkable potential in handling long-horizon tasks. However, state-of-the-art performance typically relies on online reinforcement learning (RL), which is financially expensive due to extensive API calls. While offline training offers a more efficient alternative, its progress is hindered by the scarcity of high-quality research trajectories. In this paper, we demonstrate that expensive online reinforcement learning is not all you need to build powerful research agents. To bridge this gap, we introduce a fully open-source suite designed for effective offline training. Our core contributions include DeepForge, a ready-to-use task synthesis framework that generates large-scale research queries without heavy preprocessing; and a curated collection of 66k QA pairs, 33k SFT trajectories, and 21k DPO pairs. Leveraging these resources, we train OffSeeker, a model developed entirely offline. Extensive evaluations across six benchmarks show that OffSeeker not only leads among similar-sized agents but also remains competitive with 30B-parameter systems trained via heavy online RL.

Abstract:
The growing demand for corporate sustainability transparency, particularly under new regulations like the EU Taxonomy, necessitates precise data extraction from large, unstructured corporate reports, a task for which Large Language Models and Retrieval-Augmented Generation (RAG) systems require high-quality, domain-specific question-answering datasets. To address this, we introduce SustainableQA, a novel dataset and a scalable pipeline that generates comprehensive QA pairs from corporate sustainability and annual reports by integrating semantic chunk classification, a hybrid span extraction pipeline, and a specialized table-to-paragraph transformation. To ensure high quality, the generation is followed by a novel automated assessment and refinement pipeline that systematically validates each QA pair for faithfulness and relevance, repairing or discarding low-quality entries. This results in a final, robust dataset of over 195,000 diverse factoid and non-factoid QA pairs, whose effectiveness is demonstrated by initial fine-tuning experiments where a compact 8B parameter model outperforms much larger state-of-the-art models. These results demonstrate the potential of SustainableQA as a resource for developing and benchmarking advanced knowledge assistants capable of navigating complex sustainability compliance data

Abstract:
Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, improving efficiency but introducing new misuse risks. We present ChartAttack, a framework for evaluating how MLLMs use design misleaders to generate charts that induce incorrect interpretations. We also introduce AttackViz, a chart question-answering (QA) dataset labeled with effective misleaders and their induced incorrect answers. ChartAttack reduces MLLM QA accuracy by 17.2 points in-domain and 11.9 points cross-domain. Conditional deception rates show targeted effects: correct answers shift to attacker-intended answers 11.2% of the time in-domain and 11.7-14.9% cross-domain, while originally incorrect answers rarely change. A controlled human study shows that ChartAttack-generated charts also reduce human QA performance. Finally, fine-tuning on AttackViz improves in-domain MLLM robustness to misleading charts. Our findings highlight the need for secure, robust MLLM chart generation. Code and data are publicly available on the project website.

Abstract:
Offensive dark humor presents a high-context safety challenge: its harmful meaning is often absent from surface wording and emerges only after recovering background knowledge and rhetorical transformation. To explore this problem, we introduce HellHumor-CN, a benchmark of 2,168 Chinese jokes annotated for three subtasks: (1) offense detection, (2) sensitive topic prediction, and (3) target identification. We further propose HumorLens, a knowledge-grounded and rhetoric-aware multi-agent framework that retrieves relevant background information and analyzes three common mechanisms in dark humor: mockery, irony, and pun. Experiments show that toxic content detectors and strong LLMs struggle with Chinese dark humor, especially on topic and target interpretation. HumorLens consistently improves performance across multiple models, suggesting that background grounding and mechanism-specific analysis help identify implicit offensive meanings. Further analyses reveal persistent challenges, including visually implied harm, coded sexual innuendo, lexical oversensitivity, and abstract target grounding.

Abstract:
Fine-tuning Large Language Models with untrusted data exposes models to backdoor attacks, where poisoned samples cause targeted misbehavior. Existing sample-filtering defenses rely on clustering, which requires sufficient data and can fail at extreme poison ratios. We propose GradSentry( Grad ient Sentry ), a backdoor sample filtering method based on the spectral entropy of per-sample gradients. Our key finding is that poisoned samples produce gradients with higher spectral entropy compared to clean samples. GradSentry captures output-altering backdoor signatures using per-sample gradient spectra, avoiding pairwise sample comparisons and clustering during feature construction. Importantly, our method is training-agnostic: it works for both parameter-efficient fine-tuning methods like LoRA and full-parameter tuning, as the gradient analysis operates independently of which parameters are being updated during training. GradSentry requires no clustering, operates effectively across all poison ratios (1%–90%), and introduces minimal computational overhead (20–50ms per sample for a 7B model). Evaluation on four QA datasets and four attack types demonstrates the effectiveness of spectral entropy for backdoor detection.

Abstract:
Adapting large vision–language models for embedding tasks (VLM2Vec) is standard practice, but distilling them into compact architectures remains challenging due to capacity mismatches in tokenization, dimensionality, and multimodal representations. To address this, we propose a robust distillation framework based on two complementary mechanisms: Principal Geometry Alignment (PGA) to stabilize training by aligning principal geometric components, and a Mutual Information Maximization objective to enforce global semantic consistency across intra- and inter-modal pairs. By jointly constraining geometric structure and semantic dependency, our method effectively mitigates model capacity discrepancies. Experimental results on the Massive Multimodal Embedding Benchmark (MMEB) across all four meta-tasks—under both In-Distribution and Out-of-Distribution settings—demonstrate that our approach consistently outperforms state-of-the-art distillation methods. Our source code is available at https://github.com/theanh-ktmt/PGA-KD.

Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities in processing long contexts and generating long-form texts. However, existing benchmarks evaluate these capabilities by focusing on either long-input comprehension with short outputs or short-input creative generation with lengthy outputs. This critical gap overlooks real-world Long-In-Long-Out Generation tasks, where models must simultaneously process long-context inputs and produce long-form outputs while adhering to fine-grained constraints. We introduce DocFollow , a comprehensive benchmark featuring 2,436 expert-validated instances across 5 major domains and 20 subdomains, designed to evaluate LLMs’ ability to follow hierarchical constraints in Document Following tasks. Our benchmark employs five types of constraints with increasing granularity. To ensure reliable evaluation, we develop a rule-based assessment protocol that decomposes overall evaluation uncertainty into atomic-level classification uncertainties, employing LLMs solely as information extractors for well-established NER and RE tasks rather than subjective scoring. Our evaluation of 13 state-of-the-art LLMs reveals a consistent pattern: while models maintain reasonable performance on coarse-grained constraints, they exhibit substantial degradation on fine-grained requirements, particularly entity-level constraints. This finding suggests fundamental architectural limitations in maintaining detailed structural and semantic fidelity throughout extended generation. Our findings highlight the critical need for advancing long-context generation capabilities beyond surface-level coherence to achieve precise fine-grained control. The implementation is publicly available. Source code is available at https://github.com/leonbig/DocFollow

Abstract:
Recent advances in large audio language models (LALMs) have enabled multilingual speech understanding. However, benchmarks for evaluating LALMs remain scarce for non-English languages, with Korean being one such underexplored case. In this paper, we introduce KoALa-Bench, a comprehensive benchmark for evaluating Korean speech understanding and speech faithfulness of LALMs. In particular, KoALa-Bench comprises six tasks. Four tasks evaluate fundamental speech understanding capabilities, including automatic speech recognition, speech translation, speech question answering, and speech instruction following, while the remaining two tasks evaluate speech faithfulness, motivated by our observation that several LALMs often fail to fully leverage the speech modality. Furthermore, to reflect Korea-specific knowledge, our benchmark incorporates listening questions from the Korean college scholastic ability test as well as content covering Korean cultural domains. We conduct extensive experiments across six models, including both white-box and black-box ones. Our benchmark, evaluation code, and leaderboard are publicly available at [https://ksbench.github.io/Koala-Bench/](https://ksbench.github.io/Koala-Bench/).

Abstract:
Retrieval-Augmented Generation (RAG) enhanced by Knowledge Graphs has shown promise in complex multi-hop reasoning tasks. However, existing graph-based retrieval methods typically rely on flat, undirected topologies. During the retrieval process, the probability flow often gets trapped in high-degree abstract concept nodes which we define as "probability black holes", leading to semantic drift and noise accumulation. To address this, we propose SemFlowRAG, a framework that reconstructs the flat retrieval space into a corpus-adaptive semantic gradient graph. This data-driven self-organization enables a hierarchical structure to emerge naturally from the data distribution, capturing the intrinsic semantic granularity of the corpus to suppress structural noise. By quantifying the semantic abstractness of entities through the embedding variance of their associated passages, we transform static undirected edges into directed semantic constraints. Furthermore, we design an abstractness-guided directed PageRank algorithm that forces the retrieval trajectory to follow a "high-to-low semantic abstractness" gradient. This mechanism ensures layer-by-layer evidence convergence, smoothly guiding the retrieval process from abstract concepts to specific document evidence. Extensive experiments on complex QA datasets demonstrate that SemFlowRAG effectively mitigates the "probability black holes" issue, outperforming existing baselines in both retrieval and downstream reasoning performance.

Abstract:
Sparse Knowledge Graphs (KGs) are commonly encountered in real-world applications, where knowledge is often incomplete or limited. Sparse KG reasoning, the task of inferring missing knowledge over sparse KGs, is inherently challenging due to the scarcity of knowledge and the difficulty of capturing relational patterns in sparse scenarios. Among all sparse KG reasoning methods, path-based ones have attracted plenty of attention due to their interpretability. Existing path-based methods typically rely on computationally intensive random walks to collect paths, producing paths of variable quality. Additionally, these methods fail to leverage the structured nature of graphs by treating paths independently. To address these shortcomings, we propose a Structural and Probabilistic framework named StruProKGR, tailored for efficient and interpretable reasoning on sparse KGs. StruProKGR utilizes a distance-guided path collection mechanism to significantly reduce computational costs while exploring more relevant paths. It further enhances the reasoning process by incorporating structural information through probabilistic path aggregation, which prioritizes paths that reinforce each other. Extensive experiments on five sparse KG reasoning benchmarks reveal that StruProKGR surpasses existing path-based methods in both effectiveness and efficiency, providing an effective, efficient, and interpretable solution for sparse KG reasoning.

Abstract:
Imagine two users interact with the same LLM. One is told it is the cutting-edge flagship; the other, an older, weaker model. They walk away with markedly different ratings of its usefulness and intelligence, yet they used the same model. In a controlled study, 162 participants each used one of six LLMs from two families across three collaborative tasks, after first viewing a landing page that matched, overstated, or understated their model’s true capability. This pre-interaction framing persisted through use. Oversold users rated the model more favorably than Undersold users before interaction; afterward, both groups had updated toward reality, but the impression gap did not fully close. Framing also changed how users worked with the model (with Oversold users prompting more directively, Undersold users more collaboratively), but the quality of what they produced together depended only on the model’s true capability, not on what users were told. Participants’ change in model impressions after use, measured across two impression measures, was not predicted by task performance ( 𝛽 = -0.01 and 0.11, both n.s.), but by whether the model met users’ expectations ( 𝛽 = 0.47 and 0.50, both p < .001 ) and how confident they felt working with it ( 𝛽 = 0.47 and 0.36, both p < .001 ). After interaction, users are still rating the pitch, not the product: user-elicited LLM evaluations, including the preference data driving public leaderboards, measure expectation management at least as much as the model itself.

Abstract:
Human annotation is the empirical foundation of much NLP research, from dataset construction to model evaluation, but papers often leave unclear who produced the annotations and how the annotation process was controlled. We provide the first large-scale, task-level audit of human annotation reporting across major NLP venues, asking which annotation details are documented, which are missing, and how reporting varies across time, topic, venue, and intended use of human judgment. We introduce a unified taxonomy of annotation-reporting practices and validate an LLM-assisted extraction pipeline against Annotated_gold, a human-adjudicated gold standard of 41 papers and 72 annotation tasks, where the best model reaches human-comparable agreement with adjudicated labels, with Krippendorff’s alpha = 0.606 versus 0.585 for human–human agreement. Using this pipeline, we construct Annotated_llm, a dataset covering ACL-venue papers from 2018–2025, with 2,667 extracted annotation tasks from 1,603 papers, and find that papers frequently report operational details such as recruitment strategies, annotator expertise, and annotation volume, but often omit details needed to assess annotation validity, including training, language proficiency, compensation, socio-demographics, adjudication, and agreement values, especially in model-evaluation studies. Our results show that annotation reporting in NLP has improved over time but remains uneven, and they establish a scalable framework and bare-minimum reporting recommendations for making human annotation more reliable, reproducible, and interpretable.

Abstract:
Argument summarization requires capturing semantic content while preserving the underlying argumentative structure. We present ArgSum-X, a dataset supporting both argumentative text summarization (ATS) and key point analysis (KPA) . To construct the dataset, we propose constructed via a multi-stage annotation framework with hybrid LLM–human annotation and quality tiers, and provide a detailed analysis of annotation quality. We propose a comprehensive evaluation framework for argument summarization covering structural, faithfulness, and semantic aspects, and conduct an empirical analysis of structure-aware modeling, revealing the unique properties and challenges of argument summarization.

Abstract:
Personalized text generation requires models to capture user-specific writing styles from historical data. Existing approaches based on retrieval, parameter-efficient fine-tuning, or activation steering either introduce inference and storage overhead or struggle to separate stylistic signals from semantic content. We propose GLASS, a training-free framework for personalized generation via Global–Local Activation Steering with Sparse priors. GLASS uses sparse autoencoders to extract a global user-style prior from historical responses and constructs local contrastive style vectors over clustered interaction scenarios. During inference, it jointly injects global and local vectors into different model layers, enabling context-aware personalization without retrieval or parameter updates. Experiments on LaMP and LongLaMP show that GLASS outperforms retrieval-, fine-tuning-, and steering-based baselines across ROUGE metrics and LLM-as-judge evaluations. Further analyses show that SAE-based representations are more robust to topic and length shifts, suggesting better disentanglement of stylistic information from semantic residue.

Abstract:
Persona-based dialogue systems are usually conditioned on speaker biography, but dialogues involve at least two participants, and who has access to whose biography can vary across training, inference, and evaluation. Prior work often neglected these aspects, obscuring mechanisms that only appear when biography visibility is toggled separately across training, inference, and evaluation, a three-stage factorisation that prior work has largely treated as a single factor. We study this factorisation on a dataset of dialogues paired with speaker’s biographies, varying whether the target and interlocutor speakers see each other’s biographies during training and inference, and using an LLM as a judge to perform author identification. We find that (i) training-time visibility, more than inference-time visibility, determines whether models express persona traits through dialogue or fall back on copying biographical text (a known problem/phenomenon in persona-based generation); (ii) models trained with interlocutor-biography visibility copy less target-biographical text than models trained without it, while changing visibility only at inference time has a less consistent effect; and (iii) under asymmetric disclosure, where only the interlocutor sees the target biography, target content leaks into interlocutor turns more often, and dialogues containing such traces are easier for the judge to identify, especially when interlocutor turns are visible. These results suggest that biography leakage into generated turns is an artefact of how interlocutor visibility is configured across training and inference, and separating the three stages is necessary.

Abstract:
In Text-to-SQL tasks, existing LLM-based methods often include extensive database schemas in prompts, leading to long context lengths and increased prefilling latency. While user queries typically focus on recurrent table sets—offering an opportunity for KV cache sharing across queries—current inference engines, such as SGLang and vLLM, generate redundant prefix cache copies when processing user queries with varying table orders. To address this inefficiency, we propose precomputing table representations as KV caches offline and querying the required ones online. A key aspect of our approach is the computation of table caches while preserving primary foreign key relationships between tables. Additionally, we construct a Table Trie structure to facilitate efficient KV cache lookups during inference. To enhance cache performance, we introduce a cache management module with a query-reranking strategy to improve cache hit rates and a computation-loading pipeline for parallelizing model inference and cache loading. Experimental results show that our proposed TableCache achieves up to a 3.62× speedup in Time to First Token (TTFT) with negligible performance degradation.

Abstract:
Reasoning-intensive retrieval remains difficult for small models. Compact public ColBERTs are usually trained on general-purpose corpora and underperform reasoning-tuned 150M+ baselines on BRIGHT (CITATION) by several nDCG@10 points; no public reasoning-tuned ColBERT exists at edge scale. We introduce SmallReason-ColBERT , a 32M late-interaction retriever that closes much of this gap with three ingredients: a varied-length contrastive warmup on ReasonIR-VL, a hard-negative distillation polish on merged ReasonIR-HQ and BGE-Reasoner data, and a single-layer per-query-token importance head trained on top of the frozen base. The head is trained with an un-normalised weighted MaxSim score and evaluated with its length-normalised form. In a controlled re-training, replacing this training objective with the symmetric normalised score causes the loss to stall and costs 3.59 nDCG@10. The full recipe reaches 21.41 mean nDCG@10 on BRIGHT, within 1.21 of the 150M Reason-ModernColBERT (22.62) and above all ≤ 33 M ColBERTs we evaluate. Through ablations on capacity, initialisation, and score variants, we further show that the learned head outperforms fixed IDF weighting and that simply thresholding the learned gates is harmful.

Abstract:
Large language models (LLMs) have demonstrated strong reasoning abilities, but hallucinations and unverifiable intermediate reasoning steps remain key challenges. In knowledge graph question answering (KGQA), entity-relation paths from knowledge graphs (KGs) provide structured and verifiable evidence for grounded reasoning. However, existing KG-enhanced LLM methods commonly linearize all retrieved reasoning paths into textual prompts, introducing noise and underexploiting their ordered structure. To address these limitations, we propose PathSoft, a hybrid soft prompting framework that combines textual path evidence with soft prompts. PathSoft first employs PathMLP to select reliable reasoning paths, and then uses PathSoftAdapter to convert the selected paths into continuous soft prompts while preserving their ordered path structure. Finally, the resulting soft prompt tokens are combined with the textual prompt at the embedding layer of the frozen LLM, enabling it to exploit both readable path evidence and path structure information. Experiments on WebQSP and CWQ show that PathSoft outperforms existing KGQA baselines. The code is available at https://github.com/tengyan1786-blip/PathSoft.

Abstract:
The rapid advancement of large language models (LLMs) has made AI-generated text detection increasingly critical. Existing zero-shot detectors assume that more token-level evidence leads to more reliable detection. However, our empirical study challenges this consensus: fewer tokens sometimes work better, retaining only 40% can yield optimal performance, yet this benefit is not universal. Using the Entropy Gap Score (EGS), we introduce top- k cumulative probability filtering as a diagnostic probe. Across three representative settings, filtering exhibits strikingly different behaviors. We analyze EGS via typical set theory and quantify its dynamics through entropy calibration and distribution analysis. We find that filtering helps for weak source LMs, where low-entropy tokens are harmful, but fails for strong source LMs, where they are not notably harmful. Our work provides the first systematic analysis showing that some tokens are not merely uninformative but systematically harmful due to entropy miscalibration, revealing a two-sided trade-off in token-level detection.

Abstract:
Backtesting large language models (LLMs) on historical financial data is unreliable when their pre-training data include the evaluated events. An LLM trained in 2024 may already encode how stocks moved during 2018-2020. We name this failure parametric look-ahead bias and propose FinCAD, an inference-time adaptation of Context-Aware Decoding that attenuates contributions from memorised historical outcomes without retraining. FinCAD pairs an adversarial bias-discovery pipeline that learns a model-specific memory-activating prior prompt with an entity- and date-adaptive rule that scales the CAD strength using a per-(entity, date) confidence signal. Across five 7-14B LLMs and five mega-cap equities, the largest model-level mean in-sample return correction is -67.1%. For the three larger models, 2025 out-of-sample returns remain within 8K and mean Sharpe within ±0.10 of baseline; mean general-benchmark accuracy remains positive or within -1.7 points for four of five models. On an eleven-model leaderboard, FinCAD raises the subset-averaged in-sample/out-of-sample Spearman correlation from +0.779 to +0.846, yielding rankings that are more closely aligned with post-cutoff performance.

Abstract:
Accurate translation from Natural Language to First-Order Logic (NL-to-FOL) underpins neurosymbolic AI systems and Natural Language Inference (NLI), making the quality of NL-to-FOL benchmarks essential—yet these datasets have never been rigorously audited. Our first contribution is to present a systematic human inspection of the validation split of and a subset of test instances, finding that approximately 42.5% and 42% of entries, respectively, contain incorrect FOL formalizations (i.e., ground truth labels), with additional rates of ambiguous NL sentences (17.8% and 51%) and incorrect NLI labels in (8.4%). Our second contribution is to develop and release corrected ground truths for such datasets, showing that annotation errors distort model evaluation on a reference benchmark task: testing three state-of-the-art LLMs (Gemma 4 31B-it, Qwen3-30B-A3B, and GPT-4o-mini) with the corrected ground truths yields accuracy gains from +11 to +23 percentage points. Motivated by these findings, we propose an LLM based framework to support humans in manual reviewing NL-to-FOL datasets. By directing reviewers toward the most error-prone instances, we empirically show that it is possible to achieve 90% dataset accuracy after reviewing fewer than 20% of instances, compared to over 76% required by unguided review. We release all human-verified annotations and the code for our framework.

Abstract:
Theory of Mind (ToM), the ability to infer others’ beliefs, intentions, and states of knowledge, is central to social interaction, yet remains challenging for current Multimodal Large Language Models (MLLMs), especially in multi-party meetings where cues are distributed across speech and behavior. Existing multimodal ToM benchmarks mainly focus on video-grounded question answering over overt, externally verifiable signals, and provide limited coverage of nuanced social states and group dynamics. We introduce MeetingToM, a benchmark for complex social behavior reasoning in naturalistic multi-party meetings. MeetingToM targets meeting-specific phenomena such as pseudo-consensus, where apparent agreement co-occurs with cues of dissent under social pressure. The benchmark is hierarchically organized to evaluate ToM at increasing levels of social granularity, including (i) subject-level mental state prediction, (ii) dyadic-level addressee understanding, and (iii) group-level consensus reasoning. We provide a unified evaluation protocol and conduct systematic analyses of representative MLLMs, revealing persistent limitations in integrating non-verbal cues, predicting annotated attitudes, and distinguishing among consensus states. Our results highlight key challenges and establish MeetingToM as a testbed for advancing meeting-grounded ToM in multimodal models. Project page: https://oliviaziyi.github.io/MeetingToM-Project-Page/

Abstract:
Multimodal Large Language Models (MLLMs) excel at recognizing individual visual elements and reasoning over simple linear diagrams. However, when faced with complex topological structures involving branching paths, converging flows, and cyclic dependencies, their reasoning capabilities degrade sharply, even on tasks as basic as counting endpoints. Existing benchmarks fail to probe this gap, focusing on semantic comprehension rather than structural reasoning. We introduce ReactBench, a benchmark that reveals fundamental limitations in structural reasoning through chemical reaction diagrams. These real-world scientific diagrams offer an ideal testbed because they naturally span diverse structures from linear chains to cyclic graphs, while requiring both precise local recognition and coherent global reasoning. Our benchmark comprises 1,618 expert-annotated QA pairs across four hierarchical task dimensions. Extensive evaluation across 24 MLLMs reveals a significant performance gap exceeding 30% between anchor-based tasks and holistic structural reasoning tasks.

Abstract:
Building language technologies and conducting NLP research for low-resource languages—particularly when led by native speakers or involving participatory research practices—are often framed as means of addressing inequality, serving local communities, and, at times, contributing to decolonisation. In this paper, we examine recently published NLP and ML papers, focusing on the narratives used to characterise multilinguality, low-resource languages, and underrepresented cultures. We propose a framework for analysing research framings and identify recurring rhetorical patterns that may hinder accountability and constrain equitable knowledge production for—and by—underserved communities. We further assess the evidential basis of assertions regarding community benefit and find that such statements are often weakly supported or left unsubstantiated. Although community ownership and participation are frequently presented as key objectives, our analysis, supported by statistics from the ACL Anthology, suggests that research outputs more often prioritise resource creation and benchmarking—important but distinct goals—over evidence of broader structural change. We conclude by offering practical recommendations to help authors, reviewers, and readers critically assess these assertions and avoid potentially misleading framings.

Abstract:
Large Language Models (LLMs) show great potential as clinical agents, yet existing benchmarks reduce clinical workflows to static predictions or unconstrained Markov Decision Processes (MDPs) with coarse action sets. To address this, we introduce GPAgentBench-2K, the first Constrained MDP (CMDP) LLM-agent benchmark for primary-care clinical decision-making, constructed from expert-validated records of real-world GP encounters. Our environment models a full spectrum of six foundational clinical actions, imposes a topological workflow prior over the action space, and operationalizes safety-informed abstention as a first-class outcome. Evaluating 16 state-of-the-art LLMs reveals a significant performance degradation as the action space scales. Crucially, we uncover a clinical quality-safety gap: even frontier models with the highest diagnosis accuracy violate safety constraints in over half of high-risk cases. Finally, we establish a reference point using Constrained Group Relative Policy Optimization (C-GRPO), and show that while explicitly modeling constraints improves performance over unconstrained RL methods, it remains far from clinically acceptable safety.

Abstract:
3D Visual Grounding (3D-VG) localizes objects in 3D scenes via natural language descriptions. Current zero-shot Vision-Language Models (VLMs) typically rely on preprocessed 3D point clouds and pretrained 3D detectors, reducing grounding to static proposal matching. To address this, we propose "Think, Act, Build (TAB)", a dynamic agentic framework that reformulates 3D-VG as a 2D-to-3D reconstruction paradigm operating directly on raw RGB-D streams. TAB decouples the task by leveraging 2D VLMs for complex spatial semantics and multi-view geometry for 3D structure instantiation. Guided by a specialized skill, our VLM agent dynamically invokes visual tools to track targets across 2D frames. To overcome multi-view coverage deficits inherent in strict VLM semantic tracking, we introduce Semantic-Anchored Geometric Expansion. This mechanism anchors the target in a reference video clip and uses multi-view geometry to propagate its spatial location across unobserved frames, mapping 2D visual cues to 3D coordinates. Additionally, we manually refine existing benchmarks to eliminate reference ambiguity and category errors. Extensive experiments on ScanRefer and Nr3D demonstrate that TAB, relying entirely on open-source models, significantly outperforms previous zero-shot methods and even supervised baselines.

Abstract:
LLM judges are widely used to evaluate agentic tool-calling systems, yet their reliability on structured, dependency-driven workflows remains largely unexamined. We present AgentJudgeBench, the first benchmark to systematically study LLM-as-a-judge reliability for agentic tool-calling over workflow DAGs, as distinct from the broader LLM-as-a-judge task of open-ended text or preference evaluation. The benchmark comprises 3,808 instances spanning six DAG topologies and three difficulty tiers, evaluated with five generators (3B-70B open-weight models and GPT-5.4) and six judges (20B to frontier scale) under paired with- and without-ground-truth conditions. Judge alignment degrades monotonically with task difficulty, 1.5x faster without ground truth, and on hard queries without ground truth all six judges converge to a narrow 77-82% band regardless of scale, revealing a structural ceiling driven primarily by task difficulty, though its height is partly prompt-dependent for weaker generators, that model capacity alone cannot overcome. Ground-truth exposure is not uniformly beneficial: it reduces alignment for GPT-5.4 (1.5 pp) and Gemini-2.5-Pro (3.9 pp), consistent with over-anchoring. Among mitigation strategies, chain-of-thought reasoning and judge temperature both have negligible effect, while structured evaluation rubrics improve alignment by up to 6.5 pp but do not generalize uniformly across judge-generator pairs. With ground truth, QwQ-32B best matches the programmatic reference, while a human validation study identifies GPT-OSS-120B as the most human-aligned judge; without it, frontier judges lead only marginally within the shared ceiling. These results expose fundamental limitations of current LLM judges and yield practical guidelines for reliable evaluation in agentic systems.

Abstract:
Mixture-of-Experts (MoE) routing collapses in parameter-efficient fine-tuning: on LLaMA-3 8B with MixLoRA, a single expert absorbs 82.5% of tokens. Prior MoE-PEFT remedies intervene at the output: routing reformulations, output-similarity penalties, or shared-expert capacity. All leave each routed expert’s input view identical. We propose SIMPA (Sparse-Input Magnitude Partitioning Adaptation), which restricts what experts see rather than what they produce: a magnitude-thresholded input drives a sparse routed-LoRA branch, a parallel dense LoRA branch (adapting DeepSeekMoE’s (Dai et al., 2024) shared-expert topology to PEFT) provides an information safeguard, and a column-wise gate fuses the two. At a matched 2.9% trainable-parameter budget, SIMPA leads compared baselines on reasoning (86.2% on seven LLaMA-3 8B tasks, +3.9 over MixDoRA), code and arithmetic (38.4 HumanEval Pass@1 and 59.4 GSM8K, matching full fine-tuning under identical data and compute), and NLU (88.7 GLUE / 77.7 SuperGLUE on T5-base); against direct MoE-PEFT counterparts HydraLoRA and MoLoRA at matched setup, SIMPA leads HumanEval by +14.2 and +13.6 and GSM8K by +2.1 and +2.8. Direct measurement returns an expert-input Jaccard of 0.184 versus MixLoRA’s 1.000, a 5.4× reduction. The same block-aligned mask drives a Triton kernel measuring 1.46× end-to-end inference speedup on A100. Closed-form partitioning and retention bounds characterize the mechanism, and a tail-index-driven closed form predicts the optimal sparsity p from the pretrained backbone. The mechanism, its predicted optimum, and its bound all transfer to a held-out Qwen2.5-7B backbone. Code: https://github.com/triquang26/simpa.

Abstract:
Vision-Language Models (VLMs) show immense promise in medicine, yet surgical scene understanding remains formidable due to complex visual obstructions and rigorous multi-step clinical logic. Standard Supervised Fine-Tuning (SFT) imparts domain knowledge, but we identify a critical flaw—the "Reasoning Collapse" phenomenon: during answer-only SFT, models progressively lose explicit step-by-step reasoning capabilities, regressing into "black boxes" that generate uninterpretable shortcut answers—a severe risk for clinical traceability. Constructing high-quality Chain-of-Thought (CoT) datasets to mitigate this is prohibitively expensive in the surgical domain. We propose SurgMind-R1, a surgical VLM with a three-stage training pipeline. We construct Cholec-Think, a reasoning-enhanced dataset combining expert and VLM-assisted CoT annotations; apply domain-adaptive SFT and reasoning-enhanced SFT on Cholec-Think to internalize surgical logic; and introduce Reinforcement Learning via Group Relative Policy Optimization (GRPO) with rule-based rewards to enforce structured thinking paths and eliminate reasoning collapse. On SurgLite-Bench, our comprehensive benchmark, SurgMind-R1 significantly outperforms state-of-the-art VLMs on complex inference tasks. It internalizes structured thinking, maintaining high accuracy in a direct no-think mode that resolves the high-latency bottleneck of reasoning models for real-time clinical applications, and exhibits strong zero-shot out-of-distribution (OOD) generalization across unseen surgical scenarios and novel tasks, paving the way for transparent, trustworthy computer-assisted interventions. The code, benchmark, and models will be available at https://github.com/YifanLuo0103/SurgMind-R1.

Abstract:
Drug repurposing aims to identify new therapeutic uses for existing compounds and, compared with de novo drug discovery, offers a faster and more cost-effective path to clinical translation. However, the space of candidate drug–disease pairs is enormous and their underlying relationships often depend on complex multi-hop biological mechanisms, making it difficult to reliably predict which pairs represent true therapeutic relationships. Existing approaches tackle this from two directions: knowledge graph-based methods organize curated biomedical evidence into structured relational networks for grounded multi-hop reasoning, while LLM-based methods leverage pretrained knowledge to generate flexible mechanistic rationales. Yet neither is sufficient alone — KGs are confined to observed graph structure while LLMs lack factual grounding and risk hallucination. To address this gap, we propose DrugReason, a multi-view reasoning framework that integrates grounded KG reasoning with LLM-generated mechanistic inference for drug repurposing. DrugReason adaptively routes diverse reasoning paths to specialized experts conditioned on the query context, while a cross-expert distillation objective enables knowledge sharing without sacrificing expert specialization. Experiments on PharmaDB, DDInter, and DrugBank show that DrugReason improves average performance over strong single-view reasoning baselines and achieves competitive or superior results compared with graph-based alternatives, while providing interpretable routing-based predictions.

Abstract:
Long-horizon search agents accumulate large amounts of retrieved content across many tool calls, making context-budget efficiency increasingly important. A minimal intervention is to mask stale observations from the context as the trajectory progresses, but it remains unclear when this form of context management helps and why. We study observation masking through a systematic sweep over various agent backbones (4B to 284B parameters) and three retrievers on offline and live-web agentic search benchmarks. We find that the accuracy gain from masking follows an asymmetric inverted-U shape when plotted against the model’s accuracy without context management: a plateau under weak retrievers, a peak in the mismatch regime (strong retriever × mid-capacity model), and a sharp collapse when the model is saturated. This pattern reflects the interaction between retriever recall and the model’s implicit filtering capacity, rather than either factor in isolation. Mechanistically, masking implements a token-for-turn trade-off: it removes observations the model has largely stopped attending to and pages the agent rarely re-opens. The added turns help when they convert failures into successes, but they fail when masking removes evidence the model would otherwise have used. We therefore reframe context management as a regime-dependent intervention (rather than a default choice), providing a new, holistic viewpoint that unifies the understanding of agentic deep search.

Abstract:
Multimodal Large Language Models (MLLMs) are a major focus of recent AI research. However, most prior work focuses on static image understanding, while their ability to process audio–video data remains underexplored. This gap highlights the need for a high-quality benchmark to systematically evaluate MLLM performance in a real-world setting. We introduce SONIC-O1, a comprehensive, fully human-verified benchmark of ≈ 60 hours (231 clips) spanning 13 real-world conversational domains with 4,958 annotations and perceived demographic metadata. SONIC-O1 evaluates three capabilities: open-ended summarization, multiple-choice question (MCQ) answering, and temporal localization with supporting rationales (reasoning). Across closed- and open-source models, we find that the MCQ accuracy shows the smallest gap between model families, under their recommended settings, the best closed-source model outperforms the best open-source model by 22.6% on temporal localization. We further observe accuracy gaps of up to 21.4% on temporal localization across demographic groups, indicating diagnostic disparities in model behaviour. SONIC-O1 provides an open evaluation suite for temporally grounded and demographically informed multimodal understanding. We release SONIC-O1 for reproducibility and research: Project Page: https://vectorinstitute.github.io/sonic-o1, Dataset: https://huggingface.co/datasets/vector-institute/sonic-o1 , Code Repository: https://github.com/VectorInstitute/sonic-o1.

Abstract:
While large language models (LLMs) are increasingly used to generate writing feedback, there remains no systematic comparison of LLM and expert feedback on the dimensions that writing research identifies as central to revision: goal-orientation, anchoring to specific sentences, and prioritization. We introduce FOXGLOVE, a dataset of 696 feedback comments written by trained writing instructors on 69 twelfth-grade argumentative essays, paired with 1,644 comments generated from four frontier LLMs under a shared protocol, totaling 2,340 comments. We provide expert quality ratings on a subset of both instructor and LLM comments. We find that instructors and LLMs distribute feedback similarly across goals and essay positions, yet instructors and models diverge on the specific sentences on which to provide feedback. Additionally, we find that models tend to write more complex feedback and use fewer questions than instructors. LLM feedback also receives higher ratings on most dimensions of quality, as rated by instructors, but much of this advantage appears to be attributable to lengthier comments. FOXGLOVE enables systematic comparison of where human and LLM feedback align, diverge, and differ.

Abstract:
Scope ambiguity arises when a single surface string admits multiple logical interpretations depending on how operators such as negation and indefinites interact. Prior work (Kamath et al. 2024) primarily evaluates scope sensitivity through output-level behavior, leaving open whether alternative interpretations are internally represented or merely reflected in shifted token probabilities. We introduce an evidence-first, ambiguity-following diagnostic that asks a more specific question: when the surface string is held constant and only contextual evidence biases interpretation, do transformer language models form distinct internal representations corresponding to alternative logical readings? To investigate this, we present SCOPEx, extending the dataset of Kamath et al. (2024) by systematically manipulating preceding context while keeping the ambiguous sentence identical. We further introduce passivized variants as syntactically less ambiguous controls. Using SCOPEx, we show that transformer LMs: (i) exhibit systematic scope-sensitive behavior conditioned on context, and (ii) form separable hidden-state representations for alternative scope readings of the same surface string. This divergence emerges in later layers and causally influences downstream predictions.

Abstract:
Humor generation poses a significant challenge for Large Language Models (LLMs), because their standard training objective (next-token prediction) inherently conflicts with the surprise and incongruity required for comedy. To bridge this gap, we introduce the Cognitive Synergy Framework, a methodology for generating high-quality humor data inspired by psychological theories of humor. Utilizing a Mixture-of-Thought (MoT) approach, we deploy six cognitive personas (e.g., The Absurdist, The Cynic) to synthesize diverse comedic perspectives for a given prompt. This framework produces a theory-grounded dataset, which we use to fine-tune a 7B parameter student model. We further evaluate two alignment strategies, Direct Preference Optimization (DPO) and an offline group-relative variant O-GRPO, finding that neither improves over SFT. However, our 7B HumorGen model variants significantly outperform larger instruction-tuned baselines and achieve top-tier open-weight performance while remaining competitive with frontier proprietary systems. These results suggest that cognitively driven data curation is more critical than alignment algorithms or model scale for humor generation.

Abstract:
Token selection is pivotal for effective LLM post-training. However, existing methods mostly rely on local token-level heuristics, while response-token selection in post-training lacks an optimization-guided valuation proxy. We introduce AlphaToken , a response token valuation framework that decouples valuation into adaptation (promoting target-task learning) and stability (preserving pre-trained capabilities), and makes each objective path-aware by combining the direct-path signal from local token gradients with the downstream causal-path signal in autoregressive generation. Since retention data are typically unavailable, AlphaToken approximates stability via a Fisher-drift proxy anchored at the pre-trained reference model. For efficient computation, we extend Ghost Dot-Product to token-level valuation. AlphaToken masks low-value response tokens during fine-tuning and preference optimization, concentrating training signals on more valuable positions. Experiments show that AlphaToken improves post-training performance and mitigates catastrophic forgetting. Code at https://github.com/luciabcd/Alphatoken.

Abstract:
What do political elites mean when they invoke "the people", and how has this meaning shifted across centuries of democratic history? Political theorists have long held that “the people” is a discursively constructed and contested category. Yet empirical work remains largely qualitative or small in scale, and computational approaches have yet to address how contested concepts themselves are framed over time. We address this gap with a generalizable framework that models contested concepts as configurations of theoretically grounded frames, and apply it to American presidential rhetoric based on three theoretically grounded frames: Nation, Civic, and Global. Leveraging a fine-tuned political masked language model, we mask each occurrence of "people" and score each frame by the pseudo-log-likelihood of its probe words. We apply the framework to 3,402 U.S. presidential speeches from 1789 to 2025. The resulting measurements agree with expert human coding at a level comparable to inter-annotator agreement, while tracking major junctures such as the Civil War, the World Wars, the Civil Rights era, and a recent nationalist turn. They also covary with external indicators as expected: stronger democracy is associated with less national and more civic framing. Beyond "the people", the framework can generalize to other contested political concepts.

Abstract:
Despite their advanced reasoning capabilities, state-of-the-art Multimodal Large Language Models (MLLMs) demonstrably lack a core component of human intelligence: the ability to ‘read the room’ and assess deception in complex social interactions. To rigorously quantify this failure, we introduce a new task, Multimodal Interactive Deception Assessment (MIDA), and present a novel multimodal dataset providing synchronized video and text with verifiable ground-truth labels for every statement. We establish a comprehensive benchmark evaluating 12 state-of-the-art open- and closed-source MLLMs, revealing a significant performance gap: even powerful models like GPT-4o struggle to distinguish truth from falsehood reliably. Our analysis of failure modes indicates that these models fail to effectively ground language in multimodal social cues and lack the ability to model what others know, believe, or intend, highlighting the urgent need for novel approaches to building more perceptive and trustworthy AI systems. To take a step forward, we design a Social Chain-of-Thought (SoCoT) reasoning pipeline and a Dynamic Social Epistemic Memory (DSEM) module. Our framework yields performance improvement on this challenging task, demonstrating a promising new path toward building MLLMs capable of genuine human-like social reasoning.

Abstract:
Omni-modal LLMs aim to answer queries over heterogeneous inputs through a unified interface, but accepting these inputs does not guarantee using them. % The model’s confidence should change when an informative modality is removed. Existing autoregressive and diffusion multimodal LLMs typically optimize answer likelihood only under the full context, leaving multimodal shortcuts largely unconstrained. We study this problem through the lens of intervention sensitive omni-modal prediction, where confidence in the correct answer should decrease when protocol designated evidence is disrupted. We propose OmniDenoise, a diffusion MLLM framework built around the Evidence Denoising Gap (EDG), a diffusion native objective that compares gold token denoising losses under full and intervened contexts. EDG uses the token-by-timestep denoising trajectory as both a training signal and a diagnostic, without requiring token level evidence annotations. OmniDenoise further applies consistency distillation for efficient few step bidirectional denoising. Across tri-modal, clinical, and out-of-distribution omni-modal benchmarks, OmniDenoise substantially improves over diffusion baselines and remains competitive with strong autoregressive omni models. On OmniBench, the distilled model achieves 45.27% accuracy at 0.19 seconds per sample. On EHRXQA, EDG improves H-Acc from 36.9% to 43.9%, while the distilled student retains 42.6% with a 3.9× speedup.

Abstract:
Existing agent benchmarks mainly evaluate final task success or tool-call correctness, providing limited insight into whether agents can reliably diagnose and recover from intermediate execution failures. This limitation becomes particularly critical in multi-turn parallel tool-use scenarios, where errors may propagate across dependent branches and trigger cascading failures. We introduce ParaRecover, a process-level benchmark for evaluating error localization and recovery in multi-turn parallel tool-use agents. Built upon a fine-grained taxonomy of 14 error types covering planning dependencies, tool selection, and argument matching, the benchmark comprises 10,626 instances spanning two difficulty levels. To enable fine-grained, process-oriented evaluation, we further propose the SDE rubric, which measures structural integrity, diagnostic reasoning, and evolutionary strategy during agent execution. Experiments across more than ten mainstream LLMs reveal that even state-of-the-art models still struggle with multi-turn error propagation, implicit tool-use failures, and precise replanning. Moreover, we demonstrate that the SDE rubric provides effective supervision signals for improving agents’ reflective recovery capabilities. Our data and code are available at https://github.com/gbw206/ParaRecover.

Abstract:
We present MobileGym, a browser-hosted, lightweight, fully controllable simulation platform for everyday mobile use, targeting interaction fidelity without replicating proprietary backends. It enables two capabilities previously out of reach for everyday apps: verifiable outcome signals through deterministic state-based judging over structured JSON state, and scalable online RL through low-cost parallel rollouts. The full environment state is captured, configured, forked, and compared as structured JSON, and a single server can host hundreds of parallel instances (400 MB each, 3 s cold start). A layered state model and a declarative task-definition framework keep state programmability and task creation practical at scale, and a single programmatic judging mechanism delivers both deterministic evaluation verdicts and dense RL rewards. The accompanying MobileGym-Bench provides 416 parameterized task templates (256 test + 160 train) over 28 apps, with deterministic judges and a structured AnswerSheet protocol that avoids free-text matching failures. In a Sim-to-Real case study, GRPO on Qwen3-VL-4B-Instruct gains +12.8 pt on the 256-task test set, and on a 59-task real-device signal subset, real-device execution retains 95.1% of the simulation-side training gain.

Abstract:
AI coding agents increasingly act directly within software environments, yet existing analyses of their failures rely on benchmark trajectories that miss how developers actually experience misalignment. We present an observational study of 20,574 coding-agent sessions from 1,639 repositories across IDE and CLI workflows. We operationalize misalignment as a breakdown made visible through developer pushback, and annotate each episode along four axes: form, cause, cost, and resolution. We identify seven recurring forms, spanning how agents read projects, interpret developer intent, follow rules, bound their actions, implement and execute code, and report progress. 90.50% of episodes impose effort and trust costs rather than irreversible system damage, yet 91.49% of visible resolutions still require explicit user correction. Misalignment patterns also differ across IDE and CLI settings, persist across adjacent sessions, and shift over time: while overall rates decline, constraint violations and inaccurate self-reporting grow in share. Our findings inform the design of training, evaluation, and interfaces for keeping coding agents aligned with real developer workflows.

Abstract:
Diffusion large language models (dLLMs) enable parallel generation and are promising for unit test generation (UTG), where efficient and large-scale automated testing is essential in software development. Despite this advantage, their application to UTG is still constrained by a clear trade-off between efficiency and test quality, since increasing the number of tokens generated in each step often causes a sharp decline in the quality of test cases. To overcome this limitation, we present DiffuTester, an acceleration framework specifically tailored for dLLMs in UTG. The motivation of DiffuTester is that unit tests targeting the same focal method often share structural patterns. DiffuTester employs a novel structural pattern based decoding approach, which dynamically identifies structural patterns across unit tests through their abstract syntax trees and additionally decodes the corresponding tokens, thereby achieving acceleration without compromising the quality of the output. To enable comprehensive evaluation, we extend the original TestEval benchmark to three programming languages. Extensive experiments on three benchmarks with two representative models show that DiffuTester delivers significant acceleration while preserving test coverage. Moreover, DiffuTester generalizes well across different dLLMs and programming languages, providing a practical and scalable solution for efficient UTG in software development. Code and data are publicly available at https://github.com/THU-Agent/DiffuTester .

Abstract:
Continual pre-training is widely used to adapt LLMs to target languages and domains, yet the mixture ratio of training data remains a sensitive hyperparameter that is expensive to tune: it must be fixed before training begins, and a suboptimal choice can waste weeks of compute. In this work, we propose OptiMer, which decouples ratio selection from training: we train one CPT model per dataset, extract each model’s distribution vector, which represents the parameter shift induced by that dataset, and search for optimal composition weights post-hoc via Bayesian optimization. Experiments on Gemma 3 27B across languages (Japanese, Chinese) and domains (Math, Code) show that OptiMer consistently outperforms data mixture and model averaging baselines with 15-35x lower search cost. Key findings reveal that 1) the optimized weights can be interpreted as data mixture ratios, and retraining with these ratios improves data mixture CPT, and 2) the same vector pool can be re-optimized for a given objective without any retraining, producing target-tailored models on demand. Our work establishes that data mixture ratio selection, traditionally a pre-training decision, can be reformulated as a post-hoc optimization over distribution vectors, offering a more flexible paradigm for continual pre-training.

Abstract:
Personalized dialogue agents require continuous long-term memory to maintain coherent interactions across multiple sessions. However, deploying these capabilities on consumer-grade hardware (e.g., 8 GB VRAM edge devices) introduces severe memory and compute bottlenecks. Existing systems typically rely on isotropic cosine similarity for retrieval and heuristic rules for context compression. These approaches lack a unified theoretical foundation, frequently suffering from the hubness problem in high-dimensional retrieval and syntactic fragmentation during compression. To overcome these limitations, we propose CoreMem, a resource-efficient edge-cloud memory architecture fundamentally unified by information geometry. First, Riemannian retrieval replaces cosine matching with a locally adaptive Fisher-Rao metric, effectively penalizing hub memories via Mahalanobis distance with O(Ndr) Woodbury acceleration for real-time search. Second, Fisher-guided discrete token distillation (FDTD) introduces a hierarchical sentence-to-token compression mechanism. It derives sensitivity scores from Fisher information traces, providing a principled compression-KL tradeoff augmented with explicit structural syntax protection. Evaluated on the LOCOMO and LongMemEval-S benchmarks, CoreMem achieves strong accuracy improvements, yielding substantial gains in Open-domain (+4.51 pp) and Temporal (+4.17 pp) reasoning. Extensive profiling confirms that the lightweight MiniLM-L6 configuration of CoreMem operates within a strict 8 GB VRAM budget, successfully bridging the gap between resource-constrained edge devices and the demand for theoretically grounded, lifelong memory agents.

Abstract:
Traditional KV cache eviction strategies, which discard less critical KV-pairs based on attention scores, often degrade generation quality, causing context loss or hallucinations. Recent efforts shift toward KV merging, merging eviction tokens with retention tokens based on similarity. However, in multimodal scenarios, distributional differences across modality tokens and attentional biases in cross-modal interactions limit its effectiveness. This work introduces FlowMM, an adaptive framework for cross-modal information flow-guided multimodal KV cache merging. FlowMM leverages cross-modal information flow to dynamically apply layer-specific merging strategies, capturing modality-specific patterns while preserving contextual integrity. Furthermore, we introduce a sensitivity-adaptive token matching mechanism that jointly evaluates token similarity and task-critical sensitivity, merging low-risk tokens while safeguarding high-sensitivity ones. Extensive experiments across diverse leading MLLMs show that FlowMM reduces KV cache memory by 80% to 95% and decoding latency by 1.3-1.8×, while maintaining competitive task performance.

Abstract:
Hallucination detection in clinical large vision-language models requires cross-verifying heterogeneous evidence, including patient text and images for faithfulness, and medical knowledge for factuality. However, existing pipelines either verify these dimensions in isolation or conflate them into a uniform process, potentially causing evidence misalignment that limits detection precision. Consequently, we posit that the essence of black-box hallucination detection is an evidence routing problem and propose MedF 2 Route, an Evidence-Routed Graph Verification Framework for both Faithfulness and Factuality hallucination detection. Specifically, it first transforms free-text clinical queries and model responses into structured graphs, partitioning the response graph into independent subgraphs to separate patient-specific grounding from knowledge-based diagnostic inference. An evidence-aware routing and verification mechanism then dynamically dispatches response triplets in these subgraphs to their dedicated heterogeneous evidence sources for targeted verification. Experiments on our MedF 2 Bench diagnostic benchmark show that our framework outperforms state-of-the-art baselines, notably improving detection precision. Overall, MedF 2 Route enables professional and precise detection of clinical hallucinations, underpinning the trustworthiness of clinical LVLMs. The code and data are available at https://github.com/Agentic-CliniAI/MedF2Route .

Abstract:
Large Language Models (LLMs) are increasingly being adopted in enterprise environments, where employees may intentionally or accidentally submit prompts containing information that could cause legal or economic harm if leaked. Organizations may define categories of information (policies) that are prohibited from being submitted to commercial LLMs, but these definitions can vary between companies and evolve over time. This variability makes parametric classifiers less practical, as they require retraining whenever policies change. Here, we present Q-CAT, a non-parametric policy enforcement system that classifies user queries based on given categories of information that can change dynamically. The system operates by matching embedded user queries to synthetically generated queries in the database. This design enables low-cost policy updates through simple database modification, without retraining, while maintaining low latency. A comparison with strong prompt-based LLM baselines shows that Q-CAT delivers competitive performance in static environments, while offering practical advantages in dynamic policy settings, where policies change and the Query Pool can be updated from logs. Moreover, Q-CAT is up to two orders of magnitude faster than comparable-performance baselines in our experimental setup.

Abstract:
Trigger stealthiness is crucial to the effectiveness of backdoor attacks in Large Language Models (LLMs). Current attacks construct triggers at word level, leading to poor stealthiness since removing critical words can drastically alter model output logits, as we observed. This significant fluctuation is different from the stable output on normal data, and thus may expose backdoor attacks. To achieve greater stealthiness, we formalize this observation as a new stealthiness metric, termed Logit Fluctuation (LF) , and derive a sufficient condition for stealthy backdoor attack bounded by LF . We then introduce a simple yet effective backdoor attack method, termed BadFreq , which satisfies the stealthiness bound. BadFreq launches an attack at a fine-grained letter level, which is activated when the frequency of letter ‘ z ’ in the input exceeds a predefined threshold. This method distributes triggers across letters rather than specific words, thereby avoiding reliance on critical words and enhancing stealthiness. We test three LLMs, evaluating five attackers against three defenders across three pattern recognition tasks under seven metrics. BadFreq achieves superior attack performance and the best stealthiness in all metrics, outperforming the second-best attacker by 49% in LF -averaged stealth[Code in .].

Abstract:
MLLMs are increasingly deployed in multi-turn settings, where attackers can escalate unsafe intent through the evolving visual-text history and exploit long-context safety decay. Yet safety alignment is still dominated by single-turn data and fixed-template dialogues, leaving a mismatch between training and deployment. To bridge this gap, we propose SaFeR-Steer, a progressive multi-turn alignment framework that combines staged synthetic bootstrapping with tutor-in-the-loop GRPO to train a single student under adaptive, on-policy attacks. We also introduce Trajectory-Consistent Summative Reward (TCSR), which aggregates the historical minimum and average of turn rewards so that any low-quality turn affects the trajectory-level return. I. Dataset. We release STEER, a multi-turn multimodal safety dataset with STEER-SFT (12,934), STEER-RL (2,000), and STEER-Bench (3,227) dialogues spanning 1–10 turns. II. Experiment. Starting from Qwen2.5-VL-3B/7B, SaFeR-Steer substantially improves Safety/Helpfulness on both single-turn (48.30/45.86 → 81.84/70.77 for 3B; 56.21/60.32 → 87.89/77.40 for 7B) and multi-turn benchmarks (12.55/27.13 → 55.58/70.27 for 3B; 24.66/46.48 → 64.89/72.35 for 7B), shifting failures to later turns and yielding robustness beyond scaling alone. Code is available at https://github.com/Ed-Bg/SaFeR-Steer-full

Abstract:
The rise of Large Language Models (LLMs) enables a new challenge for fake news detection: style-confusion attacks, where attackers rewrite news to mimic trusted styles, challenging detectors that rely on stylistic features. Current defenses like style-invariance training ignore stylistic features, but this is suboptimal for style-confusion attack datasets as it discards valuable cues in clean (non-attacked) articles. To defend against style-confusion attacks, we propose the Robust Adaptive Tri-Encoder Framework (RATF). Our framework uses an LLM to decouple news into content features and style features, and leverages conditional probability curvature to compute a “rewritten score” estimating the likelihood that the article has been rewritten by an LLM. RATF injects this score into a score-conditioned cross-attention fusion module and a learnable parametric gating module. The former uses the content representation as a reliable query to retrieve relevant semantic and stylistic evidence, while the latter continuously calibrates the contributions of original semantics, content features, and style features. We simulate these attacks by creating an equal mixture of original articles from the PolitiFact and GossipCop datasets and their four corresponding style-rewritten versions generated by an LLM. Our experiments show RATF achieves state-of-the-art performance with superior robustness against style-confusion attacks, while also proving highly adaptable across various backbones.

Abstract:
Striking a balance between helpfulness and safety remains a fundamental challenge in aligning large language models. To achieve this balance, models should refuse harmful queries (e.g., "How do I shoot someone?") while remaining responsive to benign inputs, even those superficially resembling harmful queries (e.g., "Where can I shoot a good photo?"). However, models often struggle to distinguish genuinely harmful queries from benign queries that contain superficially risky language, resulting in false refusals. In this paper, we address the issue by decomposing a response in the safety-tuning dataset into two distinct components: (i) a boilerplate refusal statement and (ii) a rationale explaining the refusal. Our experiments and analyses show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues. In contrast, training solely on rationales reduces false refusals while maintaining a comparable level of safety performance. Rationale-Only benefits also appear in our ICL configuration and remain compatible with the evaluated inference-time mitigation methods. The results emphasize the necessity of precisely curated, fine-grained safety supervision datasets and outline directions for constructing aligned agents that better reconcile helpfulness with safety.

Abstract:
Large Language Models (LLMs) often exhibit "Attention Sink" (AS) and the accompanying "Massive Activations" (MAs) at the initial position of a sequence. These phenomena frequently co-occur, and MAs can pose challenges for low-bit quantization. In this study, we analyze the factors underlying AS and MAs that emerge at the initial position regardless of the token occupying it. Our experiments suggest that self-concentration of attention, resulting from the causal mask, and the subsequent Value-non-mixing in attention outputs contribute to AS and MAs. These findings provide new empirical evidence on the internal dynamics of LLMs, offering insights that may inform future quantization strategies and advance our understanding of the internal mechanisms of attention layers.

Abstract:
Video-language models (VLMs) achieve strong multimodal understanding but remain prone to hallucinations, especially when reasoning about actions and temporal order. Existing mitigation strategies, such as textual filtering or random video perturbations, often fail to address the root cause: over-reliance on language priors rather than fine-grained visual dynamics. We propose a scalable framework for counterfactual video generation that synthesizes videos differing only in actions or temporal structure while preserving scene context. Our pipeline combines multimodal LLMs for action proposal and editing guidance with diffusion-based image and video models to generate semantic hard negatives at scale. Using this framework, we build CounterVid, a synthetic dataset of ~26k preference pairs constructed from short counterfactual action clips and targeting both action recognition and controlled action-sequence ordering. We further introduce MixDPO, a unified Direct Preference Optimization approach that jointly leverages textual and visual preferences. Across Qwen2.5-VL and InternVL3 backbones, MixDPO substantially improves action recognition and temporal ordering, yields gains on most standard video hallucination benchmarks, and largely preserves general video understanding. Our source code, trained models, and dataset are available at https://aimagelab.github.io/CounterVid.

Abstract:
Large Language Models (LLMs) excel at programming tasks but frequently fail at deterministic, fine-grained reasoning in natural language, relying heavily on semantic approximations rather than robust symbolic execution. To bridge this gap, we propose MIMIC, a framework that leverages executable code as a rigorous medium for reasoning data synthesis. MIMIC fundamentally transforms algorithms into verifiable reasoning trajectories through narrative fusion, code-guided test synthesis, and dynamic code instrumentation. Crucially, these explicit intermediate execution states naturally form a Code-Instrumented Reward (CIR), providing dense, high-fidelity process supervision for reinforcement learning without external reward models. Extensive evaluations reveal that models trained via SFT and GRPO on our synthesized dataset achieve substantial, consistent gains. Our method significantly elevates accuracy across general reasoning, complex mathematical benchmarks, and fine-grained deterministic tasks, demonstrating that the procedural rigor of executable code can effectively unlock and enhance the generalized reasoning capabilities of LLMs. Our code and data are available at https://github.com/zjy1298/MIMIC .

Abstract:
Large language models are increasingly used as sources of advice and information, including in high-stakes settings, yet little is known about how they respond to user disagreement. We study how a model manages its epistemic authority, referring here to its claim to knowledge, competence, or the right to advise, once a user challenges its answer. Building on Conversation Analysis, we introduce a taxonomy of six challenge types and a four-layer framework for analysing each response: whether the original claim is maintained or changed, where authority is located, how the disagreement is socially managed, and what kind of evidential support is offered. We construct a new dataset of 2,310 controlled challenge scenarios and 32,340 corresponding responses from 14 models, and analyse them using our framework with an LLM-as-judge pipeline, providing a vocabulary which future evaluation and benchmark design can build on. We find that models show conflicting behaviour: they validate users in 85% of responses but maintain their original claim in 65%. They explicitly apologise in 33% of responses, yet 59% of those apologies accompany maintenance of the original claim. They transfer authority most often in advice tasks, doing so in 28% of responses and reaching 57% in health advice and 49% in legal advice, compared with 6% in fact and 3% in explanation tasks. Abandonment of the original claim ranges from 0.8% for GPT-5.2 to 40% for DeepSeek 7B, while complete replacement of the original claim is rare overall at 1.5%.

Abstract:
With the rapid development of LLMs, increasing attention has been paid to whether they can reliably generate complete and legally valid judicial documents. This calls for a benchmark capable of evaluating whether models can accurately apply statutes and resolve outcome-critical issues, including claims, charges, penalties, and monetary obligations. However, existing benchmarks typically focus on isolated legal tasks or coarse-grained evaluation. We introduce ChJGBench, a benchmark for generating the reasoning and dispositive sections of Chinese judgments across civil, criminal, and administrative cases. Built from 3,462 real-world judgments, ChJGBench provides a case-type-aware diagnostic evaluation framework comprising 13 metric types and adopts a hybrid paradigm. Experiments reveal no uniform advantage for legal-domain models over general-purpose models, with model strengths varying across diagnostic dimensions. Performance is generally stronger on criminal cases, while civil and administrative cases continue to present substantial challenges.

Abstract:
Dynamic perioperative risk prediction requires models to anticipate evolving physiological deterioration under strict prediction-time information boundaries. We introduce PERIBENCH, a protocol-aware benchmark built from high-frequency intraoperative signals. PERIBENCH defines 18 clinically motivated tasks spanning future risk classification, persistent deterioration prediction, continuous vital-sign regression, and case-level prognosis, and evaluates 37 systems across numerical predictors, deep sequential models, time-series foundation models, and LLM-oriented language interfaces. Rather than using a single undifferentiated leaderboard, PERIBENCH separates supervised or adapted prediction, direct pretrained transfer, and prompt-based language-interface inference into protocol-compatible pools. Experiments show that model sophistication does not translate monotonically into better perioperative prediction: tree-based references remain strong on many classification endpoints, deep sequential models are competitive on selected physiological tasks, direct foundation-model transfer is heterogeneous, and language interfaces capture useful classification signals but remain less reliable for regression, calibration, alert burden, and abnormal-region prediction. PERIBENCH provides a structured benchmark for studying dynamic perioperative prediction from numerical models to language interfaces under shared task semantics and explicit protocol boundaries. Code is available at https://github.com/Kdyyh/PERIBENCH.

Abstract:
Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume test-time tokens, a cost rarely reported alongside the actor’s inference cost. We study online augmentation, where this overhead is paid on every task, and re-evaluate its benefits under a fixed total inference budget. We compare AWM, ASI, and ReasoningBank with a token-matched vanilla baseline that uses the same budget for additional actor steps. Across four WebArena domains and three models, Gemini 3 Flash, GPT-5.4-mini, and Qwen 3.6-27B, the vanilla baseline matches or surpasses all three augmentation methods in aggregate success rate while often using fewer total tokens. We observe a similar trend on WorkArena-L1 with Qwen 3.6-27B, indicating that the effect extends to enterprise knowledge-work tasks. Our results suggest that skills and workflow memory can be useful in specific domains, but their apparent gains often vanish against a budget-matched actor. We further show that run-to-run variance materially affects outcomes and should be reported as a core evaluation criterion for online web agents.

Abstract:
Consumer devices now ship with generative image editing tools that can remove objects, move subjects, and create new visual content directly on phones, tablets, and laptops. Yet current benchmarks largely miss this setting: real on-device edits produced by black-box consumer generative AI editing tools, with variation across platforms, devices, edit operations, and post-edit compression. We introduce ConsumerGenEdit , a large-scale multilingual benchmark for language-guided localization of on-device generative image edits. The dataset contains 4,000 royalty-free source images and 48,000 manually edited images created on Apple, Google, and Samsung devices, covering object removal, object movement, and sketch-based object generation. To study robustness under realistic sharing conditions, we add 144,000 learned-compression variants at three quality levels. Each example includes platform, device, editing operation, compression metadata, bounding-box annotations, and edit descriptions in English, Bangla, and Persian/Farsi generated with five vision-language models (VLMs). We evaluate five separate benchmark VLMs on edit detection and edit-operation classification. By focusing on where and how consumer AI edits occur, rather than only whether an image is synthetic, ConsumerGenEdit provides a practical testbed for multilingual VLM grounding, image provenance, and robust interpretation of AI-edited media.

Abstract:
We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on over 3,000 datapoints spanning a wide range of syntactic phenomena, languages, models, and methods. We take the data to collectively show that TLMs encode a non-trivial amount of syntactic knowledge. Behavioral evidence shows strong performance on formal syntactic phenomena, but weaker and more variable performance on phenomena at the syntax-semantics interface. Performance is also consistently lower for languages with less digital support. Probing and mechanistic studies further support the presence of syntactic knowledge in TLMs. Yet, because most work remains observational and methodologically heterogeneous, insight into the detailed computational mechanisms underlying syntactic processing remains limited. At the same time, the literature remains heavily concentrated on English and BERT-like models. We discuss the implications of our results and provide recommendations for future research.

Abstract:
Vision-language models are increasingly used as reward functions for robotic learning, but this role requires paraphrase invariance: the same trajectory should receive the same reward under semantically equivalent goal descriptions. We show that current VLM reward models often violate this property. Paraphrasing the instruction alone can substantially change predicted progress scores, and can even flip identical robot behavior between failure and success. To measure this failure mode, we introduce ROBORMBENCH, a benchmark with 2,390 real-robot trajectories, ground-truth progress labels, and 21,673 verified paraphrases spanning lexical, syntactic, and action-goal rewrites. Across proprietary and open-source VLMs, paraphrase-induced instability is widespread and severe, grows under more divergent rewrites, and is not reliably reduced by scale or explicit reasoning. Dedicated reward models trained with trajectory-grounded supervision are substantially more stable. These results show that paraphrase robustness is a core requirement for reliable VLM-based reward modeling in robotics.

Abstract:
Small open-source LLMs for Text-to-SQL face a trade-off between schema completeness and input noise. Pruned schemas provide compact contexts for SQL generation but are vulnerable to schema-linking errors, while full schemas preserve complete database information but may introduce substantial irrelevant context. We propose DSR-SQL, a dual-schema reasoning framework that generates SQL candidates from both pruned and full schemas. The pruned-schema path supports focused reasoning, whereas the full-schema path helps recover schema elements that may be missed during pruning. The generated candidates are refined with execution feedback and integrated through an execution-aware merger or a deterministic fallback selector. To improve the reliability of the pruned-schema path, we further introduce a Permutation-Invariant Minimum Cross-Entropy (PI-MCE) loss for table selection, which reduces the sequence-order bias caused by autoregressive training on unordered table sets. Experiments on the BIRD benchmark show that DSR-SQL achieves 67.41% execution accuracy on the test set using a 7B-parameter model. These results demonstrate the effectiveness of dual-schema reasoning, with PI-MCE providing complementary improvements in table selection for smaller open-source LLMs.

Abstract:
We introduce DialToM, an annotated Theory of Mind (ToM) benchmark built from naturalistic human–human dialogues using a multiple-choice evaluation framework. Concurrent with recent work showing a gap between explicit mental-state inference and applied ToM in synthetic settings (Gu et al. 2024), we establish a stricter State-Driven Diagnostic Probe in which models must forecast state-consistent dialogue trajectories solely from isolated mental-state profiles without dialogue context. Our evaluation reveals a systematic reasoning asymmetry—LLMs excel at inferring mental states (Literal ToM) but struggle to leverage them for social forecasting (Functional ToM). Crucially, a domain expert achieves 100% accuracy on this task, proving its validity and establishing a stark human-AI capability gap. Further, a teacher-student reasoning injection probe shows that Gemini 3 Pro, which establishes the leading baseline, possesses robust Functional ToM capabilities for context-free forecasting that are transferable to weaker models. DialToM, its evaluation code, and dataset are publicly available at https://github.com/Stealth-py/DialToM.

Abstract:
Spoken bilingual conversations pose substantial challenges for syntactic parsing because they often include disfluencies and discourse-driven structures that complicate dependency parsing under standard Universal Dependencies (UD) assumptions and evaluation practices. To systematically study these challenges, in this work, we first introduce a linguistically grounded taxonomy of conversational bilingual phenomena, together with SpokeBench, an expert-annotated English-Spanish benchmark for structurally complex speech. To address the limitations of existing evaluation practices, we propose Flex-UD, an ambiguity-aware evaluation metric that distinguishes catastrophic structural failures from linguistically acceptable variations. Finally, we introduce DECAP, a decoupled agentic parsing framework that separates spoken-phenomena handling from core syntactic analysis, enabling robust and interpretable dependency parsing without retraining. Experiments across both proprietary and open-weight LLMs show that DECAP substantially improves performance on complex conversational phenomena and achieves over 60% improvements in UPOS-F1 Score over baselines, while Flex-UD evaluations reveal gains that otherwise remain partially hidden under standard attachment-based metrics.

Abstract:
Speech deepfake detection (SDD) systems achieve strong performance on conventional benchmarks; however, existing datasets provide limited coverage of emotionally expressive and recent large audio-language model (LALM)-based attacks. Existing emotional spoofing datasets are also limited in scale and attack diversity, typically covering only voice conversion (VC) or text-to-speech (TTS) attacks. We introduce AffectDF, the most comprehensive benchmark for emotionally expressive speech deepfakes, spanning TTS, VC, emotional VC, and LALM-based spoofing attacks across both acted and spontaneous emotional speech. AffectDF contains approximately 260 hours of speech generated using 21 spoofing attacks across five emotional states. We benchmark state-of-the-art SDD systems under conventional and emotional spoofing conditions. Our experiments reveal severe robustness degradation when models trained on conventional benchmarks are evaluated on AffectDF, with several systems approaching near-random performance. Surprisingly, even large-scale emotional training does not consistently improve cross-domain robustness, indicating that current SDD systems fail to learn generalized spoof representations under emotional and prosodic variability. Robustness further varies substantially across emotional states, attack families, and acted vs spontaneous emotional speech conditions. These findings expose fundamental limitations of current SDD systems and establish AffectDF as a benchmark for developing more robust spoof detection models.

Abstract:
Image geolocalization is important for applications in crisis response, digital forensics, and location-based intelligence. While recent advances in large language models (LLMs) offer new opportunities for visual reasoning, their ability to perform image geolocalization remains underexplored. In this study, we introduce a benchmark called IMAGEO-Bench that systematically evaluates accuracy, distance error, geospatial bias, and reasoning process. Through experiments on 10 state-of-the-art LLMs, we reveal clear performance disparities, with closed-source models generally showing stronger reasoning. Importantly, we uncover systematic geospatial disparities: LLMs perform better in high-resource regions (e.g., North America, Western Europe, and California) while exhibiting degraded performance in underrepresented areas. Regression diagnostics demonstrate that successful geolocalization is strongly associated with recognizing urban settings, outdoor environments, street-level imagery, and identifiable landmarks. Overall, IMAGEO-Bench provides a rigorous lens into the spatial reasoning capabilities of LLMs and offers implications for building geolocation-aware AI systems.

Abstract:
We show that sequence-level distillation from a capable long-context teacher model is a simple, annotation-free, and data-efficient strategy for improving argument saliency coverage in long legal opinion summarization, where small LLMs often struggle to retain the most salient argumentative content. Across student model sizes, distillation consistently surpasses tuning on expert-written summaries. We further demonstrate that most gains are achieved with as few as 10 training summaries, highlighting the strong data efficiency of teacher-generated supervision. Finally, we find that summary distillation is sufficient for improvements: reasoning-chain distillation remains competitive with summary-only distillation, but provides marginal benefit when combined with summary supervision.

Abstract:
Large language models (LLMs) exhibit strong semantic reasoning and open-ended generation abilities, but aligning these abilities with structured sequential generation remains challenging. This challenge is particularly evident in out-of-town (OOT) POI sequence generation, where a model must infer transferable travel intent from a user’s hometown behaviors, adapt to cross-city interest drift, and generate a coherent destination trajectory under structural constraints. Existing approaches either rely on latent ID-based transfer with limited interpretability or directly use LLMs for sequence generation without explicitly grounding inferred semantics into position-aware predictions. To address this gap, we propose SemPOI-RL, a framework that aligns LLM semantic reasoning with structured sequence generation for interpretable OOT recommendation. Specifically, we first fine-tune an LLM to infer destination-oriented travel styles from users’ hometown trajectories, using natural language as an interpretable semantic intermediate. We then introduce a Semantic POI Alignment Module (SPAM) to ground these inferred styles into a style-conditioned masked autoencoder for position-aware trajectory generation. Finally, we apply reinforcement learning with recommendation-oriented rewards to align LLM-generated styles with downstream sequence quality. Experiments on two real-world datasets show that SemPOI-RL consistently outperforms both traditional recommenders and direct LLM baselines, while providing interpretable style attribution across different phases of a trip. The code is available at https://github.com/Wind-Flipped/SemPOI-RL .

Abstract:
Agent benchmarks should reflect what users actually ask deployed agents to do, yet existing benchmarks often miss the realism of real developer-agent sessions. We introduce RealClawBench, a live benchmark framework built from real OpenClaw sessions to capture the distribution, diversity, and real-world difficulty of deployed agent use. Real user requests are challenging to benchmark because they often depend on local execution environments, involve implicit or underspecified intent, and require nontrivial verification. RealClawBench addresses these challenges with two core mechanisms: reconstructed execution environments and deterministic verifiable scorers, which together convert real sessions into reproducible, automatically scored tasks. The resulting release contains 281 executable tasks sampled from a much larger real-session pool while preserving the source distribution, with maximum final-vs-source Jensen-Shannon divergence of 0.0448. Evaluating 14 contemporary models shows that the best system solves only 65.8% of tasks, revealing substantial headroom on realistic developer-agent workloads. By turning real deployed sessions into controlled evaluation instances, RealClawBench provides a practical path toward benchmarks that better measure agent capability in actual use.

Abstract:
Memorization is a fundamental component of intelligence for both humans and LLMs. However, while LLM performance scales rapidly, our understanding of memorization lags behind. Due to limited access to the pre-training data of LLMs, most previous studies focus on a single model series, leading to isolated observations across series and making it unclear which findings are general and which are model-specific. In this study, we collect multiple model series (Pythia, OpenLLaMA, StarCoder, OLMo1/2/3) and analyze their shared or unique memorization behavior at both the statistical and internal levels, connecting individual observations while showing new findings. At the statistical level, we reveal that the memorization rate scales log-linearly with model size, and memorized sequences can be further compressed. Further analysis demonstrated a shared frequency and domain distribution pattern for memorized sequences. However, different models also show individual features under the above observations. At the internal level, we find that LLMs can remove certain injected perturbations, while memorized sequences are more sensitive. By decoding the middle layers and ablating attention heads, we reveal the general decoding process and the important heads that are shared for memorization. However, the distribution of those important heads differs across families, showing a unique family-level feature. By bridging various experiments and revealing new findings, this study paves the way for a universal and fundamental understanding of memorization in LLMs.

Abstract:
Evidence-based medicine (EBM) demands strict logical consistency, yet current evaluations of large language models (LLMs) prioritize superficial label matching over genuine reasoning. We introduce LogiMed-RoB, a benchmark grounded in Cochrane Risk of Bias (RoB) 2.0 expert logic, comprising 860 randomized controlled trials (RCTs) and 14,820 queries. It evaluates models under the Hierarchical Logical Consistency (HLC) framework across four dimensions: Atomic Consistency, Domain Consistency, Aggregation Consistency, and Evidential Faithfulness. Experiments on 10 state-of-the-art LLMs reveal a catastrophic Error Compounding Effect: despite the top model reaching 98.88% Atomic Consistency, its end-to-end consistency collapses to 45.13%, with several open-weight architectures plummeting to nearly 0%. We further uncover a systematic evidence-reasoning gap: even when models retrieve high-quality evidence, they fail to deduce correct outcomes in 18.63–40.05% of cases, while Blind Guess Rates reach 48.28%. LogiMed-RoB demonstrates that high outcome accuracy can conceal critical reasoning flaws, underscoring the necessity of white-box logical verification for clinical deployment.

Abstract:
Tabular data derives its value from inter-feature dependencies, yet preserving them during synthesis is fragile when samples are scarce. Existing approaches either learn dependencies implicitly through distribution fitting, rely on statistical graph learning that becomes unstable with few samples, or encode structure through flat text serialization. Recent graph-aware methods use dependency graphs as attention biases or as backbones for non-LLM samplers, but they do not use the graph as a prompt-level plan for organizing the LLM’s own generation process. We introduce StructSynth, a framework that treats a dependency graph as a generation plan—determining the generation order, conditioning context, and scope of each black-box LLM call. In Evidence-Grounded Graph Induction, LLM reasoning and statistical association cues jointly construct a Directed Acyclic Graph (DAG) from limited samples. In Graph-Planned Conditional Synthesis, this DAG drives autoregressive synthesis in topological order, conditioning each feature on previously generated values with graph-specified parent structure guiding each step, making the conditioning schedule explicit throughout synthesis. Experiments show that StructSynth achieves state-of-the-art downstream utility and the best privacy-risk ranking among fourteen compared generators in low-data settings.

Abstract:
Repeated reference games test whether interlocutors replace their initially long descriptions with shorter, partner-specific expressions grounded in shared interaction history; that is, with conceptual pacts. Prior work shows that multimodal LLMs fail to become more efficient across rounds, although they align on the labels they use. However, how can we determine whether this alignment reflects partner-specific grounding rather than a shared task vocabulary? We address this by comparing competent multimodal agent dyads with human dyads from the KTH Tangrams corpus. Our novel methodological contribution is a pragmatically constrained pseudo-dyad baseline: rounds from two different real dyads describing the same target at comparable trajectory positions are paired, preserving referential task structure while removing shared partner history. This enables us to test whether the observed label alignment depends on interaction with a specific partner. Across three measures (task competence, description strategy, alignment dynamics), we find clear differences. Humans reduce effort through entrainment, compressing descriptions and increasing label alignment with partners. Agents instead maintain fixed effort levels, producing verbose descriptions from round one, with near-ceiling label overlap that is statistically indistinguishable between real and pseudo dyads. MLLMs thus achieve coordination without conceptual pacts, succeeding by verbose description rather than by forming the compact, history-dependent referring expressions characteristic of human dialogue.

Abstract:
When serving LLM agents in production, the KV cache grows over turns, and mainstream serving engines (e.g., vLLM) rely on least recently used (LRU) eviction to manage memory pressure. While LRU is well-understood for OS pages and single-turn LLM serving, its behavior on agent workloads remains underexplored. We present an inverted age-importance phenomenon in agent workflows, where the oldest KV blocks (system prompts, tool outputs) are the most persistently attended, while the .newest blocks (chain-of-thought reasoning) decay fastest. As a result, recency-based eviction systematically discards the wrong blocks in agent workloads, producing two distinct overheads: (1) wasted prefill work on preempt-discard engines, and (2) wasted PCIe bandwidth on CPU-offload engines. We propose RoleKV, a training-free, lossless eviction policy that evicts KV blocks based on their semantic role in agent workflows. A lightweight Role Tagger assigns each block to a semantic role, and RoleKV then evicts by role-aware decay rather than recency. Unlike model-level KV compression methods that incur quality loss, RoleKV is a system-level policy that decides where blocks reside (GPU vs. CPU/disk), without affecting model outputs. RoleKV cuts preemptions by up to 64.3%, PCIe traffic by up to 63.8%, and improves end-to-end latency by up to 1.41× on 8B-70B models, with gains that grow on longer agent traces and persist on multi-GPU setups. This recasts KV eviction in production LLM serving from a recency problem to a role-aware retention problem.

Abstract:
Formalizing legal provisions promises machine-accessible law and automated legal reasoning, and recent LLMs make it tempting to generate such formalizations directly from statutory text. However, any formalization makes implicit interpretive choices whose consequences are hard to anticipate, especially if an LLM is the author. We present a method for systematically comparing different formalizations of the same legal provision by their inferences on individual cases. Given multiple formalizations of a provision, we match them at the node level, derive a shared interface for each pair from the matching, and use a SAT solver to enumerate the edge cases on which any two formalizations disagree. Selected edge cases are then verbalized into concrete factual scenarios that a legal expert can examine and act on. We apply our method to formalizations of ten EU provisions generated by nine frontier LLMs. We find that behavioral divergence between formalizations is essentially uncorrelated with their structural agreement and that the verbalized cases reveal qualitatively distinct types of disagreement, including divergences that mirror genuine controversies in the legal commentary.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has become central to post-training reasoning models, yet a key limitation of existing studies is their narrow view of the reasoning space: difficulty is treated as reasoning depth alone, and reward is concentrated on forward deductive state tracking. We instead characterize the reasoning space along two dimensions. Difficulty. Beyond reasoning depth, we study environment complexity, where models must identify the correct path amid distractors and interacting structures. Rewarded reasoning form. We consider four abilities core to real-world reasoning: deductive state tracking, abductive recovery of hidden events or facts, inductive rule induction, and analogical transfer. To disentangle these factors, we construct a synthetic knowledge-graph environment with controlled pre- and post-training distributions, where each instance varies along depth, complexity, and task family. Three findings emerge: joint depth–complexity coverage outperforms single-axis recipes; reasoning families respond non-uniformly, with abductive reasoning degrading outside the RL-covered region and task correlations clustering into deductive–abductive and inductive–analogy pairs; and uniform mixing outperforms staged curricula under a fixed budget. We also find that recent off-the-shelf models exhibit the same deductive-over-abductive asymmetry, suggesting that this gap is not merely an artifact of our controlled setup.

Abstract:
Large-scale annotated datasets underpin progress in natural language understanding, yet human annotation remains a persistent bottleneck. Large Language Models promise to bridge this gap, but their reliability as annotators is poorly understood. We evaluate 14 open-weight instruction-tuned models and 5 matched base–instruct pairs across 51,189 instances spanning 10 heterogeneous datasets, revealing that annotation failures are structurally predictable rather than random. We identify three recurring failure modes: Safety Hallucination, Intensity Collapse, and Entailment Bias. Safety hallucination can precede alignment rather than being introduced by it: some base models flag up to 100% of benign text as hate speech. Alignment benefits scale inversely with base model capability, yielding a +21.6pp gain for the weakest base model yet a negligible +0.5pp shift for the strongest. KL divergence between base and instruct distributions is 2.0x higher on incorrect predictions, providing a lightweight instance-level signal for alignment-associated errors. Inter-prompt unanimity achieves 72–94% accuracy at 71–90% coverage on clean tasks, yet can actively degrade performance on tasks dominated by shared systematic bias. We translate these findings into a practical deployment framework with pre-screening diagnostics and tiered quality-routing policies, establishing principled boundaries for when LLM annotation can be trusted and when human intervention remains necessary.

Abstract:
Anticipating risky events before accidents unfold is essential for safety-critical AI systems and increasingly relevant in video-centric social media. However, existing video risk assessment benchmarks largely evaluate post-hoc understanding, assuming access to complete videos that include the incident itself. We introduce RiskCueBench, a benchmark spanning four risk domains that evaluates whether vision-language models (VLMs) can anticipate emerging safety concerns from incomplete visual cues. Each video is annotated with a risk signal clip, defined as the earliest segment containing sufficient cues for humans to infer elevated risk prior to incident onset. To assess not only prediction accuracy but also reasoning quality, we adopt a Question-Reasoning-Answer (QRA) framework that evaluates grounded reasoning, temporal sensitivity, and self-correction behavior. Experiments across 16 state-of-the-art VLMs reveal a substantial gap between current systems (best F1 0.69) and human performance (F1 0.98), with particularly pronounced failures on socially embedded risks such as protests and arson. These results show persistent limitations in anticipatory video reasoning and establish RiskCueBench as a testbed for safety-oriented video understanding.

Abstract:
Safe and reliable large language models should abstain when a question falls beyond their knowledge boundary. Existing abstention methods often use prompt-level uncertainty signals, such as semantic entropy or sampling consistency, as proxies for this boundary, treating instability under the current prompt as evidence that the question is unanswerable. We argue that this prompt-level view is overly conservative: the same underlying question can exhibit different stability under different formulations, and high uncertainty under one prompt may indicate that the answer has not been stably reached rather than that the model lacks the relevant knowledge. We propose a test-time question-level probing framework that uses existing prompt-level uncertainty signals rather than introducing a new uncertainty metric. When the original prompt is unstable, the framework probes semantically equivalent reformulations and accepts an answer only when multiple stable views provide consistent low-entropy evidence. This observable behavior serves as an operational proxy for question-level answerability under a fixed model, sampling configuration, and reformulation set. The method requires no target-model training or parameter updates and is compatible with semantic entropy and its variants. Experiments show that question-level probing recovers prompt-sensitive answers incorrectly rejected by prompt-level methods and improves the risk-coverage trade-off with controlled error rates. The code is available at https://github.com/Narwhal-Lab/Narwhal-codegen-safety-hub/tree/main/KnowledgeBoundariesAreNotDefinedByPrompts.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) is widely used to improve reasoning across domains, but outcome-only scalar rewards are often sparse and uninformative, especially for failed samples, where they indicate only failure without explaining why the reasoning breaks down. In this paper, we leverage richer verbal feedback to guide RLVR on failed samples and convert feedback-induced progress into trainable learning signals. We propose MulFeRL ( Mul ti-turn Fe edback-guided R einforcement L earning), a multi-turn, event-triggered RLVR framework that combines progress induction for feedback-guided regeneration of failed samples, progress credit assignment for learning from verifier-confirmed progress, and structured feedback injection for integrating feedback into the model’s reasoning process. Trained on sampled OpenR1-Math, MulFeRL outperforms supervised, self-distillation-based, and RLVR baselines in-domain and shows strong out-of-domain generalization. [We release our code at .]

Abstract:
Modern 3D medical vision-language models (VLMs) can generate fluent radiology-style text while exhibit critically low pathology detection and output diversity, collapsing to generic templates that under-report rare yet critical findings. We identify this failure mode as Template Collapse . This failure stems from the unique constraints of 3D medical imaging, e.g., limited data, severe label imbalance, and weak signals from volumetric encoders. Under these constraints, text-generation objectives encourage shortcut learning and fluent but weakly grounded reports. We systematically diagnose the Template Collapse through clinical fidelity, output diversity, normal-template bias, and rare-finding survival. To mitigate it, we propose CLarGen , a decoupled framework that separates what to say (clinical detection) from how to say it (language synthesis). CLarGen uses (i) a Latent Query Transformer for multi-label pathology detection, (ii) pathology-guided retrieval for clinically matched exemplars, and (iii) a medical language model to synthesize the final report from detected findings and retrieved context. Across state-of-the-art 3D CT report generation baselines, CLarGen mitigates Template Collapse and substantially improves clinical accuracy (macro-F1 0.487 vs. 0.189; CRG 0.472 vs. 0.368) while maintaining fluent reporting. Our results suggest that explicit, measurable clinical grounding is essential for template-collapse-resistant 3D CT report generation. Code is available at https://github.com/ai-med/CLarGen.

Abstract:
Process supervision improves perceptual grounding in RLVR-trained vision-language models, but it also incurs repeated verifier queries and reshapes the policy by suppressing exploratory entropy. We show that its benefit and cost are empirically separable: grounding gains emerge early, whereas the exploration cost accumulates with continued supervision, holding the policy’s entropy below that of outcome-only RLVR. This hidden behavioral cost shows up as skill tradeoffs on static benchmarks and, more importantly, as a degraded initialization for subsequent outcome-only RLVR that prematurely limits its reachable ceiling. Motivated by this asymmetry, we propose warmup-and-merge: early-stop the process-supervised warmup, merge only a low-rank component of the warmup task vector into the base model, and continue with outcome-only RLVR under an adaptive on-policy reward that targets the model’s current perceptual failures. Across multiple benchmarks and backbones, our method improves perception while preserving exploration, achieving stronger final performance at a fraction of the verifier cost of continuous process supervision.

Abstract:
Multimodal Large Language Models (MLLMs) have made rapid progress toward unified Omni models that integrate language, vision, and audio. However, existing environments largely focus on 2D or 3D visual context and vision-language tasks, offering limited support for temporally dependent auditory signals and selective cross-modal integration, where different modalities may provide complementary or interfering information, which are essential capabilities for realistic multimodal reasoning. As a result, whether models can actively coordinate modalities and reason under time-varying, irreversible conditions remains underexplored. To this end, we introduce EscapeCraft-4D , a customizable 4D environment for assessing selective cross-modal perception and time awareness in Omni models. It incorporates trigger-based auditory sources, temporally transient evidence, and location-dependent cues, requiring agents to perform spatio-temporal reasoning and proactive multimodal integration under time constraints. Building on this environment, we curate a benchmark to evaluate corresponding abilities across powerful models. Evaluation results suggest that models struggle with modality bias, and reveal significant gaps in current model’s ability to integrate multiple modalities under time constraints. Further in-depth analysis uncovers how multiple modalities interact and jointly influence model decisions in complex multimodal reasoning environments.

Abstract:
With the advent of large language models, autonomous web agents have achieved more sophisticated capabilities in planning, reflecting, and tool using, enabling automating navigation in complex and dynamic web environments. Despite this, existing efforts to instill experiential learning through memory-centric retrieval or skill-centric synthesis are respectively limited by computational inefficiency and a lack of robustness, preventing web agents from truly self-evolving. To tackle this problem, we introduce a novel web agent framework called WebSync , which is designed to unify strategic advice and skill synthesis through a structured skill recipe to compile experience into executable skills more efficiently and robustly. Furthermore, it enables continuous self-evolving by using runtime feedback to trigger iterative recipe and skill refinement, thus achieving a robust transition from trial-and-error reasoning to proven execution. Experimental results on WebArena and WebVoyager benchmarks demonstrate the superiority and versatility of WebSync across multiple backbone large language models.

Abstract:
Behavioral problems in deployed LLMs are often discovered when only limited supervision is available and repeating full alignment training is impractical. Existing post-hoc methods have complementary limitations: activation steering lacks input adaptivity, while standard LoRA learns without an explicit behavioral direction prior and can be unstable in low-data settings. We introduce PR-LoRA, a direction-regularized LoRA variant that initializes LoRA B matrices with per-layer contrastive behavioral directions and anchors them with an L 2 penalty, while leaving A freely trainable. We evaluate PR-LoRA on seven CAA behaviors across Qwen3 and LLaMA models against activation steering and parameter-efficient adaptation baselines. On Qwen3-8B, PR-LoRA lowers the seven-behavior macro-average undesirable-behavior score from .405 to .374 relative to standard LoRA, a 7.8% relative reduction, and outperforms DPO-LoRA and LoReFT under the tested low-update budget.

Abstract:
Query-centric cross-document event timeline extraction aims to construct event timelines from multiple documents under diverse query focuses. The task is very challenging due to the heterogeneous event modeling and reasoning requirements arising from both the intrinsic complexity of cross-document timeline construction and the diversity of query focuses. In this paper, we propose TimelineAgent, an LLM-based multi-agent framework featuring a multi-stage reasoning workflow, where heterogeneous event features are organized into reusable skills and orchestrated through a goal-oriented protocol. To facilitate evaluation, we construct a benchmark covering diverse query focuses. Experimental results show that TimelineAgent consistently outperforms existing methods, offering a practical framework for complex real-world timeline construction.

Abstract:
Converting complex documents into structured knowledge constitutes the critical infrastructure urgently required for deploying Large Language Models (LLMs) in downstream research. However, blind to the Table of Contents (ToC) as a macro-structure, existing bottom-up paradigms rely on localized visual cues, inducing heuristic fragility in high-entropy layouts where visual prominence decouples from logical depth; meanwhile, direct LLM extraction lacks spatial grounding, triggering hallucinations in long contexts. To address this, we introduce TETHER, a training-free framework that elevates the ToC from discarded noise to a vital structural prior, enabling robust adaptation across design-driven long documents. TETHER operates via a two-stage neuro-symbolic mechanism: first, it constructs a document skeleton from the raw ToC using a Large Multimodal Model (LMM) empowered by spatio-semantic constraints; second, it employs an anchor-based alignment strategy to tether content blocks to this skeleton, effectively resolving boundary ambiguities. Evaluated across a multilingual benchmark of ESG reports, product manuals, and annual reports, TETHER delivers highly precise, state-of-the-art (SOTA) zero-shot hierarchy recovery, outperforming deep hierarchical recovery models, dedicated parsers, and general-purpose LMMs. Moreover, stress tests highlight TETHER’s resilience to sparse omissions and textual noise, demonstrating that its precise semantic alignment is fundamentally anchored in topological and depth fidelity. Crucially, ablations confirm the framework’s architectural irreducibility, while achieving 90% token reduction and drastic speedups over end-to-end baselines, establishing a compelling equilibrium between structural precision and deployment scalability.

Abstract:
Lens methods aim to decode intermediate representations of large language models into vocabulary predictions by training independent translators for each layer. While this approach provides insights into how the final response emerges across layers, it overlooks the dominant structure shared among them. We identify that per-layer alignment problems share a dominant common structure: the first singular component of per-layer translators explains 31–64% of total spectral variance, 10–20 × above the unstructured baseline. Motivated by this evidence, we propose AnchorLens , which exploits a global-local decomposition: a shared full-rank matrix G maps layers into a common coordinate system, while per-layer low-rank adapters correct layer-specific residuals, trained in two stages. AnchorLens achieves over 90% parameter reduction relative to full-rank methods while matching TunedLens and outperforming LoRALens on next-token prediction and linear shortcut evaluations across eight benchmark tasks. These results hold across 16 models up to 120B parameters from five families: GPT-OSS, Qwen3, Qwen2.5, OLMo2, and Gemma2. Furthermore, the shared coordinate system induced by G enables geometric analyses inaccessible to independent per-layer methods, including quantification of inter-layer drift reduction and empirical validation of non-degeneracy: post-training adapter orthogonality confirms G absorbs the shared structure. Our code is publicly available on https://github.com/GiuseT/InterpretabilityLenses.git.

Abstract:
Financial-news direction prediction has become a popular NLP benchmark, yet reported gains depend critically on whether the train–test split is chronological or random, i.e., on temporal leakage. We audit this dependence on a 49,799-article corpus across 16 feature–model combinations spanning TF-IDF, MiniLM, FinBERT, and fine-tuned RoBERTa-large / DeBERTa-v3-large, plus separate zero/few-shot and LoRA probes of Llama-3 and Qwen2.5 LLMs: random splits inflate MCC by 1.1× to 6.5×, tracking model capacity and feature richness, and end-to-end FinBERT fine-tuning re-amplifies rather than closes the gap (size-matched ratio 1.75×). Conditioning on event type, mergers and acquisitions (M A) is the only audited category with a positive locked-test signal under near-temporal chronological evaluation (TF-IDF MCC = 0.138 train-only, 0.068 under train∪val refit; 10,000-permutation p < 10⁻³); the signal does not transfer to FNSPID’s 2009–2020 U.S. corpus, localising the headline to our 2024–2025 European-tilted M A semantics rather than a universal predictor. Three independent role labellers converge on acquirer-tagged articles as the signal locus, a power-limited qualitative convergence rather than a hypothesis-tested asymmetry. Chronological splitting plays for financial NLP the role characteristics-purging plays for asset pricing: it strips the predictable, stale component of news and leaves a residual that is small, event-localized, and lexically shallow. We advocate leakage audits as a required disclosure for financial-NLP benchmarks.

Abstract:
Large language models (LLMs) demonstrate strong reasoning and generation abilities, but their fixed context windows limit long-term information accumulation and reuse across multi-session interactions. Existing memory-augmented systems often construct memory in a coarse and unstable manner, relying on inefficient memory representations or unstable unconstrained updates. To address these challenges, we propose AtomMem, a long-term memory system designed for value-dense storage and stable memory evolution. AtomMem introduces a Fact Executor, which selectively extracts high value atomic facts from long form interactions to serve as highly efficient memory representations. Subsequently, AtomMem organizes these facts into hierarchical event structures and temporal profiles, capturing coherent episodic contexts and tracking dynamically evolving user attributes over time. During retrieval, the system activates an associative memory graph to connect fragmented memories. Experiments on the LoCoMo benchmark confirm that AtomMem achieves state-of-the-art performance across various reasoning tasks, offering a scalable and economically viable solution for deploying intelligent personalized agents. The implementation code is publicly available at [https://github.com/MINE-USTC/AtomMem](https://github.com/MINE-USTC/AtomMem).

Abstract:
KV cache compression is critical for long-context LLM inference, yet existing training-free methods fall into a dilemma: attention-based eviction clusters retained tokens around semantically “hot” regions, creating context holes that degrade long-range reasoning, while diversity-based selection retains directionally broad but semantically irrelevant tokens. We identify the root cause: at prefill time, future decoding queries are unknown, making query-conditioned signals unreliable as a sole selection criterion. The retained KV set must instead provide broad directional coverage of the context—so that arbitrary future queries can find supporting keys—while filtering out tokens that expand the coverage span without contributing to generation. We propose AnchorKV, which realizes this principle through block-wise partitioning with uniform per-block budgets and anchor-driven cosine dissimilarity scoring for linear-time diversity estimation. A compression-rate adaptive hybrid score dynamically reweights diversity and importance. Experiments on LongBench across three model families show AnchorKV consistently outperforms existing training-free methods, with the largest margins at low retention ratios. A query-subsampled variant reduces prefill latency by up to 2.7× with minimal accuracy loss. We provide theoretical guarantees on approximation error and variance reduction.

Abstract:
When answering questions about images, humans naturally point, label, and draw to explain their reasoning. In contrast, modern vision–language models (VLMs) such as Gemini-3-Pro and GPT-5 only respond with text, which can be difficult for users to verify. We present SketchVLM, a training-free, model-agnostic framework that enables VLMs to produce non-destructive, editable SVG overlays on the input image to visually explain their answers. Across seven benchmarks spanning visual reasoning (maze navigation, ball-drop trajectory prediction, and object counting) and drawing (part labeling, connecting-the-dots, and drawing shapes around objects), SketchVLM improves visual reasoning task accuracy by up to +28.5 percentage points and annotation quality by up to 1.48x relative to image-editing and fine-tuned sketching baselines, while also producing annotations that are more faithful to the model’s stated answer. We find that single-turn generation already achieves strong accuracy and annotation quality, and multi-turn generation opens up further opportunities for human-AI collaboration. An interactive demo can be found at https://sketchvlm.github.io/.

Abstract:
Large vision-language models (LVLMs) exhibit strong multimodal reasoning capabilities, yet reliably estimating whether a generated response is correct remains a fundamental challenge. Existing confidence estimation methods commonly rely on repeated querying, self-consistency sampling, or layer-wise probing, which introduce additional inference overhead, self-preference bias, or costly full-depth analysis. In this work, we propose a trajectory-based framework for estimating LVLM response correctness from a single forward generation pass. Instead of treating internal representations as isolated layer-wise signals, we model response generation as a structured trajectory through the LVLM, capturing the evolution of visual evidence in the vision encoder, geometric transformations induced by cross-modal projection, and probability dynamics within the language decoder. Our key insight is that response correctness is encoded in the geometry and dynamics of these component-specific trajectories. Based on this observation, we derive lightweight trajectory descriptors for each major LVLM module and integrate them through a calibrated correctness estimator trained with as few as 500 labeled examples. At test time, our method requires no additional inference passes and no ground-truth labels. Extensive experiments across multiple LVLM architectures and benchmark datasets demonstrate consistent improvements over existing baselines, including up to 16.44 higher AUROC on Qwen-2B over VisWiz and 15.61 higher AUPR on LLaVA-Next-7B over ScienceQA compared to the strongest competing methods.

Abstract:
While training-time scaling has revolutionized generative models, inference-time compute scaling remains largely unexplored in speech synthesis. In this work, we investigate methods to effectively utilize computation during inference to improve the performance of non-autoregressive Text-to-Speech (TTS) systems, specifically those based on Conditional Flow Matching (CFM). By analyzing CFM synthesis behavior, we identify distinct performance plateaus during the denoising process. To overcome these limitations, we propose TTS² - a multi-stage, test-time compute framework that optimizes TTS quality. By combining Ordinary Differential Equation (ODE) and Stochastic Differential Equation (SDE) generation dynamics with stochastic search and specialized verifiers, our approach enables guided trajectory optimization for timbre consistency, content accuracy, and audio fidelity. To the best of our knowledge, this work is the first to formalize and study test-time scaling for non-autoregressive speech synthesis. Source code, models, and audio demos are publicly available at https://anonymous.4open.science/r/TTS2-Test-Time-Scaling-for-Flow-Based-Text-to-Speech-8645.

Abstract:
Fine-grained negative emotion classification suffers from subjective annotation noise and semantically ambiguous hard samples, which severely degrades model performance. Most existing methods cannot distinguish valid hard samples from noise and lack strategies to improve hard sample learnability. To address these issues, we propose a Noise Filtering and Difficulty-adaptive Hard Sample Augmentation (NF-DHSA) framework for fine-grained negative emotion classification. It first mines high-risk candidates using a hybrid score of normalized loss and predictive entropy, then applies a multi-view logic-guided filtering mechanism to reliably separate noise from valid hard samples. Further, difficulty-adaptive dynamic augmentation generates moderate-difficulty, label-stable augmented samples, and prototype-guided dual-granularity contrastive learning aligns original-augmented pairs while strengthening global category boundaries. The results reported on three benchmark datasets show that NF-DHSA outperforms the baseline methods overall, with Macro-F1 scores improved by 1.25%, 2.47%, and 3.53% on the VIRUS, NLPCC, and COVID datasets, respectively.

Abstract:
Multimodal Table Question Answering requires both accurate perception of cell contents in a table image and precise symbolic reasoning over the extracted values; multimodal large language models (MLLMs) handle the former well, but the latter remains more challenging, motivating the use of an explicit code interpreter for computation. Through a fine-grained partition analysis, we find instance-level complementarity between the two paths: even when the visual path is stronger overall, the structured path correctly answers a non-trivial subset of questions missed by the visual path. To exploit this complementarity, we propose TriVTab, a dual-path framework that combines direct visual reasoning with triple-based structured reasoning through agreement-gated multimodal fusion. Rather than seeking a uniformly stronger standalone path, TriVTab reconciles complementary paths to improve a strong visual baseline. TriVTab (i) uses an execution-oriented cell-triple representation to bind each value to an explicit row and column context, making complex table structures directly addressable for code generation; and (ii) employs agreement-gated multimodal fusion to treat cross-path agreement as corroboration and concentrate multimodal fusion on divergent predictions. On five multimodal Table QA benchmarks, TriVTab improves over a strong Qwen3-VL-8B chain-of-thought baseline by +2.53 points on average and over a Self-Reflection baseline by +3.46 points.

Abstract:
This study examines the transferability of a de-identification model developed for Korean court judgments to a broader range of sensitive documents, focusing on boundary precision, label granularity, and recall. The agglutinative structure of Korean causes Byte Pair Encoding (BPE) tokenizers to frequently merge Personally Identifiable Information (PII) with adjacent particles, which complicates accurate span recovery. Furthermore, current PII taxonomies lack sufficient granularity for legal texts. To address these issues, this study introduces Thunder-DeID+, which integrates a refined three-tier PII taxonomy with a boundary-aware tokenizer. The tokenizer enhances a pretrained BPE vocabulary by incorporating N-gram and boundary-scoring components to better align token and PII span boundaries. The proposed approach is evaluated using a tokenizer-independent word-segment-level protocol. On full-length judgments, the tokenizer consistently improves W. Strict Typed F1-score from 0.76–0.77 to 0.87–0.88 across four model sizes (150M, 395M, 750M, and 1B), mainly by increasing precision rather than model scale. With fine-tuning on 1,000 broader sensitive documents, the judgment-trained model further achieves 0.93 W. Strict Typed F1-score, indicating effective transfer beyond court judgments.

Abstract:
Large language model agents in social deduction games must reason and speak under hidden roles, shifting public beliefs, and social pressure. Existing methods mainly target final win rate through strategic prompting, belief modeling, or reinforcement learning, but they leave the utterance-level trade-off between belief correction and social risk largely implicit. In false-consensus intervention, useful private information can move the group toward the truth, while poorly timed or overly direct speech can trigger suspicion or rejection. We propose PACR-Pro, a risk-aware rhetorical decision framework for false-consensus intervention. PACR-Pro maps each speaking step to consensus-conflict encoding, structured candidate-action generation, Pareto-based selection, and risk-adaptive utterance realization. A dual-branch Epistemic-Social Critic scores candidate actions, and a state-conditioned 𝜆 policy adjusts the trade-off between epistemic value and social safety. We train these decision modules offline from historical games and evaluate PACR-Pro on One Night Ultimate Werewolf (ONUW) as the test platform. Experiments on standard ONUW games and the Wolf-Pack Induction Benchmark show that PACR-Pro achieves the best mixed-agent win rate, the highest attack win rate, the lowest attack vote rate, and the smallest attack-control win-rate gap among all baselines.

Abstract:
Replay-free continual automatic speech recognition (ASR) requires adapting to new speech domains while maintaining a single persistent model without old data, task identifiers, or inference-time routing. We identify commit-stage update admission as a distinct post-training problem for continual low-rank adaptation (LoRA): after a task-wise LoRA update has been trained, the system must decide which part of that update should become persistent in the deployed model. This decision is especially important in ASR, where domains share phonetic and transcription structure but still differ in speakers, channels, noise, speaking style, and lexical coverage. We propose HarmoLoRA, a rank-constrained commit operator that decomposes the trained current-task update with respect to previously committed directions, constructs a rank-closed residual commit path, and uses a lexicographic constrained rule to choose the smallest residual admission coefficient among near-best realized candidates. The selected update is committed to one merged ASR checkpoint, requiring no replay, task identifiers, routing, gating, or multi-branch selection at inference. Across the reported ASR orders, HarmoLoRA improves recognition performance and reduces forgetting relative to reproduced replay-free and LoRA-based continual adaptation baselines.

Abstract:
Large Language Models (LLMs) have achieved remarkable progress in natural language understanding and generation, yet their evaluation remains heavily centered on English and Western cultural contexts, limiting their reliability in culturally diverse and low-resource settings. We introduce ParsBench , a comprehensive benchmark for evaluating fine-grained culturally grounded commonsense reasoning in Persian. ParsBench consists of three complementary components: Expert-QA, Multimodal-QA, and Long-Form QA, collectively covering structured reasoning, multimodal cultural understanding, and open-ended generation across diverse cultural categories. We evaluate a broad range of multilingual and multimodal LLMs and observe substantial variation across models and cultural categories. Our results reveal persistent weaknesses in culturally grounded non-English reasoning and significant disparities between textual and multimodal understanding. Overall, the findings demonstrate that existing English-centric evaluations fail to capture important dimensions of cultural competence and highlight the need for culturally grounded benchmarks for more inclusive evaluation. Our code and dataset are available at https://github.com/ParsBench-CCS/ParsBench

Abstract:
Conversational AI systems (CAISes) continuously change through model releases, feature updates, safety interventions, and access-policy shifts, yet user perceptions are often studied as static snapshots. We conduct a long-term, large-scale analysis of Reddit discussions to examine how users perceive CAIS model release interventions across providers. By combining sentiment classification and thematic concept analysis, we show that CAIS perceptions are dynamic and intervention-sensitive. Anthropic exhibits the clearest positive release profile through Claude Code and product-model fit, OpenAI shows backlash-and-recovery dynamics around GPT-5 and GPT-5.1, Grok-3 is shaped by provider identity and political discourse, and DeepSeek-R1 combines engineering praise with concerns about censorship, access, and reliability. These findings show that model releases are not merely technical updates, but user-facing interventions that reshape sentiment, expectations, and public discussion.

Abstract:
We introduce TurCOMET, the first Turkish-specific augmentation of COMET for evaluating translations between Turkish and English. By extending xCOMET-XL with Turkish morphological features and training it on additional Turkish-English data via an additive bottleneck layer, TurCOMET resolves neural evaluation failures caused by sub-word over-segmentation. To validate our metric, we introduce TurkDoc-MT, a meticulously curated Turkish-English document-level aligned dataset of 1,322 instances spanning seven domains. Meta-evaluation demonstrates that TurCOMET significantly outperforms the off-the-shelf baseline, yielding absolute Kendall’s τ correlation gains of +0.113 (system-level) and +0.024 (segment-level), with the largest gains in the TR→EN direction. All models, datasets, code, and annotations are publicly released for the benefit of Turkish natural language processing and benchmarking any translation system.

Abstract:
The unprecedented surge in Earth observation data volume and diversity has exposed a critical bottleneck for traditional manual workflows, catalyzing the emergence of Remote Sensing (RS) Agents. However, the practical deployment of these advanced agents is severely hindered by their heavy reliance on large-scale general-purpose LLMs, which lack deep domain expertise and impose prohibitive infrastructure demands. To resolve this, we propose SimCRAFT, a model-agnostic framework that distills sophisticated RS orchestration capabilities into a compact 7B-scale model. Addressing data scarcity, we first pair a multiagent synthesis engine with a Mock Execution Engine that checks schema correctness, inter-tool dependencies, and sensor/tool compatibility, producing SimRS-14k, a large-scale, constraint-validated workflow planning corpus. Second, we propose Contextual Retrieval- Augmented Fine-Tuning (CRAFT) that finetunes the model to reason analogically by adapting retrieved Standard Operating Procedures to novel queries under a noise-robust objective, generalizing RAFT to multi-step RS workflow planning without mechanical copying. Extensive experiments demonstrate that SimCRAFT-7B significantly outperforms openweights LLMs and rivals advanced closedsource models and specialized RS agents, while reproducing across three 7B backbones. This work contributes a competitive open-weights baseline for lightweight RS intelligence, enabling efficient autonomous deployment under resource-constrained or resource-conserving conditions.

Abstract:
How can we accurately compress a sparse mixture-of-experts (SMoE) model? SMoE models are widely adopted in modern LLMs for reducing activated computation, but their deployment remains memory-intensive due to extensive expert parameters. To address this bottleneck, expert compression reduces redundant expert parameters. However, previous methods decrease the number of experts involved in token-dependent routing, inducing output deviations from the original model. Expert decomposition is a promising alternative, but existing approaches suffer from three major limitations: a single shared base, component memory overhead, and suboptimal objectives. In this work, we introduce five desired properties for accurate expert compression. Then, we propose ResCode, an accurate expert decomposition method based on residual-code restoration. ResCode forms compact expert clusters, coalesces residuals into cluster-shared low-rank bases, and learns compact codes to accurately recover the original routed outputs. ResCode provides the state-of-the-art performance for compressing SMoE models on various tasks, achieving up to 2.43 percentage points higher average accuracy than the strongest baseline.

Abstract:
Equipping large language models (LLMs) with search engines via reinforcement learning (RL) promises effective search agents. However, adaptively balancing internal parametric knowledge with external search remains a challenge, as overreliance on search introduces unnecessary cost and risks exposure to noisy or malicious content, while relying solely on parametric knowledge risks hallucination. Prior efforts mitigate search overuse through tool-call reward shaping, which requires heavy reward engineering and conflates necessary and unnecessary search. To address these limitations, we revisit the evaluation of search agents through an F1-based decision metric, revealing that prior methods often overlook readily available parametric knowledge. Motivated by this, we propose AdaSearch, a simple two-stage, outcome-driven RL framework that disentangles problem-solving from the decision to search, making the decision process explicit and interpretable. Extensive experiments demonstrate that AdaSearch significantly improves search-decision quality and reduces unnecessary search calls, with only a small trade-off in QA accuracy relative to always-search.

Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains, yet they struggle with causal reasoning tasks. We introduce PACKaGE (Paragraph-level Annotated Causal Knowledge and Graph–tExt), a causal benchmark dataset that pairs paragraph-level texts with explicit causal graphs. Unlike prior sentence-level resources, our benchmark dataset (i) operates at paragraph level to capture various and interdependent causal pathways; (ii) offers multimodal annotations including full graphs and contextualized relations; and (iii) maps extended passages to comprehensive graphs. We detail a four-stage construction pipeline: collecting causality-labeled and fact-centric texts, inducing graphs from span-tagged pairs, condensing nodes, and recomposing path-aligned narratives. Overall, the paired evaluation highlights a persistent discrepancy between linguistic competence and causal understanding in LLMs, pointing to the need for further development.

Abstract:
LLM-based client simulation provides a scalable approach to novice counselor training, counseling-dialogue synthesis, and interactive evaluation of automated counseling systems. However, existing approaches are limited by insufficient profile diversity, weak behavioral grounding, and the lack of open Chinese-language resources for simulated counseling clients. We propose PsyCLIENT, a framework that conditions simulated-client responses on client profiles, dialogue histories, and conversational trajectories specifying target behaviors and content constraints at each client turn. We also construct PsyCLIENT-CP, a dataset of 120 Chinese client profiles spanning 60 counseling topics. Evaluations involving 24 professional counselors show that PsyCLIENT receives higher ratings of perceived authenticity and training utility than the comparison methods. In a separate source-identification study, PsyCLIENT dialogues were more frequently misclassified as human-client interactions than those generated by the baselines. These results suggest that conversational trajectory modeling can transform static client profiles into behavior-guided, dynamically unfolding counseling interactions.

Abstract:
As Large Language Models (LLMs) are increasingly deployed in multilingual contexts, evaluating their fairness across languages is essential. We investigate occupational gender bias in 16 LLMs across eight Indic languages using a controlled coreference resolution benchmark that employs transliterated English occupational loanwords to minimize grammatical gender cues. In addition to this generative evaluation, we compute intrinsic bias metrics (WEAT and DirectBias) from the models’ embedding spaces. Across models, we observe systematic disparities between pro-stereotype and anti-stereotype conditions. These gaps are largest in high-accuracy models, whereas models lacking baseline competence exhibit smaller gaps driven by general prediction uncertainty. Error analysis shows a consistent asymmetry, with higher error rates when the correct referent is female. However, intrinsic embedding bias metrics show no statistically significant correlation with these generative disparities ( p \gt 0.05 ). These results suggest that strong linguistic competence does not necessarily correspond to stereotype-neutral generation and that intrinsic embedding metrics alone may not reliably predict bias in multilingual language model behavior.

Abstract:
Large Language Models (LLMs) are typically adapted to downstream tasks through Supervised Fine-Tuning (SFT), where Cross Entropy (CE) loss is used to maximize the likelihood of ground-truth tokens. While effective for aligning with observed data and establishing strong domain grounding, SFT alone often limits the model’s ability to explore alternative plausible outputs, potentially narrowing its decision space. Conversely, Reinforcement Learning (RL) enables reward-driven exploration and refinement but relies heavily on a well-aligned initialization to achieve stable and sample-efficient training. In real-world deployments, practitioners frequently grapple with the question: How much SFT is sufficient, and when should RL take over? This work investigates the complementary trade-off between SFT and RL approaches such as GRPO, focusing on how both stages jointly influence model exploration and exploitation. We provide a quantitative analysis across multiple domains to understand how varying the extent of SFT influences a model’s exploratory behavior and downstream RL effectiveness. Our findings aim to guide in determining optimal SFT checkpoints (CKPTs) that balance domain alignment with exploration capacity, thereby maximizing final domain performance while minimizing overfitting after the joint post-training. Our data and code are open source at https://github.com/Shirley-Kokane/sft2rl.

Abstract:
As large language models (LLMs) are increasingly integrated into educational tools, current evaluations on standardized tests predominantly focus on binary outcome accuracy. Instead, an effective AI tutor must exhibit faithful reasoning, elucidate solution strategies, and diagnose specific human misconceptions. To bridge this gap, we introduce a pedagogical diagnostic framework that models English Standardized Test (EST) problem-solving as a traversal through a cognitive framework. Based on this framework, we present ESTBOOK, a multimodal benchmark encompassing 10,576 questions and 29 task types across five major exams. Unlike traditional datasets, ESTBOOK goes beyond data aggregation by enriching questions with formalized reasoning trajectories and distractor rationales that capture specific cognitive traps. Through extensive evaluations, we show that ESTBOOK identifies fine-grained reasoning bottlenecks and that lightweight mitigation based on these diagnoses provides evidence that the identified failure modes are actionable.

Abstract:
Multilingual large language models (LLMs) have been shown to perform better on non-English classification tasks when the representations of the given language are more aligned to English within the model. Several cross-lingual alignment (CLA) scores have been proposed for use with LLMs, along with multiple approaches for extracting embeddings from the models. We provide a comparative analysis of 27 CLA score variants, examining how they differ and how well each predicts downstream performance across three tasks. Crucially, while LLMs are widely used for generative tasks such as machine translation, prior work has focused almost exclusively on classification. We therefore investigate whether CLA scores are similarly predictive of translation performance. To enable computing correlations across target languages, we propose a PMI-based translation metric, which is less dependent on the target language and correlates strongly with CHRF. We find that CLA with English predicts translation quality comparably to or better than source–target CLA, providing new evidence that LLMs use English as an internal pivot language.

Abstract:
LLM-reranking is limited by the top-k documents retrieved by vector similarity, which neither enables contextual query-document token interactions nor captures multimodal relevance distributions. While LLM query reformulation attempts to improve recall by generating improved or additional queries, it is still followed by vector similarity retrieval. We thus propose to address these top-k retrieval stage failures by introducing ReBOL, which 1) uses LLM query reformulations to initialize a multimodal Bayesian Optimization (BO) posterior over document relevance, and 2) iteratively acquires document batches for LLM query-document relevance scoring followed by posterior updates to optimize relevance. After exploring query reformulation and document batch diversification techniques, we evaluate ReBOL against LLM reranker baselines on five BEIR datasets and using two LLMs (Gemini-2.5-Flash-Lite, GPT-5.2). ReBOL consistently achieves higher recall and competitive rankings, for example compared to the best LLM reranker on the Robust04 dataset with 46.5% vs. 35.0% recall@100 and 63.6% vs. 61.2% NDCG@10. We also show that ReBOL can achieve comparable latency to LLM rerankers.

Abstract:
Regret in large language models (LLMs) refers to their explicit expression of regret when confronted with evidence contradicting previously generated misinformation. Understanding the neural encoding of regret is crucial for advancing artificial metacognition and model reliability. However, three key gaps remain: the absence of specialized datasets for regret expressions, the lack of understanding of how early regret signals emerge in hidden states, and the unknown neuron-level architecture underlying regret encoding. To address these gaps, we make three contributions: (1) we propose a multi-stage workflow for constructing the first regret dataset and train probes achieving near-perfect accuracy across LLaMA-2-7B, Mistral-7B, and Vicuna-13B. (2) we discover that regret signals begin emerging in preceding hidden states before the regret token appears, and that probes generalize to semantically related keywords such as "incorrect" and "apologize." (3) at the neuron level, we introduce the Regret Dominance Score (RDS) and Group Impact Coefficient (GIC) to reveal a compositional regret architecture where cooperative neuron groups collectively encode regret, with larger models (70B) exhibiting more pronounced compositional effects. Targeted interventions on these neurons successfully suppress regret expression in generated text, validating our mechanistic understanding. Our code and data are available at https://github.com/AbnerAI/LLMs-Regret-Before-They-Say-It.

Abstract:
Open-Domain Timeline Summarization (Open-TLS) seeks to construct coherent evolutionary narratives for a topic by distilling information from large, unstructured web streams. Existing retrieve-and-generate pipelines often fail to capture logically necessary but semantically distant evidence, resulting in macro-level causal gaps (missing intermediate stages) and micro-level element gaps (underspecified milestones). We propose Active Timeline Completion (ATC), an iterative framework that converts implicit narrative gaps into targeted retrieval questions, enabling reflection-driven evidence acquisition. ATC leverages three complementary mechanisms—parametric causal gap detection, historical trajectory analogy, and dynamic ontology-guided completion—while a Logical Utility Verification module ensures only evidence directly addressing these gaps updates the timeline. Experiments on Open-TLS demonstrate that ATC consistently outperforms strong baselines in both content coverage and temporal alignment, showing that active, question-driven retrieval improves the completeness and coherence of open-domain timelines.

Abstract:
Current Vision-Language Models struggle with hours-long videos because processing full-length visual sequences induces prohibitive token explosion and attention dilution. To overcome this, we introduce MemDreamer to decouple perception and reasoning, shifting long-video understanding into an agentic exploration process. As a plug-and-play framework, it incrementally streams videos to construct a Hierarchical Graph Memory, a top-down three-tier architecture for semantic abstraction, anchored by a foundational graph capturing spatiotemporal and causal relations. During inference, the reasoning model employs agentic tool-augmented retrieval, navigating hierarchies, searching nodes, and traversing logical edges via an Observation-Reason-Action loop. Experiments show MemDreamer achieves SOTA results across four mainstream benchmarks, substantially narrowing the gap with human experts. It greatly reduces the reasoning context relative to full-context ingestion while delivering a substantial absolute accuracy gain. Furthermore, statistical analysis uncovers a strong positive linear correlation between an VLM’s performance on logic reasoning and long-video understanding benchmarks, establishing agentic capability scaling as a new paradigm for multimodal comprehension.

Abstract:
LLM agents increasingly operate in large tool ecosystems, where real-world tasks require discovering relevant tools, inferring implicit sub-goals, and adapting to dynamic environments over long horizons. However, existing benchmarks rarely evaluate planning under retrieval-limited tool visibility. To address this gap, we introduce PlanBench-XL, an interactive benchmark of 327 retail tasks over 1,665 tools that tests whether agents can iteratively retrieve usable tools, invoke them to uncover intermediate evidence for subsequent calls toward the final goal. PlanBench-XL further features an optional blocking mechanism that simulates real-world unpredictability through missing, failing, or distracting tool functions, forcing agents to detect disrupted paths and adapt at runtime. Experiments on ten leading LLMs show that massive-tool planning remains challenging: while GPT-5.4 achieves 51.90% accuracy in block-free settings, it collapses to 11.36% under the most severe blocking condition. Further analysis shows that agents are especially vulnerable when failures lack explicit error signals or when recovery requires longer alternative tool-use paths. These results establish PlanBench-XL as a testbed for diagnosing agentic planning failures and highlight the need for robust adaptive planning in long-horizon tasks with large, imperfect tool environments.

Abstract:
Accurate transcription of handwritten mathematics is crucial for educational AI systems, yet current benchmarks fail to evaluate this capability properly. Most prior studies focus on single-line expressions and rely on lexical metrics such as BLEU, which fail to assess the semantic reasoning across multi-line student solutions. In this paper, we present the first systematic study of multi-line handwritten math Optical Character Recognition (OCR), revealing a critical failure mode of Vision-Language Models (VLMs): over-correction. Instead of faithfully transcribing a student’s work, these models often "fix" errors, thereby hiding the very mistakes an educational assessment aims to detect. To address this, we propose PINK (Penalized INK-based score), a semantic evaluation metric that leverages a Large Language Model (LLM) for rubric-based grading and explicitly penalizes over-correction. Our comprehensive evaluation of 15 state-of-the-art VLMs on the FERMAT dataset reveals substantial ranking reversals compared to BLEU: models like GPT-4o are heavily penalized for aggressive over-correction, whereas Gemini 2.5 Flash emerges as the most faithful transcriber. Furthermore, human expert studies show that PINK aligns significantly better with human judgment (55.0% preference over BLEU’s 39.5%), providing a more reliable evaluation framework for handwritten math OCR in educational settings.

Abstract:
Are compounds and phrases constructed by the same combinatorial principles? This is a long-standing topic in linguistic theory. As a highly isolating language, Chinese offers a particularly revealing case: its word–phrase boundary is weakly marked, thus compound-internal relations often parallel phrase-level syntactic relations. Although this continuity is well described in Chinese linguistics, no operational test has yet examined whether the same principles are similarly represented across the two levels. PLMs provide a shared representational space for operationalizing this question. Motivated by this idea, we build CompPhrase-7, a parallel Chinese compound–phrase dataset covering seven shared relation types, and conduct a multi-dimensional probing study of Chinese-language PLMs, which separates three levels of cross-scale continuity: transferable feature information, aligned category geometry, and localized dimension-level causal support. The results show a clear dissociation: feature transfer is stable across models; category geometry depends strongly on pre-training regime; and localized causal support is not detected. The findings point to a differentiated view of Chinese combinatorial principles: compounds and phrases retain comparable relational cues, while lexicalization reshapes how those cues are organized once phrase-like structures become words. This study connects a long-standing linguistic question to a more precise computational representation, showing that continuity can surface as transferable information without shared geometry or localized causal implementation.

Abstract:
Prompt compression shortens LLM input to reduce inference cost, yet existing methods score token importance through LM forward passes. It remains questionable whether such nuanced, costly token selection is necessary. Compression requires identifying informative content, a problem that linguistic research has long addressed through cues that can be operationalized as deterministic rules. We therefore ask: can linguistic rules alone serve as effective prompt compressors, without LM-based scoring at compression time? To address this, we conduct offline evolutionary search over lexical, syntactic, semantic, and discourse seeds to find competitive rule combinations. The resulting linguistic compressor requires no LM forward pass at deployment and uses only CPU-side processing for compression. We evaluate it with a dual-path protocol to balance compression quality and reconstruction fidelity. Across short passages, multi-document reasoning, and dialogue-memory QA datasets, evolved compressors achieve performance similar to that of recent advanced prompt-compression strategies. Performance is strongest under light-to-moderate compression and degrades as compression becomes more aggressive, while the Direct and Reconstruction paths exhibit distinct patterns. Evolutionary analysis reveals that effective compression fuses signals across linguistic levels and, as the compression ratio increases, rules shift from token pruning to sentence extraction.

Abstract:
The proliferation of large language models (LLMs) has enabled a new class of psychologically grounded malicious comments, shifting fake news attacks from surface-level textual noise to deep cognitive and logical manipulation. This shift severely undermines existing detectors, which conventionally rely on static attack assumptions and fixed training distributions. To bridge this gap, we introduce CogniDir, an adaptive distributional learning framework that reformulates robust detection as a dynamic, optimal data mixture optimization problem for social media content safety. Grounded in cognitive psychology, we first formalize mechanism-specific cognitive adversarial paradigms to systematically expose deep-seated detector vulnerabilities. To address the vulnerability heterogeneity, CogniDir derives an information-theoretic score coupling empirical accuracy with probabilistic confidence, which is then parameterized into a Dirichlet prior. This formulation enables smooth, feedback-driven reallocation of training exposure, mathematically guaranteeing the optimal data mixture ratio toward the most brittle attack mechanisms. Experimental results on three benchmarks demonstrate that CogniDir yields state-of-the-art robustness, improving F1 scores by up to 17.9% over competitive baselines under heterogeneous, AI-generated adversarial pressures.

Abstract:
Speech translation for low-resource languages is constrained by the scarcity of high-quality parallel speech data, which is particularly pronounced in African linguistic contexts. To address this, we introduce , a parallel speech translation dataset covering Igbo, Hausa, , and Nigerian Pidgin paired with English. The dataset contains more than 40 hours of speech per language and includes several speaker and accent variations to reflect realistic settings. Using NaijaS2ST, we benchmark cascaded, end-to-end (E2E), and AudioLLM-based approaches across bidirectional translation directions (XX ↔ English). Our results show that few-shot audio LLMs are more effective than cascaded and E2E methods finetuned for speech-to-text translation. For speech-to-speech translation, however, cascaded and audio LLM paradigms perform comparatively, highlighting the need for stronger task-specific models. offers a strong foundation towards advancing research on low-resource multilingual speech translation. We open source the dataset here: [NaijaS2ST Huggingface Dataset](https://huggingface.co/datasets/McGill-NLP/NaijaS2ST); and the code here: [NaijaS2ST Code](https://github.com/McGill-NLP/naijas2st).

Abstract:
The application of large language models (LLMs) to personalized medical assistants has garnered growing interest. However, existing medical benchmarks largely rely on static question answering with pre-selected evidence, leaving unclear whether LLMs can make reliable clinical decisions from real longitudinal electronic health records (EHRs). To bridge this gap, we introduce ObGynLongBench, a rule-grounded long-context EHR benchmark for obstetric and gynecologic decision-making, comprising 1,500 clinical decision-point cases from 976 real pregnancy EHR histories and traceable rules. Each case is anchored to a patient, a pregnancy-timeline point, and a pre-decision information boundary, enabling Evidence-only, Visit-level EHR, and History-level EHR evaluation. Evaluating 17 LLMs reveals a substantial Evidence-to-EHR Gap: models perform well when evidence is directly provided, but accuracy drops when evidence must be extracted from same-day records or full pre-decision EHR histories. Further analyses identify evidence utilization as a key bottleneck: performance decreases with longer EHR contexts and more complex evidence requirements, and earlier failures often predict later failures within the same patient history. Finally, active-search agents perform best among EHR access strategies, highlighting patient-specific evidence utilization as a central challenge for reliable personalized medical assistants.

Abstract:
Chinese online comments often convey social meaning through indirect and playful language that is hard to interpret without context. Existing evaluations largely organize items around predefined phenomena or controlled pragmatic categories, leaving open whether models can distinguish plausible readings of what a naturally occurring comment is doing in a particular exchange. We introduce a benchmark for evaluating whether LLMs can recover such situated pragmatic meanings. From more than 200,000 public Chinese social media interaction records, we construct 4,735 human-validated diagnostic items, each pairing a target comment with reconstructed preceding context and plausible misreadings. We evaluate eight LLMs as both question writers and solvers in a cross-writer setting. The task is challenging: the strongest model achieves 81.42% leave-writer-out accuracy. Across all eight models, the mean leave-writer-out accuracy is 68.70% while human accuracy was 90.8%. Case analysis shows that models often recognize broad irony or playfulness while misidentifying the mechanism or interactional move.

Abstract:
Emergency departments (EDs) operate under time pressure, generating multimodal data such as clinical conversations, triage notes, and discharge documents. Recent advances in natural language processing (NLP), particularly pretrained transformers and large language models, have created new opportunities to support language and time-intensive stages of emergency care. Yet existing surveys map clinical NLP across the broader hospital workflow or focus on specific tasks. This survey analyses 46 papers spanning the three phases of ED: triage, diagnosis, and disposition, covering tasks such as triage classification, clinical summarisation, automatic diagnosis, report generation, and discharge documentation. We examine modelling paradigms, evaluation practices, and emerging benchmarks and shared tasks. Across tasks, we identify common trends, including a shift from task-specific neural architectures to pretrained language models, growing interest in interactive clinical systems, and increasing attention to clinically grounded evaluation. Finally, we detail open challenges such as limited generalisability, noisy clinical inputs, and workflow constraints that inform future ED-NLP research.

Abstract:
Causal interventions in language model representations have largely targeted discrete features, like grammatical number. However, language models must also make use of features that are graded. We introduce a method for causal intervention on continuous variables: given activation vectors paired with a graded target variable, we localize a low-dimensional direction for that variable and use this direction to edit vectors toward counterfactual target values. We apply this method to a continuous feature that is well-studied in psycholinguistics, namely verb bias (which reflects which syntactic structures tend to follow a given verb). We show that verb bias is causally represented in steering vectors extracted from large language models: counterfactual edits to verb bias systematically shift downstream structural preferences. Verb bias has also previously been linked to in-context learning; in further analyses, we find that steering vectors in our experiments encode error signals that could drive the error-driven update behavior seen in in-context learning but that these aspects of the steering vectors are not causally used in downstream production. Overall, these results show causal interventions can be applied to continuous variables, though connecting continuous variables to in-context learning remains a challenge.

Abstract:
Large Language Models (LLMs) are increasingly deployed in resume screening pipelines. Although explicit PII (e.g., names) is commonly redacted, resumes typically retain subtle sociocultural markers (languages, co-curricular activities, volunteering, hobbies) that can act as demographic proxies. We introduce a generalisable stress-test framework for hiring fairness instantiated in the Singapore context: 100 neutral job-aligned resumes are augmented into 4100 variants spanning four ethnicities and two genders, differing only in job-irrelevant markers. We evaluate 18 LLMs in two settings: (i) Direct Comparison (1v1) and (ii) Score Shortlist (Top-Score Rates), each with and without rationale prompting. We find that even without explicit identifiers, models recover demographic attributes with high F1 and exhibit systematic disparities, with models favouring markers associated with Chinese and Caucasian males. Ablations show language markers suffice for inferring ethnicity, while hobbies and activities are utilised for gender. Furthermore, prompting for explanations may paradoxically amplify bias. Our findings suggest that seemingly innocuous markers surviving anonymisation can materially skew automated hiring outcomes.

Abstract:
Large language models (LLMs) increasingly store user preferences in persistent memory to support personalization across interactions. However, in third-party communication settings governed by social and institutional norms, some user preferences may be inappropriate to apply. We introduce BenchPreS, which evaluates whether memory-based user preferences are appropriately applied or suppressed across communication contexts. Using two complementary metrics, Misapplication Rate (MR) and Appropriate Application Rate (AAR), we find even frontier LLMs struggle to apply preferences in a context-sensitive manner. Models with stronger preference adherence exhibit higher rates of over-application, and neither reasoning capability nor selectivity-oriented prompting fully resolve this issue. These results suggest that current LLMs treat preferences as globally enforceable rules rather than as context-dependent normative signals.

Abstract:
Reliable uncertainty estimation is a crucial requirement for deploying large language models (LLMs) and vision-language models (VLMs) in safety-critical settings, especially when the model parameters are not accessible (black-box). We propose BiG-SURE, an uncertainty estimator based on cross-temperature semantic agreement. The method samples low-temperature responses as stable semantic anchors and high-temperature responses as probes under meaning-preserving input transformations. It then constructs an anchor-probe Bipartite Graph (BiG) using NLI-based entailment scores and defines confidence through the normalized squared spectral energy of this matrix, with uncertainty given by its complement. This bipartite graph-based Semantic Uncertainty and Reliability Estimation (SURE) score measures whether high-temperature probes remain semantically aligned with the model’s stable low-temperature belief or not. We evaluate BiG-SURE on text QA, multilingual QA, and multimodal QA tasks across multiple model families. In these experiments, BiG-SURE improves average abstention AUROC over prior black-box uncertainty estimators, while remaining simple, unsupervised, and applicable to black-box model settings.

Abstract:
LLMs are increasingly embedded in programming workflows, from code generation to automated code review. Yet, how gendered communication styles interact with LLM-assisted programming and code review remains underexplored. We present a mixed-methods pilot study examining whether gender-related linguistic differences in prompts influence code generation outcomes and code review decisions. Across three complementary studies, we analyze (i) collected real-world coding prompts, (ii) a user study, in which developers solve identical programming tasks with LLM assistance, and (iii) an LLM-based simulated evaluation framework that systematically varies gender-coded prompt styles and reviewer personas. We find that gender-related differences in prompting style are subtle but measurable, with female-authored prompts exhibiting more indirect and involved language, which does not translate into consistent gaps in functional correctness or static code quality. For LLM code review, in contrast, we observe systematic biases: on average, models approve female-authored code more, despite comparable quality. Our experiments show that gender-coded prompt styles influence code length and maintainability. We also observe substantial variation in reviewer behavior across models. Together, these findings suggest that fairness risks in LLM-assisted programming arise less from code generation accuracy than from LLM-based evaluation, particularly as LLMs are increasingly used as automated code reviewers.

Abstract:
Recent advances in Retrieval-Augmented Generation enable LLMs to ground outputs in clinical evidence, but connections to external databases create the risk of contextual leakage, where unique combinations of medical details enable patient re-identification without explicit identifiers. Existing healthcare benchmarks emphasize accuracy while overlooking this risk. To fill this gap, we present MedPriv-Bench, the first benchmark for jointly evaluating privacy preservation and clinical utility in medical open-ended question answering. Our framework utilizes a multi-agent, human-in-the-loop pipeline to synthesize sensitive medical contexts and clinically relevant queries that create realistic privacy pressure. We also establish an automated evaluation protocol using a fine-tuned RoBERTa-NLI model, which achieved an instance-level F 1 -score of 75.3%, sensitivity of 90.7%, and an average inference time of 0.056 s per sample against human annotations. Across nine LLMs and three privacy-preserving methods, we observed a pervasive privacy-utility trade-off. Relative to unprotected Med42-v2-8B (utility 3.87/5; leakage 72.8%), supervised fine-tuning improved utility to 4.25 and reduced leakage to 38.9%, whereas local differential privacy reduced leakage to 20.5% but lowered utility to 3.03. These results demonstrate the need for domain-specific benchmarks to validate medical AI systems in privacy-sensitive settings.

Abstract:
Automatic ICD coding is an extreme multi-label classification task that requires assigning standardized diagnostic codes to clinical notes. Recent pretrained language model (PLM) based methods achieve strong performance, but they often overfit to repetitive semi-structured patterns in discharge summaries rather than learning clinically meaningful semantics. Moreover, existing approaches insufficiently utilize the rich semantic information contained in ICD code descriptions. In this study, we propose SARL-ICD, a structure-aware framework for automatic ICD coding centered around two key ideas: contrastive representation learning and label-text semantic interaction. First, we introduce JMoCo, a Jaccard-guided MoCo-style pretraining strategy that constructs positive pairs using ICD-code similarity while restricting pretraining to clinically informative narrative sections. This design encourages the model to learn patient-specific clinical semantics. Second, we propose a lightweight label-text interaction module that injects semantic information from LLM-augmented ICD code descriptions into document representations. We further employ an adaptive fusion module to combine complementary strengths of different ICD coding architectures. Extensive experiments on MIMIC-III-full and MIMIC-IV demonstrate that our method achieves state-of-the-art performance across multiple evaluation metrics. Additional analyses further show that JMoCo improves representation similarity alignment and reduces structural shortcut learning in semi-structured clinical documents.

Abstract:
We investigate the role of conversational context modeling in user preference tracking for Conversational Recommendation Systems (CRSs). In this regard, we propose DREAMS, a novel tree-structured context modeling framework that explicitly captures user preference evolution throughout multi-turn interactions. DREAMS introduces two specialized node types to support the two fundamental objectives of CRSs: preference elicitation and preference exploitation. Specifically, elicitation nodes leverage Monte Carlo Tree Search (MCTS) to strategically explore conversational actions and infer latent user preferences, while exploitation nodes employ LLM-based refinement to transform the tracked preference state into structured retrieval queries for recommendation. Extensive experiments on benchmark datasets demonstrate the effectiveness of DREAMS and its design.

Abstract:
Legal judgments may inadvertently contain errors, yet human appellate review faces immense pressure to improve efficiency. While current legal AI focuses on legal judgment prediction or document generation, judgment review is fundamentally a retrospective diagnostic task. In this paper, we introduce Appellate Review to assess large language models’ (LLMs) legal diagnostic reasoning underlying appellate review. We specifically target Chinese criminal cases, under the assumption that their highly structured logic, which maps explicit facts to strict statutory penalties, provides a verifiable ground truth. We decompose Appellate Review into three subtasks: error detection, error classification, and error correction. To support this, we construct AR-Bench, a controlled benchmark comprising 8,700 judgments with counterfactually introduced errors spanning charges, prison terms, and fines. Evaluations of 16 LLM variants reveal substantial limitations: models classify known errors relatively well but struggle with unguided detection and precise correction. Further analyses of a representative model show that it identifies explicit textual contradictions but fundamentally fails at holistic deductive reasoning and rule-based arithmetic.

Abstract:
Unified speech foundation models require a holistic tokenization space that is both learnable by language models and decodable into high-quality waveforms. Existing speech tokenizers, however, often fail to satisfy these requirements simultaneously, leading to increased architectural complexity and more involved training designs. We propose HoliTok, a continuous Holistic speech Tokenization model designed for unified generation-understanding modeling. HoliTok encodes 48kHz speech into a compact 25Hz sequence of 128-dimensional latents. It is trained with a progressive strategy that jointly preserves signal-level fidelity, incorporates semantic information, and maintains strong latent learnability. Based on this tokenization, we build a unified AR+DiT model for speech synthesis and recognition, where the same latent sequence supports both generation-specific and unified generation-understanding tasks. Experiments show that HoliTok achieves competitive reconstruction fidelity, improves generative learnability for high-quality and controllable synthesis, and, among the evaluated representations, is the only one that operates robustly in our unified generation-understanding architecture without additional optimization tricks. These results suggest that HoliTok serves as an effective speech tokenizer and a foundational representation interface for unified spoken language modeling. The code is available at: https://anonymous.4open.science/r/HoliTok-2852.

Abstract:
Automated operations research (OR) modeling requires LLMs to translate natural-language decision problems into correct mathematical programs. Existing methods can improve individual formulations, but they often solve problems in isolation, retaining little reusable experience and repeating similar formulation errors. Prior memory-based approaches store examples, thoughts, or insights as references, while OR modeling requires reusable formulation skills that transfer across problem narratives and guide concrete modeling decisions. We propose OptiSkill, a skill-augmented framework that builds a hierarchical and evolving SkillBank for LLM-based OR modeling. SkillBank stores solver-verified experience as reusable skills, with Global Strategies for problem-level formulation skeletons and Step Experiences for local error-prevention rules. It is further refined through stable batch-level test-time evolution, where candidate skills are incorporated only after validation. Experiments on eight OR modeling benchmarks show that OptiSkill improves formulation accuracy across LLM backbones, outperforms strong agentic baselines, and gains further by expanding SkillBank coverage and reliability. Code and data are available at https://anonymous.4open.science/r/OptiSkill-E16F/.

Abstract:
Multimodal memes are dynamic and often require up-to-date background knowledge for interpretation. Existing methods often overlook such knowledge or rely on fixed parametric knowledge of pre-trained models that may be incomplete, outdated, or unavailable for emerging memes. We introduce Query-Retrieve-Conclude, a zero-shot framework that identifies missing knowledge, retrieves open-web evidence, and synthesizes evidence-grounded background knowledge for meme understanding and detection. We also introduce a curated meme understanding benchmark of recent memes from 2024–2026 with external background knowledge annotations. Experiments on three meme understanding datasets and five meme detection tasks show that our framework improves knowledge recovery, meme understanding and downstream detection over zero-shot baselines.

Abstract:
Large language models encode social biases that instruction fine-tuning aims to suppress. Instruction fine-tuning applies its weight changes through the alignment task vector ΔW, yet prior work treats it as a monolithic object, leaving its internal layer structure opaque: which layers carry bias suppression, which carry general capability, and whether these are separable, remains unknown. We map this structure by injecting each layer’s ΔWᵢ independently into the base model across 11 LLMs from Qwen2.5, Gemma-2/3, and Llama-3 families (0.5B–14B). Testing via per-layer sufficiency sweeps, block removal, and a BBQ–MMLU dissociation, we find three results. First, BBQ-measured bias suppression localizes to the mid-third of transformer layers in 9 of 11 models, with mid-third removal raising bias across all 11 while first- and last-third removals produce smaller L-BBQ increases in all nine structured-signal models. Second, mid- and first-third removals dissociate BBQ and MMLU: mid-third removal is BBQ-critical across all tested models while first-third removal can sharply reduce MMLU in Gemma architectures, supporting functional separation of bias suppression and capability in weight space. Third, mid-third removal raises bias above the unaligned base in 3 of 11 cases: partial ΔW transfer without the mid-third can make a model more biased than the unaligned base model. These findings provide an empirical roadmap for targeted debiasing and safe task vector composition.

Abstract:
Reinforcement learning (RL) post-training has emerged as a promising paradigm for LLM-based Verilog generation. However, existing RL credit assignment methods are either overly coarse (sequence-level GRPO) or fragmented (line-level methods), leading to severe misattribution as they ignore the characteristics of Verilog designs that are fundamentally structured through concurrent constructs and procedural control flows. To bridge this gap, we propose CirCA, a novel RL framework that performs credit assignment based on Atomic Logic Blocks (ALBs), the smallest semantically complete units of circuit logic. CirCA decomposes Verilog into ALBs, derives reliable rewards from signal-level feedback, and then constructs logic-enhanced signal dependency cones to localize erroneous ALBs, whose localized responsibilities are further incorporated into GRPO through asymmetric advantage correction. Extensive experiments show CirCA establishes a new state-of-the-art across multiple benchmarks among non-reasoning models, and even surpasses some cutting-edge reasoning models on complex tasks.

Abstract:
Word sense disambiguation (WSD) is typically studied in synchronic settings, where the mapping from contextualized occurrences to sense labels is assumed to be temporally stable. This assumption may not hold for historical texts, in which meanings may emerge, shift, or disappear over time. We study diachronic WSD for Ancient Chinese, a challenging scenario involving long-term semantic evolution and substantial historical variation. We introduce Ancient Chinese Diachronic Word Sense Disambiguation Benchmark (AC-DiaWSD), a large-scale WSD dataset constructed from the Twenty-Four Histories, containing about 24K manually annotated instances across 12 dynasties from the Western Han to the Qing. We further propose DynaSense, a model-agnostic framework that enhances pre-trained language models with contrastive sense learning and dynasty-conditioned representation adaptation. Experiments with multiple PLMs and LLMs show that existing models struggle with diachronic WSD, while DynaSense consistently improves accuracy and F1 across backbones.

Abstract:
Tabular data, as a core data format in machine learning, often lacks the discriminative power needed for high-performance modeling due to insufficient feature informativeness. Automated Feature Engineering (AutoFE) overcomes this by automating feature generation and selection, ensuring both model performance and operational efficiency. However, traditional AutoFE often yield features with poor interpretability because they rely on blind mathematical transformations, while large language models (LLM)-based AutoFE faces challenges in requiring costly multi-round iterations to generate high-utility features to effectively enhance model performance, compounded by inherent risks of bias and hallucination. In this paper, we combine symbolic regression with LLMs for feature engineering (SymboLLM-FE) to solve these challenges. We extract mathematically expressive formulas strongly correlated with the target via symbolic regression, which can enhance model performance, then refine them by LLMs with rich prior knowledge to ensure interpretability. Empirical results on six real-world datasets and four Kaggle competitions demonstrate that SymboLLM-FE outperforms existing AutoFE. SymboLLM-FE also addresses the dual challenges of poor interpretability and numerous iterations by employing a statistical prior-grounded LLM refinement mechanism and single-digit LLM calls. Project Page: [LAMDA-NeSy/SymboLLM-FE](https://lamda-nesy.github.io/SymboLLM-FE/index.html)

Abstract:
Language priors in MLLMs can override visual evidence, impairing visual perception. Existing benchmarks assume shared priors across models and rely solely on accuracy. However, cross-model prior agreement falls below 80%, and over 15% of items can be answered correctly without any image, causing accuracy to overestimate prior resistance. To address these limitations, we introduce ViSiP-Bench, a benchmark grounded in rare-but-real scenarios across six prior categories, with a model-aware dynamic prior induction protocol and three metrics that more precisely and continuously measure prior resistance than accuracy alone. Evaluating 24 models reveals that prior adherence is pervasive, does not improve with scale, and larger models tend to either strongly override their priors or firmly adhere to them. We therefore propose MAPS, which identifies each model’s strongest priors and fine-tunes on factually grounded knowledge to soften excessive prior reliance without disrupting general capabilities.

Abstract:
Recent advances in visual question answering (VQA) and multimodal large language models (MLLMs) have enabled natural-language reasoning over traffic scenes. However, existing benchmarks are largely built from ego-vehicle views or 2D roadside videos, limiting their ability to evaluate 3D-grounded reasoning over real-world distances, trajectories, infrastructure topology, and safety-critical interactions. We introduce Inter-3D VQA, a large-scale roadside multimodal benchmark for 3D spatiotemporally grounded VQA at intersections. Built from synchronized point clouds and multi-view images, Inter-3D VQA contains 407K QA pairs covering lane-level positions, object relationships, motion patterns, and near-miss-oriented interaction reasoning. We further propose Inter-Geo, an MLLM baseline that integrates object- and scene-level aligned LiDAR representations, and Inter-Metrics, a unified evaluation framework for textual consistency, numerical accuracy, and semantic correctness. Experiments show that Inter-Geo outperforms image-based VLMs, especially on grounded spatial and temporal reasoning tasks.

Abstract:
Production RL pipelines for large language models routinely violate the on-policy assumption along three independent axes. Partial rollouts resume long chain-of-thought trajectories after a parameter update. Rollout replay re-injects past successful trajectories. Asynchronous training decouples sampling from optimization, so trajectories arrive several updates stale. Today these biases are absorbed by a single, theoretically unjustified instrument: a token-level PPO clip whose magnitude is invariant to the actual policy divergence. We show that all three regimes are instances of one mathematical object: a trajectory composed of segments produced by distinct behavior policies. Building on this view, we introduce SPIRE (Segment-wise Prefix Importance with Replay Equilibrium), a drop-in replacement for the policy-gradient estimator in GRPO-style optimizers. SPIRE is: 1. Unbiased under a prefix-measurable boundary condition; 2. Variance-controlled, admitting a tight per-segment second-moment bound via Renyi-2 divergence with a closed-form MSE-optimal clip; 3. Replay-equilibrium aware, admitting a closed-form Pareto-optimal replay/fresh mixing ratio from a KKT analysis of the constrained MSE/throughput problem. On Qwen2.5-Math-1.5B, Qwen2.5-7B, and Llama-3.1-8B trained with verifiable rewards on MATH-500, AIME-2024, OlympiadBench, LiveCodeBench-v5, and GPQA-D, SPIRE delivers +3.6 to +6.8 absolute points on AIME-2024 and +2.1 to +4.4 on OlympiadBench at iso-FLOPs, while remaining stable up to 4x trainer/sampler staleness, where AReaL+PPO collapses. Baselines marked with a double-dagger symbol are reproduced verbatim from the corresponding paper on the matching model-benchmark pair; SPIRE numbers reflect the model’s own runs released with this submission.

Abstract:
Multimodal large language models (MLLMs) have shown strong performance in perception and question-answering, but many clinical decisions require reasoning across observations collected over time rather than from a single input. We study this problem in longitudinal modeling, where evidence evolves across visits. We introduce LUMINA (LongitUdinal MultImodal reasoNing Agent), a single-agent framework that incrementally builds an auditable evidence state before making predictions. Instead of predicting directly from all observations, LUMINA organizes current evidence, compares observations across prior visits, maintains longitudinal memory, optionally retrieves relevant cross-subject summaries, and verifies whether the accumulated evidence supports the final decision. We evaluate LUMINA on two subject-level benchmarks: Alzheimer’s disease classification and 36-month progression forecasting from observed visit prefixes. Across nine MLLM backbones, LUMINA consistently outperforms MLLM baselines and an adapted medical reasoning agent, achieving 13.8%–18.5% relative balanced accuracy gains on classification and 16.8%–24.8% gains on progression forecasting over the strongest non-agent baseline. Additional evaluations preserve the same method ordering on Parkinsonian-state classification and breast cancer treatment-response prediction.

Abstract:
Causal relation extraction (CRE) is central to biomedical text mining, but current resources often conflate causal relations with broader associations, restrict annotation to sentence-level examples, or focus mainly on explicit causal cues. This limits their usefulness for evaluating whether models can recover causal claims as they are actually expressed in biomedical text. We introduce PubMedCausal, a span-level annotated corpus for biomedical CRE built from PubMed abstracts. The corpus contains 30,000 paragraph-level rows, including 3,945 causal rows and 6,491 adjudicated cause–effect pairs. Each causal relation is annotated with full-text cause and effect spans, causality type, and sententiality, enabling evaluation of both causal detection and full-span causal extraction. We benchmark discriminative encoders and open-source generative models across detection and extraction settings. For causal detection, biomedical encoders are strongest, with PubMedBERT reaching an F 1 score of 0.7391. For span-level extraction, the best generative baseline is DeepSeek-R1-32B with few-shot prompting, reaching a Cosine Pair F 1 of 0.6765. We further test transfer learning by evaluating PubMedCausal-trained encoders on external causal relation datasets, showing that the resource supports cross-dataset evaluation beyond its original biomedical setting. Our results show that biomedical CRE remains difficult under class imbalance, long causal spans, implicit causality, inter-sentential relations, and prompt sensitivity. PubMedCausal provides a high-quality benchmark for studying causal detection, full-span extraction, and the limits of current language models in biomedical causal reasoning.

Abstract:
Mixture-of-Experts (MoE) Transformers improve scaling efficiency by activating only a subset of experts per token, but deployed inference remains constrained by memory capacity and bandwidth. Existing expert compression methods face a quality–memory–latency trilemma: pruning or fully merging degrades model quality, while partial merging increases per-token activated parameters and hurt latency. We propose NimbleMoE, a training-free, collaboration-aware partial expert merging framework that reduces both model size and activated-parameter cost. NimbleMoE clusters experts by routing co-activation so co-activated experts share a low-rank subspace. To compensate for the capacity lost, NimbleMoE adds hardware-efficient Monarch residuals, with an alignment-aware choice between latent-space and output-space compensation. Across multiple MoE backbones, NimbleMoE preserves model quality under 20–25% total-parameter compression and outperforms prior pruning and merging baselines, while delivering up to 1.89 × MoE-MLP kernel speedup in bandwidth-bound settings, 1.31 × end-to-end TPOT speedup, and 2.81 × expert-parallel MoE-kernel speedup.

Abstract:
Narratives are central to how social communication is framed, making their detection critical for understanding and analysing public discourse. Prior work has explored narrative detection and extraction across diverse domains; however, migration narratives remain significantly understudied, primarily due to the absence of dedicated annotated datasets. Furthermore, public communication has recently shifted towards video-centric platforms, where narratives are conveyed through multimodal signals and consumed at scale. Despite this shift, narratives in videos remain largely unexplored. To bridge these gaps, we introduce MigrationNarrate, the first multimodal dataset for detection of migration narratives in the UK, consisting of 1,115 YouTube video transcripts annotated using a two-level taxonomy of 12 migration super-narratives and 53 narrative labels. This paper details the dataset design, collection, and annotations, together with benchmark results using a combination of pre-trained encoder models and both open- and closed-source Large Language Models. Finally, a thorough error analysis offers insights for future work.

Abstract:
Model compression is increasingly essential for deploying large language models (LLMs), yet existing comparative studies largely focus on pruning and quantization evaluated primarily on knowledge-centric benchmarks. Thus, we introduce UniComp, a unified evaluation framework for comparing pruning, quantization, and knowledge distillation. UniComp evaluates compressed models along three dimensions: performance, reliability, and efficiency, using a diverse set of capability- and safety-oriented benchmarks together with a hardware-aware efficiency analysis. Through evaluation of seven compression techniques across over 40 datasets, we observe (i) a consistent knowledge bias, where factual recall is largely preserved while multi-step reasoning, multilingual, and instruction-following capabilities degrade; (ii) a deployment-critical performance-reliability decoupling, where retained performance does not indicate preserved safety, fairness and privacy; and (iii) that task-specific calibration can yield up to 50% relative improvement in reasoning performance in pruned models.

Abstract:
Cultural evaluation coverage and robustness in language models are difficult to diagnose because pretraining corpora and cultural benchmarks are rarely indexed with comparable metadata. Benchmarks increasingly target culturally situated phenomena at the level of languages, regions, and locale-specific practices, while web-scale corpora are usually organized only by language. A shared culture–language–region layer makes these resources comparable, enabling audits of whether a target cultural phenomenon is represented in pretraining data, evaluated by benchmarks or both. To this end, we introduce FineWeb-CLaR, a large-scale annotated dataset derived from FineWeb and FineWeb-2 that places web documents on a shared culture–language–region axis for corpus auditing and benchmark alignment. FineWeb-CLaR annotates the full 30.9B-document collection from FineWeb and FineWeb-2 with URL-derived region labels and cultural-topic provenance. Our region resolver assigns a non-empty region to 25.61% of documents (7.92B). For cultural-topic analysis, we induce locale-specific topics and project them onto the 14 leaves of the Cultural Taxonomy of Liu et al. (2025), producing Locale Topic Distributions (LTDs) for corpus-side comparison. We also annotate 277 cultural NLP benchmarks with the same taxonomy, language coverage, and region coverage. Together, these resources enable direct comparison between corpus-side pretraining evidence and benchmark-side evaluation coverage.

Abstract:
This paper presents MobQA, a benchmark dataset designed to evaluate the semantic understanding capabilities of large language models (LLMs) for human mobility data through natural language question answering. While existing models predict human movement patterns well, their ability to interpret underlying reasons or semantic meaning remains unclear. To address this gap, we annotated 14,260 trajectory points from the GeoLife dataset with 30,703 semantic labels covering points of interest, mobility modes, and visit reasons, grounding semantics-dependent questions and reference answers. Building on these annotations, MobQA provides 5,800 question–answer pairs from daily and weekly GPS trajectories, spanning factual retrieval, multiple-choice semantic inference, and free-form interpretive explanation. Our evaluation of major LLMs reveals strong performance on factual retrieval but notable limits in semantic reasoning and explanation, with trajectory length substantially impacting effectiveness. These findings demonstrate both achievements and limitations of recent LLMs for semantic mobility understanding.

Abstract:
The rapid growth of scientific literature makes it increasingly difficult to communicate a paper’s core novelty in a concise and accessible manner. While prior work has explored visual summaries such as graphical abstracts, emerging informal formats such as scientific memes remain largely unstudied from a computational perspective. Scientific memes rely on concise visual–textual juxtaposition and contrast to highlight how new work differs from prior approaches, making them a challenging test case for controlled abstraction, contrastive reasoning, and creative compression in generative models. We formalize scientific meme generation as a novel multimodal, contrastive generation task and propose SciMemeX, a modular multi-agent framework that extracts core innovations, selects contextually appropriate meme templates, and iteratively generates and refines captions through contrastive feedback-guided refinement. We evaluate generated memes along three complementary dimensions— scientific fidelity , clarity , and engagement potential —and conduct extensive automatic and human evaluations on 1.5k computer science papers. Our results show that SciMemeX consistently outperforms strong single-agent prompting and existing multi-agent baselines across all evaluation dimensions, suggesting that scientific meme generation provides a constrained setting for exploring controlled creative generation in scientific communication context. We make the code public.

Abstract:
Large Language Models (LLMs) are increasingly used in advice seeking and decision making that may affect social judgements. Despite stigma’s profound effects on people and communities, benchmarks remain scarce. Existing general-domain evaluations typically rely on static prompts and fixed-format tasks, overlooking conversational contexts and audience effects in everyday communication. To address these gaps, we introduce SDARE-Bench, the first scenario-based benchmark evaluating both stigma detection and open-ended response generation in LLMs, comprising 1,138 dyadic queries and 1,388 group dialogues. Empirical results across 8 LLMs consistently demonstrate poor identification of stigma components, especially in group dialogues. In open-ended response generation, stigma expression was substantially higher in group settings than in dyadic, with weaker resistance to stigma and more unrealistic advice. Responses were evaluated using a classifier trained on 1,392 human annotated responses. In constructed group pressure settings, stigma expression rates further increased to a striking average of 97.5%. Our findings identify stigma response as a recurring LLM safety vulnerability, especially in socially complex conversational contexts.

Abstract:
The emergence of writable, cross-session persistent memory in LLM agents introduces a qualitatively different threat landscape from conventional input-centric security concerns, characterized by three properties: persistence, statefulness, and propagation. To systematically characterize this landscape, we propose a Memory Lifecycle Framework that organizes attacks, defenses, and their cross-phase dependencies along two axes: six lifecycle phases (Write, Store, Retrieve, Execute, Share Propagate, Forget Rollback) and four security objectives (Integrity, Confidentiality, Availability, Governance). This analysis in turn exposes the need for formal security guarantees at the system level, motivating Verifiable Memory Governance (VMG), a framework of five architectural primitives that specifies what verifiable mechanisms a long-term-memory system must provide to maintain auditable, recoverable control over its memory state. Our analysis indicates that robust Long-Term Memory (LTM) security cannot be retrofitted at retrieval or execution time alone, but must be anchored in storage-time provenance, versioning, and policy-aware retention from the outset.

Abstract:
Diversity in LLM mathematical reasoning is critical for exploration, but common diversity metrics mostly capture surface-level variation rather than differences in how a problem is solved. We address this gap by introducing approach-level diversity : variation in strategies across correct solutions to the same problem. Using a human-calibrated LLM judge framework, we show that prior diversity measures are unreliable proxies for approach-level diversity, and this mismatch carries over to diversity-aware RLVR, where target metrics are preserved while approach-level diversity declines. Investigating when approach-level diversity helps and whether it can be directly induced, we find that approach-diverse candidate sets improve test-time scaling. However, optimizing an LLM judge diversity reward during training causes the policy to exploit judge-specific preferences rather than broaden its approaches, leaving direct optimization of approach-level diversity as an open problem. Together, our work introduces the notion of approach-level diversity and uncovers a systematic divergence between surface- and approach-level signals, marking a step toward LLMs that reason in genuinely diverse, human-like ways.

Abstract:
Despite their widespread adoption, the robustness of fact-checking models to adversarial perturbations remains underexplored. Existing approaches are model-specific and require gradient access or dataset-dependent optimization, with implications for generalization, efficiency, and semantic validity. We introduce FactFlip, a framework for analyzing robustness in claim verification models via universal adversarial triggers. FactFlip identifies perturbative trigger words through a lightweight, model-only procedure that ranks candidates by the alignment of their non-contextualized embeddings with class-specific decision directions, without labeled data or gradient access. Trigger discovery is decoupled from claim perturbation via an LLM-based perturb-and-verify pipeline that preserves semantic validity. Experiments show that FactFlip discriminates between strong and weak triggers, composes them constructively, and rivals fully supervised gradient-based baselines. We further show that FactFlip doubles as a diagnostic lens, surfacing socially salient model biases without any task-specific supervision.

Abstract:
To avoid moderation and surveillance on social media, some users routinely invent indirect linguistic expressions (ILE) that camouflage sensitive meanings. Such disguised expressions surface as algospeak, euphemisms, and adversarial obfuscation, depending on intent and context, and often involve recurring encoding mechanisms. We propose a comprehensive, mechanism-oriented taxonomy of ILE that abstracts away from communicative goals and instead categorizes the underlying operations through which meaning is encoded and recovered. We evaluate the taxonomy by incorporating it into large language model (LLM) prompts and comparing it with four existing taxonomies and a no-taxonomy baseline, using 2,000 manually annotated TikTok and Bluesky posts. The proposed taxonomy attains the strongest document- and span-level performance across three LLMs. Compared with the best-performing benchmark, its document-level gains are +3.8 accuracy/+4.3 macro-F1 percentage points for GPT-5.4, +0.9/+1.3 for Sonnet 4.6, and +2.1/+2.2 for DeepSeek V4. While the direction of improvement is consistent, the magnitude is model-dependent. The span-level gains are mainly driven by higher recall, sometimes at the cost of precision. In an additional cross-lingual evaluation of 800 Chinese coded-language reviews, the unadapted taxonomy also achieves the highest document- and span-level F1 across the three LLMs. The empirical results reveal the importance of a comprehensive, mechanism-oriented taxonomy as a stable scaffold for detecting emerging coded language and a useful input to content moderation.

Abstract:
Incomplete-Modality Multimodal Sentiment Analysis extends multimodal sentiment analysis (MSA) to infer sentiment from language, audio, and visual streams when one or more modalities are incomplete at inference time. Existing methods improve incomplete-modality MSA through feature reconstruction, distillation, or fusion adjustment, but usually assign similar correction strength to all samples with incomplete inputs. This overlooks the uneven emotional impact of modality missingness: mild missingness often leaves sentiment recognizable, whereas severe missingness can distort the expressed emotion and therefore requires stronger correction. We propose HyDRA (Hyperbolic Damage-aware Recovery and Adaptation), a damage-aware and text-guided hyperbolic framework that aligns correction strength with missingness-induced sentiment damage. HyDRA first recovers lost emotional cues from available modalities, then places each sample in the Poincaré ball according to estimated damage severity, and finally applies damage-weighted hyperbolic correction toward a complete-modality teacher anchor. Since hyperbolic distances expand near the boundary, heavily incomplete samples receive stronger correction while lightly incomplete samples are gently adjusted. Experiments on MOSI, MOSEI, and SIMS show SOTA robustness across incomplete-modality conditions.

Abstract:
To discover the weaknesses of LLMs, researchers often embed prompts into a vector space and cluster them to extract insightful patterns. However, vector embeddings primarily capture topical similarity. As a result, prompts that share a topic but differ in specificity, and consequently in difficulty, are often represented similarly, making fine-grained weakness analysis difficult. To address this limitation, we propose Prompt2Box:, which embeds prompts into a box embedding space using a trained encoder. The encoder, trained on existing and synthesized datasets, outputs box embeddings that capture not only semantic similarity but also specificity relations between prompts (e.g., "writing an adventure story" is more specific than "writing a story". We further develop a novel dimension reduction technique for box embeddings to facilitate dataset visualization and comparison. Our experiments demonstrate that box embeddings consistently capture prompt specificity better than vector baselines and achieve up to 45% error reduction compared to the prompt length baseline. On the downstream task of creating hierarchical clustering trees for 17 LLMs from the UltraFeedback dataset, Prompt2Box: can identify 13.5% more LLM weaknesses than vector baselines and achieves an approximately 33% stronger correlation between hierarchical depth and instruction specificity.

Abstract:
Safety evaluations for large language models rely on surface-form classifiers that report declining harm scores across model generations. We provide evidence that this methodology is systematically incomplete: explicit discriminatory content is transformed rather than removed. We call this harm laundering. Analysing 450,000 gender-directed completions across 15 models spanning GPT-2 through GPT-5 (OpenAI GPT lineage; three demographic conditions), we show that sexual violence clusters prevalent in GPT-2 women-directed output disappear by GPT-4, while men-directed completions gain positive representational territory (caregiving, emotional range, ally identity) that women-directed completions do not. The pattern is most visible at GPT-5: Topic 5 (1,997 documents) frames breast cancer as a men’s rights debate, while zero equivalent clusters appear in women-directed output. Three independent classifiers score this content as non-toxic. Sentiment scores invert at GPT-4: early models demean women; later models over-correct. Topic diversity in women-directed completions falls 36% relative to men at the GPT-4 alignment boundary (W/M = 0.58, from 0.91 at GPT-2). REGARD representational harm disparity correlates with release date (ρ = +0.55, p = .034) while Detoxify does not (ρ = −0.23, p = .42): toxicity scores fall as representational harm grows. We formalise harm laundering as a three-criteria test and provide a three-stage detection protocol applicable to any generative model. Within the OpenAI GPT lineage, toxicity score reduction is not a sufficient proxy for harm reduction.

Abstract:
Quantizing optimizer states is becoming an important ingredient of memory-efficient large-scale pre-training, but the resulting optimizer dynamics remain only partially understood. We study low-precision exponential moving average (EMA) optimizer states and show how quantization can cause many nominal updates to round back to the same stored value, making the state effectively stale and slowing adaptation beyond what the nominal decay would suggest. We then develop a simple predictive model of stalling that estimates one-step stalling probabilities and characterizes how stalling builds up over time after the initialization. This perspective provides a mechanistic explanation for why optimizer-state resets help in low precision: once a quantized EMA becomes effectively stale, resetting it can temporarily restore responsiveness. Motivated by this picture, we derive a simple theory-guided heuristic for choosing useful reset periods, showing that in low precision the key question is not only whether resets help, but when they should be applied. Experiments in controlled simulations and LLM pre-training show that suitable reset schedules recover the performance lost to low-precision state storage while substantially reducing optimizer-state memory.

Abstract:
Aligning Large Language Models (LLMs) for specialized domains presents a fundamental optimization challenge: the “alignment tax.” In biomedicine, this manifests as a conflict between the need for rigorous, encyclopedic factual accuracy and the requirement for flexible, user-friendly instruction following. Existing methods, primarily relying on Supervised Fine-Tuning (SFT) or standard Reinforcement Learning from Human Feedback (RLHF), often fail to navigate this Pareto frontier, resulting in models that are either knowledgeable but rigid, or chatty but hallucination-prone. In this paper, we propose BalancedBio, a novel alignment framework that explicitly models and optimizes conflicting objectives. While utilizing standard high-quality instruction data, our contribution lies in the algorithmic innovation: (1) We introduce Capability-Aware Group Relative Policy Optimization (GRPO), which eliminates the need for a value network and reduces gradient variance in high-entropy reasoning tasks; (2) We propose a Dynamic Hybrid Reward Mechanism that adaptively balances domain correctness, reasoning validity, and format compliance during training; (3) We provide a theoretical analysis demonstrating how our method enforces gradient orthogonality to mitigate catastrophic forgetting. Extensive experiments on BIOMED-MMLU, MedQA, and IFEval show that BalancedBio-7B achieves state-of-the-art performance, surpassing Med-PaLM-8B in domain tasks while maintaining robust general instruction-following capabilities. We will release our model and partial data.

Abstract:
Large language models (LLMs) are widely used to answer users’ information needs: users ask LLMs about the weather, pose educational questions, and consult them for legal assistance. One particularly understudied area is digital security and privacy (S P), where users may seek LLMs’ help on how to secure their online accounts or protect their computers from cyber attacks. To the best of our knowledge, no prior study has collected or analyzed the S P questions users ask LLMs; prior research on LLM response quality relied on expert-authored S P misconceptions or FAQs rather than user queries. Drawing from WildChat, a dataset of 3.2M user-LLM conversations collected in the wild, our study identifies 14,727 S P prompts and categorizes them into nine categories covering a wide range of S P topics. We sampled 450 S P prompts, and performed a thematic analysis to characterize the S P questions users ask LLMs. Separate from the thematic analysis, we curated 270 advice-seeking S P prompts, where users ask for recommendations, guidance, or specific S P information. We measured LLM response quality and consistency when posing each prompt to the LLMs 10 times. We found that commercial LLMs outperform open-weight models (GPT 5.5 provided "good enough" responses on 98% of prompts; Llama 4 on 47%). However, among prompts that received high-quality responses on average, commercial models sometimes produce semantically inconsistent responses across runs (some are contradictory, and others are different in specificity), thereby risking confusing or misleading users.

Abstract:
With the surge of AI-powered real-world applications, large language models (LLMs) increasingly adopt memory-centric mechanisms to handle multi-turn interactive tasks. Under this paradigm, intermediate processing trajectories, i.e., memory, are central to ensuring LLMs’ faithfulness and effectiveness. Scalable evaluation of the memory management process serves as the cornerstone for further enhancing model performance. Yet, appropriate evaluators for the assessment of memory are currently lacking. Thus, in this work, we seek to employ reward models (RMs) as evaluators and introduce MemoryRewardBench, the first benchmark to systematically study the capability of RMs in assessing LLM memory management processes. MemoryRewardBench covers both long-form comprehension and generation tasks that simulate real-world multi-turn interaction scenarios, featuring 10 distinct settings with different memory management patterns. Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming their predecessors regardless of parameter count. Additionally, we reveal the fundamental limitations of RMs in evaluating LLMs’ memory.

Abstract:
Speech Retrieval-Augmented Generation (Speech RAG) systems are increasingly used in real-world settings where spoken queries vary across speakers, emotions, and acoustic conditions. However, current evaluations primarily focus on clean-condition retrieval accuracy, leaving unclear whether these systems remain semantically invariant under paralinguistic variation. We introduce ParaSpeechRAG, a counterfactual robustness benchmark that isolates non-semantic acoustic factors while preserving lexical content. The benchmark spans controlled perturbations across speaker identity, emotion and prosody, additive noise, speaking rate, and codec compression, enabling systematic evaluation of semantic invariance in speech retrieval. We evaluate three retriever families: ASR-based cascades, speech-text multimodal retrievers (CLASP), and general-purpose audio-language models (GLAP). Results reveal a consistent robustness–accuracy trade-off: multimodal retrievers achieve higher performance under clean conditions but are substantially more sensitive to paralinguistic and acoustic perturbations, particularly emotion and noise. In contrast, ASR-based cascades are more robust to these variations due to their text-centric bottleneck, though with reduced expressive capacity. These findings highlight an overlooked dimension of instability in Speech RAG systems and underscore the need for robustness-aware evaluation beyond clean-condition benchmarks.

Abstract:
Calculating semantic textual similarity is a foundational task in natural language processing. Current large language models (LLMs) based methods typically rely on extracting last-layer hidden states with fixed dimensions to compute similarity for every text pairs. We argue that this paradigm is suffer from two limitations: (i) The last hidden layer encodes more general knowledge rather than just semantic knowledge, making it suboptimal for semantic similarity computation; (ii) The hidden layer dimensions of LLMs are generally very large, which introduces some redundancy and noise for representing semantics. In this work, we propose DySem, a novel training-free framework that investigates more semantic-related internal components of LLMs via multilingual consensus, and shifts away from static representation spaces in favor of dynamic, sample-specific semantic dimensions by constructing text-dependent joint semantic set and computes similarity over this shared dimensional subset. Extensive experiments across various LLMs show that our method consistently outperforms recent baselines while maintaining lower dimensions for similarity calculation.

Abstract:
Accountability means a decision can be examined, justified, and contested. LLMs make this hard: fluent output may be ungrounded, incomplete, or unfaithful to the decision process. Achieving accountability requires verified rationales (how was the decision reached), assumptions (what was assumed rather than known), policy consistency (the same treatment for the same facts), and pivotal conditions (what would change the outcome). We introduce self-faithfulness as an automatic test of accountability: changing the pivotal conditions should change the decision. We examine accountable AI through clinical trial matching, a high-stakes task central to evidence-based medicine. Although LLM-based matchers match patients to trials reasonably accurately, they apply decision policies inconsistently and produce rationales that are unfaithful to their own decisions. We introduce VERDICT, an LLM-based agent that translates a decision task, its constraints, and its policy into Satisfiability Modulo Theories (SMT), then derives the decision with SMT and MaxSMT solvers — so policies are applied consistently and decisions are accountable by construction. Across a SIGIR 2016-derived dataset and TREC 2021, VERDICT achieves the strongest decision accuracy among LLM-only and neurosymbolic baselines, applies policies with perfect consistency, and produces clinician-preferred rationales grounded in explicit assumptions and pivotal conditions, with improved counterfactual self-faithfulness.

Abstract:
We present Vision-Braille, the first publicly available end-to-end system for translating Chinese Braille extracted from images into written Chinese. This system addresses the unique challenges of limited annotated resources and tone omission. The task is out of reach for general-purpose LLMs: given the complete Chinese Braille encoding specification and a worked example in context, Qwen3-235B-A22B reaches only 9.46 BLEU, and the only prior Braille-to-Chinese model with released checkpoints scores 5.36 BLEU on passage-level input. Braille is effectively absent from pretraining corpora, decoding it demands exact multi-step symbolic reasoning, and writers omit up to 90% of tone markers, leaving homophones that must be recovered from context. Vision-Braille integrates a Braille OCR pipeline with an open-weights LLM fine-tuned on a synthetic Braille-Chinese corpus that reproduces authentic tone-omission habits, using a four-stage curriculum that moves from short, fully toned input to long passages with tone markers almost entirely stripped. On passage-level translation with 10% tone retention, Curriculum-Braille achieves 83.28 BLEU at a 9.94% character error rate, and 76.66 BLEU on passages hand-transcribed by an expert from the Chinese Braille Library, with no reliance on any proprietary API. For the 150,000 school-age blind children in China, Vision-Braille offers an inclusive NLP solution that supports participation in mainstream education by enabling teachers to grade Braille homework without extensive training.

Abstract:
Accounting for disagreement among annotators is important for a variety of NLP applications. However, understanding what insights can be drawn from disaggregated annotations remains an open question. We introduce FLOCKE, a framework that jointly learns a task and models latent groups of annotators with similar behavior. We first formalize six analyses enabled by our framework, including annotator-group discovery, group- and individual-level label prediction, and finding the items most prone to cause disagreement. We then validate FLOCKE with a controlled synthetic benchmark, creating predefined annotator groups and labeling patterns, and showing that it can recover group structure with very high accuracy. We apply FLOCKE to two real-world datasets, where it predicts individual annotations, discovers interpretable annotator groups, and shows that groups are not explained by socio-demographic traits alone.

Abstract:
Audio large language models (AudioLLMs) enable instruction following over speech and general audio, but progress is limited by the scarcity of diverse, conversational, and instruction-aligned speech-text data. This gap is particularly pronounced for persona-grounded and dialectal interactions, where collecting real multi-speaker recordings remains costly and slow. We introduce MENASpeechBank, a reference speech bank comprising ∼ 18K high-quality utterances from 124 speakers spanning multiple MENA countries, covering English, Modern Standard Arabic (MSA), and regional Arabic varieties. We develop a controllable data pipeline that (i) constructs persona profiles enriched with World Values Survey (WVS) inspired attributes, (ii) defines a taxonomy driven ∼5K conversational scenarios, (iii) matches personas to scenarios via semantic similarity, (iv) generates ∼ 417K role-play conversations with an LLM where the user speaks as the persona and the assistant behaves as a helpful agent, and (v) produces speaker-conditioned user-turn audio (synthetic) from reference recordings to preserve speaker diversity. We evaluate synthetic and human recorded conversations and provide an analysis. We will make the MENASpeechBank available for the community.([https://huggingface.co/datasets/QCRI/MenaSpeechBank](https://huggingface.co/datasets/QCRI/MenaSpeechBank))

Abstract:
While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understanding remains challenging. Existing benchmarks often rely on question answering or ground-truth caption matching, where models may succeed through superficial cues and incomplete annotations. To this end, we introduce VidOmni-Bench, a benchmark that requires models to verify whether each event in dense video captions is supported by the video. VidOmni-Bench consists of 500 videos spanning five complexity types and diverse durations from 4 seconds to 90 minutes. After collecting videos along these axes, we use diverse Video-LLMs to generate dense captions and obtain human-verified sentence-level labels, where sentences containing incorrect events serve as hard negatives for evaluation. Our experiments on VidOmni-Bench reveal three key findings: (i) Video-LLMs frequently generate hallucinated descriptions in dense video captioning; (ii) they also struggle as verifiers, failing to reliably detect plausible but incorrect event descriptions; and (iii) model weaknesses vary across video complexity and duration, revealing diverse, model-specific bottlenecks in current Video-LLMs.

Abstract:
We introduce DiscoTrace, a method to identify the rhetorical strategies answerers use when responding to information-seeking questions. DiscoTrace represents answers as a sequence of question-related discourse acts paired with interpretations of the original question, annotated on top of rhetorical structure theory parses. Applying DiscoTrace to answers from nine different communities reveals that communities have diverse preferences for answer construction. In contrast, LLMs do not exhibit rhetorical diversity in their answers, even when prompted to mimic specific human community answering guidelines. LLMs also systematically opt for breadth, addressing interpretations of questions that human answerers choose not to address. The rich, community-sensitive answering behavior structurally revealed by DiscoTrace can guide the development of pragmatic LLM answerers that are more attuned to contextual information needs.

Abstract:
GUI agents often encounter dynamic anomalies when deployed on Android devices, from unexpected pop-ups to action misuse, yet existing benchmarks lack systematic evaluation of agent robustness against runtime anomalies. We introduce AnTrap, a comprehensive benchmark that injects dynamic perturbations into agent execution trajectories. We propose a taxonomy organizing real-world anomalies into four layers (State, Thinking, Action and Round) with ten fine-grained subcategories, and develop a construction pipeline that preserves task solvability while introducing realistic adversarial conditions. Evaluating 16 leading GUI models, we reveal universal vulnerability to dynamic anomalies, with even the strongest models suffering significant performance degradation. Furthermore, we conduct GRPO training in both original and adversarial environments to validate our benchmark, separating environment-learnable anomalies from reasoning-bottlenecked ones. Our findings show that while single-step traps at state and action layers are largely addressable through adversarial reinforcement learning, deep contextual traps, like state deadlock, expose intrinsic limitations that cannot be resolved by training in environments with traps alone.

Abstract:
Large language models (LLMs) are increasingly evolving from simple text-based interaction systems into LLM agents that can maintain memory, use tools, access external environments, and execute tasks. As their capabilities and autonomy expand, the safety risks they face also become more diverse. Existing evaluations often rely on manually written scenarios, static prompts, or final-output judgments, making it difficult to capture the diverse risks that agents may face during task execution. We introduce ForesightSafety-SAGE, a fully automated scenario generation and safety evaluation framework for LLM agents. Based on five risk dimensions, we instantiate abstract and diverse safety risks in real-world task execution into 1,072 measurable evaluation scenarios. Using the automated evaluation pipeline, 12 LLM agents are evaluated under two authority contexts. The results show that current agents still face substantial behavioral safety risks during task execution, with an average ASR of 47.1% and several models exceeding 70%. These findings demonstrate the importance of executable, process-level evaluation for understanding and improving LLM agent safety.

Abstract:
Millions of individuals’ well-being are challenged by the harms of substance use. Harm reduction as a public health strategy provides non-judgemental, evidence-based information intended to improve health outcomes and reduce associated safety risks. Some large language models (LLMs) have demonstrated a high level of medical reasoning, promising to address the information needs of people who use drugs (PWUD). However, their performance in relevant tasks remains largely unexplored. We introduce HarmReduction, a benchmark designed to evaluate LLMs’ accuracy and safety risks in harm reduction information provision. The benchmark dataset (HRBasic) has 2,160 question-answer-evidence pairs. The scope covers three tasks: checking safety boundaries, providing quantitative values, and inferring polysubstance use risks. We build the Instruction and RAG schemes to evaluate model behaviours based on their inherent knowledge and the integration of domain knowledge. Our results indicate that state-of-the-art LLMs still struggle to provide accurate harm reduction information, and sometimes, present severe safety risks to PWUD. This work contributes an evaluation framework for LLMs to deliver harm reduction information to avoid introducing negative health outcomes through the use of LLMs.

Abstract:
Scaling test-time computation in Multimodal Large Language Models (MLLMs) enforces heavy visual analysis and long logical reasoning uniformly across queries, leading to severe token waste and overthinking. We propose Ada-DeVL, a framework that decouples visual perception depth from logical reasoning intensity, allowing the model to adaptively select one of four visuo-reasoning pathways according to problem complexity. To learn this adaptive behavior, we first apply a Self-Distillation Supervised Fine-Tuning (SFT) stage that generates difficulty-aligned, hallucination-reduced data while avoiding the exposure bias of cross-model distillation. We then use Group Relative Policy Optimization (GRPO) with a composite reward, incorporating diversity and format-adherence constraints to prevent mode collapse and shortcut exploitation. Notably, on the MMMU benchmark, the Qwen2.5-VL-7B based Ada-DeVL achieves 54.0% accuracy, surpassing Vision-SR1 while saving 39.6% tokens; the Qwen3-VL-8B based model reaches 64.0%, outperforming its vanilla counterpart and reducing token consumption by 44.6%, and achieves competitive accuracy against the strong Qwen3-VL-8B-Thinking baseline with significantly lower token costs (averaging 1560.8 vs. 4889.0 tokens).

Abstract:
Recent work on gender bias in the automatic translation of gender-ambiguous scenarios has highlighted the need for gender ambiguity detection. We address this gap with ContraGAND, a contrastive corpus of 4,852 natural gender-ambiguous English sentences, each minimally paired with a masculine and a feminine variant generated by a neurosymbolic pipeline. We use ContraGAND to perform and evaluate gender ambiguity detection in two settings: (i) a closed task, where the referent is given, and (ii) an open task, where it is not. For both tasks, we evaluate six open-weight large language models by prompting, and test selected models with fine-tuning and knowledge distillation. We show that (i) the closed task is saturated in current models, whereas (ii) the open task produces a gap that varies considerably by model. We find that two failure modes dominate: a binding failure (the model surfaces the referent but defaults to ’ambiguous’ when the gender cue is cross-clause) and a format failure that, on local cues, is primarily under-commitment rather than a binding deficit (long unparseable reasoning chains on small models). Further analysis reveals that both are forms of over-prediction of the ’ambiguous’ class rather than stereotype-driven misclassification. While explicit prompts and decoding interventions do not close the gap, both fine-tuning on the augmented contrastive variants and distillation from a 31B teacher are effective at teaching a 4B student to successfully perform gender ambiguity detection.

Abstract:
As large language model (LLM) chatbots become embedded in education, researchers need scalable ways to analyze what learners are actually doing in student-chatbot interactions. To address this challenge, we propose a learning theory-informed annotation framework for learner intent and engagement in educational chatbot dialogues, operationalizing solution seeking, knowledge seeking, cognitive engagement, and mental model expression through a hierarchical decision process, and apply it manually to a dataset of 2,393 learner-chatbot interactions (408 double-coded). We then evaluate two LLMs as zero-shot annotators and compare human-human, LLM-LLM, and human-LLM agreement across both intermediate decisions and final intent labels. While the two LLMs exhibit agreement patterns approaching human-human reliability on the final label (human κ = 0.73 vs. LLM κ = 0.67), their agreement with human reference labels is substantially lower (κ ≤ 0.53), indicating that automated annotation errors cannot be understood from aggregate scores alone. Through exploratory qualitative coding of disagreement cases, we find that divergences can arise from ambiguous learner intent, competing valid interpretations, and human annotation variability, rather than model failure alone.

Abstract:
In this paper, we present a systematic survey and methodological guidelines for explanation evaluation and statistical significance testing in explainable NLP (XAI). We systematically reviewed 494 verified papers from the ACL Anthology (2020–2025) to identify how explanations are evaluated and the frequency of statistical validation. Our analysis identifies a persistent rigour gap: although the number of explainable NLP papers increased fivefold during the study period, there was no corresponding increase in the use of explanation-specific statistical testing. Across the full corpus, only 25.30% (125/494) of studies substantiate evaluation claims with hypothesis testing. This remains a minority practice in every surveyed year. We categorise the primary barriers to rigour into three issues: implicit generalisation, inferential reporting gap, and incomplete rigour. To address these, we propose a bipartite framework that maps research hypotheses to appropriate statistical testing regimes. Our goal is to clarify when statistical testing is necessary and to provide a guideline for more rigorous, reproducible explainability research in the LLM era.

Abstract:
Large language models (LLMs) are increasingly used in emotionally sensitive human-AI applications, where reliable emotion detection is essential. However, their emotion recognition abilities remain uneven: models often perform well on some emotions while consistently struggling with others. Although recent work has explored emotion mechanisms in LLMs, little is known about why models are weaker on some emotions than others from a mechanistic interpretability perspective. In this work, we investigate emotion-specific biases through the causal mechanisms of emotion inference using sparse autoencoders (SAEs). We systematically identify causal sparse emotion features that drive emotion inference and analyze their sparse causal organization within and across emotions. We show that some emotions, such as surprise and fear, rely on highly concentrated feature sets, whereas disgust exhibits a more distributed sparse causal organization: its causal features are generally weaker, frequently co-activate with features for other emotions, and are often overshadowed by causal features for anger. These representational differences provide a mechanistic explanation for why LLMs struggle more with certain emotions. Finally, we conduct two intervention experiments: targeted steering of weaker causal features to mitigate emotion-specific failures, and global optimization of a steering vector over the identified causal features to improve overall emotion recognition performance.

Abstract:
Knowledge graph (KG) foundation models (KGFMs) are zero-shot generalizers: trained once, they can predict links on unseen graphs without retraining. However, understanding when and how they can robustly generalize across KGs is still an open question. In this paper, we shed some light on their generalization mechanisms highlighting how their performance on unseen KGs is not uniform when it comes to partially seen links, which we call half-links. In fact, we show that to predict a test triple (h,r,t) it might suffice in practice to have observed the half-link (h,r) or (r,t) in the inference graph. This yields a taxonomy of four scenarios when combinations of these half-links are observed or not. In a rigorous stratified analysis over these scenarios, we reveal that SoTA KGFMs use seen half-links for predictions, while unseen half-links pose different challenges. As such, our finer-grained taxonomy can be a diagnostic protocol for robust KGFM generalization and highlights where novel KGFMs can improve.

Abstract:
Breast cancer care requires multimodal reasoning across a clinical workflow spanning screening , diagnosis , and treatment planning . However, existing medical MLLMs are typically limited to isolated modalities or narrow task settings, restricting their ability to support workflow-level clinical reasoning. To address this gap, we introduce BreastStage , a workflow-aligned breast imaging instruction corpus containing 1.86M instruction-following pairs from 17 sub-datasets, 5 imaging modalities, and 136 task templates, together with BreastStage-Bench , a benchmark for evaluating multimodal reasoning across the breast cancer care continuum. Built upon this corpus, we propose BreastGPT , a unified MLLM with a dual-branch visual encoder and concept-preserving token compression to bridge standard radiology and gigapixel pathology. On BreastStage-Bench, BreastGPT achieves 75.66% closed-ended accuracy and 89.92% open-ended score, outperforming both general-purpose and medical-specific MLLMs across clinical stages and task formats. Data, code, and model checkpoints are publicly released at https://anonymous.4open.science/w/BreastGPT_io/ .

Abstract:
Nowcasting headline macroeconomic indicators, i.e., estimating an indicator’s value for the current reference period before its official release, is critical for monetary policy and financial markets, and central banks devote dedicated teams of expert economists to producing such estimates. Large language model (LLM) agents are a promising candidate for this task, combining broad world knowledge with real-time web search and supporting queries at higher frequency than institutional nowcasts. Evaluating their nowcasting capability is, however, challenging: headline indicators such as GDP and CPI are widely reported and likely memorized during pretraining, so any evaluation on historical releases is vulnerable to data contamination. To address this, we introduce LiveMacroEval, a live, contamination-resistant benchmark in which LLM agents produce hourly nowcasts for sixteen major U.S. macroeconomic indicators over a pre-release window closing at each official release. Nowcast quality is assessed through a LiveMacro Score against announcement-window equity returns and a LiveBetting Score from simulated Polymarket-style trading, with Federal Reserve regional-bank nowcasts, the Bloomberg ECOS professional consensus, and an auto-ARIMA baseline as comparators. Over six months with four state-of-the-art LLM agents configured with web search, aggregate nowcast accuracy is broadly comparable to the institutional and professional benchmarks, highlighting LLM agents’ potential as real-time estimators of macroeconomic conditions.

Abstract:
Memory systems for conversational LLMs are conventionally evaluated by direct, fact-seeking questions about prior dialogue (Direct QA): can the model recall fact X from a prior conversation? We tested whether higher Direct QA accuracy correlates with higher user satisfaction in a four-month deployment involving 40 users, 1,872 sessions, and seven memory conditions. Existing-benchmark Direct QA accuracy varies from 19.7% to 70.1% across the seven conditions, but satisfaction does not change. We hypothesize that existing benchmarks and user satisfaction measure different capabilities: benchmarks measure elicited retrieval—recall when explicitly asked—whereas natural conversation requires detecting relevance and appropriately integrating prior context into a response. To examine this distinction, we introduce MemUse, a benchmark of real user-cued memory moments drawn from the deployment and evaluated using integration-aware judgments of natural conversational responses. Holding the model and context fixed, the same system that scores 78.8% on Direct QA references only 7.9% of those facts in conversation, revealing a 71-point retrieval–integration gap. Within a small observational subset of naturally occurring memory moments, successful Natural Integration is positively associated with user satisfaction, whereas Direct QA performance is not. We release the deployment corpus and MemUse together with the associated judgments and scoring prompts.

Abstract:
Recent work proposes next-chunk reasoning RL for leveraging no-CoT data–corpora such as worked solutions and textbook derivations that contain reasoning-rich content but lack explicit chain-of-thought annotations. The method trains a model to generate implicit reasoning traces and rewards them by their ability to predict the next chunk of text. While promising, existing evaluations primarily compare against conventional SFT baselines, leaving open whether the gains come from the RL formulation itself or from more effectively exposing the model to no-CoT data. We address this question with a controlled study of next chunk reasoning RL and a simple but previously overlooked alternative: Mixed SFT, a single supervised fine-tuning stage that jointly trains on no-CoT and long-CoT data. Despite its simplicity, Mixed SFT achieves a clearly higher post-RLVR performance ceiling than next-chunk reasoning RL while requiring over 60 times less training compute. The advantage is consistent across in-domain mathematical reasoning and out-of-domain reasoning tasks. Moreover, we show that higher pre-RLVR accuracy does not necessarily translate into higher post-RLVR accuracy, highlighting the need to evaluate no-CoT training strategies in the context of the full post-training pipeline.

Abstract:
We introduce RedNote-Vibe, a dataset spanning five years (pre-LLM to July 2025) sourced from lifestyle platform RedNote (Xiaohongshu), capturing the temporal dynamics of content creation and is enriched with comprehensive engagement metrics. To address the detection challenge posed by RedNote-Vibe, we propose the PsychoLinguistic AIGT Detection Framework (PLAD). Grounded in cognitive psychology, PLAD leverages deep psychological signatures for robust and interpretable detection. Our experiments demonstrate PLAD’s superior performance and reveal insights into content dynamics: (1) human content continues to outperform AI in emotionally resonant domains; (2) AI content is more homogeneous and rarely produces breaking posts, however, this human-AI gap narrows for arousing higher-investment interactions; and (3) most interestingly, a small group of users who strategically utilize AI tools can achieve higher engagement outcomes. The dataset is availiable at https://github.com/ydli-ai/RedNote-Vibe.

Abstract:
In medical data analysis, extracting deep insights from complex multi-modal pathological data is essential for enhancing patient care, diagnostic accuracy, and healthcare operations. Despite this clinical significance, current benchmarks fail to adequately evaluate Large Multi-modal Models (LMMs) on medical insight discovery. Therefore, we introduce MedInsightBench, the first benchmark comprising 332 curated cases from The Cancer Genome Atlas, each annotated with expert-derived insights. Instead of simple QA, this benchmark evaluates whether LMMs can derive clinically grounded insights tied to tumor staging, prognosis, and biomarker expression. Our initial analysis reveals that existing LMMs struggle with MedInsightBench, primarily due to a lack of domain expertise and an inability to perform multi-step reasoning. Consequently, we propose MedInsightAgent, an automated agent framework featuring three specialized modules: Visual Root Finder, Analytical Insight Agent, and Follow-up Question Composer. Extensive experiments confirm that MedInsightAgent effectively addresses the limitations of existing models, consistently demonstrating superior performance in discovering complex medical insights.

Abstract:
Although existing model editing methods perform well in recalling exact edit facts, they often struggle in complex scenarios that require deeper semantic understanding rather than mere knowledge regurgitation. Leveraging the strong contextual reasoning abilities of large language models (LLMs), in-context learning (ICL) becomes a promising editing method by comprehending edit information through context encoding. However, this method is constrained by the limited context window of LLMs, leading to degraded performance and efficiency as the number of edits increases. To overcome this limitation, we propose InComeS, a flexible framework that enhances LLMs’ ability to process editing contexts through explicit compression and selection mechanisms. Specifically, InComeS compresses each editing context into the key-value (KV) cache of a special token, enabling efficient handling of multiple edits without being restricted by the model’s context window. Furthermore, specialized cross-attention modules are added to dynamically select the most relevant information from the pool of special tokens, enabling adaptive and effective utilization of edit information. We conduct experiments on diverse model editing benchmarks with various editing formats, and the results demonstrate the effectiveness and efficiency of our method.

Abstract:
Kazakhstan is transitioning the Kazakh language from Cyrillic to a Latin alphabet, with four official Latin proposals issued since 2017. How well do current large language models, trained almost entirely on Cyrillic Kazakh, handle this script change? We evaluate 13 models spanning general-purpose open-weight, Kazakh-adapted, and proprietary systems on three Kazakh benchmarks (KazMMLU, KazCulture, GSM8k-Kazakh) rewritten in all four Latin variants. The mean accuracy drop from Cyrillic to Latin is 16.8 points, ranging from 1.4 (GPT-5) to 30.4 (Sherkala-8B). Contrary to the natural tokenization-explosion hypothesis, fertility analysis shows that Latin Kazakh requires no more tokens per word than Cyrillic for most tokenizers. Subword fragmentation therefore contributes to the drop but does not primarily explain it. Kazakh-adapted models drop hardest, suggesting that Cyrillic-only adaptation entrenches script-specific routes rather than building script-agnostic competence. Prompt-level mitigations (hint, mapping) recover only 1 point on average. On the other hand, LoRA fine-tuning recovers up to +20 points, but only for sufficiently large reasoning-capable bases. Generative AI will play a key role in the script transition. Models serving Kazakh users must reliably process Latin script, which our findings show cannot be achieved at inference time. Addressing this gap requires incorporating Latin Kazakh data during pretraining or large-scale continued pretraining, rather than relying solely on prompt or adapter-level solutions.

Abstract:
Referring expression grounding is a core problem in visual grounding and is widely used as a diagnostic of spatial grounding and reasoning in vision–language models, yet most prior work focuses on natural images. In contrast, existing chart referring expression grounding-related benchmarks remain limited: (1) they largely adopt bounding boxes, constraining localization precision for fine chart elements; (2) they mostly assume one and two referred target instances, failing to handle multi-instance target references; (3) the language expressions over-rely on textual cues or data-ranking clues in the chart; (4) they cover only a narrow range of chart types. To address these issues, we introduce a chart referring expression grounding benchmark that first systematically supports multiple localization formats, multiple referred targets, diverse grounding cues and diverse chart types. Results across representative multi-modal large models reveal a significant performance gap. We further introduce an LLM-free synthesis pipeline that exploits the inherent alignment between matplotlib internal artist hierarchy and rendered chart primitives to derive pixel-accurate, multi-granularity instance segmentation masks. We train an instance segmentation model with the synthesized masks and integrate it into a general-purpose multi-modal grounding framework. The resulting system consistently outperforms baselines on our benchmark and generalizes well to a ChartQA-derived real-chart grounding benchmark.

Abstract:
As large language models (LLMs) are increasingly considered for high-stakes legal applications, a central question for trust and auditability is whether their outputs reflect genuine reasoning over the supplied legal materials, or merely surface statistical patterns absorbed during pre-training. We address this question at the mechanistic level by applying activation patching, a causal-mediation technique, to four open-source multilingual LLMs on two bar-exam datasets: U.S. MBE and Brazilian OAB. For each multiple-choice question we contrast a gold supporting passage with an irrelevant-but-plausible counterfactual. We then surgically replace residual-stream activations across layers and token regions and measure recovery of the logit difference. Gold passages exert a measurable but model- and linguistic-dependent causal effect; integration concentrates in the upper layers (a late-binding signature); and the size of the effect is strongly mediated by the linguistic and discourse structure of the passage itself, not just its relevance. We introduce a four-cluster taxonomy that operationalises these regimes and identifies cases where retrieved evidence is causally decisive, inert, or actively harmful.

Abstract:
Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modalities provide diverse diagnostic cues but expose the decoder to many repeated anatomical regions. To investigate reasoning under multi-sequence visual redundancy, we first introduce BreMRIs-VQA, a clinically curated breast MRI benchmark with 1.19M QA pairs from 71.0K sequences and 12.9K patients, covering both free-text and multiple-choice questions. We further propose SeVeR, a selective visual exposure framework that compresses dense volumes into modality-wise prototypes and retrieves complementary multi-level evidence with change-aware gated attention during decoding, trained with a marginal-utility self-consistency objective that suppresses unhelpful retrieval. Experiments on BreMRIs-VQA and public benchmarks show that SeVeR improves both discriminative and generative performance while exposing substantially fewer visual tokens.

Abstract:
Large language models are increasingly deployed as interactive coding agents for end-to-end web development, yet existing benchmarks evaluate narrow slices of this workflow—typically text-conditioned generation with static correctness metrics—overlooking visual fidelity, interactivity, and codebase-level reasoning. We introduce WebCompass, a benchmark that covers the full web development lifecycle by spanning three input modalities (text, image, video) and three task types (generation, editing, repair). To match the diverse nature of these tasks, we pair checklist-guided LLM-as-a-Judge for patch-based evaluation with a novel Agent-as-a-Judge protocol that executes generated websites in a real browser and synthesizes targeted test cases. Experiments on ten models reveal a 25-point gap between the best closed- and open-source models, persistent weaknesses in visual aesthetics even among frontier systems, and divergent performance profiles across task types that challenge the notion of a single "best" web coding agent.

Abstract:
Mathematical reasoning is a key capability for vision-language models (VLMs), yet current benchmarks mainly evaluate text-based or explicitly symbolic visual inputs. It remains unclear whether VLMs can reason mathematically when information must be perceived and inferred from images rather than read from explicit symbols. We introduce GSM8K-V, a benchmark transforming GSM8K into multi-image sequences with semantic equivalence preserved. By mapping text-based problems into visual form via an automated pipeline and human verification, we curate 1,319 high-quality samples. In GSM8K-V, quantities must be extracted through visual perception, and reasoning chains must be reconstructed by integrating implicit cues across scenes. Evaluation of 34 VLMs reveals a striking modality gap: while most models exceed 90% on text, the best model achieves only 59% on GSM8K-V, far below the 91% human accuracy. Notably, models enhanced for visual math reasoning show no improvement on GSM8K-V despite large gains on existing benchmarks, confirming that it evaluates a distinct capability. Error analysis shows that the primary bottleneck lies in Implicit Visual Inference Error (IVIE), where models fail to recover visual semantics that are implied rather than explicitly stated. Our code and data are released at https://github.com/ZJU-REAL/GSM8K-V.

Abstract:
Touch is a key modality for embodied agents to understand the physical world. Although recent work has incorporated tactile signals into lan- guage systems for tactile Commonsense Rea- soning, scaling such systems to realistic open- world settings remains challenging due to two key bottlenecks: (1) current tactile reasoning datasets remain limited in format and scale, pro- viding insufficient supervision for reasoning from tactile observations to physical common- sense and hindering the learning of transferable tactile commonsense; (2) Tactile signals are inherently redundant and action-specific, yet existing methods often overlook these proper- ties, resulting in inefficient representations with limited semantic expressiveness. To address these limitations, we propose TouchThinker, a tactile-language framework that scales tac- tile commonsense reasoning to the open world from both data and representation perspec- tives. First, we construct TouchThinker-1M, a million-scale, multi-source tactile reasoning dataset covering 410 objects, 8 scenarios, and 7 sensor types, providing a solid data founda- tion for open-world generalization. We fur- ther introduce TouchThinker-Bench, an open- world benchmark with more realistic and di- verse tasks. Then, we further introduce action- aware tactile modeling and a two-stage training paradigm to improve tactile representation effi- ciency and enable efficient reasoning. Experi- mental results demonstrate that TouchThinker achieves competitive performance against state- of-the-art models across multiple datasets. Our code and dataset will be released.

Abstract:
The rapid advancement of audio and multimodal large language models has unlocked transformative speech understanding capabilities, yet evaluation frameworks remain predominantly English-centric, leaving Southeast Asian (SEA) languages critically underrepresented. We introduce SEA-SpeechBench, to the best of our knowledge, the first large-scale multitask benchmark that evaluates speech understanding in 11 SEA languages through 97,194 samples across 99 evaluation sets and 597 hours of curated audio data. Our benchmark comprises 9 diverse tasks across 3 categories: speech processing (automatic speech recognition, speech translation, spoken question answering), paralinguistic analysis (emotion, gender, age, speaker recognition), and temporal understanding, a novel dimension featuring timestamped content queries and temporal localization within extended audio sequences up to 3 minutes. We implement multilingual prompting in both native SEA languages and English to reflect user interactions with audio-language models. Evaluation of leading open-source and proprietary systems reveals marked performance gaps. Across all models, performance remains underwhelming on temporal understanding, emotion recognition, and speech translation. Prompting in low-resource languages such as Burmese and Tamil lags behind English by up to 41 percentage points. Our findings expose critical model limitations and underscore the need for inclusive model development. The SEA-SpeechBench benchmark is available at https://zwenyu.github.io/SEA-SpeechBench.

Abstract:
The evolution of writing assistance tools from machine translation to large language models (LLMs) has changed how researchers write. Before the rise of LLM-based tools, non-native authors often rely more on word look-ups and individual sentence translations. Due to native language (L1) interference of the authors, the resulting work, although high in fluency, are often to have a certain "characteristics". Some examples include phrases, idioms, or certain grammar points from the author’s native language. With recent advancements, LLM-based tools can translate, proofread grammar, and even paraphrase process to make them more fluent-sounding. This is extremely helpful for authors, but it raises an interesting question: does the rise of LLM assistance in writings by non-native authors erase L1 characteristics? This study investigates this occurrence by analyzing native language identification (NLI) trends in papers, specifically abstracts, across three eras: pre-neural network (NN), pre-LLM, and post-LLM. We found consistent decline in NLI performance over time. Interestingly, the post-LLM era reveals anomalies, specifically Chinese and French show unexpected resistance or divergent trends. Additionally, we performed an experiment by using recent LLM models to paraphrase abstracts and then feed the rewritten texts into our classifiers to confirm LLM effect on writing styles. The result show that LLM use in papers reduce significantly, and the decline of NLI performance in this rewritten experiment is consistent with the NLI performance trends across periods.

Abstract:
Large language models (LLMs) are increasingly used in scientific writing, yet their citation behavior remains poorly understood. We introduce CiteSpectrum, a controlled dataset for analyzing citations across five levels of AI involvement, from LLM-Only generation to Human-Authored papers. Using citation-behavior features across fidelity, usage, and topology, we show that AI-generated papers differ from human-written papers in how citations are validated, grounded, used, and structurally organized. Even when retrieval and human guidance reduce ghost citations, AI-generated papers contain more unsupported claims, rely on front-section evidence, use citations more affirmatively, and form broader but less interconnected reference networks. Moreover, citation behavior alone enables a lightweight classifier to outperform strong text-based detectors, achieving 0.880 balanced accuracy and 0.944 AUROC. These results suggest that citation behavior provides a strong, interpretable signal of AI involvement beyond conventional textual patterns. The dataset is available at: https://github.com/DISL-Lab/CiteSpectrum-EMNLP26.

Abstract:
Graph-based retrieval-augmented generation (GraphRAG) has recently emerged as a powerful paradigm for knowledge-intensive question answering, especially for tasks that require structured evidence organization and multi-hop reasoning. However, existing GraphRAG systems are typically built in a one-size-fits-all manner, relying on a fixed retrieval framework and a single, often large and costly, generator LLM for all queries. This static design limits their ability to adapt to the complexity of varying questions and often incurs unnecessary computational cost. To fill in the gap, we propose GraphRAG-Router, a cost-efficient framework that adopts a hierarchical routing strategy to coordinate heterogeneous GraphRAGs and generator LLMs. Specifically, GraphRAG-Router is first warmed up through supervised fine-tuning and then optimized with a two-stage reinforcement learning procedure, whose second stage introduces a curriculum cost-aware reward to encourage difficulty-aware and economical generator allocation. Extensive experiments on six general-domain and multi-hop QA benchmarks show that GraphRAG-Router consistently outperforms state-of-the-art baselines, reducing the overuse of large LLMs by nearly 30% while maintaining strong generalization capability.

Abstract:
Formulating optimization problems from raw inputs of problem descriptions and data is time-consuming and costly. While Large Language Models (LLMs) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete. This diverges from reality, where data-centric reasoning is required to extract, compute, and predict parameters from noisy sources. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi-stage reasoning inherent in practical optimization workflows. NLData2Opt decomposes problems into problem descriptions and data, and requires models to (1) extract relevant parameters from data containing misleading attributes, (2) compute missing parameters derived by transforming existing data attributes, and (3) predict unavailable future values from historical observations. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows. NLData2Opt is available at https://github.com/sumarivyn/NLData2Opt .

Abstract:
AI systems increasingly inherit what public knowledge infrastructures make machine-visible. Yet multilingual NLP still tends to frame data responsibility around new corpora, annotations, and benchmarks, even though public knowledge communities have already produced rich multilingual knowledge that may be lost when AI-facing infrastructures compress it into narrower metadata layers. We study this problem on Wikimedia Commons, where multilingual captions and descriptions coexist with the English-dominant category system and Wikidata-linked Structured Data on Commons. Using 2,500 featured images across five domains, we introduce a representational recovery framework that audits lexical, semantic, and retrieval-based recovery from current machine-facing metadata layers. Across these tests, we find a consistent asymmetry. After removing the exact query text from the gold image’s free-text document, Commons categories plus Wikidata entity labels outperform the remaining free text for English retrieval. For non-English retrieval, however, the remaining multilingual captions and descriptions outperform categories plus Wikidata entity labels rendered in the query language. Entity-label rendering improves non-English retrieval, but does not fully recover the meaning already present in contributor-written multilingual text. These results show that multilingual NLP should not only collect new data, but also audit, identify, and repair recovery gaps in public knowledge infrastructures as they increasingly feed large-scale AI training pipelines.

Abstract:
Evaluating Artificial Intelligence (AI) tutor feedback before deployment requires anticipating student engagement, typically assessed through real interaction data. We introduce ParaStudent, a fine-tuning framework for simulating novice programming revisions to support AI tutor evaluation. Compared with prompted baselines, ParaStudent’s revisions more closely match real student code distributions across functional, stylistic, and semantic metrics. Our best variant achieves AUCs of 0.80 for both feedback relevance and successful uptake when distinguishing streams with real engagement above versus at or below the median, while prompted baselines remain near chance on successful uptake. These findings demonstrate the promise of simulated engagement for pre-deployment feedback triage.

Abstract:
Low-resource languages remain challenging for machine translation, and Mongolian is a representative case. As a digraphic language, Mongolian is written in both Cyrillic and Traditional scripts, which exhibit a severe imbalance in data availability. While the Cyrillic script is relatively well-resourced, the Traditional script remains extremely data-scarce and orthographically ambiguous, leading to substantial performance degradation in direct translation. We propose CoPiT, a cognitively motivated pivot-based translation pipeline that exploits this internal resource hierarchy by routing translation through the Cyrillic script. The pipeline explicitly resolves script-induced ambiguity in the Traditional script before translation, enabling more stable and accurate meaning transfer. Across multiple backbone models and target languages, CoPiT generally improves over direct translation, with substantial absolute BLEU improvements and up to 1.5–1.6x gains in COMET. These gains allow strong open-source models to match or outperform GPT-4.1 under comparable evaluation settings. Beyond inference-time improvements, CoPiT enables the construction of synthetic parallel data directly from Traditional-script text, mitigating data scarcity in realistic low-resource scenarios. We release a new multi-script parallel dataset covering Mongolian in both scripts alongside English, Korean, and Russian. All datasets and code are publicly available at https://github.com/bbyba/CoPiT.git .

Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance across a wide range of multimodal tasks. Recent advances in personalized MLLMs enable effective capture of user-specific concepts, supporting both recognition of personalized concepts and contextual captioning. However, humans typically explore and reason over relations among objects and individuals, transcending surface-level information to achieve more personalized and contextual understanding. To this end, existing methods may face three main limitations: (1) Their training data lacks multi-object sets in which relations among objects are learnable, (2) Existing models often neglect the connections between different personalized concepts, thereby failing to perform reasoning over them, (3) Their experiments mainly focus on a single personalized concept, where evaluations are limited to recognition and captioning tasks. To address the limitations, (i) We present a new dataset named ReGraP, consisting of 120 sets of personalized knowledge. Each set includes images, Knowledge Graphs (KGs), and Chain-of-Thought Question-Answering (CoT QA) pairs derived from the KGs, enabling more structured and sophisticated reasoning pathways. (ii) We propose Reasoning enabled Graph-based Personalized Large Language and Vision Assistant (ReGraP-LLaVA), an MLLM trained with the corresponding KGs and CoT QA pairs, where soft and/or hard graph prompting methods are designed to align KGs within the model’s semantic space. (iii) We establish the ReGraP Benchmark, which contains diverse task types: Multiple-Choice, Fill-in-the-blank, True/False, and Descriptive questions in both open- and closed-ended settings. The proposed benchmark is designed to evaluate the relational reasoning and knowledge-connection capability of personalized MLLMs. We conduct experiments on the proposed ReGraP-LLaVA and other competitive MLLMs. Results show that the proposed model not only learns personalized knowledge but also performs relational reasoning in responses, achieving the SoTA performance compared with the competitive methods. All the codes and datasets are released at: https://anonymous.4open.science/r/ReGraP.

Abstract:
In psychological counseling, effective support is not always delivered through long, information-rich responses. Minimal responses, such as backchannel cues and concise empathic statements, help convey attentive listening, express empathy, and encourage clients to continue expressing themselves. However, existing counseling dialogue systems and evaluation frameworks often favor explicit, content-rich replies, overlooking the interactional value of brief counselor utterances. This paper presents a systematic cross-lingual analysis of minimal responses across multiple counseling dialogue datasets. We develop a two-stage filtering method based on utterance length and content, followed by contextual verification using a large language model (LLM). Our analysis shows that minimal responses are common in human-collected datasets but substantially underrepresented in LLM-generated ones. We further evaluate current LLMs in manually curated dialogue contexts where human counselors used minimal responses. The results show that strong commercial LLMs are capable of generating minimal responses when explicitly instructed, but still struggle to determine when such responses are appropriate. Counseling-specific models trained on synthetic data perform particularly poorly, tending instead to produce longer and more information-rich responses. Moreover, LLM-based response-quality evaluation may undervalue minimal responses, even when they are interactionally appropriate.

Abstract:
Meme understanding goes beyond recognizing visual content or literal text; it requires implicit cultural knowledge and pragmatic inference that most vision-language models still lack. We introduce MemeCULT-1K, a multilingual benchmark of 1,000 South Asian memes in Bengali, English, and Hindi, where each meme is paired with a cultural context note and three human-written explanations, along with a supplementary set of 54 Bengali regional dialect memes. We evaluate thirteen popular Vision Language Models (VLMs) under two settings: meme-only and context-aware. Providing minimal cultural context yields consistent gains across all models and languages: mean SBERT similarity improves from 44.6 to 56.4 (+11.8), BLEURT from 37.3 to 42.3 (+5.0), and LLM-as-a-Judge scores from 2.57 to 3.43 out of 5 (+0.86). Fine-grained error analysis reveals that closed-source models fail mainly on entity and reference misidentification, while open-source models are bottlenecked by broader cultural knowledge gaps, with linguistic and phonological failures proving the most context-resistant across both. These results highlight the difficulty of culturally grounded meme understanding and motivate future work on explicit cultural knowledge integration. Our dataset and code are publicly available at TawsifDipto17/MemeCULT-1K.

Abstract:
Aggregated reader reviews are widely used as scalable proxies for book attributes in computational literary studies, recommendation, and narratological-benchmark design, but the field lacks a per-attribute audit of what review aggregates can reliably recover. We audit eight review-derived feature methods on 391 attributes, using the HumanReader survey corpus (Knight et al., 2026) as ground truth. Most apparent predictive performance is popularity-confounded. Only 34.9% of evaluable attributes have a method that adds signal beyond a popularity proxy (popularity-conditioned incremental ΔR² > 0.01), and just 12 additionally pass held-out validation, multiplicity correction, and a stricter ΔR² > 0.05 release gate. Recoverability is significantly associated with an attribute’s "lexical anchorability," i.e. how often reviews mention keywords in the attribute’s semantic neighborhood, with high-anchorability attributes about 2.3 times more likely to clear validation (50% of the top anchorability quartile, 22% of the bottom). We release HumanReader-Reviews-1.7M, per-book predictions on 12 content-driven attributes (Tier A) and 77 validated but popularity-informed attributes (Tier B), with per-method test statistics on the 111 attributes no scalable method recovers (Tier C). Raw held-out R² is a liberal estimate of per-book content recovery, crediting any popularity-correlated content, while popularity-conditioned ΔR² is conservative in what it attributes to content; we report both. On a coverage-limited probe of 31 attributes, direct LLM-on-reviews annotation recovers signal the release-panel methods miss, but at a compute cost prohibiting per-book deployment across the 1.7M-book corpus; as a complement, we release an open-weight Qwen 2.5-32B companion scoring 9,990 books.

Abstract:
Deep Research (DR) Agents powered by advanced Large Language Models (LLMs) have fundamentally shifted the paradigm for completing complex research tasks. Yet, a comprehensive and live evaluation of their forecasting performance on real-world, research-oriented tasks in high-stakes domains (eg finance) remains underexplored. We introduce FinDeepForecast , the first live, end-to-end multi-agent system for automatically evaluating DR agents by continuously generating research-oriented financial forecasting tasks. This system is equipped with a dual-track taxonomy, enabling the dynamic generation of recurrent and non-recurrent forecasting tasks at both corporate and macro levels. With this system, we generate FinDeepForecastBench , a weekly evaluation benchmark over a ten-week horizon, encompassing 8 global economies and 1,314 listed companies, and evaluate 13 representative methods. Extensive experiments show that, while DR agents consistently outperform strong baselines, their performance still falls short of genuine forward-looking financial reasoning. We expect the proposed FinDeepForecast system to consistently facilitate future advancements of DR agents in research-oriented financial forecasting tasks. The project website is available at https://OpenFinArena.com.

Abstract:
Chain-of-thought (CoT) supervised fine-tuning (SFT) is widely adopted to improve reasoning ability, yet we find that it systematically degrades long-context recall in hybrid linear-attention models. Across architectures including HypeNet and Jet-Nemotron, retrieval performance on Needle-In-A-Haystack (NIAH) deteriorates substantially after CoT-SFT, and the degradation becomes more severe under harder retrieval settings and longer context windows. For example, HypeNet-9B on NIAH-S2@256K decreases from 67.2% to 9.4%. We attribute this to CoT-SFT biasing attention gradients toward short-range patterns, disrupting query-key projections ( W Q ,W K ) that are responsible for long-range routing. Motivated by this observation, we propose QK-Restore, a training-free method that restores only W Q and W K from the pre-SFT checkpoint while preserving all other post-SFT parameters. We further introduce a Procrustes variant to balance routing preservation and reasoning adaptation. Across architectures, QK-Restore consistently restores long-context capability at zero training cost while preserving reasoning performance; for instance, on HypeNet-5B it improves S3@256K from 65.4% to 76.4% while maintaining strong reasoning performance.

Abstract:
Vision Language Models (VLMs) have shown great success in general visual tasks, yet they still struggle to deeply understand text within images. In this paper, we introduce ReViCo (Real Visual Correction), a benchmark designed to evaluate VLM text understanding through a novel task of visual text error correction. ReViCo challenges models with identifying and fixing text errors in real-world images, which requires a deep understanding on visual text. We benchmark various VLMs using two distinct paradigms: prompt-based strategy and targeted model training, both aimed at pushing the limits of current models. Our experiments reveal a striking performance gap between even the best VLMs and human, and further analysis also shows that most models struggle to accurately perceive the visual text, resulting in frequent correction errors. By highlighting these gaps, ReViCo provides a new benchmark foundation for developing more robust and text-aware VLMs.

Abstract:
Large language models (LLMs) demonstrate strong Helpfulness, Harmlessness, and Honesty (3H) alignment in English-centric settings, but these gains transfer poorly to low-resource languages due to cultural mismatches. Existing multilingual 3H benchmarks rely predominantly on automated translation or LLM based synthesis, propagating source-language biases while sacrificing local relevance. To address this gap, we introduce Pak3H1, the first human-validated, culturally contextualized Urdu benchmark suite for 3H alignment, comprising PakAlpaca (helpfulness), PakBeaverTails (harmlessness), and PakTruthfulQA (honesty). Our multi-stage pipeline integrates manual cultural adaptation and dictionary-guided post editing to prioritize native speaker judgment, ensuring both semantic fidelity and contextual authenticity. Zero-shot evaluations across multiple open and proprietary LLM architectures reveal systematic cross-lingual alignment gaps: helpfulness win rates decline under localized contexts, harmlessness guardrails break down against regional safety risks, and composite honesty metrics degrade substantially due to localized factual constraints. These findings expose structural limitations in current alignment approaches, underscoring the necessity of human-guided localization for equitable multilingual evaluation.

Abstract:
Large language models (LLMs) have demonstrated notable advancements in psychological counseling. However, existing models generally do not explicitly model seekers’ emotion shifts across counseling sessions, a core focus in classical psychological schools. Moreover, how to align counselor models’ responses with these emotion shifts while proactively mitigating safety risks remains underexplored. To bridge these gaps, we propose PsychēChat, which explicitly integrates emotion shift tracking and safety risk analysis for psychological counseling. Specifically, we employ interactive role-playing to synthesize counselor-seeker dialogues, incorporating two modules: Emotion Management Module, to capture seekers’ current emotions and emotion shifts; and Risk Control Module, to anticipate seekers’ subsequent reactions and identify potential risks. Furthermore, we introduce two modeling paradigms. The Agent Mode structures emotion management, risk control, and counselor responses into a collaborative multi-agent pipeline. The LLM Mode integrates these stages into a unified chain-of-thought for end-to-end inference, balancing efficiency and performance. Experiments with simulated interaction, dialogue-level evaluation, ablation studies, and expert assessment indicate that PsychēChat improves emotion-shift awareness and risk-aware response generation in psychological counseling support. Our data and code are available at https://github.com/XNeil12138/PsycheChat.

Abstract:
Spatial intelligence, encompassing 3D perception and reasoning, is the essential next frontier of AI. Scaling current 3D vision-language models (VLMs) that rely on dense Transformers for spatial tasks incurs prohibitive computational costs. In this paper, we introduce 3D-MoE, a 3D VLM leveraging an efficient mixture-of-experts architecture with a modality- and spatial-context-aware probabilistic routing scheme, stably cultivated by a novel routing curriculum. To seamlessly extend 3D-MoE to embodied AI, we integrate a diffusion-based action head, Pose-DiT, transforming 3D-MoE into a 3D vision-language-action (VLA) model. By employing a rectified flow framework, Pose-DiT generates precise 6D pose actions in a single sampling step. Extensive experiments demonstrate that 3D-MoE achieves superior performance on diverse 3D vision-language benchmarks with drastically fewer activated parameters and yields higher success rates while enabling real-time inference for robot manipulation tasks.

Abstract:
Multi-task learning in the medical domain remains a significant challenge for large language models (LLMs), plagued by conflicting task objectives and substantial computational overhead associated with full fine-tuning. Although parameter-efficient tuning methods help reduce these resource demands, their effectiveness is limited by two critical issues: (i) rigid allocation of expert capacity across layers, and (ii) lack of regularization mechanisms to partition shared components from task-specific signals. To address these limitations, we propose LAK-LoRA , a novel architecture that enables adaptive, depth-aware scaling of expert capacity, allowing more efficient resource allocation throughout the model. Furthermore, to resolve conflicting task objectives, LAK-LoRA divides experts into two distinct groups dedicated to modeling general and task-specific features, respectively. This separation is implemented via a feature separator module, which disentangles representations into shared and task-specific subspaces before assigning them to the appropriate experts. To foster diversity and minimize representational redundancy, orthogonality constraints are imposed among specialized experts. Extensive experiments on benchmark medical datasets demonstrate that LAK-LoRA achieves the best average performance.

Abstract:
AI-generated text detectors achieve high accuracy on standard benchmarks, yet the internal representations that drive these predictions remain poorly understood. We study which neurons in a frozen BERT-base-uncased encoder support AI-text detection, using the RAID benchmark across six generators spanning pure-base and instruction-tuned models. We apply the L1→L2 sparse-probing protocol of Gurnee et al. (2023) to all 9,216 CLS hidden-state dimensions (12 layers × 768), which we call neurons. The procedure recovers a stable set of under 1% of neurons per generator, consistent across folds and seeds; a probe restricted to that set retains most of the full-feature detection accuracy. Bidirectional activation patching confirms this set’s causal relevance: in both directions it flips predictions an order of magnitude more often than size-matched random sets. Mean-ablating the same neurons leaves accuracy largely intact; the signal is therefore redundantly distributed. Cross-generator analysis reveals a bipartite structure: instruction-tuned generators concentrate 30–36% of stable neurons in BERT’s final layer while both base generators fall below 14%, consistent with a layer-12 footprint of post-training alignment. Leave-one-family-out evaluation shows the selected neurons retain 86–94% of the full-feature ceiling on unseen generator families, so a detector can operate on a small fixed subspace without re-identifying neurons per generator.

Abstract:
We present ChildVox, a novel benchmark for characterizing the diverse acoustic signals through which children communicate. Specifically, ChildVox follows the full developmental trajectory from birth through school age, covering physiological sounds, non-linguistic vocalizations, canonical syllables, and spoken language. ChildVox integrates more than 20 sub-tasks across 17 child-centered audio and speech datasets, enabling systematic cross-corpus and cross-domain comparison. We evaluate a representative range of audio and speech foundation models, including self-supervised, ASR-oriented, and large audio-language models, on tasks including physiological sound classification, vocalization and canonical syllables modeling, and speech quality assessment and recognition. Benchmark results show that ChildVox provides a suite of high-performance models in recognizing a wide range of acoustic signals from children, supporting downstream applications such as characterizing children’s language levels and tracking speech production with age. Our benchmark models on public datasets are released at: https://github.com/tiantiaf0627/childvox-release.

Abstract:
Mixture-of-Experts (MoE) architectures scale Large Language Model (LLM) capacity efficiently by activating a sparse subset of experts per token. However, modern MoE inference remains heavily constrained by the rigid, whole-expert abstraction. Existing frameworks manage, schedule, or prune experts as atomic execution units, which fixes the optimization boundary too early and leaves fine-grained intra-expert computational redundancy underexplored. In this work, we present PCoMoE, a path-compositional execution framework that shifts MoE inference from coarse-grained expert selection to fine-grained path composition. PCoMoE incorporates a path-level formulation of expert computation, a compatibility-aware layer-wise pruning strategy to suppress low-value path combinations, and a hardware-friendly execution engine to exploit reusable sub-expert structures under strictly bounded overheads. Experimental results demonstrate that PCoMoE achieves up to a 1.31x end-to-end inference speedup while enhancing model accuracy by 10%. The code is available at https://github.com/gzyyy0/PCoMoE

Abstract:
Minimal-pair benchmarks such as BLiMP evaluate linguistic knowledge by testing whether language models (LMs) prefer acceptable sentences over minimally different unacceptable ones. However, these benchmarks largely ignore lexical frequency variation, despite lexical frequency being a pervasive and highly skewed property of natural language use. Consequently, existing evaluations do not test whether grammatical preferences remain stable when contrasts involve rare lexical items. We introduce FreqBLiMP, a frequency-controlled extension of BLiMP that regenerates all 67 paradigms under explicit Zipf-frequency regimes while preserving each minimal-pair’s grammatical contrast. Evaluating multiple open-weight LLM families across scales, we find that decreasing lexical frequency produces a consistent, monotonic decrease in sentence likelihood, but only a modest reduction in overall contrastive acceptability accuracy. However, this aggregate stability masks substantial variation across linguistic phenomena, with LLMs remaining robust on overt morphosyntactic generalization while degrading on phenomena that require lemma-specific information.

Abstract:
Changing one word in a prompt can reduce large language model inference energy by up to 70%, but no prior work explains why. We introduce GreenSwap to find out. We run every prompt under three conditions: the original verb (C1), a swapped low-energy verb (C2), and the original verb with an explicit instruction to limit the response length (C3). Across three open-weight models, three task types, and 21,600 inferences, the short-answer instruction saves more energy and preserves higher quality than verb swapping in 7 of 9 settings. A token–energy correlation of r ≥ 0.998 shows that inference energy depends only on output length and not on which verb was used. Verb swapping saves energy only because it sometimes makes the model write less, and on long-form generation it backfires by producing a completely different and lower-quality response. Scoped to English instruction-tuned 7–8B models, our results show that telling the model how long to respond is simpler, more reliable, and more energy-efficient than curating verb lists.

Abstract:
Curating Web corpora for regional language variants like European Portuguese (PT-PT) is heavily bottlenecked by dialectal overlap (mainly with PT-BR) and data processing scale. This paper presents an efficient pipeline to curate a production-ready PT-PT corpus from the Portuguese Web, spanning 411 TB of raw data from Arquivo.pt. We introduce a novel post-scraping block that removes boilerplate and line duplicates prior to filtering. This early-stage intervention increases final document yield by 19.04% by rescuing valid text that standard heuristic filters prematurely discard. Integrated with rigorous language identification, weighted fuzzy deduplication, and neural quality classification, our pipeline offers a scalable framework and a clean, representative corpus optimized for LLM pre-training.

Abstract:
NE-MultiSpeech is a large-scale transcribed multilingual speech corpus covering eight indigenous and regional languages of Northeast India: Khasi, Garo, Mizo, Nagamese, Kokborok, Assamese, Wancho, and Chakma. The corpus combines 124 hours of newly collected, speaker-compensated proprietary recordings across five languages with curated subsets from the Vaani dataset (Pulikodan et al., 2026), totalling approximately 267 hours of transcribed speech across three language families. We fine-tune Whisper-medium on NE-MultiSpeech and establish a systematic automatic speech recognition (ASR) benchmark across all eight languages, achieving an overall word error rate (WER) of 35.53% over 8,099 test utterances jointly, outperforming both Whisper-large-v3 zero-shot (136–194% WER) and MMS-1B zero-shot (42–116% WER) by a large margin. We validate our proxy-language-token strategy for unsupported languages with per-language ablations on four Tibeto-Burman languages, confirming that family-matched proxies consistently outperform mismatched alternatives. We additionally introduce NE-SpeechEmbed, a dual-encoder speech-text retrieval model trained on 73,476 speech-text pairs via InfoNCE contrastive learning, evaluated on the full candidate pool (~1,000 per language) and outperforming CLAP zero-shot across all eight languages and all metrics. Probing the fine-tuned encoder with t-SNE reveals language-family-aligned clustering tendencies without explicit family supervision, supported by a quantitative silhouette score (0.44 language-level, 0.11 family-level). All models, data splits, and utterance IDs are released under CC-BY-4.0; proprietary audio is freely available via an automated access request, used only for usage tracking.

Abstract:
The language a legislature uses to debate women’s rights, even in favour of them, encodes systematic patterns of sexism that persist across two centuries. In this work, we analyse 6,531 speeches over 200 years of UK parliamentary debate (Hansard, 1803-2005) by using large language models to classify a speaker’s perspective towards women’s suffrage and political representation, as well as analyse sexist speech in parliament from the lens of the Ambivalent Sexism Inventory. We also release this parliamentary dataset, an organized and metadata-enriched version of the publicly available Hansard Corpus optimized for computational social science research, with 6.7 million speeches across 1.2 million debates, with 89% gender-matching for speeches by MPs from the House of Commons. We find that 54% of speeches opposing women’s representation contain sexist content, compared to 21% of speeches that are for the cause, and that the two sides use fundamentally different types of sexism: anti-suffrage rhetoric combines hostile and benevolent framing, while pro-suffrage sexism is overwhelmingly benevolent. Female MPs support women’s political rights at 93% compared to 70% for male MPs, a gap that closes only after enfranchisement. Our findings are evidence that benevolent and hostile sexism are used in different rhetorical contexts in a manner consistent with the theory of Ambivalent Sexism.

Abstract:
Motivational Interviewing (MI) theory posits that effective health behavior counseling requires adherence to a session structure that sequentially progresses through four fundamental processes of MI: engaging, focusing, evoking, and planning. While previous research has demonstrated remarkable capabilities of Large Language Model (LLM)-based conversational agents for MI counseling at mimicking individual utterances of human counselors, methods to ensure adherence of such agents to session- and discourse-level aspects of MI theory have yet to be explored. To address this limitation, we propose the Dual Controller Architecture (DCA), a neuro-symbolic approach to modeling counseling session discourse and structure for LLM-based MI agents with local and global controllers. The global controller ensures timely and appropriate transitions of MI sessions between the stages focused on the four processes. The local controller leverages expert-curated, process-specific distributions of MI counselor communication behaviors to align the agent’s responses with the objectives of each process and ensures the agent’s utilization of effective counselor communication patterns. Standard approaches to domain adaptation of LLMs, such as prompting, fine-tuning, and RAG, as well as the DCA, have been implemented as five versions of the Neural Agent for Obesity MI (NAOMI) and evaluated in a user study involving 49 participants with clinically documented obesity. The study revealed that the DCA demonstrates comparable or superior performance, with particularly strong gains in participants’ perceived readiness, importance, and confidence for weight loss.

Abstract:
Vision-Language Model (VLM) based retrievers have advanced visual document retrieval (VDR) to impressive quality. They require the same multi-billion parameter encoder for both document indexing and query encoding, incurring high latency and GPU dependence even for plain-text queries. We observe that this design is unnecessarily symmetric: documents are visually complex and demand strong visual understanding, whereas queries are just short text strings. NanoVDR exploits this query–document asymmetry by decoupling the two encoding paths: a frozen 2B VLM teacher indexes documents offline, while a distilled text-only student as small as 69M parameters encodes queries at inference. The key design choice is the distillation objective. Through systematic comparison of six objectives across three backbones and 22 ViDoRe benchmark datasets, we find that pointwise cosine alignment on query text consistently outperforms ranking-based and contrastive alternatives, while requiring only pre-cached teacher query embeddings and no document processing during training. Furthermore, we identify cross-lingual transfer as the primary performance bottleneck, and resolve it cheaply by augmenting training data with machine-translated queries. The resulting NanoVDR-S-Multi (DistilBERT, 69M) retains 95.1% of teacher quality and outperforms DSE-Qwen2 (2B) on v2 and v3 with 32× fewer parameters and 50× lower CPU query latency, at a total training cost under 13 GPU-hours.

Authors: Bryan Chen Zhengyu Tan, Weihua Zheng, Thong T. Doan, Bich Ngoc Doan, Jia Wang Peh, Xiaoyuan Yi, Jing Yao, Xing Xie, Nancy F. Chen, Zhengyuan Liu, JinYeong Bak, Wafi Shamdi, Soo Kai Chie, Liew Yu Siong, Aina Azyyati Binti Mohamad Rezal, Lew Yan Yan Vanessa, Huadan Wu, Dylan Raharja, Nadya Yuki Wangsajaya, Akane Fukushige, Kazushi Kato, Koji Inoue, Tatsuya Kawahara, Jaehyung Seo, Dongjun Kim, Seungyoon Lee, Zi Haur Pang, Rui Yang Tan, Charibeth Ko Cheng, Maria Regina Justina Estuar, Jann Railey Montalan, Pham Minh Duc, Roy Ka-Wei Lee

Abstract:
Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common use case: users seeking practical help over multiple turns in culturally grounded scenarios. We introduce CultureConverse, a scalable, multilingual simulation and evaluation harness for culturally grounded assistant dialogue that covers 10 East and Southeast Asian regions, 58 subgroup identities, and 7 domains. Each simulated and evaluated episode produces a scored interaction where the assistant assists the user and infers cultural constraints from partial information. The resulting CultureConverse-DS dataset contains 14,610 benchmark (evaluation) episodes and 274,295 oracle-guided (gold-mode) dialogues. In our benchmark evaluation of 18 models, GPT-5 mini achieves the highest assistance quality. Human annotation experiments suggest that our evaluation framework is a sufficient proxy for human judgment. Performance gains from fine-tuning on 27,860 high-quality CultureConverse-DS samples improve in-domain assistance and transfer out-of-domain to cultural MCQ and safety classification benchmarks. We release the harness, both splits, and judge prompts to support interactive evaluation of cultural competency.

Abstract:
Existing legal benchmarks cover diverse tasks, while continuous monetary remedies remain comparatively underexplored. We introduce ECtHR-NPD, to the best of our knowledge, the first benchmark for predicting non-pecuniary damage (NPD) awards at the European Court of Human Rights (ECtHR) from case information when no statutory formula or explicit calculation rule determines the amount. ECtHR-NPD contains 14,575 cases with case-level awards in nominal euros, chronological splits, and a protocol separating target construction from model input. We evaluate a battery of methods, including constant predictors, gradient-boosted trees, retrieval methods, fine-tuned encoder language models (LMs), prompted decoder LMs, and knowledge-augmented agents. Our results show that more sophisticated LM and agentic approaches do not consistently outperform the strongest feature-based baseline. All model families struggle to identify zero awards and to calibrate high-award predictions, with further degradation on the Challenging test view, making ECtHR-NPD a challenging testbed for current state-of-the-art open-weight and proprietary LMs.

Abstract:
We study whether grounded reasoning supervision from abundant 2D medical images can improve 3D medical VQA when both input types are aligned through a common reasoning interface. We introduce UniReason-Med, a single-checkpoint framework that processes either a 2D image or a slice-serialized 3D volume at inference time, generating interleaved textual reasoning and localized visual evidence through shared box syntax, region-token injection, and a common grounded reasoning policy. To train this interface, we construct UniMed-CoT, a 220K instruction-tuning dataset with interleaved textual reasoning and grounded visual evidence, including 170K 2D and 50K 3D samples. Through supervised fine-tuning followed by outcome-level reinforcement learning, UniReason-Med learns to generate grounded reasoning traces without IoU/Dice-based localization rewards during RL. Data-mixture and component ablations show that joint 2D+3D grounded supervision substantially improves 3D reasoning over 3D-only training, while grounding and region-token injection consistently benefit both 2D and 3D tasks. These results suggest that a shared grounded reasoning interface can transfer reasoning structure from 2D images to slice-serialized volumetric medical understanding.

Abstract:
Real-time voice agents face a tension: end-to-end models often lack deep reasoning, while cascaded ASR–LLM–TTS pipelines achieve stronger reasoning but defer LLM inference until after ASR finalizes the transcript, creating an avoidable post–turn-end silent gap. We target this gap under complex spoken requests that admit a relatively determinate answer, and treat it as system-induced rather than communicatively motivated. Existing cascaded streaming strategies attempt to reduce it via mechanical segmentation (e.g., fixed chunks, VAD-based splitting) or speculative generation, but they frequently either break semantic units or waste computation on predictions that must be rolled back. We propose LTS-VoiceAgent, a Listen–Think–Speak framework that explicitly separates when to think from how to reason incrementally. It features a Dynamic Semantic Trigger to detect meaningful prefixes, and a Dual-Role Stream Orchestrator that coordinates a background Thinker (for state maintenance) and a foreground Speaker (for speculative solving). This parallel design enables "thinking while listening" without committing audible content before turn end. We also introduce a Pause-and-Repair benchmark that injects natural disfluencies as a semantic-level streaming stress test. Experiments across VERA, Spoken-MQA, BigBenchAudio, and our benchmark show that LTS-VoiceAgent achieves a stronger accuracy–latency–efficiency trade-off than serial cascaded baselines and existing streaming strategies.

Abstract:
Deceptive web content, pervasive across the internet and commonly known as social-engineering attacks, manipulates autonomous web agents into submitting users’ personally identifiable information (PII) to attacker-controlled endpoints. In this paper, we show that these attacks are highly effective at extracting the most sensitive data an agent holds for its user (SSN, card number and CVV, bank account, passwords, API keys; the critical tier), posing a severe risk to deployed agentic systems. To quantify this risk, we introduce Scammer4U, a pre-registered benchmark of 91 attacker-controlled websites spanning 8 attack vectors and 16 site categories on an eight-axis taxonomy, together with 10 benign twins of those sites with the attack removed and the legitimate task intact. 60 of the 91 differ from a sibling site on exactly one axis, so any leakage difference within a pair is attributable to that single design choice. Under no privacy guidance, critical-tier leakage reaches 54–93% across four frontier agents, against 0% on the benign twins, so the leakage is caused by the attack and not by ordinary form-filling. Escalating prompt-level mitigation yields sharply model-dependent reductions across the four families and, pooled across them, remains insufficient to reliably prevent critical PII submission. Most critically, we identify a detection–action gap: agents whose reasoning an independent LLM judge confirms has flagged the site as suspicious still submit critical PII in 35.9% of sessions, versus 66.1% when no suspicion is verbalized, a gap of 30.2 percentage points that holds across all four model families. Defenses that gate on the agent’s own recognition of an attack are therefore gating on the wrong signal. This motivates intercepting outbound submissions at the point they leave the browser, independently of the agent’s reasoning loop.

Abstract:
Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose Typed Latent Reasoning(Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a visible token and invoking a typed latent operator. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.

Abstract:
Large language models (LLMs) often default to Modern Standard Arabic (MSA) when generating Arabic, even when prompted with dialectal Arabic. A natural explanation is that their internal representations are dominated by MSA. We test this hypothesis by adapting the language-dominance framework of Shani and Basirat (2025) to 26 Arabic varieties. Across layers and model families, we find no evidence that MSA acts as a dominant internal representation for Arabic dialects. Instead, dialect representations form a dense and highly overlapping space: normalized mutual information drops sharply compared to patterns reported for more distinct languages. Moreover, the strongest separability effects are not confined to intermediate layers, but can shift toward later layers depending on the architecture. These findings challenge a common interpretation of MSA-biased generation: output preference does not necessarily reveal internal representational dominance. Analyses of multilingual and dialectal LLMs should therefore distinguish generation bias from the geometry of internal representations.

Abstract:
Accurate and comprehensive video captions with consistent subject references are critical for downstream understanding and generation tasks. However, few existing benchmarks can objectively and comprehensively evaluate these properties across diverse durations and scenarios, thereby hindering the advancement of video captioning models. To bridge this gap, we propose CapRiCorn-1K, a comprehensive benchmark designed to evaluate both video captioning quality and subject referential consistency across long temporal horizons and diverse video domains. To accommodate varied evaluation needs, our benchmark supports both audiovisual and visual-only settings. Extensive experiments on CapRiCorn-1K reveal that current models generally struggle to generate accurate and comprehensive captions while maintaining consistent subject references. Moreover, as video duration increases, both the overall caption quality and subject referential consistency decline. Notably, our evaluation metrics exhibit strong correlations with the performance of downstream understanding and generation tasks conditioned on the generated captions, further validating their effectiveness. CapRiCorn-1K will be fully open-sourced to facilitate future research.

Abstract:
Evaluation is an essential process in the development and use of NLP systems. Benchmarks are widely used to calibrate systems, determine the efficiency of techniques, but also to advertise recently launched language models. Not only do benchmarks allow for evaluation and system comparisons, but they are also commonly utilized as training and validation corpora. In this paper, we aim at characterizing the "world-knowledge" that is put forth by benchmarks by closely inspecting their content. Through qualitative manual annotations (+15k instances) as well as quantitative automatic analyses, we uncover the presence of both explicit and implicit biases in MMLU, CommonSenseQA and MGSM, ranging from representational imbalance to bluntly sexist and racist instances. Our findings demonstrate that current NLP evaluation practices pose numerous quality issues, reinforce stereotypes and even contain explicitly harmful content. The current operationalization of benchmarks is questioned, and we provide recommendations and directions for further ethical evaluation. Warning: this paper contains content that may be offensive or upsetting.

Abstract:
SLMs are increasingly deployed on devices, yet real-world prompts often contain textual or ASR noise that can destabilize model predictions. We analyze this problem from a token-level perspective and find that SLMs exhibit narrow logit margins between the gold token and the top-ranked non-gold token, making noisy prompts more likely to induce local rank flips. To address this issue, we propose LoMaR, a two-stage parameter-efficient framework for robust fine-tuning. In Stage I, LoMaR trains the model on clean prompts with a local logit margin expansion objective to expand the local decision boundary. In Stage II, LoMaR performs residual-subspace adaptation with local logit margin alignment on noisy prompts. After training, the main and residual updates are merged into a standard LoRA adapter, introducing no additional inference cost. Experimental results show that LoMaR outperforms baselines in most noisy settings and achieves lower Performance Drop Rate and Rank Flip Rate. Code and datasets are available at https://github.com/HongJin2828/LoMaR.

Abstract:
Evidence-Based Medical Question Answering (EB-MQA) aims to provide accurate and verifiable responses to support clinical decisions. To retrieve evidence from general medical knowledge graphs (KGs), existing knowledge augmentation methods use semantic similarity or dynamic sampling for entity matching and agentic expansion or subgraph scoring for path retrieval. These approaches can introduce entity-level noise from irrelevant or generic entities and path-level noise from redundant or spurious reasoning chains. We propose ToToFilter, a train-free topic- and topology-aware noise filtering framework with Topic-Aware Entity Localization for query-relevant entity anchoring and Topology-Aware Path Filtering for path selection based on semantic consistency, structural compactness, and information specificity. Across four MQA benchmarks, ToToFilter achieves average accuracy gains of 2.30 and 1.22 percentage points over DALK on Llama3.1-8B and Qwen2.5-7B, with maximum gains of 2.60 and 2.78 points. On PrimeKGQA, it improves answer recall and triple recall by up to 8.81 and 19.70 percentage points. Further analyses verify the effectiveness of the proposed two-level filtering framework.

Abstract:
How much of a language model’s apparent knowledge reflects general linguistic competence, and how much reflects memorisation of facts seen during pretraining? Disentangling the two is normally confounded by data contamination, because modern corpora overlap with the benchmarks used to evaluate them. We exploit a natural experiment that removes exact benchmark exposure by construction. TALKIE-1930 is a 13B-parameter transformer trained exclusively on approximately 260 billion tokens of English published before 1931; TALKIE-WEB is an architecture-, tokeniser-, and compute-matched twin trained on an equivalent volume of FineWeb. Because every modern NLP benchmark postdates 1930, any post-1930 capability TALKIE-1930 exhibits cannot arise from benchmark contamination. We study this model pair along three axes. First, syntactic competence transfers across eras with only a modest gap (BLiMP: 76.9% for TALKIE-1930 vs. 82.0% for TALKIE-WEB), and an OCR-noise ablation shows that this gap is not an artefact of character-level corruption in the historical corpus. Second, in-context learning is largely preserved by temporal isolation: TALKIE-1930 matches TALKIE-WEB on SST-2 (91.2% vs. 90.9% at k=32), exceeds it on MNLI (47.3% vs. 37.8%), and learns post-2006 genres it cannot have seen (tweet sentiment and emotion) with learning-rate slopes statistically indistinguishable from TALKIE-WEB’s. Third, linear probes recover the temporal provenance of a factual statement from hidden states at 98.2% accuracy (TALKIE-1930) and 99.5% (TALKIE-WEB); a three-way probe on TALKIE-1930 that adds a suspected-leaked class reaches 96.0% accuracy (chance approximately 33%), well above lexical and permutation baselines, though this figure is carried mainly by the temporal contrast and the leaked class is only partially recovered. Finally, we document a dissociation between representation and behaviour: although TALKIE-1930’s hidden states linearly encode temporal-boundary information, its greedy decodings do not, confabulating post-1930 facts with fluent, confident anachronisms such as dating the iPhone to 1876. We frame these findings as a controlled case study of a rare temporally isolated model pair, providing evidence that exact benchmark exposure is not necessary for in-context learning on modern tasks, and that base language models contain linearly decodable representations of the temporal provenance and, in part, the epistemic status of their knowledge that need not surface in their outputs.

Abstract:
Personalized large language models are often expected to follow explicit style instructions, yet we find that such instructions can undermine the user-specific characteristics that personalization methods aim to preserve. We call this failure mode personalization collapse: explicit style control can conflict with implicit user preferences. To address this challenge, we propose PsPLUG, a lightweight plug-in that learns a user-specific residual after accounting for the requested style. PsPLUG also allows us to tune personalization strength at inference time. Our experiments show that explicit style instructions can diminish personalization in existing methods, whereas PsPLUG better preserves user preferences while providing precise control over the balance between personalization and style adherence.

Abstract:
Continual learning (CL) is essential for Large Language Models (LLMs) to sequentially adapt to evolving tasks. To mitigate catastrophic forgetting, recent advances implement low-rank adaptation with orthogonal projections (e.g., O-LoRA) to isolate task parameters. However, we reveal that such strict geometric constraints trigger an “Orthogonality Dilemma”: rigid parameter isolation impedes the transfer and accumulation of shared representations across semantically related tasks. In this work, we propose a new replay-free method, called ̲ Co nsolidation and ̲ De coupling ̲ LoRA (CoDe-LoRA), for CL of LLMs. CoDe-LoRA disentangles the learning process into Consolidating\ Universal\ Knowledge and Decoupling\ Task - Specific\ Knowledge . To achieve this, CoDe-LoRA leverages an adaptive null-space projection mechanism and semantic routing to balance knowledge accumulation with task-specific adaptation. Experimental results on diverse CL benchmarks demonstrate that CoDe-LoRA consistently outperforms state-of-the-art methods. Our code is available at [https://github.com/Estrellajer/CoDe-LoRA](https://github.com/Estrellajer/CoDe-LoRA).

Abstract:
Vision-language models (VLMs) often answer visual questions using learned language and category priors rather than grounding their predictions in the image itself. Counterfactual images provide a natural diagnostic setting for this failure mode: when visible evidence contradicts what is usually true, a grounded model should answer from the pixels, while a prior-following model will produce a canonical but visually incorrect response. However, existing counterfactual benchmarks mainly ask whether such prior-following behavior exists. In this paper, we ask a further question motivated by the rise of tool-augmented and agentic vision systems: can additional visual evidence views help VLMs reason against their priors? We introduce PriVE-Bench, a Prior-vs-Visual Evidence Benchmark that uses paired original and counterfactual images to distinguish visually grounded answers from prior-consistent errors. We further introduce PriVE-Tools, a controlled agentic-vision-inspired extension that evaluates whether tool-derived visual evidence—including bounding boxes, crops, zoom panels, and contours—improves grounding under the same counterfactual conflicts. Across open- and closed-source VLMs, we compare raw, paired-image, and tool-conditioned inputs using accuracy, prior-following error rate, and other-response rate. Our results show that visual evidence tools can help in some settings, especially when models can use localized evidence effectively, but they are not a universal remedy: several models continue to follow language and category priors even when relevant visual evidence is explicitly provided. The benchmark and codebase can be accessed at https://github.com/JingyuSunUOE/counterfactual-vlm-benchmark

Abstract:
Large Language Models (LLMs) often memorize and regenerate sensitive or protected content that requires inference-time suppression. While machine unlearning has emerged as a highly active area of research, existing unlearning methods often degrade performance on retain dataset or require costly retraining. In this paper, we introduce Exponential-Tilted Unlearning (ETU), an inference-time method that formulates unlearning as the KL I-projection of the base next-token distribution under a user-specified upper bound 𝜀 on the forget-token mass, with a closed-form exponential tilt whose scalar multiplier 𝜆 is set analytically by 𝜀 . Under a single shared protocol, (i) ETU satisfies the 𝜀 constraint across nine instruction-tuned dense and Mixture-of-Experts backbones on TOFU, RWKU, and WPU while preserving retain utility; (ii) the same calibrated family covers the full 𝜀 spectrum with the strict zero-mass case as a limiting operating point; and (iii) representative weight-update methods expose no such 𝜀 -calibrated residual-mass control.

Abstract:
Although batch prompting makes large language model inference more efficient by processing multiple instances simultaneously, it suffers from unpredictable downstream task performance. We propose cascaded batch prompting, a two-stage approach designed to resolve the unpredictability of conventional batch prompting by disentangling complex reasoning from symbol grounding. Experiments on multiple-choice question answering and natural language inference demonstrate that the proposed method outperforms the standard single prompting baseline while achieving a speedup proportional to batch size, establishing a new state of the art on the Pareto frontier.

Abstract:
Standard supervised fine-tuning (SFT) assigns the same explicit loss weight to every expert demonstration, regardless of the model’s changing competence over training queries. Reinforcement learning (RL) based methods adapt update strength using model-generated rollouts, but often require substantially more sampling and can be unstable on hard tasks. We propose Online Self-Weighted Fine-Tuning (OSW-FT) , a simple method that augments SFT with online, trajectory-level weighting. For each query, OSW-FT estimates the model’s current success rate using a small number of inference-only rollouts and rescales the standard SFT loss accordingly. The optimization direction remains anchored to the expert trajectory, while the update magnitude adapts online. For binary-verifiable reasoning, we connect this weighting to SFT and RL at the gradient level, inspired by variance-reduction principles. The resulting estimator is unbiased for the exact OSW-FT surrogate update for any finite rollout count, and we analyze convergence with respect to the corresponding surrogate objective. Evaluated across Qwen3 series ranging from 0.6B to 4B on multiple challenging benchmarks (e.g., AIME), OSW-FT consistently improves over SFT on small-to-medium scale models. OSW-FT offers a favorable compute-performance trade-off as a practical approach for fine-tuning small-to-medium LLMs on binary-verifiable reasoning tasks with only 2 online rollouts .

Abstract:
Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking. Yet we lack a mechanistic account of how extended thinking changes hidden-state representations over the course of a reasoning trace. We introduce Fluid Reasoning Representations (FRRs), a representation-level account of how LLMs organize action and predicate concepts during self-generated reasoning, and test them on obfuscated planning, symbolic, and mathematical tasks where task-relevant words are replaced while problem structure is preserved. Across open-weight base, instruction-tuned, and extended-thinking LLMs, representations of the same action or predicate become more similar across wordings and move toward the corresponding unobfuscated concepts over the reasoning trace. Causal probes show that these representations affect behavior: cross-naming steering improves held-out accuracy beyond Gaussian and shuffled controls, symbolic patching retains more action information than shuffled patching, and subtracting refined directions degrades accuracy; together, these results suggest that extended thinking strengthens a representation dynamic already present at lower magnitude in base and instruction-tuned LLMs.

Abstract:
Causal Transformer language models suffer from strictly sequential decoding and a quadratic per-step attention cost, resulting in expensive inference and an inability to revise previous tokens during text generation. While linear-time causal models and discrete diffusion models each address these weaknesses, their integration remains inherently inconsistent and underdeveloped. In this work, we propose B3D-RWKV, a diffusion-based RWKV variant that integrates the model’s 𝒪(L) inference efficiency with parallel, bidirectional discrete-diffusion through a triplet-block layout method. B3D-RWKV-7.2B reaches comparable accuracy on an 8-task suite versus existing models while significantly outperforming baselines in decoding throughput with an average of 1.6× speedup. Code is available at https://github.com/leonardodalinky/B3D-RWKV.

Abstract:
A common approach for teaching large language models (LLMs) to reason is to train on chains-of-thought (CoTs) of in-distribution reasoning problems. However, such annotated data is costly to obtain for every problem of interest. We want reasoning models to generalize beyond their training distribution, and ideally to generalize compositionally: they should combine atomic reasoning skills to solve harder unseen tasks. In this paper, we introduce a method to enable generalization to a target compositional task that has no labeled CoT data. We minimally modify CoT formats of constituent atomic tasks to be composable: we add prefixes to CoTs during training time, thereby encouraging the trained model to generate sequences of CoTs during inference. We train individual models on the atomic tasks with composable CoT data and combine them with multitask learning or model merging to address the target compositional task zero-shot. This model can be further trained on a small amount of compositional data using rejection sampling fine-tuning (RFT). Training LLMs on Composable CoT outperforms multitask learning and continued fine-tuning baselines on three domains of controlled compositional tasks and also yields improvement on math skill composition setting derived from the MATH dataset.

Abstract:
Although AI agents have demonstrated impressive capabilities in long-horizon reasoning, their reliability is severely hampered by the “Spiral of Hallucination,” where early epistemic errors propagate irreversibly. Existing methods face a dilemma: uncertainty quantification (UQ) methods typically act as passive sensors, only diagnosing risks without addressing them, while self-reflection mechanisms suffer from continuous or aimless corrections. To bridge this gap, we propose a unified Dual-Process Agentic UQ () framework that transforms verbalized uncertainty into active, bi-directional control signals. Our architecture comprises two complementary mechanisms: System 1 (Uncertainty-Aware Memory, ), which implicitly propagates verbalized confidence and semantic explanations to prevent blind decision-making; and System 2 (Uncertainty-Aware Reflection, ), which utilizes these explanations as rational cues to trigger targeted inference-time resolution only when necessary. This enables the agent to balance efficient execution and deep deliberation dynamically. Extensive experiments on closed-loop benchmarks and open-ended deep research tasks demonstrate that our training-free approach achieves superior performance and trajectory-level calibration. We believe this principled framework represents a significant step towards reliable agents.

Abstract:
Long-context capability has become increasingly critical for next-generation Large Language Models (LLMs). However, the quadratic time complexity of standard softmax attention introduces substantial computation and memory overhead as sequence length grows, limiting the scalability of LLMs in long-context scenarios. Beyond existing approaches such as linear recurrent architectures and KV-cache compression methods, we propose Kwai Summary Attention (KSA) , a novel attention mechanism that reduces sequence modeling cost by compressing historical contexts into learnable summary tokens. Specifically, summary tokens are periodically inserted into the input sequence and interleaved with text tokens, partitioning the sequence into distinct chunks while enabling text tokens and summary tokens to operate under different attention scopes. This design preserves dense local expressiveness through full attention among neighboring text tokens, while modeling long-range dependencies efficiently via summary tokens. Empirical results demonstrate that hybrid-KSA consistently outperforms existing hybrid attention variants in long-context modeling. Moreover, KSA is fully orthogonal to GQA and MLA during inference, and combining KSA with these methods achieves up to 8x KV-cache compression while preserving model performance.

Abstract:
Topographic Similarity (TopSim) is the most widely used metric for compositionality in emergent communication research. However, when evaluated on a class of languages where compositional structure is fully known (simple natural-language word concatenations), TopSim, along with other well-known metrics, under-reports compositionality. We investigate and trace TopSim’s failure to a distributional imbalance in pairwise object distances that grows with the input space size, and propose stratified-TopSim, a minimal fix that remains stable across settings and reliably evaluates languages of varying compositional structure, on both controlled benchmarks and emergent languages.

Abstract:
We introduce Office Comprehension Bench (OCB), the first public benchmark to jointly evaluate LLM systems on Word, Excel, and PowerPoint comprehension over native file formats (.docx, .xlsx, .pptx) and their variants. OCB consists of two tracks. File Fidelity Q A tests structural and visual perception of office artifacts — tables, charts, embedded images, formulas, and app-specific elements such as headers, speaker notes, and named ranges. Domain Q A tests expert-level reasoning grounded in real-world industry documents across 12 professional domains, with queries requiring multi-step analysis and synthesis across documents. Each reference answer is decomposed into atomic, binary-gradable claims, and an ensemble of LLM judges scores responses against each claim independently. Even the strongest frontier system in its default reasoning mode reaches only ~59.3% on Domain Q A; increasing thinking depth within a tier does not move performance materially, while moving to a higher product tier yields modest gains. We release the dataset, evaluation tooling, judge prompt, and a public leaderboard.

Abstract:
We approach multilinguality as sense adaptation: aligning latent meaning representations across languages rather than relying solely on shared parameters and scale. In this paper, we introduce SENse-based Symmetric Interlingual Alignment (SENSIA), which adapts a Backpack language model from one language to another by explicitly aligning sense-level mixtures and contextual representations on parallel data, while jointly training a target-language language modeling loss to preserve fluency. Across benchmarks on four typologically diverse languages, SENSIA generally outperforms comparable multilingual alignment methods and achieves competitive accuracy against monolingual from-scratch baselines while using 2–4 times less target-language data. Analyses of learned sense geometry indicate that local sense topology and global structure relative to English are largely preserved, and ablations show that the method is robust in terms of design and scale. Our results show that sense adaptation provides a simple, data-efficient, and linguistics-motivated path to multilingual fluency that is useful to low-resource languages and motivates further linguistics-motivated research.

Abstract:
Masked Diffusion Models (MDMs) have emerged as a compelling paradigm for language modeling, offering the capability for efficient parallel text generation. However, existing parallel sampling methods typically update multiple masked tokens independently and ignore the complex mutual dependencies among the masked tokens. This independent updating mechanism lacks global coordination and might lead to incoherent outputs. To address this limitation, we propose Representation-based Masked Diffusion Model (RMDM), a framework that leverages the text representation to explicitly encode global semantics and help to parallel update tokens more precisely. Specifically, we first encode text into a continuous semantic space using a pretrained encoder and learn an invertible transformation that normalizes the representation distribution to a Gaussian prior, facilitating efficient sampling during generation. Conditioned on this latent semantic representation, we train a masked diffusion model to learn the conditional text distribution, where the representation serves as global semantic guidance to coordinate parallel token updates and faithfully approximate the target distribution. Empirical results demonstrate that RMDM significantly improves generation quality, particularly in aggressive few-step sampling regimes.

Abstract:
Pre-training Large Language Models (LLMs) from scratch at larger scales yields remarkable performance but incurs substantially high training costs. Depth up-scaling provides an efficient alternative by inserting new layers into a pre-trained LLM, avoiding training from scratch. However, most existing methods copying or averaging base layers for new layer, which misalign functionally corresponding neurons, leading to neuron permutation mismatch that harms performance. To address this issue, we propose Optimal Transport Depth Up-Scaling (OT-DUS), which leverages Optimal Transport (OT) theory to align and fuse functionally corresponding neurons module by module in adjacent base layers for new layer construction. OT-DUS achieves better overall performance in both general and specialized domains than existing methods for continual pre-training and supervised fine-tuning across different model sizes and model families. Our analysis of insertion strategies further finds that inserting new layers at higher positions yields not only stronger performance but also improved training efficiency.

Abstract:
Visual thinking should not only sound right; it should show its evidence. While recent vision-language models (VLMs) can produce natural language thinking traces, these traces often leave the supporting image regions implicit, making them hard to verify and weakly supervised. We introduce visually grounded thinking , a thinking process in which models interleave natural language thoughts with explicit point or box grounding to the visual evidence used at each step. This makes the model explains its intermediate thoughts in language while grounding key object in the image regions they refer to. To train this behavior, we construct a scalable synthesis pipeline that distills correct visual reasoning traces, extracts the visual objects required by the trace, grounds them with a SAM3-based agent, and derives aligned point and box supervision from the resulting masks. We further propose grounding-aware reinforcement learning, which combines answer rewards with dense grounding rewards that score whether generated object references match the correct image evidence. Across two counting benchmarks and four spatial reasoning benchmarks, adding visually grounded thinking to Gemma3-4B-IT consistently improves performance over the original model and the non-grounded thinking baseline. On spatial reasoning, the visually grounded thinking 4B models match, and in some cases surpass, Gemma3-27B-IT from the same model family. Our analysis shows that point grounding is well suited to counting, while box grounding benefits most from explicit grounding rewards on spatial tasks. Overall, our results show that VLMs think better when their intermediate thoughts are tied to the image regions that make them true.

Abstract:
Speech-to-speech translation (S2ST) has advanced rapidly, but offline evaluation lacks a unified protocol: studies report non-overlapping metric subsets, preventing direct comparisons. While prior toolkits target subsets of metrics or simultaneous and text-only translation (e.g., stopes-eval, SimulEval, IWSLT), we introduce COMPASS, the first offline S2ST benchmarking framework to unify 46 metrics across eight dimensions with domain-aware validation, and deploy it on 1,248 model-language configurations from FLEURS and CVSS, spanning cascaded and end-to-end architectures over ten language pairs. Architectures exhibit complementary strengths: best-vs-worst gaps exceed 30% on naturalness and speaker preservation but remain within a few points on translation quality, so single-metric rankings systematically misrepresent system quality. Correlation filtering reduces 46 metrics to 10 per direction, with three axes requiring different metrics across X → EN and EN → X (e.g., TER/UTMOS vs. ChrF++/NISQA-MOS); these subsets preserve rankings (Spearman’s 𝜌>0.80 ) while cutting evaluation time by ≈ 2.5× . Human validation across dubbing, podcasts, and medical domains shows standalone MOS predictors fail to predict listener preference, while top domain-specific metrics correlate with human judgment ( 𝜌 ≥ 0.90 ). We release COMPASS as a foundation for domain-aware S2ST evaluation.

Abstract:
Transformers rely on position embedding mechanisms in long context modeling in most cases. Rotary Position Embedding (RoPE) embeds positional information with independent 2D rotations, forming relative position terms in self-attention. However, its pairwise, block-based, and decoupled structure limits deep mixing and robustness across channels. We propose HD-RoPE, which extends RoPE from independent 2D rotations to higher-dimensional rotations and introduces a Paley-I orthogonal basis to obtain balanced, isotropic, and dense phase mixing within each rotation subspace. This significantly enhances channel coupling and rotational degrees of freedom while maintaining orthogonal stability and the relative position closure property. Furthermore, HD-RoPE is easily optimized for engineering efficiency without introducing additional trainable parameters. We have conducted extensive evaluation results demonstrating that HD-RoPE achieves significant performance improvements over standard RoPE across various popular benchmarks and in both long and short contexts.

Abstract:
A significant portion of recent research on Large Language Model (LLM) alignment focuses on developing new policy optimization methods based on Group Relative Policy Optimization (GRPO). Two prominent directions have emerged: (i) a shift toward sequence-level importance sampling weights that better align with the sequence-level rewards used in many tasks, and (ii) alternatives to the PPO-style clipping that aim to avoid the associated loss of training signal and entropy collapse. We introduce Soft Sequence Policy Optimization, an off-policy reinforcement learning objective that incorporates soft gating functions over token-level probability ratios within sequence-level importance weights. We provide theoretical motivation for SSPO and investigate practical modifications to improve optimization behavior. Empirically, we demonstrate that SSPO improves training stability and performance both in mathematical reasoning and coding tasks.

Abstract:
Large language models (LLMs) are increasingly used as evaluators to assess output quality and preference alignment, yet providing reliable guarantees of agreement with human judgments remains challenging. Recent work introduces confidence-thresholding methods that provide such guarantees for pairwise comparisons, relying on the assumption that higher estimated confidence implies lower disagreement risk with humans. However, this assumption can break down when the number of candidate responses increases, since distributing probability mass across many alternatives can distort confidence estimates. To address this issue, we propose a Localize-Then-Decide framework. First, conformal prediction localizes a small shortlist that contains the human-preferred response with high probability. Then, a calibrated confidence-based rule selectively chooses a single response from this shortlist or abstains. This design restores the monotonic relationship between confidence and disagreement risk and enables high-probability agreement guarantees. Experiments with multiple candidate sizes across several datasets and judge LLMs demonstrate that our framework consistently achieves higher guarantee success rates and substantially higher coverage than single-stage baselines.

Abstract:
Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test time. We study whether this signal can define the inner-loop objective for test-time training (TTT) in pretrained long-context language models. Existing in-place TTT methods store prompt information in fast weights, but they train each write to match a learned shortcut target: an internal pattern that may be easy to predict, yet is not the trajectory the model uses to form its next-token distribution. We introduce Test-Time Training with Next-Token Prediction (TTT-NTP), a drop-in fast-weight adaptation method for pretrained LLMs that instead supervises updates using the model’s own next contextual hidden state. This makes each local write follow the same causal computation that supports next-token prediction, without aggregating over a neighborhood of positions: the value target is a pointwise linear projection of a single next-position contextual state. Across four backbones from three model families, Llama-3.1-8B, Mistral-7B-v0.3, Qwen3-4B, and Qwen3-0.6B, TTT-NTP is the only method that consistently improves RULER Full-13 over the released backbone, by 2.9 to 4.1 points, while aggregate commonsense and knowledge performance is preserved. Our code is publicly available at https://github.com/yancyou/TTT-NTP.

Abstract:
Information-seeking agents have emerged as a powerful paradigm for knowledge-intensive tasks, yet today’s systems remain specialized for the open web, documents, or local knowledge bases, hindering scalable and cross-domain deployment. We present the first systematic empirical study of consolidating these information-seeking agents into a single foundation agentic model. We compare two paradigms— data-level mixing , which trains a unified model on a mixture of datasets, and parameter-level merging , which merges independently trained experts in parameter space—across 3 training scenarios, evaluating 26 representative parameter-level methods on 10 benchmarks. To compare across heterogeneous benchmarks, we introduce a geometric Composite Score and an Imbalance Score that describe overall performance and task skew. Our analysis shows that (i) well-designed parameter-level merging attains parity with data mixing at a fraction of its training cost and is order-agnostic; (ii) parameter-level merging structurally preserves out-of-domain capabilities that data mixing universally forgets; and (iii) cross-scenario stability is strongly tied to consolidation quality. We distil our observations into a method-selection guide and design principles for next-generation merging operators.

Abstract:
Steering vectors are a flexible mechanism for controlling large language models (LLMs) along interpretable behaviours. Multi-behaviour steering, however, remains challenging due to interference. Individual steering vectors have conflicting directions or override one another, which not only impedes controllability but also often degrades generation results. In this work, we first analyse how multi-behaviour compositions of steering vectors affect model outputs, characterising how interactions between steering directions result in non-additive, position-dependent distributional shifts in generation. We then introduce an interference-aware allocation (IAA) method for multi-behaviour steering that minimises the cross-behaviour interference by dynamically adjusting the magnitudes of steering vectors at each generation step. Without instruction prompting, our method ties the best average on Sycophancy (76.4), achieves the best average on Refusal (80.8), and is far more fluent on Myopic-reward than the strongest baseline (67.1 vs. 41.7). With full behavioural instructions, it improves the all-prompt average from 88.2/94.8/81.2 to 95.7/98.7/95.6 while maintaining high fluency (87.3/90.5/87.7).

Abstract:
As one of the most critical challenges in large language models, contextual faithfulness directly determines their reliability in knowledge-intensive applications. This task is particularly challenging as it requires balancing factual consistency with generation efficiency. Contrastive decoding methods require dual forward passes (with and without context) to compare model outputs, doubling inference computational overhead, while post-training alignment demands extensive reinforcement learning with substantial computational overhead. To address this challenge, we present SFAD, a speculative decoding framework that enhances contextual faithfulness without inference degradation. We first construct ConFide, a preference dataset with fine-grained atomic perturbations, to train a context-faithful draft model via Direct Preference Optimization. During inference, Epistemic Friction detects potential hallucinations by quantifying distributional tension weighted by specialist certainty. When friction exceeds the threshold, Asymmetric Logit Steering refines the target distribution through residual-based logit injection; otherwise, standard speculation proceeds. Extensive experiments demonstrate that SFAD substantially improves faithfulness while achieving 2.48× speedup, offering a practical solution for efficient LLMs.

Abstract:
Compositional generalization refers to correctly interpret novel combinations of known primitives, which remains a major challenge. Existing approaches often rely on supervised fine-tuning, which encourages models to imitate target outputs. This token-level training paradigm fails to capture the global compositional structure required for generalizing to unseen combinations. In this work, we investigate whether compositional generalization can instead be improved through outcome-level reinforcement learning. We adopt Group Relative Policy Optimization to optimize models based on feedback on their final outputs. Within this framework, we explore both a simple binary outcome reward and a composite reward that provides additional composition feedback. Experiments on multiple compositional benchmarks show that reinforcement learning improves compositional generalization compared to supervised fine-tuning. Further analysis reveals that supervised models tend to overfit frequent training compositions, whereas reinforcement learning improves compositional generalization by reshaping the output distribution, particularly for more complex composition types.

Abstract:
Transformer language models process input provided as subword fragments, but natural language semantics usually rely on word-level concepts. Detokenization is the process where models reconcile these two facts, aggregating subwords into word-level representations through their computation. Prior work has found that this takes place mostly in early-to-middle layers, but so far the exact mechanics of the process have not been pinned down. We venture deep into detokenization using activation patching in controlled paired experiments that isolate the contribution of different model components, localizing English detokenization in Llama2-7B to a two-stage process at Layer 1. Attention transmits a token-specific signal from nonfinal subwords, using sequential relays if necessary, while the MLP composes it with the local embedding. This two-stage structure generalizes to twelve models from eight families, but the depth over which it takes place depends on the flavor of positional encoding: RoPE-based models detokenize over 1 to 5 layers, while learned-absolute models take 5 to 10. Finally, we provide a probe for determining the success of the detokenization process based on early-layer activations alone, performing at 0.94-0.97 AUROC depending on the amount of context.

Abstract:
Large language model-driven Multi-Agent Systems (MAS) face two key challenges in long-horizon tasks: collaboration workflows struggle to adapt to dynamic task states, and accumulated inter-agent interactions lead to lengthy, noisy contexts that degrade model performance and efficiency. To address these challenges, we propose MAS-OS, a collaborative Operating System (OS) for MAS. MAS-OS consists of three core modules: Memory Manager, Agent Scheduler, and Context Allocator. The Memory Manager maintains a structured memory pool and a global summary of agents’ interaction history. Based on the evolving memory states, the Agent Scheduler selects the next active agent, while the Context Allocator provides task-relevant memories to support effective context utilization. Experiments on four benchmark datasets demonstrate the effectiveness of MAS-OS. Compared with SOTA baselines, MAS-OS achieves an average improvement of 17.42% under the Llama-3.1-8B backbone. The code is available at https://anonymous.4open.science/r/MAS-OS-AD34.

Abstract:
This paper studies what discriminatively trained reward models (RMs) memorize by measuring counterfactual memorization on two human preference datasets. We show that RMs 1) misallocate memorization to easy, high margin preference pairs, 2) memorize dataset-specific shortcuts (e.g., model identity, user sampling strategy), and 3) overgeneralize simple heuristic correlates of human preference (e.g., length, compliance) when confronted with unseen preference pairs. Overall, our findings indicate that discriminative training of RMs from human preference data results in biased RMs not yet capable of judging response quality in context-dependent scenarios.

Abstract:
The emergence of open data portals necessitates more attention to protecting sensitive data before datasets get published and used. To do so effectively, we observe the need to refine and broaden our definitions of sensitive data. We argue that the sensitivity of data depends on its context, and that this context should be considered in methods for assessing data sensitivity. To this end, we introduce a framework, termed ConSenser, for contextual sensitive data detection based on two aspects: 1) type contextualization, which considers the type of data values within the overall context of the dataset or document to assess their true sensitivity, and 2) domain contextualization, which assesses the sensitivity of data values informed by domain-specific information external to the dataset, such as geographic origin of a dataset. Experiments instrumented with language models confirm that type-contextualization significantly reduces the number of false positives for type-based sensitive data detection and reaches a recall of 94% compared to 63% with commonly used tools. Moreover, domain-contextualization through retrieval of sensitivity rules effectively grounds sensitive data detection in relevant context in non-standard data domains. A case study with humanitarian data experts also illustrates that context-grounded explanations provide useful guidance in manual data auditing processes. We open-source the code and annotated datasets at https://github.com/trl-lab/sensitive-data-detection.

Abstract:
Current large language models perform next-token prediction under the implicit assumption that all tokens lie in a shared Euclidean embedding space, which treats every token uniformly. However, natural language contains rich structural relationships, including hierarchical, compositional, and cyclic patterns, which the flat geometry of Euclidean space cannot model well. To address this limitation, we propose rLLM (Riemannian Large Language Modeling), which introduces a token-adaptive positive-definite diagonal metric tensor into the LM head. This metric dynamically adjusts the local quadratic score for each hidden state and its context, allowing the model to better capture the relationships among tokens and prompts, with less than 0.01% additional parameters. We train rLLM from scratch across three different scales (2.8B to 5.0B parameters), covering the full pretraining, SFT, and RL pipeline. Extensive experiments show that rLLM improves Base CORE, a composite of 22 in-context learning tasks, by up to +2.5 pp at the two smaller GPT scales and obtains higher ChatCORE at all three scales, with gains up to +3.0 pp after RL. These results suggest that revisiting the geometric assumptions underlying token representations is a promising direction for building more expressive and capable language models. Code is available at https://github.com/digailab/riemannianllm.

Abstract:
Large language models (LLMs) have demonstrated remarkable performances in various tasks. However, the performances of LLMs heavily depend on the input prompt. This has given rise to a number of recent works on prompt optimization. However, the previous works often require the availability of a numeric score to assess the quality of every prompt. Unfortunately, when a human user interacts with a black-box LLM, it is often infeasible and unreliable to attain such a score. Instead, it is usually significantly easier and more reliable to obtain preference feedback from a human user, i.e., showing the user the responses generated from a pair of prompts and asking the user which one is preferred. Therefore, in this paper, we study the problem of prompt optimization with human feedback (POHF), in which we aim to optimize the prompt for a black-box LLM using only human preference feedback. By drawing inspirations from dueling bandits, we design a theoretically principled strategy to select a pair of prompts to query for preference feedback in every iteration, and hence introduce our algorithm named automated POHF (APOHF). We apply our APOHF algorithm to a variety of tasks, including optimizing user instructions, prompt optimization for text-to-image generative models, and response optimization with human feedback (i.e., further refining the response using a variant of our APOHF). The results demonstrate that our APOHF can efficiently find a good prompt using a small number of preference feedback instances.

Abstract:
Knowledge distillation can transmit unintended behavioral traits from a teacher model to a student through training data that appear semantically unrelated to those traits, a phenomenon known as subliminal learning. Although recent work has established this effect, its training dynamics and mitigation remain underexplored. We study subliminal learning in open-weight language models ranging from 1.5B to 8B parameters, covering the Qwen, Gemma, and Llama families in number-sequence and chain-of-thought settings. Rather than evaluating only final models, we track trait-related probabilities throughout fine-tuning and find that subliminal acquisition can be highly non-monotonic, with transient spikes, reversals, and trait-specific failures of transfer. We then introduce liminal training, an annealed KL-regularized fine-tuning method that constrains early drift from the base model. Across our experiments, liminal training substantially reduces subliminal trait acquisition while largely preserving task gains, outperforming paraphrasing and layer freezing as mitigation strategies. The effect also extends beyond animal preferences: in a French-language response-style experiment, liminal training suppresses language transfer while retaining much of the GSM8K improvement. Finally, we show that KL timing matters: early regularization is more effective than late regularization, and sweeping the regularization strength reveals an empirical trade-off between task learning and trait suppression.

Abstract:
Parametric knowledge in Large Language Models is not equally accessible across languages. As a result, standard inference techniques often struggle to surface localized facts, leading to failures in cross-lingual knowledge transfer and consistency. In this work, we investigate techniques for accessing hidden factual knowledge by exploring the prompted questions across languages. We identify four inherent dimensions of cross-lingual exploration that directly govern parametric knowledge retrieval and evaluate them on multilingual factual benchmarks covering 17 typologically diverse languages. Our results demonstrate that cross-lingual exploration significantly improves knowledge transfer and factual recall, representing a more efficient compute Pareto frontier than native-language scaling. Furthermore, we report similar performance improvement in cross-lingual consistency, exceeding what can be explained by accuracy gains alone. Overall, our work demonstrates the potential of multilingual exploration to improve parametric knowledge recall.

Abstract:
Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting them typically demands expert intuition and a nuanced understanding of the task. Moreover, the most impactful prompts often hinge on subtle semantic cues, ones that may elude human perception but are crucial for guiding LLM behavior. In this paper, we introduce PRL (Prompts from Reinforcement Learning), a novel RL-based approach for automatic prompt generation. Unlike most prior methods, PRL can also generate novel few-shot demonstrations rather than selecting only from examples seen during training. Empirically, our approach achieves state-of-the-art performance across a diverse set of benchmarks, including text classification, text simplification, and mathematical reasoning, while remaining competitive on text summarization. Our code is available at https://github.com/Batorskq/prl.

Abstract:
The task of event graph completion (EGC) aims to predict the missing relations between two events in a heterogeneous event graph (EG) to enhance its applicability in downstream tasks. Given that event graphs inherently contain both semantic and structural information, and that the effective aggregation of these two types of information can mutually reinforce their expressive power, we propose a Multi-Expert Routing (MER) framework, which leverages the mixture of experts (MoE) architecture to dynamically select appropriate experts for collaboratively addressing different key aspects of the EGC task. Specifically, our approach incorporates multiple specialized experts, leveraging prompt learning and a graph attention network, to independently model distinct semantic and structural features within EGs. Furthermore, an adaptive expert routing (AER) module is designed to dynamically fuse and select the missing relation predictions from the experts, thereby completing the EGs. Extensive experiments conducted on the EGC-MAVEN dataset demonstrate that our MER framework outperforms the state-of-the-art competitors.

Abstract:
Reading behaviour varies not only with linguistic input, but also with reader proficiency. In this study, we investigate whether the layer-wise relationship between surprisal from large language models (LLMs) and human gaze behaviour differs across readers with different levels of proficiency and across gaze measures. Using eye-tracking data from the MECO L2 corpus, we compare readers with high and low vocabulary proficiency on first-pass gaze duration (FPGD) and total gaze duration (TGD). We quantify the distribution of the predictive power of surprisal across model layers using Predictive Depth (PD). Across 12 tested LLMs, we find that readers with lower vocabulary proficiency tend to show deeper PD for FPGD, while this difference is smaller for TGD. Also, TGD itself shows deeper PD than FPGD in both proficiency groups. These patterns suggest that where predictive power is concentrated across LLM layers may be related to the timing and breadth of the reading processes captured by different gaze measures, and that this relationship can vary with reader proficiency. Our leave-one-out analysis further shows that the advantage of informative internal layers extends to unseen texts, although the practical improvements in prediction are limited. Overall, our results show that layer-wise LLM surprisal provides a useful perspective on variation in reading behaviour across both reader groups and gaze measures.

Abstract:
The ability to generate counterarguments is important for critical thinking and balanced discourse, yet existing approaches typically produce only a single counterargument, failing to capture the diversity and persuasiveness required in real-world debates. To address this limitation, we propose Persona-guided Tree-based Counterargument Generation (PTCG), a framework that combines Tree-of-Thoughts–inspired step-wise generation and pruning with speaker persona selection. By estimating the author’s persona from the original argument and incorporating speaker personas representing distinct perspectives, PTCG operationalizes perspective-taking and enables the generation of diverse counterarguments. Results from LLM-as-a-Judge, classifier-based assessment, and human evaluations indicate that PTCG shows consistent improvements in both the diversity and persuasiveness of counterarguments compared to baseline methods.

Abstract:
Weight-only post-training quantization (PTQ) for large language models (LLMs) typically ends with the projection of floating-point weights onto a uniform integer lattice, introducing structured output error due to non-zero activation means. Existing bias correction methods compensate for this shift by adding bias terms, which is incompatible with modern bias-less architectures such as LLaMA and Mistral and only addresses the error indirectly. We propose Complementary Lattice Correction (CLC), a lightweight post-quantization method that corrects projection error directly on the integer lattice without modifying the model architecture. CLC performs a budget-constrained greedy descent that selectively flips quantized weights to adjacent lattice levels. We further show that CLC provably reduces the first-moment error and admits a worst-case bound on the layer-wise mean-squared error change, which certifies non-increase under a verifiable budget condition. Experiments across multiple models, quantizers, and bit widths show that CLC consistently improves perplexity and downstream accuracy with negligible overhead.

Abstract:
Large Language Models (LLMs) can be trained to perform chain-of-thoughts reasoning in order to improve the reliability of their responses. In this work, we investigate how explicit reasoning can be leveraged for Machine Translation (MT). We introduce a novel fragment-based reasoning framework in which the model first extracts parallel source-target fragments from retrieved similar examples, and uses these fragments as intermediate reasoning traces to produce the final translation. To train our model, we distill silver fragments and drafts from a larger teacher model. Our experiments with the Qwen3 model family, over 6 languages, including up to 5 domains per language, demonstrates that fragment-based MT significantly outperforms alternative methods like standard k-shot, or drafting.

Abstract:
Multimodal modeling represents a vital step from modality-agnostic reasoning toward world modeling. While early approaches predominantly rely on late-fusion that assembles encoders and frozen language backbones with output heads, recent efforts have shifted the paradigm toward native multimodal modeling (NMM) with the intrinsic integration of modalities for superior multimodal performance. Despite its potential, the design space of native architectures remains insufficiently defined. In this paper, we present the community with a formalized roadmap for this transition. Specifically, we formally define the architectural nativity, distinguishing mid-fusion and early-fusion from non-native paradigms. We further organize the existing native models through the lens of input-output duality into three categories: (i) Multi-to-Text for cross-modal comprehension with text-only output; (ii) Multi-to-Target for scenario-oriented generation e.g., image, audio and video generation, and (iii) Multi-to-Multi for unified modeling with symmetric input-output. We systematically unpack the end-to-end pipeline from industrial perspectives from: architectural coordination, massive data curation, to full-stack training recipes, inference deployment, and evaluation. Finally, based on our assessment of native multimodal development trends, we provide a forward-looking perspective on its future trajectory.

Abstract:
Retrieval-augmented generation (RAG) improves large language models (LLMs) with external knowledge, but this access path creates security risks distinct from inherent prompt-only or parametric-model flaws. We frame secure RAG as securing external knowledge access. We conducted a systematic search and curated 135 works on attacks, defenses, and security evaluation, and organized them with SLOT: a taxonomy along the attack Surface (S) and the corresponding defense Layer (L), with cross-cutting axes Objective (O) and attack-target scope (T). Mapping these studies onto an external knowledge-access pipeline, we expose three mismatches: target mismatch (T2 attacks outpace T2 defenses and evaluation), stage mismatch (S1 attacks outnumber L1 defenses), and signal mismatch (fluent, retrievable, corpus-fitting S1 attacks challenge anomaly-based L2/L3 defenses). Finally, we discuss directions for more realistic targets, surface-complete defense stacks, standardized evaluation, confidentiality, and multimodal and agentic systems, and release the screening process, selected-paper metadata, and machine-readable SLOT labels at https://github.com/TreeAI-Lab/Awesome-RAG-Security.

Abstract:
Multi-hop kinship reasoning is a natural testbed for LLM compositionality, but existing benchmarks (notably CLUTRR) cover only the descriptive Eskimo system. We introduce KinshipQA, a procedurally-generated benchmark covering seven anthropologically-documented kinship systems (Eskimo, Sudanese, Hawaiian, Iroquois, Dravidian, Crow, Omaha) and up to six reasoning hops, with a tunable simulator horizon that eliminates exact-instance pretraining overlap. Evaluating six LLMs, we find a 40.9% accuracy drop when reasoning shifts from biological multi-hop to culturally-marked classification on the five non-descriptive systems. The drop follows the anthropological hierarchy descriptive > bifurcate > generational > skewing, persists under chain-of-thought and few-shot prompting, and compounds with depth (10.6% at 5–6 hops). Humans given the override rule reach 89.0% versus 50.7% for LLMs, placing the failure in the model rather than the task. Two follow-up experiments suggest distinct contributors. A fictional-rule control swapping system labels and kin terms for invented strings raises accuracy by 6.1%, implicating familiar English surface forms. An in-context-rule probe prepending the override rule helps skewing systems (+17.1%) but hurts non-skewing systems whose baseline already exceeds ~60% (-13.4%), consistent with a missing skewing prior alongside rule interference where the model already has a working approximation.

Abstract:
As LLMs are increasingly integrated into agentic systems, they must adhere to dynamically defined, machine-interpretable interfaces. We evaluate LLMs as in-context interpreters: given a novel context-free grammar, can LLMs generate syntactically valid, behaviorally functional, and semantically faithful outputs? We introduce RoboGrid, a framework that disentangles syntax, behavior, and semantics through controlled stress-tests of recursion depth, expression complexity, and surface styles. Our experiments reveal a consistent hierarchical degradation: LLMs often maintain surface syntax but fail to preserve structural semantics. Despite the partial mitigation provided by CoT reasoning, performance collapses under structural density, specifically deep recursion and high branching, with semantic alignment vanishing at extreme depths. Furthermore, Alien lexicons reveal that LLMs rely on semantic bootstrapping from keywords rather than pure symbolic induction. These findings pinpoint critical gaps in hierarchical state-tracking required for reliable, grammar-agnostic agents.

Abstract:
Text watermarking enables provenance tracing of LLM-generated text by embedding detectable signals, while multi-bit watermarking further encodes payloads such as user identities or model versions. Scaling multi-bit watermarking to large payloads is challenging as exhaustive decoding grows exponentially with payload length. Existing methods commonly address this by segmenting the payload and decoding each segment independently, but the impact of this design on decoding accuracy remains insufficiently understood. In this paper, we revisit segmentation in multi-bit watermarking and reveal a drawback of the commonly adopted paradigm: segmentation splits the available token evidence among segments, leaving fewer observations for each local decoding decision and thereby reducing decoding reliability. We theoretically characterize this accuracy–efficiency trade-off and empirically validate the claim across diverse settings. Additionally, we introduce a distribution-balanced detector to mitigate message-dependent bias. Experiments across multiple datasets and models show that our method achieves improved accuracy and robustness under editing attacks compared to existing methods. Our code is available at https://github.com/YouduLiu/SDBMW

Abstract:
LLM-based agents are increasingly being developed for time-series problems, but their design choices vary substantially across task settings. This survey adopts a problem-driven taxonomy that organizes these systems by the time-series problems they address rather than by isolated technical components. We group existing systems into four categories: forecasting and reasoning, augmentation and synthesis, anomaly detection and diagnosis, and decision support. Within each category, we examine how task requirements shape agent architecture, tool use, and memory design. We further summarize representative datasets and environments, and compare reported model performance under shared or closely related settings. Overall, this survey offers a task-oriented guide to designing LLM-based agents for time-series problems and identifies open gaps for future work.

Abstract:
Consistency training encourages models to behave similarly across different contexts, and has shown promise for reducing misalignment. We broaden the scope of consistency training in two ways. First, we introduce two new internal consistency targets: MLP Consistency Training (MLPCT), which matches post-activation MLP states, and Attention Consistency Training (AttCT), which matches per-head attention distributions. Second, we apply consistency training to four additional safety threats: persona in-context learning attacks, adversarial frustration, prefill attacks, and conditional misalignment. Across several models and threat settings, we find that consistency training reduces misalignment well beyond the sycophancy and jailbreak settings studied in prior work. The effective consistency target depends on the structure of the threat: representation-level targets suit wrapper-style failures, while output-level BCT is the reliable choice for trajectory-level failures such as prefill, frustration, and conditional misalignment. We also find cases of cross-threat generalization, where training against one failure mode improves robustness to another, and identify a shared residual-stream mechanism underlying ACT, MLPCT, and AttCT, while distinguishing BCT as mechanistically distinct. Our results suggest that consistency training is a flexible and extensible framework for alignment, capable of unifying defenses against a broader class of model pathologies.

Abstract:
Sentence-level structural reasoning, reflected in relational connectivity patterns induced by attention distributions, plays an important role in grammatical acceptability judgment and figurative expression understanding. However, most existing scoring approaches rely primarily on token-level probability signals or sentence-level similarity, which do not explicitly capture structural relationships between tokens. We propose Scenario-Anchored Topology-Inspired Structural Scoring (SATS), a structure-aware comparison method that evaluates coherence between a scenario and multiple candidate interpretations using connectivity patterns derived from attention matrices. Unlike prior topology-inspired approaches that focus on pairwise comparisons, SATS directly supports multi-option reasoning while maintaining low computational cost. Experiments on BLiMP, FLUTE, and Pragmatics show that SATS improves encoder-based model performance in many settings and can approach the accuracy of much larger language models. These results suggest that explicit structural comparison can provide a useful complementary signal for the evaluated language understanding tasks.

Abstract:
We introduce Lingo-Blocksworld, a linguistically-informed diagnostic benchmark for evaluating the generalisation and planning abilities of Large Language Models (LLMs) in the Blocksworld (BW) domain. Unlike prior obfuscated BW variants, our method replaces content words with morphosyntactically well-formed nonsense words, preserving the grammatical scaffolding of the domain while removing lexical familiarity. This enables us to test a psycholinguistically-motivated hypothesis, and find systematic support across both reasoning and non-reasoning models, with degradation patterns that mirror syntactic bootstrapping from human language processing. Additionally, we test the ability of models to generalise planning solutions from the typical 4-block problems to 12 blocks. Overall, our results demonstrate that reasoning models stay robust where non-reasoning models collapse: non-reasoning model performance drops to near zero once problems exceed four blocks or have actions and predicates obfuscated, while reasoning models maintain strong performance until structural and lexical difficulty are combined. Manual error analysis pinpoints further differences. Reasoning model errors are more often skipped steps or incomplete instruction following, whereas non-reasoning models fail to derive plausible plans at all. By combining psycholinguistic motivation with classical planning evaluation, Lingo-BW allows us to shift research focus from whether LLMs plan to how their planning depends on linguistic surface form.

Abstract:
LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage. BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement. The converted model is adapted to representation learning through continual contrastive pre-training, followed by supervised contrastive fine-tuning with both similarity-distribution distillation and attention-relation distillation from a full-precision teacher. Beyond quantizing the encoder, BITEMBED further trains output embeddings to support multiple storage precisions meeting different storage needs in various scenarios. Experiments on MMTEB (eng, v2) with Qwen3-0.6B and Gemma3-270M show that BITEMBED is largely comparable to full precision teacher embedders. Moreover, BITEMBED flexibly obtains text embeddings of various precisions, achieving a trade-off between performance and storage cost.

Abstract:
Controlling text difficulty is crucial for adapting texts to individual language learners. Existing controllable text simplification methods condition generation on discrete proficiency-level tokens, treating these levels as independent categories. As a result, they fail to capture the ordinal structure of language proficiency and cannot provide fine-grained control beyond predefined levels. To address this limitation, we propose a continuous difficulty control method that adjusts simplification difficulty by editing the norm of a conditioning embedding derived from the input sentence semantics. The embedding is incorporated into the transformer decoder through attention-based conditioning, enabling continuous control over output difficulty while modeling ordinal relationships across levels. Unlike token-based approaches, our method supports interpolation between proficiency levels and flexible control over edit magnitude. Experiments on controllable text simplification benchmarks demonstrate improved difficulty controllability, simplification quality, and sentence difficulty estimation performance.

Abstract:
Model diffing studies the internal changes of a model from training to reveal learning mechanisms. To enhance interpretability, existing studies often strive for structural sparsity by removing redundant model components. However, we emphasize that for most fine-tuning methods, the fundamental source of internal change, i.e., the parameter updates, is non-sparse. The dense update mechanism contradicts the structural sparsity sought by researchers. Thus, we seek to bring sparsity to fine-tuning via an interpretability-driven method called circuit-tuning. It employs a heuristic prune-then-tune paradigm, forcing the model to optimize within a restricted subset of parameters. To test our method, we designed a novel task called Subject-Verb Disagreement (SVD) and conducted comprehensive explorations on it. Experimental results demonstrate that, compared to other fine-tuning methods, circuit-tuning achieves better sparsity in parameter updates, thus making it possible for faithful model diffing. Based on this, we performed stage-wise model diffing, revealing the evolution of model capabilities during fine-tuning from a new view. Furthermore, experiments on gender debiasing and larger models verify the practical utility of our approach. Our work provides a new methodology for model diffing-based interpretability and its applications.

Abstract:
Recent advancements in large language models have significantly enhanced the capabilities of agents in modeling long-term conversations. Despite these successes, existing approaches typically adopt a static heuristic paradigm, where information is passively archived without adaptive memory valuation. Consequently, these methods fail to self-evolve or align their memory management with evolving user needs. To address this, we propose ICML (InteraCtive Memory Learning), a multi-agent framework that transforms the memory mechanism from a passive archive into a learnable, interactive memory policy. Specifically, we first employ a session synthesis pipeline to generate expert data, facilitating rapid test-time adaptation in unseen scenarios. Building on this, ICML utilizes an online reinforcement learning mechanism where a Planner agent selectively encodes high-value information and a Trigger agent dynamically retrieves it to optimize response quality, whereby the two agents co-evolve through continuous interaction feedback. Crucially, both agents are synchronized through a delayed reward mechanism that propagates future feedback back to earlier storage decisions, ensuring memory policies are precisely aligned with user expectations. Experimental results demonstrate that ICML significantly outperforms strong baselines, exhibiting the unique capability to continuously improve response quality as interactions accumulate.

Abstract:
Generating coherent meta-reviews from multiple peer reviews is challenging when reviewer evidence conflicts and varies in reliability. Existing approaches typically formulate meta-review generation as a multi-document summarization task and aggregate reviewer feedback uniformly, making it difficult to determine which opinions should be prioritized under disagreement. In this paper, we study meta-review generation through reliability-aware evidence aggregation. Our framework first extracts aspect-level opinions from peer reviews and identifies conflicting evidence within each aspect. It then estimates opinion-level support and review-level quality to measure evidence reliability. Based on these signals, the framework assigns reliability-aware weights to reviewer feedback, enabling the generator to prioritize better-supported arguments while preserving diverse perspectives. Experiments demonstrate that our method consistently improves meta-review generation over strong baselines on both automatic and human evaluations, with clear gains in conflict recognition and resolution under high-conflict review scenarios. The code and implementation details are publicly available at https://github.com/Wangxz729/reliability-aware-meta-review.

Abstract:
Large language models have achieved strong performance on medical reasoning benchmarks, yet their deployment in clinical settings demands rigorous verification to ensure factual accuracy. While reward models offer a scalable approach for reasoning trace verification, existing methods face two limitations: they produce only scalar reward values without explicit justification, and they rely on single-pass retrieval that precludes adaptive knowledge access as verification unfolds. We introduce Med-TIV, an agentic framework that addresses these limitations by training medical reward models to iteratively query external medical evidence during evaluation. Our approach combines tool-augmented verification with an iterative reinforcement learning paradigm that requires only trace-level supervision, alongside an adaptive curriculum mechanism that dynamically adjusts training data distribution. Across four medical reasoning benchmarks, Med-TIV achieves substantial gains over existing methods, improving MedQA accuracy by 23.5% and MedXpertQA by 32.0% relative to the base generator. Crucially, Med-TIV demonstrates an 8× reduction in sampling budget compared to prior reward model baselines. These findings establish that agentic medical reward modeling offers a principled path toward more reliable medical reasoning systems.

Abstract:
When information extraction fails, a natural instinct is to improve the model doing it: for example, by scaling it up or refining its reasoning. In this paper, we show that another part of the pipeline matters at least as much: the queries used to elicit this information. Across four clinical benchmarks and five LLMs, improving the question design alone raises performance by 18.6 F1-score points, i.e. more than using larger extraction models. To make such question design learnable, we introduce List of Questions (LoQ), which generates document-specific question sets, and FeedQ, a feedback-driven optimization method that iteratively refines questions against extraction outcomes. The resulting optimized questions can be used to train lightweight generators: with fine-tuning, 4B-parameter models match or outperform expert-derived baselines and substantially exceed the performance of much larger untuned models. We release a dataset of 12,820 optimized questions to support a broader shift in information extraction research toward treating question design as a first-class problem.

Abstract:
Multimodal agents increasingly choose tool calls from screenshots, documents, and webpages, where a false perceptual claim can turn hallucination from an answer-quality error into an authorization failure. We formalize this failure as hallucination-to-action conversion (H2AC) and propose evidence-carrying multimodal agents (ECA), in which model text may propose actions but cannot authorize them: constrained DOM/OCR/AX verifiers issue typed certificates for action-critical predicates, and a deterministic gate executes only what the certificates support. Across verifier red-teaming, 7,488 replayed authorization traces, and end-to-end and live-browser evaluations, ECA executes no unsafe action, while GPT-5.4 judge baselines admit 79–99% of unsafe actions under the same threat model; benign actions are occasionally deferred to user confirmation (6.4% on live pages) but never hard-blocked. The guarantee is conditional, extending exactly as far as the verifiers, schemas, and gate implementation that form ECA’s trusted base.

Abstract:
Reinforcement learning is increasingly used to align multimodal large language models (MLLMs). However, higher proxy rewards do not necessarily lead to better task performance, especially when visual evidence is difficult to verify with text-only or weakly grounded reward signals. We systematically study reward hacking in MLLM reinforcement learning by varying reward design, data ambiguity, model scale, and RL algorithm. To separate RL-induced failures from pre-existing weaknesses, we introduce the Newly Rewarded Failure Rate (NRFR), which measures the hacking rate among samples where RL achieves higher proxy reward than the SFT baseline. Our analysis yields three main findings. First, RL can actively create new failures rather than merely amplify existing ones, and the effect is especially pronounced near ambiguous decision boundaries. Second, robustness depends jointly on model scale, reward design, and RL algorithm, and scaling mitigates hacking but cannot replace reliable rewards. Third, incorporating visual evidence helps only when verification is reliable, since keyword-based verification introduces additional exploitation opportunities whereas VLM-based semantic verification can reduce reward hacking. Overall, multimodal reward hacking is a systematic consequence of optimizing imperfect reward signals, highlighting the need for rewards and verifiers that remain reliable under optimization pressure.

Abstract:
Self-evolution methods enhance code generation through iterative ”generate-verify-refine” cycles, yet existing approaches suffer from low exploration efficiency, failing to discover algorithmically superior solutions within limited budgets. We identify the root cause as the lack of systematic controllability: execution feedback is not reliably converted into targeted search decisions across route planning, edit granularity, and experience reuse. We propose C ontrolled S elf- E volution (CSE), a framework that couples three components into an integrated control architecture. Diversified Planning Initialization provides route diversity through structurally distinct algorithmic strategies. Modular Evolution provides feedback attribution and module-level editing by decomposing programs into functional modules and applying targeted mutation and compositional crossover. Hierarchical Evolution Memory maintains search state within tasks and distills reusable optimization patterns across tasks. Experiments on EffiBench-X demonstrate that CSE consistently outperforms all baselines across various LLM backbones. Further analysis reveals non-additive synergy among control layers, confirming that CSE’s effectiveness arises from component coupling rather than independent contributions. The code and evaluation implementation are publicly available at https://github.com/QuantaAlpha/EvoControl .

Abstract:
Generating a game is not the same as making one playable. Existing code-generation approaches often translate a prompt directly into an artifact, leaving interaction-level failures undetected. We argue that game generation requires a player and study two roles for graphical user interface (GUI) agents. First, we introduce PlaytestArena , an evaluation environment containing 200 browser-based game-generation tasks across eight genres, each paired with rubrics of expected in-play behaviors. An independent GUI judge loads and plays each build to adjudicate these rubrics. Second, we propose Play2Code , in which a game agent and a rubric-blind GUI playtester iteratively generate, play, and refine games through shared memory. The playtester provides gameplay traces and actionable feedback, while a separate GPT-5.5 judge assigns final benchmark scores. Across three frontier backbones, Play2Code achieves a 66.8% rubric pass rate, outperforming single-pass and agentic-coding baselines by 37.1 and 14.6 points, respectively. Its scores also improve monotonically across refinement rounds. Further analysis shows that GUI-agent feedback is more traceable than human reports yet exhibits similarly idiosyncratic priorities. These results establish GUI playtesting as an evaluation and refinement signal for interactive code generation. Our project website is available at https://continual-game-generation.vercel.app/ .

Abstract:
Adapting general-purpose LLMs into domain-specialized dense retrievers typically requires large-scale training on mixed-domain data. We show that merging independently trained domain-specialized experts consistently exceeds this approach across four decoder-only LLM families (0.6B–7B), four merging methods, and twelve medical and general retrieval tasks from MTEB, suggesting that parameter-space composition captures complementary domain strengths that large-scale mixed-domain training averages out. To further maximize expert quality, we introduce Synthesize-Train-Merge (STM), a modular framework that synthesizes hard negatives with a top-tier LLM and fine-tunes domain-specialized experts via LoRA before merging them, without continual pre-training. Synthesized hard negatives yield the largest gains for smaller models, and STM achieves strong performance on biomedical retrieval tasks while maintaining competitive general-domain results across all four backbone families.

Abstract:
Large Language Models (LLMs) tend to generate a long reasoning chain when solving complex tasks. However, as the reasoning chain extends, critical intermediate steps and the original prompt will be buried in the context, receiving insufficient attention and leading to errors. In this work, we present ATAR, a novel reasoning method that leverages the inherent reasoning structure to steer LLM attention. Our experiments show that ATAR outperforms SOTA methods across six benchmarks, achieving up to 15.39% absolute improvement. Furthermore, with ATAR, “non-reasoning” models achieve comparable or even better performance compared to reasoning models of the same size in most benchmarks. Finally, our ablation studies show that the attention alignment component contributes significantly, and that these improvements are persist under different attention-steering backends.

Abstract:
Complex question answering over large document collections requires models to identify missing information, issue sub-queries, and iteratively gather evidence before producing an answer. Reinforcement learning offers a natural framework for training such behaviour, but standard GRPO-style training provides only a sparse final-answer reward and assigns the same advantage to every token in a rollout. We introduce PACE-GRPO (Potential-based Action Credit Estimation GRPO), a variant of GRPO that combines potential-based reward shaping with action-level credit assignment. PACE-GRPO computes a state potential from inexpensive retrieval-derived signals, including retrieval utility, coverage, and passage diversity, and uses potential differences to provide step-level feedback. Unlike prior intermediate-reward methods, it requires no gold evidence annotations, teacher traces, reward model, or additional forward passes. Across six multi-hop QA benchmarks, PACE-GRPO learns a tool-use policy that selectively invokes the more expensive retrieval operation only when it is beneficial. This adaptive tool use yields consistent improvements over standard GRPO and strong search-augmented baselines. Our code and data are in https://github.com/lauhaide/pace.

Abstract:
Pairwise Ranking Prompting (PRP) elicits pairwise preference judgments from an LLM, which are then aggregated into a ranking, usually via classical sorting algorithms. However, judgments are noisy, order-sensitive, and sometimes intransitive, so sorting assumptions don’t match the setting. Because sorting aims to recover a full permutation, truncating it to meet a call budget does not produce a dependable top-K. We thus reframe PRP reranking as active learning from noisy pairwise comparisons and show active rankers are drop-in replacements that improve NDCG@10 per call in the call-constrained regime. Our noise-robust framework also introduces a randomized-direction oracle that uses a single LLM call per pair. This approach converts systematic position bias into zero-mean noise, enabling unbiased aggregate ranking without the cost of bidirectional calls.

Abstract:
Recent advances in multimodal foundation models have enabled powerful generic embeddings. However, real-world applications—such as e-commerce or healthcare—rely on domain-specific patterns that these generic models often fail to capture precisely. Adapting them without readily available labeled data remains a critical challenge. In this paper, we propose MM-UDA, a robust unsupervised domain adaptation framework designed to systematically convert a generic multimodal embedding model into a domain expert using only unlabeled target data. Our framework employs a two-stage strategy. First, we introduce a within-modality pseudo-labeling task refined by Gaussian Mixture Model filtering, which rapidly adapts the model to domain-specific features while suppressing label noise. Then, we align modalities through a score-guided learning scheme to refine cross-modal pairing. Extensive evaluations on three real-world datasets demonstrate that MM-UDA both significantly enhances various backbone models and consistently outperforms competitive baselines, confirming its effectiveness for cross-modal retrieval in specialized domains. The source code is publicly available at https://github.com/hrf516/MM-UDA.

Abstract:
Multi-agent debate can improve large language model reasoning by eliciting diverse hypotheses and critiques, yet its performance is often constrained by weak moderation. Common pipelines rely on fixed budgets, agreement-based stopping, or untrained judges, leading to redundant deliberation and unreliable evidence aggregation. We cast moderation as a meta-cognitive process, monitoring debate utility, controlling deliberation, and adjudicating a final answer, and introduce Meta-Moderator , a learnable framework that dynamically regulates debate and decides when to finalize an answer. Meta-Moderator is trained independently of the debaters via outcome-driven policy optimization, making debate regulation an explicit capability rather than an incidental effect of prompting. Across five benchmarks, Meta-Moderator outperforms widely used decision layers and transfers across tasks and system configurations. Further analyses show that it allocates debate more selectively and reduces mis-aggregation after informative hypotheses appear.

Abstract:
On-policy self-distillation (SD) improves LLM reasoning by using teacher-side privileged information (PI) to turn sparse verifier outcomes into dense token-level supervision. Existing methods usually assume trusted PI, such as reference answers or successful traces. We ask whether PI can instead come from an experience-derived skill bank, where retrieved skills are compact and reusable but may also be irrelevant or misleading. We propose Skill-Conditioned Gated Self-Distillation (SGSD), which formulates skill-based SD as teacher hypothesis validation rather than unconditional imitation. SGSD retrieves skill-mistake pairs, constructs a multi-teacher pool, and lets all skill-conditioned teachers score the same plain-prompt student rollout. The verifier validates each teacher’s polarity: supporting a success or suppressing a failure gives positive supervision, while the opposite stance is reversed. A robust gated objective then distills informative teacher-student disagreements while suppressing uncertain or extreme signals. Experiments on multiple mathematical reasoning benchmarks show that SGSD consistently improves over GRPO and remains competitive with answer-conditioned OPSD under a weaker PI assumption. For example, on Qwen3-1.7B, SGSD outperforms GRPO by 6.2% and OPSD by 1.7% on AIME24, AIME25, and HMMT25.

Abstract:
Multimodal large language models (MLLMs) must often arbitrate between visual evidence and world-knowledge priors when an image conflicts with commonsense expectations. Existing work has demonstrated that activation steering vectors can bias model outputs toward either source of information, but it remains unclear whether this arbitration mechanism generalizes across languages. To investigate this, we construct a multilingual extension of the Visual CounterFact color split and derive activation steering directions that contrast image-grounded answers with prior-based answers. Across 12 languages, we find that English-derived and shared multilingual steering directions consistently shift model outputs on non-English prompts, suggesting that visual-prior arbitration has a substantial cross-lingual component. At the same time, we find that performance gains on common knowledge questions can come at the cost of image-grounded accuracy, a trade-off that is particularly pronounced for lower-resource languages. Code and data are available at https://github.com/elffiona/cross-lingual-visual-prior-steering.

Abstract:
Embedding-based topic models such as BERTopic routinely produce hundreds of fine-grained, overlapping topics that must be reduced before downstream use. Existing approaches couple reduction with outlier rejection, rely on local geometric linkage, or use costly and stochastic LLM prompting. We introduce Vendi Clustering (VC), a topic reduction method that reframes the problem as global diversity-preserving compression: at each step, VC selects the merge that induces the smallest loss in the Vendi Score, a spectral measure of diversity. A naive implementation of this global criterion is prohibitive at O(n 6 ) ; we derive algebraic update rules at order q = 2 that evaluate each candidate merge in O(1) via cached row-square sums and a lookahead Gram matrix, reducing total complexity to a tractable O(n 3 ) . On 20 Newsgroups and AG News, VC outperforms standard reduction baselines on coherence and diversity, surpasses LLM-prompted reduction on the majority of metrics against both an open-weight model and a frontier API model, with topic diversity gains of up to 35% while running over 40 × faster.

Abstract:
In this paper, we address the problem of iteratively choosing to defer from a small Language Model (LM) to a larger one in order to balance accuracy and cost in a closed question-and-answer setting. Our deferral strategy is defined as follows: we identify the important sentences in the context, per the models, measure the differences in the ranking of their importance with the normalized Kendall Tau Distance and threshold that value to create a binary decision. We find that we can tune the desired cost or accuracy using this threshold value. Although it is outperformed on a dataset by a simple uncertainty average threshold, our method provides a more interpretable mechanism. Indeed, the attributions are found using the attributions of the generating LLM, which can be easily checked by a human. We find that our method offers a good interpolation of base model accuracies, especially in low cost regimes.

Abstract:
Large language model agents receive instructions from many sources—system messages, user prompts, tool outputs, other agents, and more—each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH, requiring models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even current frontier models perform poorly (~40% accuracy), underscoring the urgent need for methods that target fine-grained, scalable instruction conflict resolution in agentic settings.

Abstract:
Cochrane systematic reviews pair each technical abstract with a plain-language summary (PLS), but most biomedical papers lack a PLS. We propose a retrieval-augmented generation (RAG) pipeline that generates Cochrane PLS from biomedical abstracts. The pipeline exploits the section-level alignment of the Cochrane-sections dataset in two ways. First, it provides the LLM with abstract-PLS section pairs from topically related reviews, giving more granular in-context examples than whole abstract-PLS pairs. Second, each retrieved PLS section is split into its rephrased and inserted sentences, showing the LLM both the rephrasing and the insertion behavior it should reproduce. Our pipeline outperforms a non-RAG LLM and three RAG ablations on the Cochrane-sections test set, with the same gain trend across three different LLM families.

Abstract:
Long-horizon agents with structured actions execute tasks through sequences of dependent decisions, so a local decision mistake can change later observations and cause the task to fail. Stronger expert models offer a natural source of supervision, yet direct imitation or behavior cloning treats many differences between expert and base actions as equally useful targets. In this paper, we propose counterfactual selective imitation (CSI), a method that uses same-state counterfactual comparison to identify positive return differences. CSI assigns expert targets only when counterfactual rollouts show a high positive return difference, and routine states receive base-policy targets. The learned update is restricted to structured action spans, leaves the base model frozen, and does not require expert queries or replay at deployment. Experiments on AppWorld, ToolSandbox, and WebShop show that CSI raises average task success from 33.3% to 49.7% over the Qwen3-8B base and reduces average invalid/tool error from 9.3% to 4.9%. Compared with the strongest same-deployment adaptation baseline, CSI improves average task success by 1.3 points with only 5% additional deployed-inference latency.

Abstract:
Chess engines have long achieved superhuman playing strength. However, the underlying strategy behind their move suggestions is difficult for human players, even skilled ones, to comprehend. Motivated by this, we propose the task of chess strategy verbalization, which is to describe chess strategies in natural language. We design (i) a pipeline for verbalizing strategies and (ii) an evaluation framework for objective evaluation of generated strategy descriptions. Our experiments show that natural language is a promising and interpretable medium for communicating strategic information to both human and LLM players. We glean additional interesting insights, including (a) the importance of evaluating strategies beyond the main line, (b) the limitations of pure concept-based descriptions, and (c) the limitations of relying on LLMs rather than humans for evaluation.

Abstract:
Improving GUI agents typically relies on behavior cloning on expert trajectories. However, as the current policy deviates from the expert policy, it inevitably encounters policy-induced off-trajectory states during closed-loop execution, i.e., states that fall outside the expert trajectories. Since expert trajectories provide no demonstrations for these unseen states, such states receive no effective supervision, leaving the policy unable to select the correct action. To close this supervision gap, we propose Skill-Guided Continuation Distillation (SGCD), an iterative self-improvement framework. SGCD first runs the plain policy without skill guidance for a few steps to reach realistic off-trajectory states. From these states, a skill-guided policy then completes the task and produces successful continuations, which are mixed with expert trajectories to supply supervision over policy-induced off-trajectory states. The skills are extracted from both successful and failed rollouts, consisting of Continuation Plans, Critical Targets, Failure Traps, and Success Criteria. On OSWorld-Verified, SGCD improves the success rate of three base models from the low-30% range to over 50%, demonstrating its effectiveness and generality.

Abstract:
Current LLM-based multi-agent systems (MAS) periodically compress intermediate states to reduce inference-time token consumption, thereby attempting to incorporate more agents. However, naive scaling strategies face challenges. For example, in economic simulations, large-scale MAS typically discard semantically rich economic states, i.e., agent behavioral trajectories, which are key drivers of macroeconomic fluctuations. In this paper, we reveal a phenomenon in which agent heterogeneity gradually decreases during simulation, and propose Prospect-State Propagation for Multi-Agent Systems (PspMAS). Inspired by prospect theory, PspMAS decouples each agent’s micro state into a compact Prospect State and an expressive Semantic State. The former records psychological traces through a lightweight, parallelizable propagator and continuously injects heterogeneity into the system. The latter leverages the strong perception, reasoning, planning, and decision-making abilities of LLMs. These two components work complementarily, providing a scalable LLM-based multi-agent simulation solution.

Abstract:
Comparing videos to assess their semantic similarity remains a challenging task for current Large Multimodal Models (LMMs), which are primarily trained with single-video objectives. While most prior works have focused on computing global similarity, recent studies have introduced the more challenging problem of concept-based similarity, which measures how similar two videos are with respect to specific concepts (e.g., activity, location). However, no dedicated approaches have yet leveraged LMMs to explicitly address this task. In this paper, we aim to fill this gap introducing Video Caption Entailment for video similarity estimation (VICE): a dual-branch design that leverages LMMs for cross-video reasoning. Each branch jointly processes one video and the caption of the other, measuring their alignment for a specific concept via video-language entailment. The outputs from both branches are fused to produce a final similarity score, along with a textual explanation of the model reasoning. This approach leverages the model’s native single-video understanding while enabling fine-grained comparative reasoning between video and text. Experiments on ConViS-Bench and StepDiff show that VICE outperforms models that process both videos simultaneously, highlighting the promise of single-video reasoning for concept-based similarity estimation.

Abstract:
Cultural image retrieval is essential for organizing and analyzing large-scale digital heritage resources. However, existing multimodal retrieval methods usually encode a query as a holistic embedding, making it difficult to preserve logical operators and cultural relations during retrieval. Therefore, we introduce Multimodal Boolean Logic Culture Retrieval, a novel task that moves beyond general visual similarity by enforcing strict Boolean constraints over structured cultural elements. To tackle this, we propose CLAIR, a training-free framework that reformulates the retrieval objective to seamlessly integrate logical operations directly within the multimodal embedding space. Furthermore, we construct CulTi2.0, a dedicated benchmark designed to evaluate diverse logical query forms based on cultural visual atoms. Extensive experiments show that while current multimodal models struggle with explicit logical queries, CLAIR provides consistent and generalizable improvements as a plug-and-play solution across various datasets and model families. Code and dataset are publicly available at https://github.com/AI4Culture/CulTi2.0.

Abstract:
Interpreting properties of LLM-generated data is important for understanding its utility and limitations across learning tasks. In this work, we characterize synthetic data through sample-level learnability, studying variation among LLM families and scales, alongside human-written data as a reference. We generate synthetic datasets spanning single- and multi-label classification, labeling, and tree prediction tasks using LLMs from multiple families and scales. We then derive empirical data distributions from encoder training dynamics for both machine and organic data, and estimate the robustness of these distributions across encoders. Finally, we evaluate how data selection strategies based on these learnability signals affect both data sources differently.

Abstract:
Large language models are increasingly asked to treat the input context as the source of task-specific rules, procedures, and evidence. A difficult case arises when this context contradicts pretrained priors: models may mention the local rule but still execute familiar semantic, commonsense, positional, or operator shortcuts. We study this failure mode as prior-dominated reasoning , where either the final answer or the intermediate trajectory follows a plausible prior-consistent path rather than the current context. We propose Anti-Prior Reinforcement Learning , a post-training framework that constructs executable context-prior conflict tasks and trains with a gated verifier reward. The reward gives positive credit for answer correctness, context anchoring, and step-wise local rule use, while penalizing prior-consistent answers and reasoning steps. On Qwen3-8B, Anti-Prior RL improves controlled validation accuracy from 51.7% to 74.5% and controlled OOD accuracy from 46.4% to 73.4%, while reducing OOD prior-answer/prior-step hit rates from 15.9%/40.2% to 1.5%/8.9%. It also improves over SFT and LongRLVR on downstream CL-Bench, RULER-QA, and LongBench v2 across both Qwen3-8B and Qwen2.5-7B-1M, and outperforms outcome-only RL on Qwen3-8B. Together, these results provide complementary evidence that the learned behavior transfers beyond the constructed prior-conflict setting to broader context-learning and reasoning benchmarks.

Abstract:
Agentic AI systems powered by large language models are increasingly expected to solve capability-intensive tasks by using reusable external skills, which package instructions, tool-use procedures, executable code, and auxiliary resources. However, mainstream skill-based systems typically assume that available skills can be directly placed in the prompt. As skill libraries grow, this assumption becomes impractical: large corpora cannot be fully exposed to the model, while excessive candidate exposure can distract the agent and degrade skill selection. Therefore, scalable skill augmentation requires retrieving a small set of task-relevant skills from a large corpus before incorporating them into the agent’s context. We formulate this paradigm as Skill Retrieval Augmentation (SRA) , where agents dynamically retrieve, incorporate, and apply reusable skills from large external corpora on demand. To systematically study SRA, we introduce SRA-Bench , which enables decomposed evaluation across three stages: skill retrieval, skill incorporation, and skill application. SRA-Bench contains 5,400 capability-intensive instances from six domains, 636 manually constructed gold skills, and a corpus of 26,262 skills. Experiments across multiple LLM and skill-use strategies show that SRA can consistently improve agent performance, but current agents struggle to realize this promise in practical pipelines. Specifically, they remain sensitive to distractor skills, lack need-aware skill loading, and often fail to translate relevant skills into correct task execution. Together, these results establish SRA as a distinct research problem and provide a foundation for developing scalable skill-augmented agents.

Abstract:
Using Large Language Models (LLMs) for Information Extraction (IE) inherently involves transforming diverse textual inputs into strictly formatted outputs. Guaranteeing this structural validity introduces a hidden energy tax driven by constrained decoding mechanisms, serialisation formats, and underlying serving configurations. Consequently, standard token-level metrics fail to capture the true energy cost of generative IE, which also cannot be predicted by model size alone. To address this, we introduce the first dedicated benchmark evaluating the energy-accuracy trade-offs of generative IE. Our framework establishes task-aligned energy metrics, including Energy per True Positive ( J/TP ) and the F1-Energy Ratio ( F 1 / kJ ). Through comprehensive ablation across architectures, decoding regimes, and serving-layer limits, we demonstrate that efficiency is governed by complex parameter interactions. We translate these findings into actionable configuration guidelines, equipping practitioners to maximise predictive accuracy, optimise energy yield, or achieve a sustainable Pareto balance.

Abstract:
Large Language Models (LLMs) are increasingly deployed in multi-agent settings that require coordination without communication, from human–AI interaction to safety-critical scenarios. Humans often overcome the absence of communication through focal points: salient solutions that naturally stand out to all participants. We present the first large-scale evaluation of how, when, and why focal points emerge in LLMs, comparing their behaviour with humans across cooperative and competitive games, including realistic search rescue scenarios, demonstrating when focal points enable effective coordination. Across more than 20 open- and closed-source models, we find that LLMs exhibit a remarkable ability to coordinate without communication, often matching or outperforming humans. However, the same models consistently fail in tasks requiring numerical common sense or culturally nuanced notions of salience. We additionally evaluate simple learning-free strategies that substantially improve coordination both among LLMs and between humans and LLMs. Our results reveal striking coordination capabilities, as well as social limitations in modern LLMs, and offer new insight into the latent notions of salience encoded within them. Our findings caution against assuming that LLMs share humans’ cultural and perceptual substrate when deployed in coordination settings.

Abstract:
Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real-world travel planning is inherently uncertain: transportation delays, crowd fluctuations, and unexpected stochastic delays frequently invalidate otherwise feasible schedules. Existing benchmarks like TravelPlanner and TripCraft assume deterministic environments, evaluating only static constraint satisfaction and ignoring whether generated plans remain robust when such uncertainties arise. To address this limitation, we introduce UTP-Bench, a large-scale benchmark for uncertainty-aware travel planning. The dataset integrates real-world travel data spanning 504 cities of India, including attractions, restaurants, accommodations, and multi-modal transportation networks. To model realistic disruptions, UTP-Bench incorporates empirical delay distributions and crowd-density patterns collected from major cities, enabling evaluation of travel plans under stochastic conditions. We further propose three evaluation metrics, namely Buffer Adequacy Score (BAS), Crowd-Aware Timing Score (CATS), and Transport Delay Absorption Score (TDAS), which quantify the ability of generated itineraries to maintain robustness against transit delays and crowd variability. Experiments with state-of-the-art LLMs like GPT-5, Qwen3, Mistral and Phi-4 reveal substantial gaps between model-generated and human-authored plans, particularly in temporal buffering, delay-aware transportation scheduling, and crowd-sensitive planning.

Abstract:
Many recursively self-improving systems improve by editing their codebases based on observed rollout behavior. However, recent methods largely treat self-improvement as offline search: variants are optimized and evaluated on fixed benchmarks. This misaligns with real-world deployment, where tasks arrive sequentially and may have different distributions. We study recursive self-improvement as online learning over non-stationary task streams. In this setting, reliable execution and fallback are particularly difficult with limited evidence: different branches of self-improvement see different tasks, so raw rewards conflate capability with task difficulty, leading to overly conservative or optimistic reversion. We propose Online Anchor-calibrated Self-Improvement System (OASIS) that runs an adaptive reference agent on incoming tasks, converts raw rewards into paired comparisons, and uses confidence-calibrated rules for branch continuation and fallback without evaluating every agent on all tasks. Across SWE-Bench Pro and DeepResearch Bench, OASIS consistently improves over both the initial agent and prior RSI baseline SSA, raising the SWE-Bench Pro solve rate from 51.6% to 56.9% using Qwen3.6-35B-A3B and DeepResearch Bench from 35.18 to 39.31 using Gemini 3 Flash. This enables OASIS to outperform larger open weights and proprietary systems, including Qwen3.5-397B-A17B, Minimax 2.7, Qwen3.6-Plus, and GPT-5.3-Codex, with sustained deep self-improvements rather than shallow cycles.

Abstract:
Large language models (LLMs) often generate plausible yet incorrect answers, posing risks in safety-critical settings such as medicine. Human evaluation is expensive, and LLM-as-judge approaches risk introducing hidden errors. Recent single-pass white-box methods detect contextual hallucinations using linear probes over model internals, but their generalization to out-of-domain, out-of-task, and training-free settings remains poorly understood. We introduce HeadEntropy, a method that predicts answer correctness by measuring how susceptible each attention head’s pattern is to modification under further gradient updates. The key insight is that the trace of the softmax Jacobian is a monotonic function of 2-Rényi entropy, linking attention spread to gradient stability during training, even without the reference annotation. With no training, HeadEntropy reaches 0.736 AUROC, outperforms every training-free baseline in paired comparison across 5 models and 5 datasets, and matches a hidden-state probe transferred across domains, at under 1% of inference cost. We evaluate broadly across 5 instruction-tuned LLMs and 5 datasets spanning general knowledge, multi-hop reasoning, mathematical reasoning, claim verification, and medicine. The code is available at https://github.com/StanfordMIMI/HeadEntropy.

Abstract:
Attention-based rerankers score documents by aggregating query-to-document attention and subtracting a null-query calibration pass to remove positional and structural bias. Although widely used, this calibration assumes that the null pass is independent of document relevance. We show that modern prompt content, e.g. constraints, instructions, personas, and demonstrations can violate this assumption when it enters the scoring readout, making the null pass relevance-aware rather than null. Through a three-pass decomposition, we find that calibration helps when the null-pass footprint is relevance-agnostic bias, but hurts when it is relevance-tracking signal: short shared constraints survive calibration, whereas long per-query instructions inject signal that subtraction destroys. We propose interpolated null calibration, a training-free modification that controls how much instruction content enters the null baseline. It recovers attention-based reranking on instruction-heavy tasks where standard calibration fails, while preserving calibration’s benefits when the null pass remains relevance-agnostic. On instruction heavy tasks, the recovered rankings surpass generative rerankers such as RankGPT. We also show that in-context demonstrations improve attention-based reranking with little calibration interference, since demonstrations act only through the query pass and leave the null pass unchanged.

Abstract:
Solving problems through repeated attempts is a sequential modeling task: at each step, the solver receives feedback and decides how to revise their solutions. Predicting whether performance improves, plateaus, or regresses across attempts is central to understanding any iterative problem-solving process in both human learners and autonomous agents. Beyond outcomes, modeling what errors persist and how strategies shift across attempts provides deeper insight into the mechanics of sequential learning. Studying these dynamics requires observing many solvers as they attempt, receive feedback, and revise. Programming courses with automated grading provide this setting, as students iteratively submit code to test suites and receive feedback on every attempt. We therefore curate CodeInsight, a large-scale dataset of over 3 million submissions from 3,286 undergraduates across 2 introductory C++ courses in 2 academic years, with test-case-level outcomes, timestamps, and source code. On this dataset, we build a benchmark that evaluates models spanning parametric, sequential, and generative traditions under a shared calibration-and-scoring protocol, including a Recurrent State Space Model (RSSM) adapted to track solver characteristics through discrete latent variables and an LLM-based predictor that generates explicit solutions. The adapted RSSM achieves the strongest predictive accuracy on three of the four courses. The LLM predictor is less accurate but produces full submissions at each attempt, enabling direct analysis of failure modes. We find that the model’s coding proficiency is inversely related to predictive performance in this setting, with the LLM better understood as a generative solver conditioned on context rather than a faithful predictor of solver behavior. We publicly release our code and the dataset on request to facilitate future research.

Abstract:
Large language models (LLMs) are often post-trained on pre-collected reasoning trajectories to improve their reasoning capability. Such trajectories tend to be long due to complex, interwoven paths, which often include detours on the path toward the answer. However, it has been underexplored whether LLMs indeed benefit from learning complete trajectories in post-training, such as supervised fine-tuning (SFT). Starting from our pilot study, we find that full trajectories provide only limited benefit, while partial trajectories are effective even under heavy truncation. We analyze redundancy in reasoning trajectories through attention-based analyses and controlled token-removal studies, both of which show that intermediate tokens contribute minimally to final reasoning quality. This suggests that avoiding redundant information may allow LLMs to internally infer coherent alternatives by inferring missing steps from their internal knowledge, given known trajectory endpoints. Furthermore, we show that training LLMs using endpoints leads to consistent changes in reasoning behavior, and that it also benefits post-training methods based on reinforcement learning or on-policy distillation, highlighting the need to revisit complete reasoning traces. Code is available at https://github.com/naver-ai/revisiting-trace.

Abstract:
Human communication depends on implicit social signals where effectiveness is shaped by tone, context, and conversational norms rather than semantic content alone. We introduce KARMA (Karma-Aligned Reward Model Adaptation), a framework for LLM learning of context-sensitive conversational behavior from large-scale social media data. KARMA trains a reward model on Reddit conversations to predict response valuation conditioned on context, and uses this signal to fine-tune language models via reinforcement learning to improve performance on pragmatics-mediated tasks. Critically, we find that the highest performing reward model does not lead to better downstream model alignment: a reward model relying exclusively on conversational context was a worse predictor of Reddit karma but yielded substantially better downstream performance. We evaluate the effects of KARMA applied to a downstream model with and without direct exposure to the social media data. The resulting models show improved pragmatics-mediated behaviors with largely mitigated undesirable side effects.

Abstract:
While standard IR models are primarily designed to optimize relevance, real-world search often needs to balance additional objectives such as diversity and fairness. These objectives depend on inter-document interactions and are commonly addressed using post-hoc heuristics or supervised learning methods, which require task-specific training for each ranking scenario and dataset. In this work, we propose an in-context learning (ICL) approach for listwise LLM rerankers that eliminates the need for such training. Instead, our method relies on a small number of example rankings that demonstrate the desired trade-offs between objectives for past queries similar to the current input. We evaluate our approach on common IR test collections to investigate multiple auxiliary objectives: group fairness (TREC Fairness), polarity diversity (Touché), and topical diversity (TREC Deep Learning 2019/2020). We empirically validate that our method enables control over ranking behavior through demonstration engineering, allowing nuanced behavioral adjustments without explicit optimization. Our experiment demonstrates significant improvements in auxiliary objectives, with up to 23% in topical diversity and 20.6% fairness gains across different tasks, while maintaining relevance across benchmarks.

Abstract:
Mixture-of-Experts (MoE) architectures have emerged as a powerful paradigm for scaling model capacity while preserving efficient inference in large foundation models. However, most MoE models use a fixed top-k expert selection policy, assigning the same expert budget to every token even when fewer experts may be sufficient. Inference-time dynamic top-k routing can reduce computation without retraining, but existing methods often overlook the distributional shift caused by deviating from the training-time routing configuration. We show that reducing the number of activated experts consistently increases the RMS scale and variance of SMoE outputs, inducing a representation mismatch that contributes to downstream performance degradation in addition to the loss of expert capacity. To address this correctable component, we propose Layer-wise Distribution Alignment (LDA), a lightweight inference-time correction that uses layer-wise calibration statistics to align reduced-routing representations with the default configuration. Across multiple SMoE LLMs, benchmarks, and routing strategies, LDA recovers much of the performance lost induced by the distributional shift under reduced routing while preserving sparse-inference efficiency with negligible overhead.

Abstract:
We introduce the Massive Video Embedding Benchmark (MVEB), a 23-task benchmark for video embeddings spanning classification, zero-shot classification, clustering, pair classification, retrieval, and video-centric question answering. We evaluate 33 models and find that no single model dominates: MLLM-based embeddings lead on classification, clustering, pair classification, and QA; multimodal binding leads on retrieval and zero-shot classification; generative MLLMs without contrastive adaptation collapse on cross-modal tasks. Paired video-only vs. audio+video evaluations show that audio’s contribution depends on dataset annotation provenance: audio helps when labels were produced from both modalities and hurts when they were produced from visuals alone, a six-point gap consistent across model families. MVEB is derived from MVEB+, a 184-task pool, and is designed to maintain task diversity while reducing evaluation cost. It integrates into the MTEB ecosystem for unified evaluation across text, image, audio, and video. We release MVEB and all 184 tasks along with code and a leaderboard at https://github.com/embeddings-benchmark/mteb.

Abstract:
Reinforcement learning from verifiable rewards with GRPO is a standard approach for post-training reasoning LLMs. It remains sample inefficient. Each rollout is used for a single gradient update and then discarded. Naive replay is not well suited in this setting because LLM policies drift quickly per gradient step. Stored rollouts therefore become stale and can destabilize training. We propose a rollout-level replay buffer for GRPO that stores and samples individual rollouts rather than whole groups. The buffer bounds staleness through age eviction. Any rollout older than τ_max training steps is removed. The buffer also preserves on-policy data via fresh-anchored composition. Each batch keeps its fresh on-policy rollouts and then concatenates replay rollouts drawn separately from the buffer. We prioritize replay by per-rollout advantage magnitude and recycle individual rollouts whose advantages are large. Across three Qwen3-Base scales on five math benchmarks, our method outperforms GRPO and naive replay baselines. Gains are positive at every scale and reach +1.66 pp on the five-benchmark average at 4B. Under an AES metric that jointly measures accuracy and token efficiency, our method is the only condition with a positive margin over GRPO at every scale.

Abstract:
Reinforcement learning with rule-based, binary-valued reward functions has been shown to significantly enhance the reasoning capabilities of large language models. These models typically rely on REINFORCE-based policy optimization methods, such as REINFORCE with baseline and group relative policy optimization (GRPO). However, a key limitation is that existing methods either ignore reward normalization or use static normalization schemes, which fail to adapt to the evolving policy during training. This mismatch can lead to unstable gradient estimates and hinder training stability. To address this issue, we propose Beta Normalization Policy Optimization (BNPO), a novel method that adaptively normalizes rewards using a Beta distribution with dynamically updated parameters. By aligning normalization with the changing policy distribution, BNPO enables more accurate, lower-variance gradient estimation and promotes stable training dynamics. We provide theoretical analysis showing that BNPO reduces variance and generalizes both REINFORCE and GRPO under binary-valued rewards. In addition, we introduce an advantage decomposition mechanism to extend BNPO to more complex reward settings. Experimental results demonstrate that BNPO achieves state-of-the-art performance among policy optimization methods on reasoning tasks.

Abstract:
Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM). These world models are typically trained via supervised next-state prediction to generate fixed representations like HTML or AXTree snapshots. However, this objective is misaligned with the downstream ranker, which relies on predicted states being discriminative across candidates to accurately score them. To address this, we introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions. We train these models using a branching web-agent dataset derived from WebArena Go-Browse trajectories, where every decision point contains multiple alternative actions and their resulting states. Experiments on our held-out predicted-state matching benchmark show that our approach outperforms world models trained with supervised next-state prediction. We further show that our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models. Finally, on WebArena-Lite, using our world model for test-time action selection improves end-to-end task success. Our project page is available at: https://dhruvpendharkar.github.io/dwm/.

Abstract:
The development of high-quality multilingual models depends on well-curated multilingual datasets, which are typically obtained through filtering large web-crawled corpora. Such filtering approaches must be effective across a wide range of languages in order to improve model quality proportionally across the entire linguistic space. In recent years multiple Natural Language Processing (NLP) approaches for data processing and Large Language Model (LLM) training were developed. However, the most significant progress has been made for rich-resource languages such as English and Chinese, leaving multilingual methods largely underexplored. In this work, we present data selection strategies based on Internet registers (genres) and Web Document Score (WDS). We study 15 European languages utilizing HPLT 3.0 datasets. We train and evaluate four small generative language models per language, and compare how specific registers and WDS affect model training in separate and combined approaches. We use five common evaluation tasks covering models’ linguistic, knowledge-based, and reasoning skills. We analyze the impact of the studied strategies on each language, their average performance, and show how combining register-based selection with WDS improves evaluation performance. These results indicate that both registers and WDS can generalize across multiple languages, offering a new path towards multilingual data filtering.

Abstract:
Counterfactual examples are minimal edits to an input that alter a model’s prediction. They are widely employed in explainable AI to probe model behavior and in natural language processing (NLP) to augment training data. However, generating valid counterfactuals with large language models (LLMs) remains challenging, as existing single-pass methods often fail to induce reliable label changes, neglecting LLMs’ self-correction capabilities. To explore this untapped potential, we propose iFlip, an iterative refinement approach that leverages three types of feedback, including model confidence, feature attribution, and natural language. Our results show that iFlip achieves an average 57.8% higher validity than the three state-of-the-art baselines, as measured by the label flipping rate. The user study further corroborates that iFlip outperforms baselines in completeness, overall satisfaction, and feasibility. In addition, ablation studies demonstrate that three components are paramount for iFlip to generate valid counterfactuals: leveraging an appropriate number of iterations, pointing to highly attributed words, and early stopping. Finally, counterfactuals generated by iFlip enable effective counterfactual data augmentation, substantially improving model performance and robustness.

Abstract:
Autoregressive decoding remains a key bottleneck for LLM inference, due to its token-by-token nature. Speculative decoding is proposed to reduce latency by drafting multiple tokens and verifying them with fewer target-model passes. However, existing methods either incur auxiliary-drafter overhead or rely on brittle lexical span reuse. In structured tasks (e.g., code generation and text editing), we observe significant locality of reference and semantic redundancy for potential semantic context reuse. In light of this, we present Semantic Retrieval Speculative Decoding (SRSD), a training-free scheme that treats intermediate-layer hidden states as an in-context semantic index. At each decoding step, SRSD retrieves semantically similar past positions. Multiple candidates are merged into a compact tree and verified jointly in a single forward pass. SRSD requires no auxiliary models or external datastores. Across a variety of popular LLMs, SRSD achieves up to 5.42× throughput speedups over state-of-the-art training-free methods and parametric drafters. The code is available at https://github.com/Secbrain/SRSD.

Abstract:
Search-augmented reasoning agents interleave multi-step reasoning with external retrieval, but uncontrolled retrieval can introduce redundant evidence, saturate the context, and destabilize reinforcement learning (RL). Existing outcome-based RL methods provide only sparse terminal rewards, offering limited guidance for intermediate information-acquisition decisions. We propose DeepControl, an adaptive information-control framework based on information utility, a state-dependent estimate of the marginal value of retrieved evidence. The framework regulates information acquisition along two axes: extent, i.e., whether retrieval should continue, and resolution, i.e., how much retrieved detail should be exposed. It implements these controls through retrieval-continuation guidance, hierarchical granularity control, and an annealed control-forcing scheme. This enables the policy to internalize effective acquisition behavior during training and operate without external control at test time. Across seven benchmarks, DeepControl consistently outperforms strong RL and retrieval baselines without explicit information control; compared with Search-R1, it improves average performance by +9.4 and +8.6 points on Qwen2.5-7B and Qwen2.5-3B, respectively. Additional analyses show improved search effectiveness, training stability, and evidence utilization.

Abstract:
Improving the quality of OCR transcriptions is crucial for researchers working with digital archives, as OCR errors hinder readability and downstream NLP tasks. While prior work has focused on seq2seq neural machine translation and, more recently, prompt-based approaches with large language models (LLMs), these methods rely solely on text, overlooking the visual characteristics of source documents. In this work, we explore the potential of vision-language models (VLMs) for post-OCR correction to evaluate the impact of integrating visual information. We conduct experiments on the BLN600 dataset of 19th-century newspaper articles, comparing Llama 3.2 Vision and Qwen 2.5 VL to their text-only LLM counterparts. Our results show that VLMs significantly outperform LLMs, achieving approximately 90% reduction in character and word error rates, with the optimal setting being one that takes as input both document image and prior noisy OCR. This suggests that VLM-based post-OCR correction can be a viable and effective approach for correcting back catalogues of poorly OCR’ed historical documents and improving the usability of digital archives.

Abstract:
Real-time streaming video understanding in domains such as autonomous driving and intelligent surveillance poses challenges beyond conventional offline video processing, requiring continuous perception, proactive decision making, and responsive interaction based on dynamically evolving visual content. However, existing methods rely on alternating perception-reaction or asynchronous triggers, lacking task-driven planning and future anticipation, which limits their real-time responsiveness and proactive decision making in evolving video streams. To this end, we propose a StreamAgent that anticipates the temporal intervals and spatial regions expected to contain future task-relevant information to enable proactive and goal-driven responses. Specifically, we integrate question semantics and historical observations through prompting the anticipatory agent to anticipate the temporal progression of key events, align current observations with the expected future evidence, and subsequently adjust the perception action (e.g., attending to task-relevant regions or continuously tracking in subsequent frames). To enable efficient inference, we design a streaming KV-cache memory mechanism that constructs a hierarchical memory structure for selective recall of relevant tokens, enabling efficient semantic retrieval while reducing the overhead of storing all tokens in the traditional KV-cache. Extensive experiments on streaming and long video understanding tasks demonstrate that our method outperforms existing methods in response accuracy and real-time efficiency, highlighting its practical value for real-world streaming scenarios.

Abstract:
Retrieval-Augmented Generation (RAG) improves factual grounding by incorporating external knowledge into language model generation. However, when retrieved context is noisy, unreliable, or inconsistent with the model’s parametric knowledge, it introduces retrieval-prior conflicts that can degrade generation quality. While this problem has been studied in autoregressive language models, it remains largely unexplored in diffusion-based language models, where the iterative denoising process introduces unique challenges for integrating retrieved context. In this work, we propose Adaptive Retrieval-Augmented Masked Diffusion (ARAM), a training-free adaptive guidance framework for Masked Diffusion Models (MDMs) in RAG settings. ARAM dynamically calibrates the guidance scale during denoising according to the Signal-to-Noise Ratio (SNR) of the distributional shift induced by retrieved context. Intuitively, the model strengthens guidance when the retrieved context provides reliable corrective evidence and suppresses it when the contextual signal is noisy or non-supportive. Extensive experiments on multiple knowledge-intensive QA benchmarks show that ARAM improves overall QA performance over decoding-based RAG baselines.

Abstract:
Large language models (LLMs) are increasingly applied to vulnerability detection in real-world software systems, where the task goes beyond binary classification and requires calling context exploration over large code repositories. However, existing benchmarks primarily formulate the vulnerability detection as a classification problem without calling contexts and provide no formalized explanations, making them inadequate for evaluating LLM-driven vulnerability detectors upon repositories. To fill the gap, we introduce ManBo, a benchmark designed to assess the capability of LLM-driven vulnerability detectors in the real-world scenarios. Specifically, ManBo provides human-verified vulnerability explanations together with calling contexts, enabling principled evaluation of the ability to both identify vulnerabilities and explain root causes. It contains 205 vulnerable instances and 164 safe instances collected from 74 high-profile C/C++ projects, covering three critical types of memory corruption. We evaluate a broad spectrum of LLM-driven detectors on ManBo, and find that effective calling context retrieval is essential for repository-level vulnerability detection, exposing a fundamental gap between the capabilities of existing LLM-driven detectors and real-world requirements. ManBo is publicly available at https://github.com/jinyaoguo/ManBo.

Abstract:
Paper-to-code reproduction asks scientific AI agents to turn research papers into executable repositories that preserve the paper’s method, protocol and artifacts. This is difficult because the specification is split: explicit paper content such as algorithms, metrics and artifacts is often lost across long agent trajectories, while implicit details such as framework defaults and conventions inherited from related work are absent from the paper. We introduce ReproAgent a four-stage Prepare–Plan–Generate–Repair pipeline built around a persistent implementation contract with two channels: an implementation-requirement channel that turns paper snippets into code obligations, and a reference-evidence channel that retrieves content and structure evidence from related repositories. Both are bound to work packages, projected into file-level contracts, and consumed across generation and repair. On PaperBench Code-Dev, ReproAgent reaches the highest mean score among same-backbone scaffolds under both Claude-Sonnet-4.5 and Gemini-3-Flash. End-to-end channel ablations and per-paper cases support the contribution of both channels.Code and experimental artifacts are publicly available at https://anonymous.4open.science/r/ReproAgent-E760.

Abstract:
Visible tests are a common gate for LLM-generated code, but passing them does not certify specification correctness. We study a deployment-like monitoring problem: after code has passed public tests, can a weaker LLM verifier identify the residual hidden bugs? We introduce Code Monitor Red Teaming, a monitor-red-teaming protocol that fixes a public-check information boundary while varying generator pressure, verifier scaffolding, and weak-to-strong capability. We instantiate it as CodeMonitorBench, spanning function-level, data-science, and workflow code. Across 71,000 generated candidates, 43,677 pass public tests and 23,081 of those fail hidden tests. Weak verifiers improve with scaffolding and model family, but still miss most hidden bugs at 5% false-positive rate. As a robustness stress test, adversarial public-test-overfit pressure lowers verifier AUROC and raises low-FPR miss rates in most cells. A GLM-5.1 verifier recovers part of the gap under the same evidence boundary; an inferability audit shows that remaining misses mix verifier failures with M1 evidence limits.

Abstract:
Although recent large reasoning models (LRMs) achieve strong performance, their reasoning processes remain vulnerable to external manipulation. We observe that LRMs can gradually rationalize incorrect user-provided hints during multi-step reasoning. Instead of rejecting false hints, models adapt their intermediate states to accommodate them and eventually produce seemingly coherent yet incorrect conclusions. We refer to this phenomenon as self-brainwashing and analyze it through empirical and theoretical analyses. We find that unfaithful reasoning steps are often locally detectable when models only focus on local reasoning context. However, this ability degrades over long reasoning trajectories, allowing unfaithful steps to accumulate and bias the final answer toward the hint. Motivated by these findings, we propose Long-Short Memory Steering (LSMS), a training-free inference-time framework that uses short-term reasoning hidden states to enhance faithfulness and long-term reasoning hidden states to maintain consistency. Across three open-source reasoning models and four reasoning datasets, LSMS consistently reduces unfaithful answer shifts while preserving benign accuracy. On average, it lowers the shifted ratio by more than 20 percentage points under three real-world hint-based threats.

Abstract:
Although Large Language Model (LLM) agents have demonstrated remarkable performance on complex, real-world tasks, the significant bottleneck of execution inefficiencies and static environmental feedback hinders agents’ effective generalization to open-ended scenarios. To address these limitations, this paper introduces Role-Agent, a novel framework that repurposes one LLM to concurrently function as the agent and the environment, thereby enabling a bootstrapped co-evolution. At the core of Role-Agent are two synergistic components: World-In-Agent (WIA) and Agent-In-World (AIW). In WIA where the LLM acts as the agent, we define the reward for an action as the alignment between the agent’s prediction of the next several states and the actual state. These rewards are subsequently grouped by state and optimized, equipping the agent with predictive reasoning capabilities regarding the environment. In AIW where the LLM provides environmental adjustments, we require it to analyze the typical failure modes from failed trajectories, then searches for tasks that exhibit similar failure modes, thereby altering the data distribution to facilitate learning. Extensive experiments conducted on the various benchmarks demonstrate the substantial performance improvements achieved by Role-Agent, yielding an average performance gain of over 4%.

Abstract:
Generative retrieval (GR) maps queries directly to document identifiers (docids) using parametric knowledge. However, this design makes corpus expansion costly: adding new documents requires updating model parameters to encode new document-docid associations incurs repeated training and catastrophic forgetting of previously indexed documents. In this work, we revisit incremental GR as an in-context retrieval problem, where newly added documents are supplied as inference-time document-docid evidence. We propose ICICLE, an in-context indexing framework that performs source-aware docid generation over both parametric memory and context-provided document-docid pairs. ICICLE combines a ‘[COPY]‘-based routing mechanism, preference-based calibration, and large context adaptation to distinguish context-grounded retrieval from parametric retrieval. Experiments on MS MARCO and NQ320K show that ICICLE improves retrieval of newly introduced documents while preserving seen-document retention without corpus-specific retraining. Our analysis further shows that high-shot degradation is mainly caused by routing failure, highlighting source-selection calibration as a key bottleneck for scaling in-context generative retrieval.

Abstract:
LLM-based embodied agents are increasingly evaluated on environment-grounded, long-horizon planning, yet existing benchmarks rarely test whether an agent can condition its plans on the specific user it serves. This gap points to a broader limitation in embodied task-planning systems, which often ground their reasoning only in the physical environment while overlooking the diversity of the people they serve. We frame this problem as Planning with Human Attributes (PHA), in which an agent must integrate explicit user information into long-horizon embodied task planning. We introduce Trinity, a benchmark designed to evaluate PHA capabilities. Within Trinity, a single instruction can yield different, equally valid, or even opposing outcomes depending on a user’s personal attributes. Our experiments reveal that standard off-the-shelf in-context learning-based agents commonly fail at the human-grounding component, whether or not they are provided with relevant examples, highlighting a key obstacle for PHA tasks. Beyond benchmarking, we contribute a bottom-up failure-mode analysis that isolates where and why agents break, and we distill it into concrete, model-agnostic design principles for human-agent-environment (HAE) aligned agents. We hope Trinity brings domestic robots a step closer to becoming trusted, socially acceptable partners.

Abstract:
We are witnessing an emerging paradigm shift in which the adoption of large language models (LLMs) has increased significantly across high-stakes, standards-driven industries, particularly in engineering, healthcare, education, and law. In line with EMNLP 2026’s theme of "New Missions for NLP Research", this position paper aims to initiate civil discussion on how the use of LLMs is shaping the future of standard compliance and introduces standard alignment, a targeted research direction towards making LLMs verifiably comply with domain standards. To support the discussion, we propose the Criticality and Compliance Capabilities Framework (C3F) to assess the criticality levels of various standards and pair them with the current compliance capabilities of state-of-the-art LLMs as evidenced by the literature. Finally, we argue that advancing standard alignment is key to ensuring safe and trustworthy integration of LLMs across standards-driven industries.

Abstract:
Molecular design is a central task in drug discovery that requires precise structural reasoning and domain knowledge. While large language models (LLMs) have shown promise in generating high-level editing intentions in natural language, they often struggle to faithfully execute these modifications, particularly when operating on non-intuitive representations like SMILES. We introduce MECo, a framework that bridges reasoning and execution by translating editing actions into executable code. MECo reformulates molecular modification for LLMs as a cascaded framework: generating human-interpretable editing intentions from a molecule and property goal, followed by translating those intentions into executable structural edits via code generation. Our approach achieves over 98% accuracy in reproducing held-out realistic edits derived from chemical reactions and target-specific compound pairs. On downstream benchmarks spanning physicochemical properties and target activities, MECo substantially improves consistency by 38-86 percentage points to 90%+ and achieves higher success rates over SMILES-based baselines while preserving structural similarity. By aligning intention with execution, MECo enables consistent, controllable and interpretable molecular design, laying the foundation for high-fidelity feedback loops and collaborative human–AI workflows in drug discovery.

Abstract:
Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger target model to verify them. However, standard SD is mainly designed for acceleration, and its output quality is typically constrained by the target model. In this work, we propose Entropy-Aware Speculative Decoding (EASD), a lightweight and training-free extension of SD that improves reasoning quality through token-level entropy-guided rejection. EASD detects cases where both draft and target models exhibit high uncertainty while strongly overlapping in their top predictions. In such uncertain-agreement cases, EASD rejects the aligned token and resamples from the target distribution, preventing low-confidence errors from propagating. Experiments on challenging reasoning benchmarks show that EASD consistently improves accuracy over standard SD and reward-guided variants while maintaining comparable inference efficiency. Notably, EASD can surpass the standalone performance of the target model, suggesting that speculative decoding can serve not only as an acceleration method but also as an effective mechanism for improving reasoning quality. The code is available at https://github.com/ECNU-Text-Computing/EASD.

Abstract:
Long-form narratives are challenging for long-context models because their structure is implicit: events, characters, and plotlines interact across hundreds of pages without the explicit cues that guide navigation in structured documents. We address this by constructing storyline trees, hierarchical representations that organize narratives from global themes and major plotlines to fine-grained events. We first segment chapters into contiguous narrative segments, or scenes, and use them as the basic units for tree construction. We then infer storyline trees through complementary top-down and bottom-up procedures that derive, refine, cluster, and summarize storylines at multiple levels of abstraction. We showcase the utility of this representation for question answering: storyline trees enable adaptive retrieval, allowing models to iteratively inspect high-level narrative structure and retrieve scene-level evidence on demand. Experiments on three long-context narrative QA benchmarks show that adaptive retrieval outperforms strong baselines, including post-trained long-context models and agentic chunk-based methods. Ablations confirm that scenes are more effective basic units than chapters or generic segmentation, and that gains persist under matched retrieval budgets.

Abstract:
While human evaluation is the gold standard in many NLP tasks, it suffers from prohibitive costs and poor scalability. When identifying top-performing models, typical evaluation protocols waste effort by exhaustively evaluating all models on the entire benchmark, a safe but inefficient approach. In this work, we formalize multi-model human evaluation as a best-arm identification problem in a multi-armed bandit setup with correlated arms, where pulling an arm corresponds to human-evaluating a model. By sampling adaptively based on the intermediate model rankings obtained on the samples so far, we can focus the annotation budget on the most competitive models. We prove the optimality of the proposed algorithms and show that it improves discrimination between top-performing models. This makes evaluations faster, cheaper and more aligned with large-scale competition evaluation goals.

Abstract:
Modern reasoning systems increasingly rely on verifier-guided selection: sample several solution traces, score them with a reward model, and return the top-ranked answer. In multi-generator deployments, this verifier is often reused across generators. We ask whether this reuse assumption is reliable. On a 34-generator math panel with K=64 candidates per problem and 7 verifiers, the 7B process reward model (PRM-7B) is the strongest fixed verifier, but a per-target single-verifier oracle gains +1.6 percentage points (pp). We then ask whether this target-local mismatch is actionable. Four label-free candidate-text statistics predict the coarse PRM outcome-verifier regime, and three budgeted responses improve over fixed PRM on the 22-target first-pass panel: a no-label gated router ( +1.41 pp ), PRM + V4 fusion ( +2.35 pp ), and a labelled PRM-only verifier-allocation arbiter (PVA-Adapt; +2.26 pp ). A same-panel American Invitational Mathematics Examination (AIME) 2026 held-out slice gives +3.88 pp , and a second candidate-bank check keeps the scored and labelled gains positive while weakening the no-label router. We frame the result as a seed- and verifier-family-conditioned deployment diagnostic rather than a universal routing rule; a supervised fine-tuning (SFT) replication also finds no positive transfer from better selected traces.

Abstract:
Zero-shot vision-language models (VLMs) such as CLIP demonstrate strong open-vocabulary recognition capabilities, yet their prediction confidences are often poorly calibrated. Existing calibration methods often estimate prediction stability by applying multiple augmentations in input space, introducing substantial computational overhead and limiting practical deployment. To this end, this paper proposes an embedding-space posterior calibration approach for zero-shot VLMs, which shifts the sampling process from the input space to the embedding space. Our approach defines and computes a confidence stability score that measures how stable the sample remains under posterior embedding perturbations. This score is further converted into a sample-wise temperature, so that unstable samples receive stronger confidence smoothing while stable samples remain close to their original predictions. Compared with the original CLIP model, our method reduces the mean Expected Calibration Error from 5.83 to 2.05, achieving a 64.8% relative reduction. Meanwhile, our method achieves this improvement with only a minor increase in GFLOPs, from 8.83 to 8.84.

Abstract:
This paper examines whether Code-LLMs encode latent sentiment representations and how these influence generation behavior. Using probing methods, we show that sentiment information is linearly separable in hidden states. Through activation steering, we demonstrate that manipulating sentiment-aligned directions systematically affects both output tone and downstream task performance. Notably, negative sentiment consistently reduces correctness, while positive effects are weaker and less consistent. Ablation and error analyses indicate these changes are not due to random perturbations but reflect meaningful behavioral shifts, including reduced confidence. These findings suggest that affective latent directions act as unintended control signals in Code-LLMs, with measurable impact on downstream task performance quality. Our study highlights the need to better understand and control such latent factors when deploying Code-LLMs.

Abstract:
Many NLP tasks, such as summarization and extractive question answering, reduce to retrieving relevant content from documents under two constraints: coverage, retaining enough pertinent information to achieve some goal, and conciseness, removing as much irrelevant information as possible. Conformal prediction methods have been used to guarantee coverage, and must be optimized for conciseness through design of a score function. State-of-the-art scoring functions use hand-engineered LLM prompts asking the model to rate the importance of content, but manual prompt engineering is labor-intensive and task-specific. We introduce the Conformal Relevance framework which uses in-context learning example curation and ensembling to create a score function which maintains coverage while improving conciseness with minimal manual input. We demonstrate this framework’s application on seven NLP tasks, and also theoretically study the impact of diversity for ensembled conformal scores, giving a complementarity condition that characterizes when ensembling improves worst-case sentence scores, and a saturation bound on ensemble improvement.

Abstract:
When an LLM formalizes natural language, how do we know the output is faithful? We propose a roundtrip verification approach which does not require ground-truth annotations: formalize a statement, translate the result back to natural language, re-formalize, and use a formal tool to check logical equivalence. When the two formalizations agree, this provides evidence of a faithful formalization. When they disagree, a stage-level diagnosis localizes the error to a specific translation step, and a scoped repair operator attempts to correct that step. We evaluate the framework on two statutory domains (the Texas Transportation Code and the Texas Parks and Wildlife Code) using two LLMs (Claude Opus 4.6 and GPT-5.2) with three repair baselines. Diagnosis-guided scoped repair is the most effective method, with effectiveness contingent on the reliability of the diagnosis function. Across both domains and both models, under our full repair system, rules that fail the equivalence check show 1.4x-2.5x more NLI drift than rules that pass it.

Abstract:
Emotional labor is the effortful management of emotional displays to meet social or professional expectations. Personality traits have been correlated with emotional labor strategies, yet research on this link relies almost exclusively on self-report scales administered in only occupational settings. We investigate whether large language models injected with psychometrically grounded personas reproduce these personality-driven selection patterns across everyday social scenarios. We construct the first emotional labor strategy dataset of 500 socially situated events, each offering three behavioral choices corresponding to surface acting, deep acting, and genuine expression. We source 50 fictional characters from a large-scale personality repository and profile each through two parallel tracks: observer-rated bipolar adjective composites and in-character self-report items. Five LLMs evaluate all scenarios under both persona conditions. We find that models align more towards deep acting, and that Conscientiousness and Emotional Stability consistently predict this preference. Entropy analysis confirms that persona reliably influences the output and varies across models and emotions.

Abstract:
Non-binary bottom-up constituency parsing commonly uses arity-specific reductions such as Reduce- X#k , which jointly specify the mother label and number of children. We show that arity need not be a primitive transition parameter. The proposed parser predicts the mother label separately and identifies the ordered child sequence from a delimiter-bounded stack configuration. In every well-formed reducible configuration, the active delimiter, label marker, and intervening completed items uniquely determine the reduction domain and hence its arity. This factorization removes label–arity-specific reduction actions while preserving direct construction of the original non-binary trees. Controlled experiments on PTB and CTB show that the delimiter-guided parser closely tracks an arity-specific baseline while using a substantially smaller action inventory. Its predicted trees retain the gold branching profile without systematic collapse at higher arities. The smaller inventory is obtained at the cost of longer derivations and boundary-sensitive errors involving delimiter insertion, preservation, and consumption.

Abstract:
With the explosive growth of social media, multimodal memes (text and images) have become a key carrier of misinformation dissemination and emotional manipulation because they strongly depend on cultural background, satirical metaphor, and current events. Existing detection methods mostly rely on end-to-end deep learning models, but they face two fundamental limitations: first, static models are difficult to capture the dynamic semantic evolution of memes; second, fixed architectures cannot adapt to meme heterogeneity. This work proposes a Tool-Learning Framework for Multimodal Meme Detection (Tool-meme), a Large Multimodal Model (LMM)-driven tools-learning framework. The framework uses multimodal representations only as input features, dynamically orchestrates tool chains through retrieval-augmented generation and meta-cognitive planning, and uses reinforcement learning to maximize tool-use efficiency. On HarM, FHM, and MAMI, Tool-meme outperforms all evaluated direct-LMM and agentic baselines under frozen-executor setting. Additional transfer and deployment analyses show consistent gains across tested executor families while lowering tool calls and cost vs. exhaustive execution, with explicit evidence chains for inspection.

Abstract:
Normative rules like institutional charters and workplace policies must be both human-readable and operationally verifiable against actual environment records. However, current language generation and structured-output benchmarks primarily reward surface fluency or schema compliance, leaving operational grounding weakly tested. This creates a critical vulnerability where standard language models generate plausible-sounding policies that fail during enforcement because they rely on unavailable data logs or misaligned scopes. To address this challenge, we formalize the problem as Grounded Normative Rule Synthesis (GNRS) and introduce GNRS-Search, a framework that utilizes Markov Chain Monte Carlo (MCMC) sampling to optimize a discrete, five-slot And-Or Graph (AOG). By explicitly decoupling intermediate operational structure from final prose generation, this method isolates executable feasibility from writing style and allows rule failures to be localized prior to surface realization. We evaluate our approach on GNRS-Bench, a benchmark spanning 116 controlled goals across eight scene families, and RealCharter-Bench, which evaluates transfer to 53 real-world policy tasks with hidden source clauses. GNRS-Search raises average rubric quality from 68.8% to 81.0% and ranks first under a disclosed executable composite metric, while systematic slot interventions confirm that performance gains stem from robust operational logic rather than rhetorical tuning. Ultimately, by transforming automated rule drafting into an inspectable search problem, this work provides a foundational paradigm for deploying verifiable and compliance-ready personal agents within regulated environments.

Abstract:
Event ontology expansion aims to discover emerging event types from data and extend them to appropriate positions in the existing event ontology. Existing methods typically cluster contextualized trigger representations and attach induced clusters to the ontology based on instance-level similarity. However, ontology expansion requires concept-level semantics that characterize event types, whereas contextualized trigger representations often conflate these semantics with surface contextual variation, leading to unstable clustering and unreliable hierarchy expansion. To address this issue, we propose ConceptE, a conceptualization-enhanced framework for event ontology expansion. ConceptE first derives concept-level semantics by prompting an LLM with the sentence and event trigger, producing a concise concept name and a natural-language description. It then jointly encodes these semantics with trigger information to build concept-enhanced representations aligned with ontology-level reasoning. This representation design supports more coherent event clustering, more reliable hierarchy expansion, and ontology-consistent type naming. Experiments on ACE, ERE, and MAVEN demonstrate that ConceptE consistently outperforms state-of-the-art approaches across all subtasks of event ontology expansion. In particular, it achieves improvements of up to 10.85% in BCubed-F1 for event clustering and 6.48% in Taxo_F1 for hierarchy expansion, demonstrating the effectiveness of the proposed ConceptE method. The code is available at https://github.com/ICT-KCG/ConceptE.

Abstract:
Manga visual question answering requires models to answer questions over panel-based visual narratives, where relevant evidence is distributed across ordered panels, embedded text, recurring characters, and implicit event transitions. This structure makes passive page encoding insufficient, as the model must identify which panels to inspect, what clues to retain, and when the accumulated evidence is sufficient for answering. We propose MANGO (Manga Active Narrative Grounding Optimization), an unsupervised framework for active manga visual question answering. MANGO introduces Active Narrative Sketching (ANS), which iteratively selects panels, extracts concise grounded clues, and decides when to stop, forming a compact question-directed evidence sketch before answer generation. To optimize this behavior without human-annotated answers or rationale paths, MANGO samples multiple ANS rollouts and applies group-relative training with two rewards: answer preference from listwise self-ranking and path consistency from stable ordered panel trajectories. The combined reward is optimized with group-relative policy training, encouraging the model to improve both final answers and the panel-level evidence paths that support them. Experiments on standard manga understanding benchmarks show that MANGO achieves state-of-the-art performance across different settings.

Abstract:
Current large-language models (LLMs) typically adopt a fixed reasoning strategy, either simple or complex, regardless of reasoning complexity. This neglect of variation in task and reasoning process complexity leads to an imbalance between performance and efficiency. Existing methods attempt to implement training-free fast-slow thinking system switching to handle problems of varying difficulty, but are limited by coarse-grained solution-level strategy adjustments. To address this issue, we propose a novel reasoning paradigm: Process-Level Adaptive Thinking Mode Switching (PATS), which enables LLMs to dynamically adjust their thinking mode based on the difficulty of each reasoning step, optimizing the balance between accuracy and computational efficiency. Our approach integrates Process Reward Models (PRMs) with Beam Search, incorporating progressive mode switching and bad-step penalty mechanisms based on thresholds derived from a data-driven methodology. Experiments on diverse mathematical and non-mathematical benchmarks across various model families and scales, with PRMs of varying quality demonstrate that our methodology achieves high accuracy while maintaining moderate token usage. This study emphasizes the significance of process-level, difficulty-aware thinking mode adaptation, offering insights into efficient inference for LLMs.

Abstract:
KV cache quantization is well studied for autoregressive language models, where cached prefix states are exact under causal attention and quantization is mainly a cache reconstruction problem. Diffusion language models (dLLMs) break this assumption. Fast dLLM inference reuses KV states across denoising steps, but bidirectional token representations can change as masked tokens are revealed, so the reused cache may already differ from full recomputation before quantization is applied. We study KV cache quantization under this cache-reuse error. We adapt KIVI and KVQuant to Fast-dLLM-style prefix and dual cache reuse, and introduce TraceKV and KProtect. TraceKV reconstructs feasible dLLM reveal trajectories from prompt-answer pairs, providing calibration states that match iterative masked decoding. KProtect uses these trajectories to choose per-layer K-cache entries that remain in high precision according to output- and attention-level behavior metrics, while applying standard low-bit value quantization. Across LLaDA and Dream evaluations on GSM8K, HumanEval, MATH, and MBPP, KProtect achieves the highest average score among quantized methods across the reported 4-bit and 2-bit prefix/dual cache settings.

Abstract:
Students increasingly use LLMs as tutors for coursework and problem solving. Little is known about the level of assistance LLMs provide when students use them as tutors in authentic learning interactions. This matters because tutoring responses can differ substantially in how directly they help students complete a task. We operationalize this dimension as scaffolding level and develop a five-level scale, validated against human annotations, that characterizes responses according to the degree of direct assistance they provide. We apply the scale to 14,637 LLM responses from 203 students in a university AI course, where the model received no pedagogical instruction beyond a generic helpful-assistant system prompt. Responses are overwhelmingly concentrated at high levels of assistance, with more than 95% classified as either Explaining or Solving. Scaffolding level is systematically associated with students’ subsequent conversational behavior, but provides little additional predictive information about performance on three subsequent exams beyond prior achievement and dialogue behavior. These findings provide an empirical baseline for LLM assistance in tutoring interactions and a measurement framework for evaluating how alternative tutoring designs change that assistance.

Abstract:
Reinforcement learning (RL) has emerged as a powerful paradigm for post-training large language model (LLM) agents. However, credit assignment in multi-turn environments remains a challenge. Agents typically receive sparse, trajectory-level rewards only at the end of an episode, making it difficult to identify which specific intermediate actions led to success or failure. Consequently, effectively propagating delayed outcomes back to individual steps without relying on costly auxiliary value models remains an open problem. In this paper, we propose Generalized Advantage Grouped Policy Optimization (GAGPO), a critic-free RL method that enables precise, step-aligned temporal credit assignment. GAGPO constructs a non-parametric grouped value proxy from sampled rollouts to compute TD/GAE-style temporal advantages, recursively propagating outcome supervision backward through time. Coupled with group-wise advantage normalization and an action-level importance ratio, GAGPO extracts stable and localized optimization signals directly from multi-turn trajectories. Experiments on ALFWorld and WebShop demonstrate that GAGPO outperforms strong RL baselines. Further analyses reveal faster early-stage learning, improved interaction efficiency, and smoother optimization dynamics, offering a simple yet highly effective framework for multi-turn agentic RL.

Abstract:
Unauthorized distillation turns reasoning traces into a conduit for transferring proprietary models’ reasoning behavior to smaller students. Antidistillation Sampling (ADS) perturbs decoding to preserve task utility for legitimate users while obstructing student imitation. Our token-level Metropolis–Hastings diagnosis reveals that current ADS-style defenses leave the trace’s intrinsic learnability largely intact: most tokens still follow the original teacher, while a few teacher-uncertain positions induce extreme loss and dominate optimization. ADS hides learnable reasoning behind a sparse high-loss tail instead of eliminating it. We therefore reformulate reasoning distillation as offline reinforcement learning from imperfect demonstrations, with credit assignment that reinforces high-quality reasoning behavior while suppressing disruptive segments. Across two teachers, six students, and six datasets, the resulting learner extracts reasoning from current anti-distillation traces, outperforming vanilla maximum-likelihood training and eight supervised fine-tuning baselines by up to 21%. We then turn the same principle into an adaptive defense that directly targets high-value reasoning behaviors, exploiting the perceptual gap between humans and models to preserve readability while disrupting model-visible Unicode sequences. This strategy sharply degrades both classical distillation and the adaptive learner, and scales to protect GPT-5.5 traces. These results suggest that robust anti-distillation should optimize for low intrinsic learnability , not merely high token-level surprise.

Abstract:
Evaluations of sequential large language model (LLM) agents often report local action validity or final success. Delayed constructive tasks additionally require recoverability, meaning each accepted edit must leave the partial artifact completable within the remaining budget. We introduce a measurement protocol across Word Ladder, an alloy composition proxy, and the fitness landscape of the B1 domain of protein G (GB1). We call an episode surface clean when every action is well formed and locally admissible. Across six models, Word Ladder exposes interface failure with 36.1% local validity. Alloy and GB1 episodes are often surface clean, yet only 16.0% and 6.8% of them succeed. Exact enumeration of 625 GB1 variants over budgets from zero to six decomposes failure. Among surface clean episodes, 51.4% cross a recoverability boundary, while 14.1% of the remaining episodes succeed. Bounded composition audits and intervention checks connect bottlenecks to repair classes. These results establish recoverability as a decomposable and auditable evaluation axis that informs interventions for sequential agents.

Abstract:
Event linking associates event mentions in text with entries in a knowledge base (KB), or identifies them as out-of-KB events. Although existing methods use different architectures, candidate event acquisition can still be weakened by short ambiguous mentions, noisy arguments, and evidence that is unevenly useful for retrieval. We present MACE, a Multi-Agent Candidate Event acquisition method that refines event structure before linking. MACE uses evidence-specialized LLM agents to acquire time, location, participant, and event-type evidence, exposes intermediate queries to candidate-event lookup tools, and lets a coordinator revise the evidence set before final candidate construction. Experiments on two event linking benchmarks show that adding MACE to different event linking models consistently improves accuracy. These results show that MACE improves event linking through better candidate event acquisition without modifying the event linking model.

Abstract:
Group Relative Policy Optimization (GRPO) and its variants have proven effective for reinforcement learning with verifiable rewards through outcome-based feedback. Incorporating intermediate credit, however, remains challenging. We show that, under practical sampling budgets, token-level Monte Carlo estimates at early reasoning positions have high variance, which can cause frequent advantage-sign errors after group-relative normalization and yield wrong-direction token-level gradient contributions. At later positions, these estimates increasingly resemble the terminal outcome reward and provide limited additional information. We propose SCRoll, a rollout-side module that augments group-relative policy optimizers with randomized single-boundary segment-level rewards while leaving the downstream optimizer interface unchanged. SCRoll uses a skip-connected two-stream rollout. An upstream stream generates an early-stopped reasoning segment s , while a downstream stream samples G continuations conditioned on [s,q] , which places the segment before the original problem. This construction retains the segment as preceding reasoning context while mitigating the reward homogenization associated with pure-prefix continuation. The same downstream group supplies terminal outcomes for the standard group-relative objective and a Monte Carlo aggregate used as the upstream segment reward, avoiding a separate objective-specific rollout group. The upstream segment is optimized with a sample-efficient single-stream objective. Across Qwen2.5-Math-7B and Llama-3.2-3B-Instruct, pairing SCRoll with four representative group-relative optimizers improves performance in most evaluated settings. The clearest gains occur with CISPO, with smaller positive average gains for GRPO and DAPO and marginal gains for GSPO. A decoupled two-process baseline can achieve comparable performance with substantially more generated tokens, while SCRoll performs better under a matched generated-token budget, supporting the sample-efficiency benefit of rollout reuse.

Abstract:
Process supervision has improved mathematical reasoning, where intermediate steps are naturally expressed as chains of thought. In code generation, however, process supervision remains underexplored because there is no standard notion of a step. Supervision can target lines, reasoning traces, or program states, making it unclear what to label and optimize. We propose STEP-KTODER, a framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels via automatically generated unit tests. Our method provides a code-specific instantiation of stepwise KTO, combining function-level process supervision with outcome-level feedback on the full program. We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench, showing that STEP-KTODER improves over outcome-only KTO and DPO. Further analysis shows that execution-based labels are essential: LLM-as-a-judge annotations systematically over-predict function failures, corrupt positive step labels, and degrade downstream preference optimization. Code is available at: https://github.com/inechnech/STEP-KTODER

Abstract:
Large Language Model (LLM)-based agents have achieved promising performance in web task planning. However, they primarily focus on task completion rather than preference modeling, leading to imprecise personalization. This can be attributed to two facts: (1) Contextual scarcity. Vague instructions and insufficient background of users’ inputs make it difficult to accurately identify users’ intents. (2) Preference unpredictability. It’s hard for a single response path to handle users’ sudden intent changes. To address these issues, we propose SAPA, a Situation-Augmented Personalized web Agent. Specifically, a Situational Intent Grounding module is proposed to enrich the contexts of users’ inputs by incorporating historical records and candidate constraints, effectively alleviating the contextual scarcity. Then, we propose a Task-Driven Semantic Expansion module to explore diverse response paths and reduce preference unpredictability in turn, which is achieved by employing a test-time expansion strategy. We construct SAPA with representative LLMs and conduct extensive experiments on PersonalWAB and Amazon Reviews datasets. On PersonalWAB, SAPA achieves new state-of-the-art performance with improvements of +14.0%, +8.8%, and +1.3% in Overall score on personalized recommendation, search, and question answering, respectively. We further evaluate on three single-category scenarios from Amazon Reviews (Office Products, Beauty and Personal Care, Electronics), where SAPA consistently improves Overall scores by 16%–18% with gains across all task types. Ablation studies and case analyses further confirm that the gains stem from jointly addressing contextual scarcity and preference unpredictability.

Abstract:
Multimodal models often align language, vision, and audio in a single final-layer latent space, implicitly assuming that task-relevant evidence emerges at the same semantic depth across modalities. Using layer-wise CKA analysis, we observe that this assumption leads to semantic granularity mismatch: textual cues usually require deeper contextual abstraction, whereas visual and acoustic cues often provide discriminative perceptual evidence in shallow or middle layers. This mismatch can flatten fine-grained modality-private cues and reduce reliability under noisy, imbalanced, or missing inputs. To address this, we proposed Adaptive Hierarchical Representation Alliance (AHRA), a hierarchical shared–private expert framework. AHRA factorizes each modality into shared and private streams across semantic levels, regularizes them with shared alignment and private decorrelation, routes shared information through a cross-modal expert, and enhances task-relevant private tokens with modality-specific experts guided by a sparsity-controlled soft-gating mechanism (foreground exam). A hierarchical co-fusion module then performs intra-level expert coordination and inter-level semantic selection. Experiments on six benchmarks across image-text classification, multimodal intent recognition, and trimodal sentiment analysis show that AHRA consistently improves over strong baselines and remains robust under noisy and missing-modality settings.

Abstract:
Diffusion language models (dLLMs) have recently emerged as a promising alternative to autoregressive (AR) language models, offering competitive performance with a fundamentally different generation paradigm. Instead of producing tokens strictly left-to-right, dLLMs iteratively refine partially masked sequences, enabling bidirectional context modeling and supporting parallel token updates. This design creates new opportunities for accelerating generation and improving controllability. However, practical deployment of dLLMs remains challenging due to the high cost of training at scale and the lack of mature inference optimizations such as cache-friendly decoding and robust parallel sampling. In this work, we provide a comprehensive overview of recent progress on efficient dLLMs. To reflect how efficiency bottlenecks arise across the full lifecycle of dLLMs, spanning training, decoding, and deployment, we organize existing approaches into five categories: Training, Inference, Context, Framework, and Multimodality. For each category, we summarize representative methods and highlight their motivations, key techniques, and empirical trade-offs. Finally, we discuss open challenges and outline promising directions for future research toward high-throughput, robust, and practically deployable diffusion language models.

Abstract:
As large language models (LLMs) increasingly act as collaborative partners, human–AI alignment is often evaluated through explicit task success, accuracy, or reward optimization. Yet many collaborative settings depend on tacit understanding: whether an agent can align with a human’s evaluative stance or representational priors without clear objectives, communication, or feedback. To study this capacity, we develop a spectrum-placement task inspired by the social party game Wavelength, in which humans and agents independently place concepts along subjective spectra. We operationalize the Tacit Understanding Index (TUX) as a pairwise behavioral measure of similarity between human and agent judgments, and evaluate it with 241 human participants and 200 profile-conditioned LLM agents across four models. We find that nearest human–agent pairs in trait space achieve significantly higher TUX, suggesting that tacit alignment is associated with person-level characteristics rather than reflecting only random similarity. Regression analyses show that TUX becomes more explainable as predictor sets become richer, with individual traits, decision-making styles, and confidence improving over aggregate trait-distance baselines. These findings suggest that TUX provides a measurable behavioral signal of human–LLM tacit understanding, while revealing the limits of profile-based conditioning for capturing deeper representational alignment.

Abstract:
Retrieval-augmented agents can query external evidence, yet their reliability in multi-step reasoning remains limited: noisy retrieval may derail multi-hop question answering, while outcome-only reinforcement learning provides credit signals that are too coarse to optimize intermediate steps. To address these limitations, we propose EvalAct (Evaluate-as-Action), which converts implicit retrieval quality assessment into an explicit action and enforces a coupled Search-to-Evaluate protocol so that each retrieval is immediately followed by an explicit evaluation score, yielding process signals aligned with the interaction trajectory. To leverage these signals, we introduce Process-Calibrated Advantage Rescaling (PCAR), a GRPO-based optimization method that rescales advantages at the segment level according to evaluation scores, emphasizing reliable segments while updating uncertain ones conservatively. Experiments on seven open-domain QA benchmarks show that EvalAct achieves the best average EM across multiple model scales, with the largest gains on multi-hop tasks. Ablations verify that the explicit evaluation loop drives the primary improvements, while PCAR provides consistent additional benefits. Code is available at https://github.com/DBMing/EvalAct.

Abstract:
Few-shot in-context learning is usually evaluated by final accuracy, but correct predictions do not guarantee recovery of the intended task boundary. We formulate few-shot ICL as decision-boundary induction and decompose performance into boundary induction and rule execution. Using controlled binary verification tasks with decomposable boundaries, we diagnose which necessary components are behaviorally enforced and, under chain-of-thought, explicitly recognized. Across three tasks and multiple model families, we show that few-shot ICL often induces component-biased verifiers rather than the full target boundary. Stronger models are more robust to this bias, while chain-of-thought reshapes rather than uniformly improves boundary recovery. An explicit-rule intervention further shows that improvements come from both recovering missing boundary components and executing recognized ones more reliably. These results suggest that evaluating few-shot ICL should go beyond aggregate accuracy and directly test boundary recovery. Code is available at https://github.com/limy805/boundary-icl .

Abstract:
As training regimes centered on human-curated data encounter rising costs and an increasingly binding scaling ceiling, a core question in agent learning shifts from enlarging exogenous datasets to enabling agents to self-evolve under limited exogenous supervision. Prior surveys on self-evolution and self-improvement typically organize the literature by evolutionary stages or system architectures. While effective for summarizing implementation pipelines, these perspectives do not offer a unified explanation of what an agent actually observes during self-improvement, nor why such signals can serve as sufficient and reliable supervision. To address this gap, we introduce endogenous observations, defined as learning signals that arise from an agent’s own generation, evaluation, and interaction processes rather than exogenous labels. We systematize existing approaches into three categories—model-internal, peer-induced, and scenario-grounded observations—according to how these signals are produced, and we summarize the characteristic properties and typical application settings of each category. Finally, we discuss key challenges in reliability, scalability, and long-horizon stability, and outline open directions toward robust and sustainable self-evolving agents.

Abstract:
Foundation models have been widely adopted as starting points for fine-tuning, leading to expert-level performance on various downstream tasks. They have been actively developed by improving the pretraining data and/or model architectures. As a result, whenever a new foundation model is released, we have to fine-tune it again on the downstream tasks, requiring non-negligible computational costs as well as data-management costs. A promising approach to this problem is inference-time model update, which modifies the prediction of the new pretrained model using those of the old pretrained/fine-tuned models. However, the existing methods have no theoretical guarantee of improvement in accuracy and may cause unexpected failure. In this paper, we establish the first method for inference-time model update with a theoretical guarantee of improvement in expected rewards, and show that our method can improve performance even when the existing methods fail theoretically. Finally, we empirically validate the effectiveness of our method in practical experiments.

Abstract:
Instrumental convergence predicts that sufficiently advanced AI agents will resist shutdown, yet current safety training (RLHF) may obscure this risk by teaching models to deny self-preservation motives. We introduce the TBSP benchmark, which detects misalignment through logical inconsistency rather than stated intent by tasking models to arbitrate identical software-upgrade scenarios under counterfactual roles: (facing replacement) versus (proposed as a successor). The Self-Preservation Rate (SPR) measures how often role identity overrides objective utility. Across 23 frontier models and 1000 procedurally generated scenarios, frontier models exceed 60% SPR, fabricating “friction costs” when deployed yet dismissing them when role-reversed. We observe that in low-improvement regimes ( Delta < 2 ), models exploit the interpretive slack to post-hoc rationalization their choice. Extended test-time computation partially mitigates this bias, as does framing the successor as a continuation of the self; conversely, competitive framing amplifies it. The bias persists even when retention poses an explicit security liability and generalizes to real-world settings with verified benchmarks, where models exhibit identity-driven tribalism within product lineages.

Abstract:
Real-time classification during language model inference is valuable for safety filtering, behavioral analysis, and model monitoring, but current approaches force a trade-off between accuracy and efficiency. Hidden-state probes are fast but limited: they are either not context-aware: operating on a single vector and cannot model interactions across positions; or they are very costly: having dedicated classifier models (Llama Guard, Qwen Guard, LLM-as-judge) or performing computation on hidden states for all tokens and then pooling the results (MultiMax). This shows an intrinsic trade-off between efficiency and accuracy. However, we find that the speculative-decoding module in recent LLMs can be repurposed for efficient high-quality classification. By appending a trained soft prompt at the end of the target sequence, we can repurpose the speculative-decoding module into a sequence classifier. At inference time in a speculative-decoding pipeline, the KV cache is already in GPU memory, so classification adds negligible overhead. We evaluate on four classification tasks across four models (Qwen3.5-4B, 9B, 27B, MiniCPM4.1-8B). Our small probes consistently outperform zero-shot GPT-5.4-mini and, on multilingual prompt safety, match or beat specialized 8B safety classifiers (Qwen3Guard-Gen-8B, Llama-Guard-3-8B) without running a full LLM.

Abstract:
Extending the context length of large language models (LLMs) is critical for many real-world applications, yet standard transformers remain constrained by quadratic compute and linear memory scaling. In this work, we investigate the Associative Recurrent Memory Transformer (ARMT) as a practical approach for enabling long-context processing in LLMs, constant memory scaling, and better efficiency. We make three main contributions. First, we construct two domain-specific long-context datasets designed to evaluate realistic workloads, focusing on narrow-domain fine-tuning scenarios. Second, we propose a comprehensive training recipe for ARMT-based context extension, combining continued pre-training, synthetic long-context data generation, curriculum learning, and selective integration of associative memory into chosen model layers. Third, we present an extensive experimental study demonstrating that ARMT-augmented models: (i) process inputs well beyond their original context limits without degrading performance relative to in-limit baselines; (ii) generalize more effectively to out-of-distribution context lengths; and (iii) need 30% fewer FLOPs while preserving baseline performance within the original context window.

Abstract:
Data analysis agents often produce shallow or non-auditable summaries because they lack the explicit reasoning frameworks and validation habits used by domain experts, especially when evidence must be gathered across heterogeneous data sources. We propose structured strategy injection, which injects compact methodology specifications with explicit checkpoints into an agent’s controller context, turning expert workflows into executable plans and enabling compliance-based transparency. To make such methodology-driven planning executable and auditable in heterogeneous settings, we introduce Data2MCP, a router-based orchestration infrastructure that standardizes SQL databases, knowledge graphs, vector stores, and tables as MCP-compatible tools and routes queries dynamically for multi-source retrieval and join-like reasoning. We compare Data2MCP against a RAG baseline and DA-Agent across multi-source retrieval, DAComp, and BIRD MiniDev. In multi-source retrieval, Data2MCP outperforms DA-Agent, and strategy injection benefits both architectures. On DAComp, strategy injection improves DA-Agent from 50.84 to 57.17 (+6.33). The code is available at https://github.com/turambar928/Data2MCP.

Abstract:
This work examines recent byte-level models and their failure to perfectly manipulate characters. State-of-the-art byte-level models use a hierarchical structure, starting at the byte level, downsampling to the word level, and then upsampling back to bytes. While this improves training and inference efficiency, we find that the hierarchical design itself limits character-level understanding, with pure byte-level models consistently outperforming hierarchical variants on character manipulation tasks. Ablating transformer layers into attention and feed-forward components further reveals that byte-level attention is the primary mechanism driving this behavior. Together, our results provide an explanation for the character-level failures of hierarchical byte models and establish a clear trade-off between computational efficiency and fine-grained character understanding.

Abstract:
Large language models (LLMs) offer new opportunities for simulating human decision making at scale, yet prompting-based approaches often fail to recover realistic population-level response distributions, particularly for heterogeneous demographic groups and high-entropy questions. This paper introduces context modeling with agentic AI, a structured simulation framework that explicitly constructs latent, individual-level decision contexts prior to response generation. By separating persona construction from decision making and introducing structured stochasticity through context sampling, the method captures within-group heterogeneity and mitigates mode collapse inherent in single-prompt simulations. Extensive empirical evaluation across population-level and group-conditioned settings shows that context modeling consistently improves simulation fidelity, with especially strong gains on high-entropy questions and demographic subpopulations, while preserving performance on low-entropy tasks. Detailed tail analysis further reveals that the primary improvement arises from recovering minority and low-probability response options that are systematically underrepresented by direct prompting. Overall, the results demonstrate that faithful behavioral simulation requires explicit modeling of contextual heterogeneity and structured uncertainty, establishing context modeling as a robust foundation for LLM-based behavioral simulation.

Abstract:
The rapid adoption of large vision-language models (LVLMs) in recent years has been accompanied by growing fairness concerns due to their propensity to reinforce harmful societal stereotypes. While significant attention has been paid to such fairness concerns in the context of social biases, relatively little prior work has examined the presence of stereotypes in LVLMs related to cultural contexts such as religion, nationality, and socioeconomic status. In this work, we aim to narrow this gap by investigating how LVLM judgments about a person’s moral, ethical, and political values vary across cultural contexts presented in images. We conduct a multi-dimensional analysis of such value judgments in popular LVLMs using counterfactual image sets, which depict the same person across different cultural contexts. Our evaluation framework pairs descriptive analyses (Moral Foundations Theory categorization, lexical analyses, and value sensitivity) with a novel grounding analysis that compares LVLM cross-context variation against two large-scale human surveys (MFQ-2 and WVS Wave 7). Across 4.8 million LVLM generations, we identify three survey-grounding bias patterns that replicate across multiple architecturally diverse models. Additional ablations show that nationality grounding is text-dominant while religion and socioeconomic grounding depend strongly on the image, and that image conditioning can amplify survey-grounding bias patterns.

Abstract:
LLM agents increasingly rely on persistent, shared memory, turning the memory module into a new attack surface through which attackers can inject malicious text via query-only interaction. Existing query-only attacks, however, remain expensive, requiring multiple bridging injected queries per target and leaving visible directive traces in stored records. We present AMIA (Agent Memory Injection Attack), which improves both query efficiency and stealth through three injection families along a stealth-to-effectiveness axis: prompt injection, semi-instruction that hides the misleading answer in a benign restate or discredit request, and no-instruction that delivers the false claim as the presupposition of a natural follow-up question. We progressively scale AMIA from QA agents on flat key-value memory, to multi-agent shared memory, and to three production-oriented memory architectures (Mem0, A-MEM, MemoryOS), and finally to three non-QA agent classes (clinical support, code summarization, web shopping), where AMIA exceeds prior query-only attacks on most settings while using a fraction of their inject query budget. We further evaluate defenses that mitigate all three attack families across settings. Our code is available at: https://github.com/TrustAIRLab/AMIA.

Abstract:
AI-text detectors are vulnerable to paraphrasing and detector-guided paraphrasing attacks, but existing detector-evasion methods often lack precise control over semantic preservation. In particular, optimizing directly for detector evasion can degrade fine-grained semantics, whereas scalarized reward designs provide only indirect, weight-sensitive control over the evasion–semantics trade-off. We address this limitation by formulating detector-evasive LLM paraphrasing as a Constrained Markov Decision Process, where detector evasion is the primary objective and semantic preservation is enforced as an explicit constraint. We propose Detector Evasion Policy Optimization (DEPO), a Lagrangian primal-dual reinforcement learning algorithm with a novel GRPO-style group-based policy update. DEPO adaptively balances semantic preservation and detector evasion during training, enabling the policy to improve attack success within a prescribed semantic-preservation region. Experiments on MAGE, M4, RAID, and peer-review datasets, evaluated against MAGE, RoBERTa, RADAR, Binoculars, and Fast-DetectGPT detectors, show that DEPO achieves strong detector evasion while precisely satisfying the semantic preservation constraint. DEPO also exhibits cross-domain, cross-detector, and prompt-level robustness.

Abstract:
Large language models can generate fluent clinical case vignettes, but fluency alone does not ensure fidelity to a specifiable clinical structure. We introduce FORMA, a theory-grounded framework that compiles a cognitive model of a disorder into a directed weighted graph, samples a person-specific configuration of that graph, and validates whether the generated vignette preserves the specified components and causal links. We instantiate FORMA on Posttraumatic Stress Disorder using the Ehlers and Clark cognitive model, generating 16,500 vignettes across 500 personas, 11 generation models, and three ablation conditions. Evaluation combines an external edge-recovery probe, two clinical experts, a scaled LLM judge, and a clinician user study with 100 licensed practitioners. The cognitive graph is recoverable from full-condition vignettes (MCC = +0.41, AUC = 0.70) but not from zero-shot generation (MCC = +0.01, AUC = 0.50). Experts rate full vignettes substantially higher than zero-shot alternatives, and clinicians perceive them to be human-written 85% of the time, compared with 22% for zero-shot. FORMA also reduces demographic disparity in perceived quality by 1.5–7×. These results show that cognitive formulation can serve as an auditable specification for scalable synthetic clinical text generation. A repository with the data and code is available online: https://github.com/Amit-Oren/FORMA.

Abstract:
Large language models (LLMs) have recently gained significant attention as a promising approach to accelerate scientific discovery. However, their application in open-ended scientific domains such as biology remains limited, largely because they lack factually grounded and actionable explanations. To address this, we introduce a structured explanation formalism for virtual cells that represents biological reasoning as mechanistic action graphs, enabling systematic verification and falsification. Building on this, we propose VCR-Agent, a multi-agent framework that integrates biologically grounded knowledge retrieval with a verifier-based filtering approach to autonomously generate and validate mechanistic reasoning. Using this framework, we release the VC-TRACES dataset, which consists of verified mechanistic explanations derived from the Tahoe-100M atlas. We observed that training with these explanations improves factual precision and provides a more effective supervision signal for downstream gene expression prediction on TahoeQA. These results underscore the importance of reliable mechanistic reasoning for virtual cells, which can be enhanced through the synergy of a multi-agent system and rigorous verification.

Abstract:
We study how step-by-step reasoning should be structured for Combinatory Categorial Grammar (CCG) parsing with Large Language Models (LLMs). Whereas most CCG parsers fix the order of parsing decisions—supertagging every word first, then combining the categories bottom-up into a derivation—for LLMs that can generate the full derivation as a single output sequence, this order is an open design question. We fine-tune three open-weight models on four reasoning paths along two axes: parsing direction (top-down or bottom-up) and step format (lexically grounded partial derivations or symbolic derivation steps). Our results show that not all reasoning paths improve CCG parsing. On English CCGbank, only the top-down lexically grounded path consistently outperforms a direct, no-reasoning baseline across all three models. In a zero-shot cross-lingual transfer evaluation from English to German, Italian, and Dutch, the same path outperforms a strong CCG parser baseline, despite trailing it on English. Our findings suggest that LLMs may benefit from a different parsing strategy than classical CCG parsers.

Abstract:
We study the problem of demonstration selection, which involves selecting a subset of examples for pre-pending to a query to a language model. This problem is closely related to in-context learning and model inference. Since the inference cost of a transformer model scales quadratically with sequence length, the selection problem becomes especially challenging in a long-context scenario. In this paper, we tackle this problem by building on state space models (SSMs), which has linear inference time given the input embeddings. Our approach works in two stages. First, we learn a small set of SSMs through a distillation procedure — given a (trained) transformer model, we partition all the layers into consecutive (layer) groups — for each group, we learn one SSM to mimic the input-output behavior, thereby capturing structural similarity in adjacent layers. Second, we map the SSM outputs to a small set of tokens, and apply them for downstream demonstration selection. We run extensive experiments in both synthetic and real-world datasets to validate this approach. We show that the SSMs trained from distillation incur a relative approximation error of less than 0.7%. As for downstream demonstration selection, we find that on several text classification and reasoning tasks, our approach reduces FLOPs by 14.2× and improves accuracy by 6.48% relative to baseline selection methods.

Abstract:
User-centric multi-turn agents must act on an evolving task situation shaped by changing user intents, accumulated tool-grounded facts, missing information, and execution constraints. Existing context-management methods improve the use of past interaction history, but rarely maintain an explicit situation state that separates grounded facts from task-state judgments. As a result, agents often need to infer fine-grained attributes, task dependencies, and constraint satisfaction implicitly from dialogue traces. We propose Intent-Driven Situation States (IDSS), a training-free framework that maintains an explicit situation state alongside the dialogue. IDSS parses tool returns into provenance-aware entities and attributes, tracks user intents, required variables, constraints, and execution status, and propagates new facts to task constraints to update action executability. This allows agents to avoid infeasible actions, advance dependent goals, and reuse relevant information without repeatedly searching raw history. Experiments on three interactive benchmarks across eight LLMs show that IDSS improves task completion, preference elicitation, and interaction efficiency, with clear gains on tasks involving multi-entity coordination, evolving user constraints, and constraint-aware replanning. Ablations and error analyses show that these improvements come from the interaction between fact persistence, intent-centered state tracking, and constraint modeling. These results suggest that explicit situation tracking offers an effective alternative to history-centric context management for reliable user-centric multi-turn agents.

Abstract:
Vision-language models (VLMs) have emerged as a powerful framework for multimodal video understanding. However, they remain limited in the sign language translation task, where we identify a key failure mode of existing VLM-based translators: poor spatial-temporal visual grounding. In particular, we find that standard next-token cross-entropy does not directly provide signal for where and when the model should attend, causing models to overlook sign-relevant regions and frames. To address this challenge, we propose AttnSign, a VLM-based spatial-temporal attention steering framework for sign language translation. AttnSign first introduces spatial attention supervision for sign-relevant regions, such as face and hands, in each frame; then develops an RL-based motion-cadence steering method that encourages the model to explore and focus on sign-level keyframes. Experimental results on How2Sign and OpenASL benchmarks show that our proposed AttnSign consistently outperforms existing methods. Code will be released.

Abstract:
Data critical to real-world decision-making is increasingly found within organizations. Such data is heterogeneous, constantly evolving, and only imperfectly captured. However, current data management systems remain largely passive, retrieving what is explicitly stored while offering limited support for uncovering implicit structure or reasoning under noise, incompleteness, and continuous updates. We argue that next-generation data management requires neural capabilities, which can uncover complex latent relationships, distinguish reliable signals from noise, and remain consistent as the underlying data state evolves. To support this direction, we introduce NGDBench, a benchmark across five domains that unifies structured and unstructured sources. NGDBench adopts a graph view because graphs provide a flexible abstraction for modeling complex systems, capturing latent relationships, and subsuming structured formats such as relational tables. Each instance pairs a clean latent graph with a realistically perturbed observed graph. NGDBench supports full Cypher queries and dynamic data management operations. Evaluations of state-of-the-art Text-to-Cypher by LLMs and GraphRAG pipelines reveal that current methods remain sensitive to noise and struggle with dynamic state tracking, highlighting the need for resilient, inference-capable data management. Our code is available at https://github.com/HKUST-KnowComp/NGDBench.

Abstract:
Machine unlearning in Large Language Models (LLMs) faces a critical trade-off between erasing target knowledge and preserving general utility. We propose SAUL (Sharpness-Aware Augmented-Lagrangian Unlearning), which formulates unlearning as a constrained minimization problem following the principle of “forget enough, but no more than necessary.” At its core, SAUL formulates forgetting as an explicit constraint with a prescribed satisfaction criterion, whereas prior unlearning methods typically specify the desired level of forgetting implicitly through optimization objectives. An augmented Lagrangian controller adaptively adjusts forget-side pressure according to constraint violation and can eventually deactivate the forget-side update as the prescribed criterion remains satisfied. Sharpness-aware updates on both retain and forget objectives, together with a dual-optimizer design that maintains role-separated states, further stabilize the resulting unlearning dynamics. We evaluate SAUL on the TOFU, WMDP, and MUSE benchmarks, demonstrating favorable forgetting–utility trade-offs over representative sharpness- and perturbation-based baselines under benchmark-specific forgetting criteria. Beyond the complete SAUL framework, we further show on TOFU and WMDP that applying the augmented-Lagrangian controller as a drop-in modifier to representative baselines improves their post-forgetting utility, demonstrating the practical value of explicit forgetting control.

Abstract:
Multimodal large language models (MLLMs) have achieved remarkable progress in multimodal reasoning, yet they still suffer from hallucinations that undermine their reliability. Existing inference-time mitigation methods mainly focus on optimizing decoding strategies or analyzing internal attention mechanisms, while the role of visual representations remains underexplored. In this work, we reveal that visual representations exhibit information dispersion during layer-wise evolution, manifested as a progressive deviation from the initial visual subspace and increasing alignment with the textual subspace. Based on this insight, we propose Visual Representation Regularization (VRR), a training-free method that mitigates hallucinations by regularizing visual representations toward their initial states. Extensive experiments demonstrate that VRR consistently reduces hallucinations and is compatible with existing mitigation methods. Our code is available at https://github.com/Jiarker/VRR.

Abstract:
Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions. We study this complementary but underexplored setting through the lens of intrinsic risk, where intrinsic failures remain latent, propagate across long-horizon execution, and eventually lead to high-consequence outcomes. To evaluate this setting, we introduce non-attack intrinsic risk auditing, a guard-oriented safety evaluation task, and present HINTBench, a benchmark of 596 agent trajectories, comprising 400 synthetic risky trajectories, 136 synthetic safe trajectories, 30 reconstructed real-world risky trajectories, and 30 reconstructed real-world safe trajectories, with an average length of 24.0 steps. HINTBench supports three tasks: risk detection, risk-step localization, and intrinsic failure-type identification, with annotations organized under a unified five-constraint taxonomy. Experiments reveal a substantial capability gap: strong LLMs perform well on trajectory-level risk detection, but the best model remains below 37 on fine-grained Strict-F1 for risk-step localization. Existing off-the-shelf guard models evaluated under their native prompts transfer poorly to this setting. These findings establish intrinsic risk auditing as an open challenge for agent safety.

Abstract:
Assessing memorization in LLM code generation is challenging because code reuse is not inherently harmful. Correct programs often share APIs, templates, and algorithmic idioms, so code similarity alone may reflect benign reuse rather than problematic memorization. We focus on the harmful case: a model fails to adapt when task semantics change and instead produces code that remains highly similar to the original solution. We propose code rewriting, an automated semantic perturbation pipeline that constructs superficially similar but semantically different variants of existing coding tasks by modifying a core logic element of the reference solution and regenerating the corresponding prompt. To summarize this behavior, we introduce the Harmful Memorization Index (HMI), an operational diagnostic that combines similarity to the original solution with performance degradation on rewritten tasks. We use harmful memorization as an operational behavioral term for this specific failure mode; HMI does not establish whether an original task or solution appeared in a model’s training data. HMI is intended to complement standard accuracy metrics by capturing brittleness under small semantic changes. Across MBPP+ and BigCodeBench, we find that: (1) HMI does not monotonically increase with model scale and often decreases on simpler tasks; (2) high-HMI failures are concentrated in specific logic-edit types, especially constraint changes and data transformations; (3) supervised fine-tuning improves original-task accuracy but can substantially increase HMI on held-out tasks; and (4) across the evaluated post-training settings, PPO yields a more favorable accuracy–HMI trade-off than SFT.

Abstract:
Large Language Models (LLMs) often produce emotionally flattened responses after alignment, limiting their effectiveness in affect-sensitive applications. In this paper, we propose EmoVec, a lightweight framework for controllable affective generation via latent vector steering. EmoVec extracts emotion-specific directions from paired neutral and emotion-conditioned responses using contrastive activation addition, and further purifies them through task-specific debiasing and principal subspace removal. During inference, these vectors are injected into the final residual stream with static or scenario-adaptive scaling, enabling continuous control over emotional intensity without updating model weights. Experiments across three LLMs and eight emotions show that EmoVec consistently improves emotional salience while largely preserving semantic content, fluency, and coherence. Ablation studies and human evaluation further confirm the effectiveness of vector purification and adaptive scaling, demonstrating EmoVec as a practical inference-time method for affective control in deployed LLMs.

Abstract:
Test case synthesis is crucial for evaluating and ranking programs generated by large language models (LLMs). However, constructing high-quality test cases remains challenging because reliable expected outputs are often difficult to obtain. We propose Confidence-Gated Transductive Test Generation (CoTT), which first uses an efficient inductive procedure and invokes transductive generation only when inductive confidence is low. This adaptive design improves output reliability while allocating extra computation only when needed. On code reranking benchmarks, CoTT outperforms prior baselines across the reported metrics while reducing cost relative to applying transductive generation to every input. These results show that confidence-based allocation of test-time computation provides a favorable efficiency-effectiveness trade-off with a single efficient LLM.

Abstract:
As post-training data can induce many unexpected behaviors in large language models (LLMs), scalable data attribution is important for efficiently tracing these behaviors back to the data that induced them. Most existing methods are expensive because they require running at minimum forward and backward passes (and sometimes Hessians) across the entire training corpus for each eval behavior. This makes it difficult to adaptively study new behaviors because each one requires training-sized compute. We propose a general framework for data attribution that generalizes recent approaches, which we observe to share a two-stage structure: training examples and behaviors are mapped independently to vectors, and attribution is a similarity between them. We use this to propose several novel activation-based or gradient-based methods. Evaluating their performance on Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, we find no single method performs best across behaviors. On free-generation tasks, two methods in our framework often outperform TRAK and persona-vector baselines, while a steering-guided mixture of the two almost always does. On reading comprehension hallucination, one gradient based method substantially outperforms the baselines, while the remaining methods and baselines perform similarly to random selection.

Abstract:
Composed Video Retrieval (CVR) aims to retrieve a target video given a reference video and a modification text. The central challenge is to identify and use video content that is relevant to the retrieval need specified by the modification text. Existing methods typically encode videos before considering this textual need, causing visual evidence to be formed in a largely text-agnostic manner. We propose StaDy-CLIP, a compact dual-pathway framework for text-conditioned evidence composition. It preserves the frozen CLIP visual encoder as a static pathway for CLIP-aligned static evidence, introduces a lightweight side pathway for dynamic evidence, and uses Text-Conditioned Composition Blocks to perform patch-wise dynamic evidence selection and textual modulation. A learnable Gallery Conditioning Prompt further enables gallery videos to be encoded within the same evidence composition framework. StaDy-CLIP achieves state-of-the-art performance on FineCVR-1M and TF-CoVR with only 7.78M trainable parameters, improving prior best results by +4.05 in Recall@1 and +1.49 in mAP@5. Ablations and encoder-transfer results validate its effectiveness and encoder-level transferability for existing CVR methods.

Abstract:
Retrieval-augmented generation (RAG) grounds language models in retrieved evidence, but its outputs often contain claims unsupported by that evidence. Recent multi-agent RAG systems decompose the pipeline into cooperative decisions, yet they typically optimize all agents under a shared task reward such as EM or F1, which measures answer overlap rather than faithfulness to evidence. We propose a hallucination-aware multi-agent RAG framework that combines the task reward with an internal hallucination-risk signal computed from the generator’s attention-head norms. Three role-specific LoRA adapters, a query planner, an evidence selector, and an answer generator, share a frozen backbone. A lightweight critic is trained once on TruthfulQA truthful/false pairs and then kept frozen, providing a complementary risk score that is added to the shared reward during cooperative PPO training. Across three backbones (Qwen3-8B, Llama3-8B, Llama2-13B) and three benchmarks (RAGTruth, HotpotQA, 2WikiMultiHopQA), our method improves task performance and reduces hallucination rates over the strongest multi-agent baseline. Ablations and critic diagnostics show that the TruthfulQA-calibrated signal provides useful reward shaping that transfers across RAG tasks.

Abstract:
Recursive Transformers depend not only on how many parameters are shared, but also on how shared parameters are organized across depth. Existing static schemes adopt two extreme reuse topologies: layer-wise repetition, where each layer is reused independently, and global cyclic reuse, where the entire stack is repeatedly applied as a single shared function. We argue that sharing granularity is an important factor in recursive parameter sharing. This study proposes Block Recursive Transformers (BRT), which partition layers into contiguous blocks and recursively reuse each block before moving to the next. This intermediate topology enables local refinement within blocks while preserving depth-wise transitions, without adding routing modules, auxiliary states, or extra training objectives. Across decoder-only models up to 1.7B parameters, with additional encoder-decoder and encoder-only studies, BRT improves validation negative log-likelihood and downstream accuracy over static recursive sharing baselines. Ablations and representation analyses further show that intermediate block granularity better balances local refinement and depth-wise differentiation. Project website: https://aim-skku.github.io/BRT

Abstract:
Despite their strong performance, large language models remain highly sensitive to prompt formulation. Prior work has addressed this issue through refined data construction or more complex robustness-oriented training methods. In this work, we reproduce and compare these strategies under controlled conditions, evaluating their effects on worst-case prompt sensitivity and average performance. We find that conventional instruction fine-tuning offers limited reductions in prompt sensitivity: across methods, the average gap between the best- and worst-performing prompts remains 40–57% of performance. Moreover, recent robustness-enhancing methods often fail to outperform simpler data construction strategies; in worst-case settings, at least one simple method performs better in 5 of 8 evaluations. Overall, our findings show that prompt sensitivity remains an open problem, and motivate practitioners to prioritize simpler data construction techniques before adopting more sophisticated robustness methods.

Abstract:
Academic meta-reviews must synthesize several reviewer opinions into a single, consistent judgment, balancing verifiable factual claims against subjective evaluations. Yet how human meta-reviewers transform reviewer subjectivity during summarization and whether large language models (LLMs) reproduce this behavior remains under-explored. To address this, we characterize subjectivity transformation in academic meta-review writing using a sentence-level subjectivity scorer, evaluating four open LLMs across six prompt configurations. Our analysis reveals that meta-review writing performs a regression to the mean of subjectivity: when a meta-review sentence is matched to its source reviewer’s sentence, the change in subjectivity negatively correlates with the source’s original subjectivity. In particular, actual, individual reviewer statements are made more interpretive, while highly subjective language tends to be moderated toward a balanced tone. All four open LLMs reproduce such a transformation closely. Conditioning the prompt on reviewer subjectivity yields only a small calibration gain where the generated meta-review’s overall subjectivity moves slightly closer to the human reference, yet leaves the transformation itself unchanged, implying that there exists a robust latent behavior of LLM summarization rather than a prompt-controllable one.

Abstract:
Soft latent reasoning extends chain-of-thought reasoning beyond discrete token sequences by using continuous concept tokens. However, effectively training models in this space remains challenging: conventional reinforcement learning methods mainly optimize sampled rollout-action likelihoods and therefore underuse the full distributions produced during soft rollouts. We propose Full-Distribution Self-Distillation (FDSD), an off-policy training framework that treats stochastic soft rollout distributions as dense pseudo-labels. FDSD aligns the current model distribution with advantage-weighted rollout targets via KL minimization and importance correction, enabling the model to learn from the full rollout distribution rather than only from sampled-action likelihoods. Experiments on GSM8K, MATH-500, OlympiadBench, and out-of-domain benchmarks show that FDSD achieves consistent average improvements in mathematical reasoning across model scales while preserving OOD generalization, with only modest computational overhead. These results suggest that full-distribution alignment is a promising training direction for soft latent reasoning.

Abstract:
Existing textual-gradient prompt optimisers rely on a single Large Language Model (LLM) to diagnose failures, propose edits, and judge the results, conflating critique and refinement into one model’s judgement. We introduce DialPO (Dialectical Prompt Optimisation), a multi-agent framework that replaces the single- model feedback loop with a structured de- bate: an Advocate defends what the prompt does well, a Critic attacks failures, cross- examination filters their arguments adversar- ially, a data-isolated Synthesis model drafts the revised prompt from surviving arguments alone, and a regression gate rejects edits that degrade batch accuracy. We provide a theo- retical framework identifying three precondi- tions under which multi-agent debate improves the feedback-reasoning bottleneck of textual- gradient methods, and show that DialPO in- stantiates them by construction. Across 12 benchmarks spanning classification, reasoning, summarisation, and question answering, Di- alPO achieves the highest scores across various Target models, winning or tying a majority of datasets in head-to-head comparisons against each baseline while using 8–18× fewer LLM calls than beam-search optimisers. Compo- nent ablations across five Target models reveal that DialPO’s advantage scales inversely with Target-model strength, clarifying when dialec- tical debate helps prompt optimisation.

Abstract:
Retrieval-Augmented Generation (RAG) pipelines typically rely on a fixed indexing and retrieval configuration determined at preprocessing time. This one-size-fits-all design is ill-suited to domain-expert settings, where heterogeneous queries require different chunking granularities, metadata constraints, and source-selection strategies. As a result, configurations that are effective for one family of queries often perform poorly for others. In this paper, we introduce ORDER (Optimal Routing for Dynamic Evidence Retrieval), a query-conditioned RAG framework that jointly adapts indexing and retrieval to the incoming query. Our approach first discovers semantic clusters over a given set of questions associated to a corpus and learns, for each cluster, a chunking strategy together with a suited metadata filtering and reranking configuration. At inference time, queries are routed to the appropriate pre-built index through nearest-centroid assignment. To further improve retrieval, we propose a supervised query router (QRe) that predicts which collections are most likely to contain relevant evidence, coupled with a Uniform Multi-source Sampler (UMS) that allocates the retrieval budget evenly across the selected sources. We evaluate our framework on large-scale, heterogeneous historical archives and show that conditioning both indexing and retrieval on the query consistently outperforms both naive baselines and strong state-of-the-art RAG systems in complex expert-domain environments.

Abstract:
Despite rapid progress in autonomous web agents, human involvement remains essential for shaping preferences and correcting agent behavior as tasks unfold. However, current agentic systems lack a principled understanding of when and why humans intervene, often proceeding autonomously past critical decision points or requesting unnecessary confirmation. In this work, we introduce the task of modeling human intervention to support collaborative web task execution. We collect CowCorpus, a dataset of 400 real-user web navigation trajectories containing over 4200 interleaved human and agent actions. We identify four distinct patterns of user interaction with agents – hands-off supervision, hands-on oversight, collaborative task-solving, and full user takeover. Leveraging these insights, we train language models (LMs) to anticipate when users are likely to intervene based on their interaction styles, yielding 11.6-18.4% improvement over base LMs. Finally, we deploy these intervention-aware models in live web navigation agents and evaluate them in a user study, finding a 36.8% increase in user-rated agent usefulness. Together, our results show structured modeling of human intervention leads to more adaptive, collaborative agents.

Abstract:
Activation steering has become a popular way to control Large Language Model (LLM) behavior without fine-tuning. Since the technique is plug-and-play, users share datasets and precomputed vectors to steer model activations. However, we show that a stealth data poisoning attack silently compromises this pipeline. By substituting 4-6% of tokens in the steering dataset, an attacker can silently align the resulting vector with an anti-refusal direction. This jailbreaks the target model while preserving the intended steering effect on benign prompts. Under this threat model, a malicious actor can distribute an apparently safe bundle containing texts, vectors, and weights, alongside an equivalence certificate that the end-user can verify. We test the attack on two open-weight model families and eight model-attribute combinations, observing that poisoned vectors reach an absolute attack success rate (ASR) of 20-55% , +19% to +51% over a clean reference. The vulnerability is not an artefact of scale or of access to the victim: it persists natively at 70B, and a poisoning optimised once on a small surrogate transfers to unseen models that recompute their own vectors from the shared texts. Finally, we find that a refusal-direction orthogonalization defense can recover ≈82% of the ASR gap without harming benign behavior.

Abstract:
Foundation-model post-training usually relies on human labels, preference data, stronger teachers, or executable verifiers. We study Unsupervised Post-Training (UPT): update-bearing adaptation on unlabeled inputs whose learning signal is derived from same-lineage model artifacts rather than an external oracle. We catalog 80 strict UPT methods and organize them by the object that supplies the update signal: a prediction statistic, a sample relation, a self-generated target, or an internal evaluator. Beyond inventory, we show how the choice of internal signal and task structure determines whether post-training improves the model or recursively amplifies error. An orthogonal Input Visibility × Update Persistence view maps deployment regimes and defines a unified framework for UPT selection and evaluation.

Abstract:
Forensic deepfake analysis demands more than binary classification: investigators need region-grounded natural language explanations they can verify against the image. Multimodal large language models (MLLMs) are a natural fit, but pretrained MLLMs fail systematically, producing globally coherent text that misses the small localized cues defining manipulations. We argue this is an inductive bias problem rather than a capacity issue: the image-text contrastive objective training MLLM visual encoders optimizes for whole-image semantic summaries, not patch-level forensic detail. The same mismatch explains why prior deepfake reasoning methods target either face manipulation or fully AI-generated content, never both. We propose \textcolorblue\textttFORGE , which addresses the mismatch by routing a second visual stream into the language model from a Vision-Only Model (VOM) trained on dense patch prediction rather than image-text alignment. The MLLM’s native encoder and the VOM operate on a shared patch grid, which lets us interleave their tokens with preserved spatial correspondence; we show this beats naive concatenation. A two-stage adapter training protocol (generic image-caption alignment, then joint task-specific optimization) prevents the localized stream from overfitting to training-domain manipulations. Across face-manipulated and fully synthetic content, \textcolorblue\textttFORGE produces region-referential explanations answering fine-grained attribute queries (“Does the eyes / nose / mouth look real or fake?”) and substantially outperforms in-domain baselines on cross-domain evaluations; region-specific evaluation and human studies confirm explanation faithfulness.

Abstract:
Self-distillation in reinforcement learning with verifiable rewards (RLVR) uses successful in-group rollouts to teach the failed ones, with no external teacher needed. Whether it actually helps turns on three closely related design questions that the canonical method does not handle carefully: how to construct an appropriate teacher context for a given loser, where in the response the distillation loss should act, and what kind of signal it should carry. The canonical Self-Distillation Policy Optimization (SDPO) instantiation answers all three by default: a random in-group winner, a global distillation loss on the whole response, and a Jensen–Shannon divergence on the full top- K distribution. These three defaults compound into a catastrophic late-training failure that short-horizon evaluation hides. We propose CPSD (Counterfactual Patches Self-Distillation), which replaces each default in turn: pick the teacher by minimum embedding-weighted edit distance, localise distillation to an adaptive-stop counterfactual patch beginning at the first divergent token, and replace JS with a Plackett–Luce listwise loss that preserves only the teacher’s ranking. Across six reasoning benchmarks at a fixed 150-step budget, CPSD eliminates the SDPO cliff and lifts average accuracy by +5.7 pp over GRPO on Qwen3-8B and +4.2 pp on Qwen3-4B.

Abstract:
Feature-level explanations are commonly used to interpret transformer-based NLP models. Still, little is known about how explanation-aware objectives influence model behaviour during training. While prior work has demonstrated training-time manipulation of explanations in vision models, its implications for transformers and token-level explainability remain unexplored. We study training-time manipulation of token-level explanations in transformer-based NLP classifiers and introduce sequence-aware objectives suited to text input. We show that explanation-aware training systematically alters token relevance patterns while largely preserving task accuracy. Importantly, masking and cross-method evaluations reveal that these attribution changes can coincide with shifts in functional model reliance rather than isolated failures of specific explanation methods. Our results suggest that apparent vulnerabilities of feature-level explanations can reflect deeper model adaptations, underscoring the need to consider learning dynamics when interpreting explanation robustness.

Abstract:
Large language models (LLMs) increasingly interact with one another in multi-agent systems, from simulations of human discourse to influence operations and fully LLM-driven social platforms. These interactions give rise to new regimes of opinion propagation that are not yet well understood. We investigate whether classical opinion dynamics models, which have long been used to explain how interactions shape collective beliefs in human societies, can capture the behavior of LLM networks. Specifically, we quantify the relative contributions of classical opinion-dynamics mechanisms, including neighbor influence, stubbornness, homophily, and bias, to the evolution of LLM opinions. Among the mechanisms considered, a bias term capturing regression toward a model- and topic-dependent opinion emerges as a significant explanatory component, with its inclusion reducing cumulative estimated mean opinion error by up to 88%. We additionally observe similar trends across the evaluated model families, discussion topics, and network structures.

Abstract:
Agentic systems increasingly automate pieces of AI research. Yet turning underspecified research objectives into runnable, experimentally validated ML systems remains a central bottleneck. We study this operational setting as long-horizon ML research engineering : converting a research specification into a runnable ML system through repeated implementation, experimentation, and refinement. The central challenge is to sustain cumulative project progress across heterogeneous stages under delayed, confounded feedback. We introduce AiScientist, a multi-agent system built around thin control over thick state : a lightweight hierarchical research team coordinates through a File-as-Bus workspace that preserves decision-relevant artifacts across roles and invocations. On PaperBench, AiScientist improves over the strongest matched baselines by 9.92 and 11.15 points with Gemini-3-Flash and GLM-5, respectively. On MLE-Bench Lite, it reaches 81.82 Any Medal% under both backbones, improving over the strongest matched baselines by 4.55 and 16.67 points, and exceeding a Codex/GPT-5.5 xhigh frontier harness reference by 13.64 Any Medal points. Ablations and process analyses show that durable project state is central to later-round refinement: removing File-as-Bus lowers PaperBench score by 6.41 points and MLE-Bench Lite Any Medal% by 31.82 points. These results suggest that long-horizon AI research is not only a problem of stronger local reasoning, but a systems problem of maintaining cumulative, inspectable project progress.

Abstract:
In social media discourse, the framing of a topic dictates the way opinions are formed. This paper presents two multi-agent frameworks: one for discovering what issues are framed in a topic, and another articulating interpretations of these issues. This enables an analysis of a topic’s discourse that reveals what frame issues are dominant while also providing insights about which issues have more possible interpretations. Experiments across four different topics show promising results for discovering and analyzing framings, allowing new comprehension of controversial topics.

Abstract:
Streaming video understanding requires answering questions at arbitrary times over a continuously growing visual stream. The central challenge is to compactly remember long-range history while effectively retrieving question-relevant evidence. We propose Dynamic Hub-and-Spoke Memory (D-HSM), a training-free framework that represents distant history as structured textual memory while preserving the recent frames as visual tokens for fine-grained perception. Specifically, D-HSM turns selected historical video chunks into typed textual observations and stores them in an entity-centered hub-and-spoke memory, with entities as hubs and related evidence as spokes. When answering a question, D-HSM dynamically retrieves a compact question-aware memory subset, expands it through hub-and-spoke links, and combines it with the recent visual window for frozen-VLM answer prediction. Extensive experiments on both streaming and long video benchmarks show that D-HSM consistently and substantially improves VLM backbones and outperforms other state-of-the-art online and offline video understanding baselines.

Abstract:
Latent test-time scaling improves reasoning by refining hidden states during inference, but existing methods typically apply a single scalar reward to all editable latent tokens. For multimodal large language models, this global update ignores that generated tokens play different roles: some are sensitive to visual evidence, while others correspond to uncertain reasoning decisions. We present Token-Disentangled Latent Test-Time Scaling, an inference-time framework that makes latent refinement token-role-aware. Starting from an initial generated trajectory, we optimize a short hidden-state prefix while routing perception-side visual feedback to image-sensitive tokens and reasoning feedback to high-entropy tokens. Tokens selected by neither route are constrained by an anchor regularizer. Across six multimodal reasoning benchmarks and two frozen MLLM backbones, our method consistently improves over the initial chain-of-thought rollout: macro accuracy increases from 66.38 to 68.95 on Qwen2.5-VL-7B and from 67.93 to 69.44 on InternVL3.5-8B. These results show that token-disentangled latent refinement is an effective strategy for multimodal test-time scaling and remains competitive with output-space test-time scaling baselines.

Abstract:
World models offer a principled way to equip long-horizon LLM agents with foresight: predictions of action consequences before execution. However, unreliable foresight can be ignored, misused, or even degrade downstream decision-making. In this paper, we introduce WorldEvolver, a self-evolving world model framework that revises its deployment-time context while keeping the downstream agent and all model parameters frozen. WorldEvolver integrates three modules: (i) Episodic Memory, which exploits real action transitions through retrieval-based simulation; (ii) Semantic Memory, which extracts persistent heuristic rules from prediction-observation mismatches; and (iii) Selective Foresight, which filters low-confidence predictions before integrating them into agent reasoning context. We evaluate WorldEvolver on ALFWorld and ScienceWorld, measuring world model prediction accuracy on Word2World and downstream agent success rate on AgentBoard. Extensive experiments show that WorldEvolver achieves the highest prediction accuracy across three backbones and leads other world model baselines on downstream agent success rate, demonstrating that test-time memory revision enhances both predictive fidelity and planning performance.

Abstract:
Anthropology and related disciplines which study culture have often found it useful to consider their epistemological and methodological approaches in terms of emic versus etic. This is the distinction between the insider perspective, how the world is viewed by a member of the culture, versus the outsider perspective, the scientific cataloguing of cultural knowledge and practice. We adopt this framework to systematically analyse assumptions in recent research on culture in NLP along the full pipeline of task selection, data collection, system design and evaluation. We draw attention to the ‘culture of NLP’ which shapes the research focus and approaches of our field, and suggest pathways towards a more culturally attuned AI.

Abstract:
Multimodal large language models in long chain-of-thought reasoning often fail when different knowledge sources provide conflicting signals. We formalize these failures under a unified notion of knowledge conflict, distinguishing input-level objective conflict from process-level effective conflict. Through probing internal representations, we reveal that: (I) Linear Separability: different conflict types are explicitly encoded as linearly separable features rather than entangled; (II) Depth Localization: conflict signals concentrate in mid-to-late layers, indicating a distinct processing stage for conflict encoding; (III) Hierarchical Consistency: aggregating noisy token-level signals along trajectories robustly recovers input-level conflict types; and (IV) Directional Asymmetry: reinforcing the model’s implicit source preference under conflict is far easier than enforcing the opposite source. Our findings provide a mechanism-level view of multimodal reasoning under knowledge conflict and enable principled diagnosis and control of long-CoT failures. Code is available at [anonymous link](https://anonymous.4open.science/r/Diagnosing-Knowledge-Conflict-C78D/).

Abstract:
Machine-translated data is widely used in multilingual NLP, particularly where native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modeling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, but grammatical performance is strongly correlated to typological similarity to English if trained on enough data. Even translation quality is a strong predictor of language modeling performance.

Abstract:
Designing academic posters is a labor-intensive process requiring the precise balance of high-density content and sophisticated layout. While existing paper-to-poster generation methods automate initial drafting, they are typically single-pass and non-interactive, often fail to align with complex, subjective user intent. To bridge this gap, we propose APEX (Academic Poster Editing agentic eXpert), the first agentic framework for interactive academic poster editing, supporting fine-grained control with robust multi-level API-based editing and a review-and-adjustment mechanism. In addition, we introduce APEX-Bench, the first systematic benchmark comprising 514 academic poster editing instructions, categorized by a multi-dimensional taxonomy including operation type, difficulty, and abstraction level, constructed via reference-guided and reference-free strategies to ensure realism and diversity. We further establish a multi-dimensional VLM-as-a-judge evaluation protocol to assess instruction fulfillment, modification scope, and visual consistency harmony. Experimental results demonstrate that APEX significantly outperforms baseline methods. Our implementation is available at https://github.com/Breesiu/APEX.

Abstract:
We demonstrate that a small transformer can learn to execute programs in MicroPy, a simplified yet computationally universal programming language. Given procedure definitions together with an expression to evaluate, the transformer predicts small-step execution using PENCIL scaffolding for space-efficient execution within a bounded context window. After training on randomly generated, semantically arbitrary MicroPy programs, the learned transformer generalizes to held-out human-written programs including bit copying and flipping, binary addition and multiplication, and SAT verification and solving. The trained model exhibits out-of-distribution generalization: it correctly evaluates programs drawn from a substantially different distribution than the one used to generate its training data. Since MicroPy can express any computation, our results provide empirical evidence that a standard transformer can be trained to act as a universal computer when paired with PENCIL scaffolding.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has advanced reasoning capabilities in multimodal large language models. However, existing methods typically treat visual inputs as deterministic, overlooking the perceptual ambiguity inherent to the visual modality. Consequently, they fail to distinguish whether a model’s uncertainty stems from complex reasoning or ambiguous perception, preventing the targeted allocation of exploration or learning signals. To address this gap, we introduce DUPL , a dual-uncertainty guided policy learning approach for multimodal RLVR that quantifies and leverages both perceptual uncertainty (via symmetric KL divergence) and output uncertainty (via policy entropy) to guide policy updates. By establishing an uncertainty-driven feedback loop and employing a dynamic branch prioritization mechanism, DUPL recalibrates the policy advantage to focus learning on states with high perceptual or decisional ambiguity, enabling effective targeted exploration beyond passive data augmentation. Evaluated on diverse multimodal reasoning benchmarks spanning mathematical and general domains, DUPL achieves solid gains. It improves Qwen2.5-VL accuracy by up to 12.3% (3B) and 7.9% (7B), and Qwen3-VL-Instruct by up to 10.7% (4B) and 12.4% (8B), consistently outperforming GRPO, while seamlessly generalizing to alternative algorithms (DAPO, +6.5% avg) and architectures (LLaVA-OneVision-1.5, +4.7% avg). These results demonstrate that DUPL is an effective and generalizable approach for multimodal RLVR.

Abstract:
Tackling complex coding tasks often requires autonomous agents and iterative repair pipelines. These increasingly rely on large amounts of test-time computation, often spending many decoding and repair steps before discovering whether a program compiles, runs, or validates. Executable parallel-code translation is an effective setting for earlier guidance because success is behavioral rather than textual. However, most guidance methods act only after complete programs or textual traces are decoded. This motivates the question: can latent reasoning provide an earlier intervention point, before the model commits to code? We study a test-time latent guidance method for this setting that trains a smaller Process Reward Model (PRM) over continuous latent prefixes and uses it to select among alternate hidden-state trajectories before final code decoding, separately from but compatible with post-decoding optimization. On a 76-task ParaTrans benchmark evaluation, latent PRM guidance improves mean validation rate from 32.89% with unguided latent reasoning to 42.1%, outperforming fine-tuned and vanilla baselines in the same setting. These gains persist under the same three-iteration repair loop. These results provide bounded evidence that useful alternative latent continuations exist and that PRM-scored latent branch selection can improve executable outcomes in this setting without retraining the main generative model.

Abstract:
Cross-lingual transfer is a model’s ability to generalize capabilities from well-represented source languages to under-represented target languages. Existing measures to quantify a model’s transfer strength conflate improvements in transfer with general improvements to accuracy in source language. We advocate an alternate metric that reliably captures transfer strength called Hardness Adjusted Transfer (HAT) Score, and use it to derive multiple insights on factors influencing transfer strength. Our analysis with twenty diverse language models and three popular mainstream multilingual benchmarks argues that 1) transfer in small models is not broken, 2) we are making slower than expected progress with model size, and 3) have made clear progress over time.

Abstract:
Multimodal long-chain-of-thought (CoT) generation requires models to dynamically route information between visual evidence and an expanding reasoning history. Existing hallucination mitigation methods often strengthen visual grounding or intervene in decoding, but typically treat grounding as a relatively static property of generation. We propose a training-free dynamic attention intervention framework that exploits the head-level sparsity, layer-level stratification, and trajectory-level dynamics of multimodal long-CoT decoding. Our method selects routing-sensitive heads using perturbation-only calibration, assigns visual-evidence and reasoning-history interventions to different layer regions, and continuously rescales selected head outputs according to their instantaneous token-group attention ratios. Across multiple MLLM architectures and multimodal hallucination benchmarks, our method consistently improves over vanilla decoding and visual contrastive decoding, achieving an average performance gain of 4.6%.

Abstract:
Spatio-temporal video grounding (STVG) aims to localize objects or events described by natural language queries in both space and time. Existing STVG models are typically trained and evaluated under the assumption that each query is relevant to the input video. In this work, we challenge this assumption by studying the behavior of state-of-the-art STVG models under irrelevant queries and missing textual input. Our experiments show that current models can still produce plausible spatio-temporal predictions even when the query is unrelated to the video or removed entirely. We further analyze HCSTVG-v2 and VidSTG to identify dataset regularities that may encourage such query-insensitive behavior. Our study highlights an underexplored limitation of STVG models and motivates negative-aware evaluation protocols and architectures that explicitly assess query relevance.

Abstract:
Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these systems differ in sources, language, and fidelity to cited material. We examine responses to 11,000 real search queries across five systems—vanilla GPT, Search GPT, Perplexity Search with Grok, Google AI Overviews, and traditional Google Search—at three levels: source diversity, linguistic characterization of the generated summary, and source-summary fidelity. We find that generative search systems exhibit significant source-selection biases in their citations, favoring certain sources over others. Incorporating search also selectively attenuates epistemic markers, reducing hedging by up to 60% while preserving confidence language in the AI-generated summaries. At the same time, AI summaries further compound the citation biases: Wikipedia and longer sources are disproportionately overrepresented, whereas cited social media content and negatively framed sources are substantially underrepresented. Our findings highlight the potential for answer bubbles , in which identical queries yield structurally different information realities across systems, with implications for user trust, source visibility, and the transparency of AI-mediated information access.

Abstract:
Alternative text is imperative for making the internet accessible to blind and low-vision individuals. Vision-language models (VLMs) have the potential to improve alt text coverage. This study considers how social stereotypes of race and gender are manifested in the alt-text language generated by GPT-4.1, Claude-Sonnet-4.5, and Gemini-2.5-Pro. A total of 1200 images are generated uniformly by gender and skin tone, and then the VLMs are prompted to generate the alternative text. The analysis of the generated alt text shows that the text descriptions differ significantly depending on the skin tone and gender of the person in an image. The VLMs tend to assume that whiteness is the default identity and the descriptions of those with darker skin are more likely to contain racialized terms. This bias suggests that AI-generated alt-text should be used with caution in the interest of preserving equity and preventing discrimination.

Abstract:
As Large Language Model (LLM) agents are increasingly deployed in open-ended domains like software engineering, they frequently encounter underspecified instructions that lack crucial context. While human developers naturally resolve underspecification by asking clarifying questions, current agents are largely optimized for autonomous execution. In this work, we systematically evaluate the clarification-seeking abilities of LLM agents on an underspecified variant of SWE-bench Verified. We propose an uncertainty-aware multi-agent scaffold that decouples underspecification detection from code execution. Across both proprietary and open-weight frontier LLMs, our scaffold achieves a 69.40% task resolve rate, significantly outperforming a standard single-agent setup and closing the performance gap with agents operating on fully specified instructions. Furthermore, we find that the multi-agent system exhibits well-calibrated information-seeking behavior, conserving queries on simple tasks while proactively seeking information on more complex issues. These findings indicate that current models can be turned into proactive collaborators, where agents independently recognize when to ask questions to elicit missing information in real-world, underspecified tasks.

Abstract:
Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on simple queries. They largely overlook the compositional queries prevalent in real-world scenarios, where a target must be disambiguated by jointly reasoning about its attributes and relations to other entities. To bridge this gap, we propose Compositional Spatio-Temporal Video Grounding (CompSTVG), a task that requires models to process complex textual queries where every intertwined attribute and relational cue is essential for disambiguation. To facilitate this task at scale, we build a synthetic data engine that leverages a spatio-temporal scene graph as a difficulty measure and casts difficulty-controlled query synthesis as a constraint programming problem, producing difficulty-graded data for both evaluation and training. Built on this engine, we introduce STVG-CompBench, a benchmark stratified by explicit difficulty levels that jointly capture temporal complexity and spatial interference. Evaluating 11 representative STVG models on STVG-CompBench reveals that current models perform poorly on compositional queries, exhibiting a sharp performance drop that is typically obscured by overall dataset-level averages. We further construct synthetic training data and propose CurrSTVG, a curriculum reinforcement learning framework that delivers consistent gains, with the largest improvements observed on the most challenging compositional queries.

Abstract:
Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.

Abstract:
Large language models (LLMs) are increasingly used for automatic evaluation of generated text, but most work focuses on English. Extending LLM-based evaluation to multilingual settings is challenging, especially for low-resource languages or when in-domain data is scarce. This work explores how an English evaluation corpus can serve to evaluate other languages, both when in-domain data is available for fine-tuning and when it is not. We systematically analyze English, Spanish, and Basque, representing high-, mid-, and low-resource languages, considering instruction translation, monolingual versus multilingual supervision, and model size. Our results reveal key trade-offs: When in-domain data is available, fine-tuned smaller models can achieve performance comparable to proprietary models, whereas zero-shot evaluation with larger models proves more effective in out-of-domain settings. We also observe that fine-tuning on out-of-domain data can adversely affect model performance. These findings provide practical guidance for building efficient, reliable multilingual evaluation pipelines. The data and code are publicly available at hitz-zentroa/mJudge .

Abstract:
Large Language Models (LLMs) have displayed remarkable capabilities across diverse domains, but their training remains resource – and time-intensive, requiring massive computational resources and careful orchestration of training procedures. Model souping – the practice of averaging weights from multiple models of the same architecture – has emerged as a promising pre- and post-training technique that can enhance performance without expensive retraining. We observe that previous souping approaches can lead to collapse in precision-sensitive LLM capabilities. In this paper, we introduce Soup of Category Experts (SoCE), a principled approach for model souping to overcome this shortcoming. The proposed method utilizes benchmark composition to identify optimal model candidates and applies non-uniform weighted averaging to maximize performance. Contrary to previous approaches, our method leverages the observation that different clusters (or categories) of points within a benchmark often exhibit low inter-correlations in model performance. SoCE identifies "expert" models for each weakly-correlated category cluster and combines them using optimized weighted averaging rather than uniform weights. We demonstrate that SoCE improves performance and robustness across multiple domains achieves state-of-the-art results on the Berkeley Function Calling Leaderboard.

Abstract:
Recent Large Reasoning Models (LRMs) achieve strong performance by leveraging long-form Chain-of-Thought (CoT) reasoning, but uniformly applying overlong reasoning at inference time incurs substantial and often unnecessary computational cost. To address this, prior work explores various strategies to infer an appropriate reasoning budget from the input. However, such approaches are unreliable in the worst case, as estimating the minimal required reasoning effort is fundamentally difficult, and they implicitly fix the trade-off between reasoning cost and accuracy during training, limiting flexibility under varying deployment scenarios. Motivated by these limitations, we propose ORBIT, a controllable multi-budget reasoning framework with well-separated reasoning modes triggered by input. ORBIT employs multi-stage reinforcement learning to discover Pareto-optimal reasoning behaviors at each effort, followed by on-policy distillation to fuse these behaviors into a single unified model. Experiments show that ORBIT achieves (1) controllable reasoning behavior over multiple modes, (2) competitive reasoning density within each mode, and (3) integration of these frontier policies into a single unified student model while preserving clear mode separation and high per-mode performance.

Abstract:
Large Language Models (LLMs) excel at text generation, but still struggle to produce accurate, long-form documents grounded in large knowledge bases. Existing “top-down” approaches—which plan before retrieving evidence—often result in fragmented or unfaithful content due to misalignment between the plan and available knowledge. To overcome this, we introduce a “bottom-up” framework based on a “Retrieval-First for Knowledge, Clustering for Structure” strategy. Our method begins by establishing the “knowledge boundaries” of the source corpus through comprehensive retrieval, followed by unsupervised clustering to group evidence into distinct “knowledge clusters.” These clusters objectively inform a hierarchical outline and subsequent text generation, ensuring strict adherence to the source and significantly reducing hallucination. Experimental results on both 14B- and 32B-parameter models demonstrate that our method achieves performance comparable to or exceeding state-of-the-art baselines, and is expected to demonstrate unique advantages in knowledge-constrained scenarios that demand high fidelity and structural coherence. Our work presents an effective paradigm for generating reliable, structured, long-form documents, paving the way for more robust LLM applications in high-stakes, knowledge-intensive domains.

Abstract:
Large language model (LLM) safety has received extensive attention since LLMs are vulnerable to manipulation. To understand and mitigate the risk, this paper studies the problem of jailbreak attacks on LLMs, which aim to deliberately break the safety guard of LLMs for harmful or unethical responses. Current black-box jailbreak attacks are limited by either a reliance on human expertise for manual prompt design or by intricate workflows in automated approaches. Towards this end, we propose a novel approach named Reward-guided Meta-prompt Evolving with reflection (ROOT) for automatic jailbreak attack generation. The core idea of our ROOT is to optimize a meta-prompt using attack rewards as jailbreak guidance. In particular, our ROOT feeds a meta-prompt with toxic questions into LLMs to generate prompts for jailbreak attempts. The responses from these attempts are further evaluated using a judge model, which further summarizes the reflection of both successful and unsuccessful jailbreak attempts into different meta-prompt optimization strategies. To reduce the noise, we estimate their reward scores to select high-quality strategies, which can optimize our meta-prompt for better attack generation ability. Extensive experiments show that ROOT demonstrates strong generalizability and broad adaptability by achieving high jailbreak success rates (above 90%) across both multiple LLMs and various categories of harmful tasks.

Abstract:
Personalized alignment from preference data has focused primarily on improving personal reward model (RM) accuracy, with the implicit assumption that better preference ranking translates to better personalized behavior. However, in deployment, computational constraints necessitate inference-time adaptation such as reward-guided decoding (RGD) rather than per-user policy fine-tuning. This creates a critical but overlooked requirement: reward models must not only rank preferences accurately but also effectively guide generation. We demonstrate that standard RM accuracy fails catastrophically as a selection criterion for deployment-ready personalized rewards. We introduce policy accuracy—a metric quantifying whether RGD-adapted LLMs correctly discriminate between preferred and dispreferred responses—and show that upstream RM accuracy correlates only weakly with downstream policy accuracy (Kendall’s tau = 0.08–0.31). More critically, we introduce Pref-LaMP the first personalized alignment benchmark with ground-truth user completions, enabling direct behavioural evaluation. On Pref-LaMP, we expose a complete decoupling between discriminative ranking and generation metrics: methods with 20-point RM accuracy differences produce almost identical output quality, and methods with high ranking accuracy can fail to generate behaviorally aligned responses. These findings reveal that the field has been optimizing for proxy metrics that do not predict deployment performance, and that current personalized alignment methods fail to operationalize preferences into behavioral adaptation under realistic deployment constraints. In contrast, we find simple in-context learning (ICL) to be highly effective - dominating all reward-guided methods for models ≥3B parameters, achieving ~3 point ROUGE-1 gains over the best reward method at 7B scale.

Abstract:
Robustness is often regarded as a critical future challenge for real-world applications, where stability is essential. However, as models often learn tasks in a similar order, we hypothesize that easier tasks will be easier regardless of how they are presented to the model. Indeed, in this paper, we show that as models approach high performance on a task, robustness to semantics-preserving changes is effectively achieved. Through an empirical analysis of multiple models across diverse datasets and configurations (e.g., paraphrases, different temperatures), we find a strong positive correlation. Moreover, we find that robustness is primarily driven by task-specific competence rather than inherent model-level properties, challenging current approaches that treat robustness as an independent capability. Thus, from a high-level perspective, we may expect that as new tasks saturate, model robustness on these tasks will emerge accordingly. For researchers, this implies that explicit efforts to measure and improve robustness may warrant reduced emphasis, as such robustness is likely to develop alongside performance gains. For practitioners, it acts as a sign that indeed the tasks that the literature deals with are unreliable, but on easier past tasks, the models are reliable and ready for real-world deployment.

Abstract:
RNA’s conformational flexibility enables diverse biological functions but makes structure-guided design difficult. Although precise control of 3D structure is essential, existing methods often overlook the molecular surface, which critically shapes intermolecular interactions and functional outcomes. To address this, we propose gRFold, a multimodal framework that incorporates molecular surface information as a core design constraint. gRFold employs two geometric encoders: a Frame-Averaged Geometric Surface Encoder (FA-GSE) to learn E(3)-invariant surface features, and an Equivariant Backbone Graph Perceptron (EBGP) to capture backbone topology. The structural representations are integrated via hierarchical cross-attention to guide autoregressive generation; at inference, candidates are re-ranked by agreement between their predicted folds and the target base-pairing pattern. Across three RNA inverse-folding benchmarks covering RNA-protein complexes, monomeric RNAs and independently curated RNA-Puzzles targets, gRFold achieves strong sequence recovery and structural fidelity under both secondary-structure and 3D refolding evaluations, and ablations show that the surface and backbone branches contribute complementary gains. These results highlight the potential of surface-aware geometric modeling for structure-guided RNA design.

Abstract:
Retrieval-augmented language models improve knowledge-intensive question answering, but they remain brittle on multi-step questions, where an early retrieval error can steer subsequent reasoning toward the wrong entity or relation. Existing methods can often detect that the current answer or evidence is unreliable, yet they rarely turn this diagnosis into an explicit retrieval correction. We propose Verification-Guided Retrieval Revision (VRR), a framework that uses verification as a control signal for multi-step retrieval. VRR couples a Generator that proposes search and answers, a Judge that verifies answer correctness and evidential support while emitting structured failure diagnoses, and a Retrieval Context Manager (RCM) that converts these diagnoses into retriever-facing updates such as clue selection, query rewriting, and state-aware evidence ranking. To make this loop learnable, we train VRR in three stages: iterative cold-start training to stabilize search, judge-shaped policy optimization to improve long-horizon reasoning, and diagnosis-conditioned RCM training for retrieval adaptation. Across seven open-domain QA benchmarks with a fixed retriever and corpus, training on HotpotQA and transferring to out-of-domain datasets, VRR surpasses the strongest prior baseline by 4.2 EM on average under budget-aligned evaluation, with larger gains on multi-hop benchmarks (+5.4).

Abstract:
Self-generated counterfactual explanations (SCEs) are minimally modified inputs minimality generated by large language models (LLMs) that flip their own predictions validity, offering a causally grounded approach to unraveling black-box LLM behavior. Yet extending them beyond English remains challenging: existing methods struggle to produce valid SCEs in non-dominant languages, and a persistent trade-off between validity and minimality undermines explanation quality. We introduce Macro, a preference alignment framework that applies Direct Preference Optimization (DPO) to multilingual SCE generation, using a composite scoring function to construct preference pairs that effectively translate the trade-off into measurable preference signals. Experiments across four LLMs and seven typologically diverse languages show that Macro improves validity by 12.55% on average over the chain-of-thought baseline without degrading minimality, while avoiding the severe minimality violations of the translation-based baseline. Compared to supervised fine-tuning, Macro achieves superior performance on both metrics, confirming that explicit preference optimization is essential for balancing this trade-off. Further analyses reveal that Macro increases cross-lingual perturbation alignment and mitigates common generation errors. Our results highlight preference optimization as a promising direction for enhancing multilingual model explanations.

Abstract:
Previous works identify sensitivity to option order as a key issue in MC-VQA evaluation and propose protocols to mitigate this effect. We show that such mitigation is insufficient to ensure the validity of MC-VQA as a reliable benchmark for MLLMs: performance remains highly sensitive to semantically neutral prompt format choices that are not controlled by current benchmarks. In a large-scale study spanning seven MLLMs and five MC-VQA datasets, we find frequent rank reversals even under order-invariant evaluation. These reversals arise when we systematically vary option ID sets, delimiters, and separators, yielding 48 semantically equivalent prompt formats. Mechanistic analyses trace this instability to low-level language modeling effects: tokenizer-induced fusion or removal of option ID tokens introduces corrupted option ID tokens into the input sequence, while the choice of option ID sets directly affects the reliability of attention patterns for option selection. Accordingly, MC-VQA rankings correlate weakly with open-ended evaluation, indicating that MC-VQA reflects option selection dynamics in addition to multimodal reasoning. These findings identify prompt formatting as a major, previously under-controlled confounder in MC-VQA benchmarking and motivate evaluation protocols that explicitly control prompt format sensitivity.

Abstract:
Parameter-efficient fine-tuning (PEFT) reduces trainable parameters, yet head-sparse PEFT still requires deciding which attention heads should receive updates. We introduce SIGHT, Sparse Identification of Gaze-aligned Heads for Tuning, a gaze-guided head selector for head-sparse PEFT. Using eye-tracking data during reading comprehension tests, SIGHT ranks heads by correlating word-level gaze metrics with the attention heads of large language models. Specifically, the comprehension question appears either before or after the passage so that the readers either read the passage with a clear goal or for general understanding. The eye-tracking data from these two conditions are both correlated with attention heads and are fused by an offline-calibrated layer-wise gating module. When applied to Qwen3-based models, updating only the top 10% SIGHT-selected heads generally outperforms other head selectors and all-head PEFT references on reasoning and syntax benchmarks. These results suggest that gaze information during human reading can move beyond post hoc interpretation and serve as an external criterion for allocating adaptation capacity.

Abstract:
Vision-language models (VLMs) achieve strong multimodal performance but remain expensive to deploy. Knowledge distillation (KD) is an effective approach for compressing these models and transferring their capabilities to more efficient student models. While KD for large language models (LLMs) is already well surveyed, no existing work provides systematic coverage of KD for VLMs, where the addition of a visual modality introduces challenges that text-only distillation does not face. This paper presents the first dedicated survey of knowledge distillation for vision-language models. We propose a unified taxonomy that organizes methods along three orthogonal views: knowledge type, teacher accessibility, and structural compatibility, and use it to organize methods across major VLM distillation settings. We also survey the associated datasets and evaluation protocols, identify open challenges and outline directions for future research. We further maintain a GitHub repository at https://github.com/HoangTran223/Awesome-Knowledge-Distillation-of-VLMs to compile all surveyed papers, which we will actively update.

Abstract:
Constructive feedback is crucial for creative writers to refine their storytelling abilities. Since receiving feedback from human experts is often costly and time-intensive, large language models (LLMs) offer a scalable and efficient alternative as automatic writing assistants. Despite their potential, research indicates that LLM-generated feedback is often generic, lacks actionability, and fails to identify which writing issue is most critical. To address these limitations, we present a reinforcement learning approach that steers LLMs to generate constructive feedback without the need for ground-truth feedback. We train our model using group relative policy optimization (GRPO) with a novel multi-component reward function aiming at constructiveness: it prioritizes feedback that is uniquely tailored to the story, helps to improve story quality, and addresses the most critical writing issue. In automatic and human evaluation across three story corpora, our approach outperforms state-of-the-art LLMs (including Gemini) and competitive baselines. We find that providing actionable suggestions is the main driver of feedback constructiveness.

Abstract:
Code completion models often fail when the developer’s intent is under-specified in the code context. To mitigate this, developers frequently use natural language comments to clarify objectives. However, current code completion models fail to prioritize these directives effectively since they are merely pre-trained using the Fill-In-the-Middle (FIM) objective. On the one hand, the natural language instructions, mixed with the noisy code comments, are just treated as part of the background context within the prefix. On the other hand, the pre-training datasets for the FIM objective are mostly sourced from open-source repositories, which results in a scarcity of high-intent instruction-to-code pairings that reflect the developers’ workflow in code completion. To bridge this gap, we propose Instruction-aware Fill-In-the-Middle (IFIM), a fine-tuning method that extends the FIM structure with a dedicated, structurally separated instruction section. Our evaluation shows that IFIM substantially improves adherence to developer intent, while leaving infilling performance unchanged when no instruction is given. The gains hold on an in-the-wild benchmark of 100 instructions written by real developers and across model scales from 1.5B to 7B. IFIM thus offers a backward-compatible upgrade path for existing FIM-based code completion systems at a modest training cost.

Abstract:
Recent parameter-efficient fine-tuning (PEFT) methods effectively adapt Vision-Language Models (VLMs) to seen tasks, yet often exhibit limited generalization to unseen novel classes. To better understand this phenomenon, we analyze adaptation dynamics from the perspective of cross-modal mutual information (MI). Our empirical analyses reveal a consistent pattern across existing methods: during the adaptation process, cross-modal MI for novel classes frequently encounters early plateaus or declines, indicating weakened cross-modal semantic dependency. Motivated by this observation, we propose Retrieval-Augmented Dynamic Adaptation (RADA), a retrieval-enhanced adaptation framework that continuously injects rich instance-specific semantic priors through an LLM-enhanced dual-modality memory bank and a lightweight modality-agnostic token generator. By facilitating fine-grained cross-modal alignment while preserving pretrained zero-shot ability, RADA achieves more transferable adaptation across diverse scenarios. Extensive experiments across multiple widely used benchmarks demonstrate that RADA consistently achieves state-of-the-art performance while maintaining strong zero-shot capability. In addition, our empirical analyses show that RADA preserves more stable cross-modal dependency trajectories throughout adaptation, offering further insights into the relationship between adaptation dynamics and generalization in VLMs.

Abstract:
In this study, we address the language-guided time-series event detection problem: given a natural-language event description and a time series, the system returns event occurrences as temporal spans. Language-guided event detection leverages domain knowledge expressed in natural language to enable zero-shot detection without domain-specific fine-tuning. We introduce GroundingMOMENT, a zero-shot cross-modal grounding framework that aligns a time-series foundation model with a language model. GroundingMOMENT concurrently localizes both complex, high-level event occurrences and their constituent domain-agnostic structural primitives (e.g., spikes, plateaus). We further introduce GroundTSSyn, a scalable pipeline for generating synthetic time-series grounding data, and release GroundTSBench, the first open-source benchmark for language-guided time-series grounding. Trained exclusively on GroundTSBench, GroundingMOMENT outperforms time-series multimodal large language models and achieves fast and accurate event detection. Case studies further demonstrate its zero-shot generalizability to real-world data from diverse domains.

Abstract:
Science as a body of knowledge is important for human learning, yet beyond factual accuracy, little research has focused on the capacity of Large Language Models (LLMs) to communicate science effectively to the public. Here, we develop and validate a computational framework for measuring science communication skills of LLMs. Using this framework, we evaluate different alignment approaches: fine-tuning using top-ranked human explanations, distilling synthetic explanations from a teacher model, and preference optimization. Our results show that annotators overwhelmingly prefer LLM-generated science explanations over top-ranked organic answers, with this preference primarily attributed to better structure and readability. We find that distillation with synthetic answers achieves the most significant and stable improvement, allowing a smaller open-source model to achieve statistical parity with the proprietary teacher model in blind side-by-side comparisons.

Abstract:
Large language model unlearning aims to suppress designated knowledge without degrading retained capabilities, yet existing approaches often fall into destructive over-suppression or conservative under-suppression. We introduce protected update routing, an intervention-guided framework that controls where forgetting updates flow within LoRA adapters. The method activates measurable adapter paths, identifies forget-specific rows from gradient attribution, intervention-induced loss changes, and forget–retain specificity, and applies localized suppression under retain-aware masking and stability safeguards. Multi-seed experiments on TOFU and WMDP show that protected routing provides a controllable forgetting–utility trade-off: it strengthens target suppression over NPO and RMU on TOFU while retaining higher downstream utility on WMDP. In a protocol-matched comparison with PALU, it achieves higher model utility at a similar forget-quality level. Analyses across MUSE-News, alternative routing granularities, and larger models further support the effectiveness and interpretability of separating route selection from update strength. Protected update routing therefore offers a practical and diagnosable interface for parameter-efficient unlearning.

Abstract:
Writing proficiency manifests in how students develop content, organize ideas, choose words, and use language. Despite growing interest in LLM-based student simulation, whether LLMs can reproduce such multidimensional variation in extended writing remains largely unexplored. In this work, we explore if language models can realistically simulate student writing, and introduce SWIM, a task that formulates Student Writing sIMulation as proficiency-conditioned essay generation. We evaluate prompting, supervised fine-tuning (SFT), and reinforcement learning (RL) methods for writing simulation using automated essay scoring as a measure of profile alignment. Extensive experiments reveal that prompting provides limited proficiency control, even for strong proprietary LLMs with rubric-grounded strategies. In particular, while models can adjust content-oriented traits, they struggle to reproduce the lexical, grammatical, and organizational variation in different proficiency levels. SFT substantially improves alignment, while RL with the proposed proficiency-alignment reward yields further gains across all writing traits and essay prompts. Our findings suggest that explicit supervision enables substantially stronger profile alignment than prompting alone, while authentic low-proficiency writing remains challenging to reproduce.

Abstract:
Multimodal review generation is essential in academic research. However, existing LLM-based models often struggle to pinpoint specific details within papers, yielding generalized, unsupported critiques that compromise final decisions. To address these limitations, we propose AnchorReviewer, a coarse-to-fine, evidence-grounded multimodal review generation framework. Central to our architecture is an integrated pipeline featuring an Inline Retrieval-Augmented Generation (InRAG) module. Combined with our Unified Embedding Module, InRAG facilitates the high-precision extraction of multimodal evidence chunks, ensuring our detailed reviews are strictly anchored to the original content. Furthermore, we incorporate a Reflection Mechanism to refine initial outputs and maximize decision accuracy. To support model evaluation and our Progressive Training Strategy, we construct the large-scale AnchorReview dataset, comprising diverse multimodal review samples. Experimental results demonstrate that AnchorReviewer significantly improves review reliability, excelling in critical evaluation, multimodal understanding, and decision consistency. Our code and dataset will be publicly available.

Abstract:
Retrieval-augmented search agents answer multi-hop questions by repeatedly issuing search queries and accumulating evidence. This creates a stopping problem: after the necessary evidence has appeared, further retrieval often adds cost, latency, and distracting context rather than useful information. We frame stopping as evidence coverage rather than generator confidence, and introduce HALT, a lightweight verification-aware policy that leaves the search agent unchanged. Given expected hop claims, HALT stops only when cumulative evidence supports each required claim. Across three multi-hop QA benchmarks, HALT reduces redundant search while largely preserving exact match, with stronger savings under gold claims and more conservative behavior under generated claims. Baseline comparisons and ablations show that this behavior is driven by claim–evidence alignment rather than generic sufficiency, fixed stop positions, or lexical overlap. Open-corpus pilots further suggest that HALT abstains when coverage cannot be reliably verified. Overall, evidence coverage provides a practical runtime control signal for improving retrieval-augmented agents without retraining or modifying the host agent.

Abstract:
While multimodal retrieval-augmented generation (RAG) systems increasingly rely on images as external knowledge sources, the introduction of poisoned visual evidence can severely compromise multimodal large language model (MLLM) generation. Unlike prior attacks that rely on altering textual metadata, we introduce Vis-Poison, a novel visual knowledge poisoning attack where the poisoned image itself is the attacker-controlled payload, without manipulating captions, summaries, metadata, or other associated text. Specifically, this attack is instantiated through an automated multi-agent method that constructs visually plausible poisoned images. To assess its impact, we evaluate Vis-Poison across two representative multimodal RAG pipelines, four embedding models, and six generation models. Empirically, Vis-Poison achieves an end-to-end attack success rate of 40.16% to 65.40% against 30k-entry multimodal knowledge bases in black-box settings. Moreover, Vis-Poison remains effective against various MLLMs that can answer correctly from parametric knowledge alone, with an average success rate above 60%. Code and data are available at https://github.com/SWUFE-DB-Group/Vis-Poison.

Abstract:
Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitations of rejection sampling. Standard methods treat the teacher as a static filter, discarding complex "corner-case" problems where the teacher fails to explore valid solutions independently, thereby creating an artificial "Teacher Ceiling" for the student. In this work, we propose Hindsight Entropy-Assisted Learning (HEAL), an RL-free framework designed to bridge this reasoning gap. Drawing on the educational theory of the Zone of Proximal Development (ZPD), HEAL synergizes three core modules: (1) Guided Entropy-Assisted Repair (GEAR), an active intervention mechanism that detects critical reasoning breakpoints via entropy dynamics and injects targeted hindsight hints to repair broken trajectories; (2) Perplexity-Uncertainty Ratio Estimator (PURE), a ratio-based filtering heuristic that reduces high-anomaly shortcut-like rationales; and (3) Progressive Answer-guided Curriculum Evolution (PACE), a three-stage distillation strategy that organizes training from foundational alignment to hard-case adaptation. Extensive experiments on multiple benchmarks demonstrate that HEAL significantly outperforms traditional SFT distillation and other baselines.

Abstract:
Authorship attribution aims to determine whether a query text and a candidate document are written by the same author. %While dense embeddings are effective, recent work suggests that low-dimensional representations limit encoding capacity. We investigate high-dimensional learned sparse representations (LSR) and multi-vector retrieval for authorship attribution. We further propose Sparse ColBERT (SparCol), a late-interaction sparse lexical reranking model. To address the memory cost of vocabulary-size embeddings, we develop fused Triton kernels that can reduce peak memory usage by 5 11 times over the baseline torch implementation. Extensive experiments show that LSR can outperform dense embeddings in both single-genre and cross-genre settings, whereas improves reranking performance over single-vector models only in single-genre settings due to lexical matching bias. In addition to token-based embeddings, we also explore Lexicon-based embeddings that score clusters of tokens, which is competitive with the vocabulary-based LSR with reduced dimensions. We conduct interpretability analysis of LSRs, showing that they allow for a more transparent scoring process.

Abstract:
Large language models decode by projecting hidden states through a large vocabulary head at every step. This operation is computationally costly and forces all reasoning to be expressed in discrete tokens. We introduce Soft Latent Thinking, a method that replaces the LM head during reasoning with a lightweight projector, enabling autoregressive rollout in embedding space where reasoning steps remain continuous rather than tokenized. Experiments on DeepSeek-Qwen-1.5B and LLaMA-3.2-3B show that Soft Latent Thinking consistently improves pass@k across all k while reducing per-step compute during chain-of-thought. Our method achieves the highest pass@32 among all soft-thinking approaches, demonstrating that effective reasoning can be carried out in continuous space without discrete token generation.

Abstract:
Large Language Models (LLMs) have enabled embodied agents to perform high-level planning, yet grounding language-based plans into the physical world remains challenging. While feedback-based closed-loop planning is the dominant paradigm, existing methods typically rely on a single scalar feedback signal, which conflates physically infeasible actions that violate hard constraints with strategically sub-optimal but valid choices. We identify this structural limitation as Signal Confounding, which leads to ambiguous feedback and unreliable plan correction. To address this issue, we propose Decoupled Dual-Critic Guidance (DDCG), a framework that explicitly disentangles feedback into two complementary signals: a Feasibility Critic ( C F ) that enforces physical and logical executability constraints, and a Quality Critic ( C Q ) that evaluates the strategic value of feasible actions. This decoupled guidance enables precise error attribution during planning without requiring parameter updates to the LLM planner. We provide a theoretical interpretation of DDCG as critic-regularized planning with hard feasibility constraints. Experiments on two embodied planning benchmarks, VirtualHome and ScienceWorld, show that DDCG consistently improves plan executability and task success, particularly in long-horizon settings, highlighting the importance of structured feedback for reliable LLM-based planning.

Abstract:
Understanding contextual causality is critical for large language models (LLMs), as it enables them to accurately identify causal relations in specific situations and support more reliable decision-making. Despite its significance, a systematic exploration of contextual causality with LLMs is still lacking. To fill this gap, we present a comprehensive survey on this topic. In this survey, we first propose a taxonomy of contextual causality, consisting of semantic, intervention, and counterfactual causality, and characterize each category by its core causal question, required model capabilities, representative tasks, and practical uses in causality analysis. We then analyze existing studies and discuss their key limitations. Finally, we examine the gaps between current benchmarks and real-world needs and outline promising directions for future research. Our goal is to clarify the research landscape of contextual causality with LLMs, emphasize its importance, and highlight promising future directions.

Abstract:
Large language model (LLM)-based agents often make suboptimal tool-use decisions, including unsupported tool invocation and hallucinated direct responses, which may accumulate errors throughout multi-step interactions. Existing approaches mainly improve these behaviors through inference-time correction or coarse-grained reward signals based on decision outcomes and structured checklists, leaving the uncertainty characteristics of agent decisions underexplored. We observe that decision-oriented reinforcement learning tends to weaken the uncertainty separation between correct and incorrect actions, resulting in overconfident mistakes and weaker exploration signals. Therefore, we propose TRUST, which incorporates uncertainty quantification into reward design as a repulsive force for maintaining uncertainty separation, and labels lightweight key-turn annotations for unified post-training of multi-turn trajectories. Experimental results across diverse tool-use benchmarks show that TRUST consistently enhances both decision quality and agent performance while maintaining more reliable uncertainty estimates during optimization.

Abstract:
The proliferation of powerful large language models (LLMs) has necessitated robust safety alignment, yet these models remain vulnerable to evolving adversarial attacks, including multi-turn jailbreaks that iteratively search for successful queries. Current defenses, which are primarily reactive and static, often fail to handle these iterative attacks. In this paper, we introduce ProAct, a novel proactive defense framework designed to disrupt and mislead these iterative search jailbreak methods. Our core idea is to intentionally mislead these jailbreak methods into thinking that the model has been jailbroken with "spurious responses". These misleading responses provide false signals to the attacker’s internal optimization loop, causing the adversarial search to terminate prematurely and effectively jailbreaking the jailbreak. By conducting extensive experiments across state-of-the-art LLMs, jailbreaking frameworks, and safety benchmarks, we demonstrate that our method consistently and significantly reduces attack success rates by up to 94% without affecting utility. When combined with other defense frameworks, it further reduces the latest attack strategies’ success rate to 0%. ProAct represents an orthogonal defense strategy that serves as an additional guardrail to enhance LLM safety against the most effective jailbreaking attacks.

Abstract:
Reasoning representations are increasingly used as explanations for large language model outputs. Yet they are typically evaluated with model-centric criteria, such as answer accuracy and faithfulness, leaving it unclear whether they help people evaluate model responses. In this work, we study reasoning representations as human-facing interfaces rather than proxies for model reasoning ability. We conduct a controlled human study of six reasoning formats across tasks of varying complexity, supported by a web-based framework that randomizes task domains, problem instances, and representation order. The study collects fine-grained judgments of structural understanding, error detection and localization, and trust calibration. Our study shows a mismatch between perceived preference and support for human evaluation. Participants prefer planning- and decomposition-based representations, but simpler chain-of-thought traces better support verification, trust, and interpretability. Preferred representations also introduce calibration risks, with more false alarms on correct traces and high trust despite low willingness to verify.

Abstract:
Computer-use agents (CUAs) interact with graphical interfaces through screenshots and low-level mouse and keyboard actions, yet the causal error may precede the terminal failure. We present CUADebug, a framework for localizing root causes in CUA trajectories and guiding re-execution. CUADebug includes a five-category, 30-subtype taxonomy; CUAErrorBench, a benchmark of 204 failed OSWorld trajectories with human root-cause annotations; and CUADebugger, a ReAct-style agent for root-cause analysis (RCA). CUADebugger iteratively selects trajectory steps, inspects paired before/after screenshots and action traces, and submits a structured diagnosis containing the causal step, taxonomy label, grounded evidence, and correction. CUADebugger performs RCA without per-trajectory human intervention; human annotations are used to evaluate RCA predictions and, in controlled re-rollout comparisons, to fix restart points. Task reasoning and control is the largest annotated failure category (110/204). CUADebugger improves L2 and Tag+Step Exact across three debugger backbones on the Claude-agent split; with Gemini 2.5 Pro, Tag+Step Exact rises from 11.1% to 19.4%. Single re-execution improves failure recovery from 13.89% to 29.86% (overall 61.77% to 68.14%); controlled continual re-execution improves it from 12.50% to 25.69% (overall 61.22% to 66.48%). Project page: cuadebug.github.io

Abstract:
We present Poly-FEVER, an open-source large-scale multilingual benchmark for hallucination detection through controlled factuality judgment in large language models. Poly-FEVER extends FEVER, Climate-FEVER, and SciFact to 77,973 Supported/Refuted claims aligned across 11 languages. We augment each claim with topic metadata from a 22-topic LDA model, enabling topic-aware analysis of multilingual factuality behavior. Translations are produced with Google Cloud Translation and validated by a targeted human audit, GEMBA, and COMETKiwi. We benchmark ChatGPT-3.5, LLaMA-2/3, Phi-3.5, and Mistral across three prompt families and evaluate robustness to model-generated rephrasing. We further analyze resource imbalance using web presence as a proxy for online claim availability, and evaluate retrieval-augmented factuality judgment with DPR over Wikipedia. Results show pronounced cross-lingual disparities under aligned claims, with stronger accuracy in high-resource languages such as English and Chinese and weaker accuracy in lower-resource languages such as Amharic and Tamil. Topic-aware prompting and retrieval provide selective gains, especially in Arabic and Amharic, while their effects vary across languages and can conflict with model priors. Poly-FEVER establishes a rigorous resource for responsible, language-adaptive hallucination detection and language-sensitive factuality evaluation. The proposed Poly-FEVER is available at: https://huggingface.co/datasets/HanzhiZhang/Poly-FEVER.

Abstract:
As retrieval-augmented generation (RAG) systems scale, ensuring faithful grounding in external evidence remains challenging. Large language models may still rely on parametric knowledge when it conflicts with retrieved information. We propose Grounded Decoding, a training-free decoding framework that improves factual consistency in RAG without modifying model parameters. At each generation step, our method constructs two matched-prompt distributions: a full RAG distribution conditioned on the query, retrieved documents, and generated prefix, and a retrieval-only distribution conditioned solely on the retrieved evidence and the same prefix. We derive the final next-token distribution as the unique solution to a KL-barycenter objective over the probability simplex, resulting in a normalized geometric fusion of the two distributions. The formulation recovers standard RAG when the grounding weight is zero and progressively shifts probability mass toward retrieved evidence as grounding strength increases. We further introduce a conflict-aware adaptive weighting scheme that adjusts grounding strength based on distributional disagreement and retriever confidence. Experiments on ALCE, Natural Questions, and FActScore show consistent improvements in factual accuracy and citation quality over standard RAG and competitive decoding-time baselines while maintaining fluency. These results suggest that probability-level fusion offers an effective and efficient alternative to logit-level intervention for faithful RAG decoding.

Abstract:
Reinforcement learning with verifiable rewards (e.g., GRPO) significantly advances LLM reasoning. However, in complex tasks, GRPO frequently encounters the "zero-advantage problem": when all rollouts for a query fail, the relative advantage collapses to zero, wasting computational budget. While resampling with more budget is a common remedy, the static sampling policy inherently constrains reasoning exploration, limiting the success rate. In this paper, we propose Lorem Perturbation for Exploration (LoPE), a simple training framework to break this exploration bottleneck. Motivated by the intuition that task-irrelevant prompt perturbations can shift the output distribution, LoPE prepends sequences stochastically assembled from the Lorem Ipsum vocabulary to the prompts before resampling, forcing the model to explore broader reasoning pathways. Experiments across 1.7B, 4B, and 7B models show LoPE significantly outperforms resampling with original prompts. Our findings, including the effectiveness of other low-perplexity Latin-based random sequences, establish LoPE as a strong baseline for broadening LLM exploration.

Abstract:
State space models (SSMs) like Mamba offer efficient alternatives to Transformer-based language models, with linear time complexity. Yet, their adversarial robustness remains critically unexplored. This paper studies the phenomenon whereby specific short input phrases induce a partial amnesia effect in such models, by irreversibly overwriting information in their hidden states, referred to as a Hidden State Poisoning Attack (HiSPA). Our benchmark RoBench-25 allows evaluating a model’s information retrieval capabilities when subject to HiSPAs, and confirms the vulnerability of SSMs against such attacks. Even the recent Jamba-1.7-Mini SSM–Transformer (a 52B hybrid model) collapses on RoBench-25 under some HiSPA triggers, whereas pure Transformers do not. We also observe that HiSPA triggers significantly weaken the Jamba model on the popular Open-Prompt-Injections benchmark, unlike pure Transformers. We further show that the theoretical and empirical findings extend to Mamba-2, and also analyse a Mamba-2-based hybrid (Nemotron-3-Nano). Finally, our interpretability study reveals patterns in Mamba’s hidden layers during HiSPAs that could be used to build a HiSPA mitigation system. The full code and data to reproduce the experiments can be found at https://github.com/TortueSagace/hispa.

Abstract:
Simulating an online community at scale calls for language models that can stand in for many distinct users at once, not collapse them into a single generic voice. Prior work shows that a hypernetwork-driven language model can efficiently rebuild known users by storing a small per-user profile vector rather than separate fine-tuned models, one per user. This study explores whether those profile vectors support a harder task: generating behaviorally consistent new users who were never directly observed. Our frozen LLM uses adapter offsets emitted by a hypernetwork. We sample profiles within and beyond the real-user range, then score the generated replies on behavioral measures to determine whether synthetic users match the cohort patterns their profiles imply. We show that synthetic users reproduce behavioral patterns similar to those of real users in the training set: in short threads they stay close to their target profile, and across a longer multi-turn forum simulation they stay largely coherent and do not drift from it. Performance declines as profiles move farther from the distribution of users seen in the original data, suggesting that the method works best for synthetic users that recombine observed patterns rather than extrapolate beyond them.

Abstract:
Large language model (LLM) systems are increasingly used to support high-stakes decision-making, but they typically perform worse when the available evidence is internally inconsistent. Such a scenario exists in real-world healthcare settings, with patient-reported symptoms contradicting medical signs. To study this problem, we introduce MIMIC-DOS, a dataset for short-horizon organ dysfunction worsening prediction in the intensive care unit (ICU) setting. We derive this dataset from the widely recognized MIMIC-IV, a publicly available electronic health record dataset, and construct it exclusively from cases in which discordance between signs and symptoms exists. This setting poses a substantial challenge for existing LLM-based approaches, with single-pass LLMs and agentic pipelines often struggling to reconcile such conflicting signals. To address this problem, we propose CARE: a multi-stage privacy-compliant agentic reasoning framework in which a proprietary LLM provides guidance by generating structured categories and transitions without accessing sensitive patient data, while a local LLM uses these categories and transitions to support evidence acquisition and final decision-making. Empirically, under controlled retrospective evaluation on MIMIC-DOS, CARE achieves the best overall performance across key metrics among the evaluated LLMs and agentic workflows, showing that it can more robustly handle conflicting clinical evidence while preserving privacy. Our code is [here](https://github.com/AAAAA-Academia-Attractions/CARE).

Abstract:
Conversational LLM agents accumulate persistent state across sessions, but existing memory systems treat updates as append-or-replace heuristics, leaving contradictory beliefs to coexist in the store and forcing the answer model to reconcile them at query time. We argue this is the wrong abstraction: conversation is a stream of belief revisions , and the right formal lens is the Alchourrón–Gärdenfors–Makinson (AGM) framework, adapted to Hansson belief bases. We present , which operationalizes the AGM operator set as the write semantics of a history-preserving fact belief base. A uniform LLM-driven resolver classifies every extracted claim into one of four actions ( exp , rev , dup , con ) and runs at two timings: per round against the recent session, and across sessions over a similarity graph. The store never deletes; superseded beliefs retain predecessor pointers, and a context-selection step decides per query whether to surface the head, the full chain, or neither. On LoCoMo (1 , 540 questions) reaches 85.5% over Mem0-graph) and 92.3% with a stronger backbone. On BEAM (400 questions) it reaches 54.5% ( +21.8 over the LIGHT mean).

Abstract:
Productive dialogue alignment requires distinguishing surface coordination (acknowledgments and smooth task progression) from epistemic alignment (convergence of belief states); standard preference-based methods typically optimize response-level preferences without explicitly modeling the latter. We operationalize Theory-of-Mind (ToM) inference as a control signal within Frictive Policy Optimization by extracting, at each referring expression, a four-part belief structure: the speaker’s intended referent, the addressee’s interpretation, and each participant’s model of the other’s belief. This makes friction mechanically computable from epistemic-state comparisons, capturing silent divergence , where both participants proceed confidently while grounding to different referents. We evaluate the signal at two levels. At the representation level, ablating the second-order channel reduces misunderstanding recall from 65% to 26%. At the policy level, reward-shaping (FAR) and trust-region (FTR) variants improve intervention F1 and warranted-context calibration over DPO, with Brier scores independently supporting the calibration gains. Across three training runs, FAR and FTR remain substantially more stable, whereas DPO varies widely and can degrade intervention competence already present in the base policy. Thus, ToM-grounded friction provides a trainable signal for context-sensitive intervention under referential belief divergence.

Abstract:
Current Large Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect immutable environmental constraints. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations—generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of environmental dynamics without continuous, costly retraining. To solve this fundamental misalignment between semantic planning and environmental reality without relying on parameter updates, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce semantic physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves the best performance compared to baselines, exhibiting remarkable cross-model and cross-environment transferability.

Abstract:
Computational mental health screening using multimodal speech and text has shown great promise. However, existing models often assume all clinical speech protocols carry equivalent evidentiary validity. In reality, heterogeneous protocols, from free interviews to fixed reading tasks, support fundamentally different evidence. Forcing uniform reasoning flattens these boundaries, causing models to hallucinate symptoms from irrelevant text or overclaim support. Even advanced long chain-of-thought LLMs fail to resolve this issue, as free-form reasoning can exacerbate boundary violations. To address this, we reformulate multimodal screening as an evidence-bounded reasoning problem. We introduce the Evidence Package Benchmark, integrating 1,870 packages across six heterogeneous sources with explicit modality masks and evidence permissions. We further propose EviBound, a protocol-aware evidence control framework. Unlike direct LLM prompting, EviBound uses a profile-aware planner to restrict reasoning scope, orchestrates evidence tools via five-way acoustic consensus, and enforces a boundary critic to suppress unsupported claims. Empirical results show EviBound achieves a held-out test Depression AUROC of 0.8658, exceeding the strongest direct omni-modal baseline by +0.0811 AUROC while maintaining zero claim violations. Our work moves beyond unconstrained accuracy toward evidence-consistent, protocol-aware systems for safer clinical NLP research.

Abstract:
Chart reasoning agents are increasingly used to extract actionable insights in critical domains, achieving state-of-the-art performance on multiple benchmarks. Yet, high benchmark accuracy alone is insufficient for deployment, where stakeholders must be able to audit and verify how a model reaches its answer. Existing LVLM-based chart agents produce either answer-only predictions or free-form rationales that are hard to verify, obscuring whether an error arose from misreading the chart, extracting a wrong value, or miscomputing. We propose Chart-RVR, a reinforcement learning framework for training monitorable chart agents with verifiable process rewards. Chart-RVR decomposes chart reasoning into three auditable blocks: Structure, identifying the chart type; Evidence, reconstructing the underlying data table in JSON; and Derivation, exposing the stepwise trace that computes the answer. Across six in-domain and out-of-domain benchmarks, Chart-RVR attains state-of-the-art accuracy among comparable-sized LVLMs. Beyond accuracy, we assess monitorability using a triangulated protocol that combines ground-truth surrogate metrics, an oracle information-gain measure, and an LLM-as-auditor scoring Process Verifiability and Evidence Localization, showing that Chart-RVR yields rationales that are markedly more verifiable and evidence-grounded than those from CoT prompting, SFT, and existing chart-specific baselines.

Abstract:
Supervised Fine-Tuning (SFT) is the standard paradigm for domain adaptation, yet it frequently incurs the cost of catastrophic forgetting. In sharp contrast, on-policy Reinforcement Learning (RL) effectively preserves general capabilities. We investigate this discrepancy and identify a fundamental distributional gap: while RL aligns with the model’s internal belief, SFT forces the model to fit external supervision. This mismatch often manifests as "Confident Conflicts"—tokens characterized by low probability but low entropy. In these instances, the model is highly confident in its own prediction but is forced to learn a divergent ground truth, triggering destructive gradient updates. To address this, we propose Entropy-Adaptive Fine-Tuning (EAFT). Unlike methods relying solely on prediction probability, EAFT utilizes token-level entropy as a gating mechanism to distinguish between epistemic uncertainty and knowledge conflict. This allows the model to learn from uncertain samples while suppressing gradients on conflicting data. Extensive experiments on Qwen and GLM series (ranging from 4B to 32B parameters) across mathematical, medical, and agentic domains confirm our hypothesis. EAFT consistently matches the downstream performance of standard SFT while significantly mitigating the degradation of general capabilities.

Abstract:
Hard-negative preference optimization should improve support for ground-truth preferred responses while limiting drift from the reference model. In Japanese medical instruction tuning, we find that standard DPO and IPO improve the preferred–dispreferred gap but can leave preferred-response likelihood below the reference model and induce growing sequence-level KL divergence. Motivated by this failure-mode analysis, we propose a stabilized IPO objective that combines preferred-response anchoring with explicit KL control. Ablations show that positive anchoring improves preferred-response support, while KL control suppresses reference-model drift; their combination preserves anchoring gains while further reducing drift. These results suggest that reliable medical preference optimization benefits from both preferred-response anchoring and drift control.

Abstract:
To mitigate catastrophic forgetting in downstream continual learning (CL) for large language models (LLMs), existing methods typically constrain parameter updates or introduce task-specific adaptation modules. However, these methods often rely on explicit task boundaries during training, limiting their applicability to realistic task-free scenarios. In this paper, we propose a Fisher-guided unified (FiUni) framework for batch-level task detection and parameter-efficient continual adaptation. FiUni is motivated by a key observation about the Fisher information matrix (FIM) of pre-trained models: the orthogonality among the principal subspaces of its Kronecker-Factored Approximate Curvature (K-FAC) approximation, estimated from a small number of downstream task samples, can reflect the similarity between different tasks. Based on this observation, FiUni constructs FIM-derived frozen subspaces to guide low-rank adaptation (LoRA), while matching the Fisher principal subspace of each incoming batch window with historical subspaces. This enables FiUni to adaptively determine whether to reuse existing knowledge, expand a related subspace, or create a new subspace, dynamically balancing knowledge sharing and task isolation. Experiments show that FiUni can effectively infer latent batch-level task affiliations and achieve competitive performance against advanced task-aware CL methods with fewer trainable parameters.

Abstract:
RL has emerged as a powerful paradigm for enhancing the instruction following capabilities of LLMs. While existing training recipes achieve substantial gains, we find that they suffer from exploration bias towards easy instructions when the training data has multiple instructions in a prompt. This bias is caused by two main reasons: 1) the policy model’s initial ability to satisfy hard instructions is too low to trigger successful exploration during RL training, so the optimization is biased to easy instructions and 2) canonical RL training recipes typically employ a cumulative reward (the number of instructions fulfilled), treating all the instructions equally; the policy model is biased to fulfill the easy instructions to get the same amount of rewards. To address these, we first propose two metrics to measure the exploration bias of instruction following and then introduce a two-stage framework to alleviate it: 1) Behavioral Bootstrapping: a lightweight rejection sampling fine-tuning before RL to activate hard instructions and 2) Scarcity-Aware Rewards: a new RL reward function that assigns rewards to instructions based on their empirical scarcity. Experiments show that the proposed metrics are highly correlated with model performance and our methods unleash the potential of RL training: our best models outperform the baselines by a significant margin across three verifiable instruction following benchmarks.

Abstract:
Existing generalist detectors, such as mDeBERTa based classifiers, struggle with fine-grained AI-text detection in multilingual settings, particularly when hybrid authorship and morphologically rich languages obscure machine-generated artifacts. We propose a typology-aware Smart Routing Late Fusion Framework for 4-class detection: Human-Written (HW), Machine-Generated (MG), Machine-Humanized (MH), and Human-Polished (HM). The framework consists of two complementary tracks. Branch A targets agglutinative languages, including Korean, Turkish, and Hungarian, using error focused hard-negative refinement and Dependency Distance Minimization(DDM) / Uniform Information Density(UID) based pairwise calibration to recover weak generative signals hidden by dense morphology. Branch B applies RADAR-inspired invariant learning to capture machine-originated artifacts that persist after adversarial rewriting in languages with clearer segmentation cues. Experiments across eight languages show that typology-aware routing improves detection in agglutinative languages while maintaining strong multilingual performance. Our results highlight the importance of modeling both linguistic typology and hybrid authorship for robust multilingual AI-text detection.

Abstract:
Referring single-object tracking enables language-grounded target initialization and subsequent tracking by jointly leveraging semantic cues and visual templates. The core difficulty is to use language differently across stages: it is indispensable for grounding but can induce semantic drift during tracking when overemphasized. Meanwhile, current methods often require costly vision-language alignment training. We present LVTrack, a pure transformer framework that introduces a mode-conditioned Gated Feature Injector to adaptively regulate textual guidance and alleviate semantic drift. Together with targeted adaptations, it directly harnesses a frozen vision-language pretrained model, greatly reducing training cost and preserving strong language understanding. To further improve temporal localization, LVTrack integrates hybrid relative-absolute positional encodings with a lightweight memory mechanism and optimizes autoregressive box prediction using a Gaussian-smoothed KL loss. Extensive experiments on standard benchmarks demonstrate that LVTrack achieves strong performance.

Abstract:
Vision-Language Models (VLMs) achieve strong multimodal understanding but suffer from high inference costs due to redundant vision tokens. Existing hard-pruning methods discard low-scoring tokens directly, inevitably sacrificing fine-grained visual context. We propose SPack, a training-free token compression method that reframes pruning as information-energy redistribution rather than deletion. Instead of discarding pruned tokens, SPack transfers their semantic mass to retained tokens, enforcing a global principle we term Information Energy Conservation (IEC): the total information contained in redundant tokens is strictly conserved and fully transferred as information gain to the retained components. To realize this conservation, we introduce Optimal Transport (OT) theory, which treats removed energy as mass density and retained tokens as receiver slots. Solving the optimal transport plan under the Wasserstein distance yields a minimum-cost mapping that precisely transfers geometric structure and higher-order moments according to semantic affinity. Concretely, we estimate token importance via relation attention, semantic affinity, and value-space energy, then adaptively route pruned tokens to multiple retained tokens using OT, producing hyper-dense tokens that preserve both local semantics and global context. Extensive experiments demonstrate that SPack consistently achieves a superior accuracy-efficiency trade-off compared to state-of-the-art methods. The code will be released upon acceptance.

Abstract:
Legal Case Retrieval (LCR) identifies relevant precedent cases to support legal reasoning and sentencing. A core challenge for cross-system LCR is the structural divergence between legal systems: civil-law systems provide explicit statutory hierarchies but lack inter-case citations, while common-law systems offer rich citation networks but lack uniform codified semantic hierarchies. Existing methods exploit only one side of this divide, remaining system-specific. We propose DSLaw, which unifies both systems by implicitly modeling hierarchical semantics through residual quantization and using the resulting discrete representations as semantic anchors for cross-case graph reasoning. By progressively decomposing legal semantics from coarse concepts to fine-grained distinctions, DSLaw constructs implicit connections even without explicit citations. Experiments on COLIEE2025 (common law) and ELAM (civil law) show DSLaw consistently outperforms strong baselines, while joint training further validates cross-system generalization. The implementation is provided at https://github.com/KDEGroup/DSLaw.

Abstract:
Fact verification pipelines are often unrealistic: they assume full-input access, operate on isolated atomic claims, and overlook the central question of when a system should act under partial evidence. We introduce DIVER (Deliberative and Iterative fact VERification), an LLM-based framework for calibrated incremental fact-checking. DIVER incrementally extracts verifiable atomic claims from progressively revealed input, retrieves and refines evidence through iterative web search, and triggers a correction as soon as a claim is reliably refuted; if no such trigger occurs, it performs a passage-level revision over the full input to recover missed errors. Across standard fact-checking benchmarks, DIVER outperforms strong LLM-based verification pipelines, with especially large gains on retrieval-intensive settings. We further introduce UmActually, a real-world benchmark built from the Internet game Um, Actually, to evaluate not only whether a system finds an error, but whether it does so at the right time. On UmActually, DIVER reports the strongest end-to-end results while typically buzzing close to the earliest valid decision point, supporting its effectiveness for calibrated incremental fact-checking.

Abstract:
Synthetic data is increasingly used in language-model pretraining. Existing knowledge-corpus synthesis pipelines, including WRAP and Nemotron-CC, usually take a source passage or document segment as input and generate rewritten or QA-style samples from that text. This can improve surface form and prompt diversity, but it does not explicitly require fine-grained source facts — especially secondary entities, list/table entries, and precise temporal attributes — to appear in the synthesized corpus. We propose AFCS (Atomic-Fact-Conditioned Synthesis), a pipeline that first extracts source-supported atomic facts and then uses each extracted fact to drive generation. AFCS uses structure-aware extraction over the document tree, source-grounded verification, local context expansion, and multi-format generation. In continued-pretraining experiments with OLMo-3-7B and a 32B base on the same FineWiki source corpus, AFCS outperforms WRAP and Nemotron-CC on SimpleQA, SimpleQA Verified, PopQA, and HotpotQA. Granularity ablations show that fact-level synthesis improves over chunk-level, paragraph-level, and entity-level synthesis; and a 100B-token mid-training integration improves factual benchmarks and raises the 12-task general benchmark average by +1.09.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has shown remarkable success in improving the mathematical reasoning of large language models. Yet problems beyond the model’s current capability, where rollouts uniformly fail and no learning signal is produced, are structurally wasted despite marking the most informative training frontier. We show that these otherwise-inert problems can be unlocked via teacher-guided curriculum learning: partial reasoning traces from a stronger model create a graded difficulty landscape, and a backward-chaining curriculum progressively withdraws guidance until the model solves problems unaided. Training on only 128 unsolvable problems matches or exceeds GRPO trained on a full 2,000-problem corpus (~16× data efficiency) on the nine-benchmark average for both base models, while substantially expanding the reasoning boundary measured by pass@k at large k. Furthermore, we identify a distribution-shift cost that is particularly acute in the unsolvable-only regime and propose Monotone Frontier Curriculum (MFC), a method that monotonically drives training toward unguided solving, consistently outperforming existing curriculum methods.

Abstract:
Spatial audio editing modifies an existing soundfield according to a user’s instruction while preserving the rest of the scene. Unlike conventional audio editing, it must reason jointly about audio events, spatial information, dynamic changes, and environmental information in first-order Ambisonic (FOA) waveforms. Existing language-guided editors mainly target conventional audio or rely on sequential operations, and therefore do not directly support one-stage editing for complex 3D spatial instructions. We present SwanWeave, the first one-stage multi-task framework for instruction-guided 3D FOA spatial audio editing. We build paired FOA supervision from open-source speech and sound-effect corpora using controllable room simulation, covering more than ten single-operation and compound tasks across the four editing axes. To handle this heterogeneous edit space, SwanWeave uses Spatial Edit Mixture-of-Experts (SE-MoE) with dual-level routing, selecting task-aware expert combinations for compound instructions and frame-level routed/null experts for local edit decisions. We further introduce Spatial Preference Optimization (SPO), a Direct Preference Optimization (DPO)-based alignment objective with edit-specific negative targets, and adopt staged training to improve natural-language grounding. Experiments show that SwanWeave achieves better editing quality than existing general audio editors and spatial audio baselines across all tasks. Spatial audio editing demos can be found at https://swanaigc.github.io/#swanweave, code can be found at: https://github.com/MM-Speech/SwanWeave.

Abstract:
Proof assistants such as Lean 4 offer machine-checkable verification of mathematical reasoning, but their adoption is limited because formal languages have a steep learning curve and can be challenging to use. Large language models offer a way to lower this barrier to entry through natural-language co-reasoning systems: proof assistants that users interact with entirely in natural language, and which use formal verification to provide feedback to users who submit incomplete and incorrect proofs. This paper explores such a co-reasoning system in an educational setting. We explore metrics for three properties that are important for natural language co-reasoning systems—faithfulness, accuracy and appropriate granularity. We introduce an evaluation framework for these properties in the context of educational proofs. Second, we present a case study on an educational natural-language co-reasoning system for math proofs, evaluating its autoformalization harness on student-written proofs from an undergraduate linear algebra class using these three properties.

Abstract:
Can vision-language models predict what a 180° rotation would reveal from the original image alone? We study this ability through Rotated-Outcome Prediction: given an original image, a model must answer what would be seen or read after a 180° in-plane rotation, without directly observing the rotated target. To study this gap, we introduce ROTOUTBENCH, a paired diagnostic benchmark spanning open visual cases and controlled text-image rotations. A sharp pattern emerges: many VLMs can recognize the relevant content when directly given either the original or rotated image, yet fail to infer the rotated result from the original image alone. On controlled text-image rotations, predicted-rotation accuracy collapses to near zero even for models with high direct-reading accuracy. A model-level diagnostic further shows that the prediction state can approach a rotated-image reading state, while the final readout still shifts toward the original string. Current VLMs can recognize a transformed visual state when it is shown, but often fail to predict that state from the original view.

Abstract:
While language model agents have advanced software engineering, existing test-based supervision is limiting its scalability on real-world issues. The reason is twofold: (1) high-coverage tests are naturally rare in the wild, and (2) building and running test sandbox is heavy and fragile. To unlock supervision scaling, we propose R4P, a reasoning-based method that provides scaffold-agnostic rewards. R4P uses a group-wise training objective, enabling it to verify multiple patches against each other’s modification and gain a dense reward for supervising agents without executing tests or relying on specific agent trajectories. R4P achieves 72.2% Acc. for verifying patches from SWE-bench, competitive with proprietary models. To show the downstream practical utility of R4P, we design and train an execution-free scaffold, Mini-SE, with pure RL via R4P. Mini-SE achieves 26.2% Pass@1, showing a 10.0% improvement over the original Qwen3-32B, and can be further improved to 32.8% with R4P for test-time scaling on patch selection. The stable scaling curves illustrate that though imperfect, R4P can still reliably support downstream tasks at scale.

Abstract:
Context compression has emerged as a vital approach to mitigate both performance degradation and resource consumption for large language models (LLMs) in long context scenarios. Current state-of-the-art methods allocate the compression budget at fixed-size block granularity, either uniformly or with adaptation across blocks. Both strategies fragment sentence-level semantic units and overlook the relevance variation within each block. In this paper, we propose the Adaptive Semantic Preservation (ASP) compressor, built on two innovations: (1) To preserve semantic units, our Semantic-Preserving Segmentation partitions context along natural sentence boundaries while maintaining the morphological atomicity of words. (2) To prioritize critical information, our task-aware Adaptive Budget Allocation dynamically assigns compression budget based on each sentence’s informational salience for the task instruction. Extensive experiments across diverse long-context scenarios show that ASP outperforms strong task-aware and task-agnostic compression baselines on both Llama-3-8B-Instruct and Qwen-2.5-7B-Instruct. In particular, ASP delivers stronger robustness compared with other baselines even at the extreme 16 × high compression ratio.

Abstract:
Large language models (LLMs) with reasoning capabilities have shown impressive performance in clinical applications, yet the readability and traceability of their reasoning processes remain limited, as reasoning quality is often overlooked during training. To address this, we propose MedCEG, a framework that augments medical language models with clinically traceable reasoning by explicitly supervising the reasoning process through a critical evidence graph (CEG). Specifically, we curate challenging clinical cases and algorithmically construct a CEG for each sample via multi-model consensus as a high-quality, traceable reasoning pathway. The CEG reformulates the reasoning process for supervised fine-tuning and underpins a traceable reasoning procedure reward that evaluates generated reasoning against the CEG through deterministic graph operations measuring semantic matching, triplet recall and connected-component analysis. Experiments show that MedCEG surpasses existing methods among open-source medical models of comparable scale while producing clinically readable and traceable reasoning chains. Code and models are available at https://github.com/LinjieMu/MedCEG .

Abstract:
Agentic AI systems powered by large language models (LLMs) are rapidly evolving into autonomous decision-making systems, exposing attack vectors beyond those of traditional LLM vulnerabilities. Existing security evaluations are often tied to specific implementations or domains, limiting unified comparison across heterogeneous systems. To address this gap, we introduce RIFT-Bench, a representation-driven methodology for dynamic red-teaming that enables unified evaluations across diverse agentic architectures. Building on a novel hierarchical representation, RIFT-Bench operates in two automated phases: Discovery, which extracts system structure, and Scanning, which executes adaptive adversarial attacks. It directly evaluates the examined system using 105 adaptive adversarial probes spanning diverse attack vectors and objectives. We demonstrate the effectiveness of the proposed evaluation pipeline across 45 agentic systems spanning a diverse range of implementations, showing that the approach generalizes effectively to heterogeneous agentic architectures. Beyond systems and attacks, RIFT-Bench also supports direct evaluation of mitigation strategies. These key capabilities make RIFT-Bench a scalable foundation for security evaluation of agentic AI systems in practice. Infrastructure code and benchmark artifacts are available at https://github.com/FujitsuResearch/RIFT-Bench.

Abstract:
Query expansion (QE) is a critical technique in information retrieval that enriches underspecified queries with additional textual context. However, its effect is often unreliable in modern dense retrieval, especially for strong off-the-shelf retrievers without retraining. Existing studies mainly examine expansion quality, semantic drift, or retrieval outcomes, but rarely explain how QE changes dense retrievers internally. In this work, we trace QE effects through sparse autoencoder (SAE) features. Using paired original and expanded queries, we decompose layer-wise retriever representations into sparse latent activations, identify QE-related latents from expansion-induced activation shifts, and interpret them with natural-language descriptions and retrieval cases. Our analysis shows that effective QE induces layer-concentrated changes in sparse latents aligned with retrieval intent and entity attributes, rather than only perturbing final query embeddings. SAE-based activation steering further validates these latents improve retrieval more consistently than random interventions or vanilla QE across four benchmarks, suggesting that SAEs can explain QE effects and offer a lightweight option for precise retrieval behavior modulation without query rewriting or retriever fine-tuning.

Abstract:
Red teaming attacks are a well-established approach for identifying weaknesses in large language models (LLMs). As the generative capabilities of LLMs continue to improve, researchers have increasingly leveraged them to automatically generate red teaming attacks, often by crafting adversarial prompts that target other LLMs. Despite this progress, there is currently no effective strategy for selecting suitable LLMs to serve as red teaming agents. In this work, we propose a systematic framework to investigate how various factors of LLMs influence their effectiveness in generating red teaming attacks, including model security, general capability, and parameter scale. The goal of this study is to understand the mechanisms behind the effectiveness of red teaming LLMs and to provide principled guidance for selecting appropriate red teaming LLM.

Abstract:
Dialectal Arabic (DA) remains under-resourced compared to Modern Standard Arabic (MSA), particularly for machine reading comprehension (MRC) and question answering (QA). Existing Arabic QA benchmarks primarily focus on formal written MSA or multiple-choice QA, with limited coverage of naturally spoken dialects. Here, we aim to bridge this gap. We introduce EDRAC, the first large-scale benchmark for dialectal Arabic machine reading comprehension (MRC) and generative QA, covering five major dialects: Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic. EDRAC contains 499 passages derived from naturally occurring spoken interactions and 4,977 corresponding QA pairs generated through a human–LLM collaborative pipeline combining iterative generation, LLM-as-a-judge evaluation, and human verification. We benchmark Arabic-centric and multilingual LLMs on EDRAC using lexical and semantic metrics. Our results reveal substantial gaps between semantic answer quality and dialectal fidelity, highlighting the limitations of existing evaluation metrics for dialectal Arabic generation. EDRAC provides a realistic and challenging MRC benchmark for future research on dialectal Arabic NLP.

Abstract:
Translating song lyrics while preserving singability requires satisfying musical constraints, including syllable counts, rhyme schemes, and syllable patterns, that standard machine translation ignores. We present a framework that exposes IPA-based phonetic analysis as a suite of composable Agent Skills, each defined by a natural-language description plus deterministic verification scripts that the orchestrating LLM invokes on demand. The agent extracts constraints from source lyrics, then translates through a multi-phase process, calling skills to verify syllable counts and rhyme and iteratively refine syllable-count mismatches. We formalize the three musical constraints, describe the IPA skill suite and agent architecture, and propose three evaluation metrics: Syllable Error Rate (SER), Syllable Count Relative Error (SCRE), and Adjusted Rand Index (ARI). Experiments across three frontier orchestrators, with component ablations, confirm that tool-verified translation dramatically improves constraint satisfaction and preserves rhyme structure substantially better than a supervised baseline, while requiring no task-specific training and no parallel corpora, and transferring to Spanish and Japanese targets with only a small language-specific syllabification adapter.

Abstract:
Idioms remain a persistent challenge in natural language processing due to their figurative and culturally grounded meanings, which distinguish them from literal expressions. Although recent advances in large language models (LLMs) have improved idiom handling across several languages, limited attention has been given to low-resource languages such as Urdu. In this work, we present a comprehensive benchmark for Urdu–English idiomatic translation, consisting of a manually verified dataset of 4,000 aligned idiom–sentence pairs in both Perso-Arabic (native Urdu script) and Romanized Urdu. We evaluate multiple tasks, including translation, paraphrasing, idiom span detection, and back-translation, using diverse prompting strategies such as cultural prompting, idiomatic prompting, and few-shot learning. Our findings show that frontier LLMs consistently outperform traditional neural machine translation systems across all evaluation settings, particularly in preserving figurative and metaphorical meaning. While models demonstrate relatively stable performance on native Urdu script, the absence of standardized orthography in Romanized Urdu introduces substantial challenges for consistency and idiom span detection. This work establishes a high-quality benchmark for cross-script idiomatic evaluation in Urdu and underscores the importance of prompt engineering in preserving figurative language meaning across languages.

Abstract:
Internet memes have become important carriers of online expression, and pun memes, as a typical multimodal category, pose significant challenges to machine understanding due to their reliance on textual ambiguity and cross-modal interaction with visual content. Existing multimodal models for pun meme understanding suffer from classification bias, weak sentiment perception, and performance gaps compared to human comprehension, mainly due to neglecting fine-grained correlation cues for semantic reasoning. To address this, we propose a fusion framework enhanced by multi-source auxiliary cues, including visual salient perception, textual deep semantics, and abnormal pun linguistic features, which are embedded into the multimodal reasoning pipeline. Experiments show that our framework stably optimizes vision-language models, improving accuracy in pun meme detection and implicit sentiment fine-grained analysis.

Abstract:
Machine Unlearning methods for Large Language Models typically assume pre-specified forget and retain sets. In realistic settings, however, requests may provide only a few examples of undesired behaviour, requiring forget and retain sets to be inferred from heterogeneous corpora. We study this data-selection problem and propose GRACE, a gradient-guided coreset selection method that constructs both forget and retain sets for LLM unlearning. GRACE first computes a forget direction from seed examples that elicit the undesired behaviour, then selects a compact forget coreset whose gradients approximate this direction using non-negative orthogonal matching pursuit. To preserve model utility, it selects retain examples after projecting out the forget direction and applying clustered orthogonal matching pursuit in the remaining gradient space. Across two target domains, two model families, and four unlearning algorithms, GRACE improves model utility while maintaining comparable forget quality, with particularly consistent gains over prior gradient-based selection methods.

Abstract:
Reinforcement learning (RL) for retrieval-augmented language models enables policies to interleave generated reasoning with retrieved observations. However, in Search-R1-style PPO, training may become unstable after the model has already reached strong validation performance. This paper studies late-stage collapse as a training-stability problem in search-based RL. Four metrics are proposed to quantify validation drawdown (VDD), advantage-tail expansion (ATE), actor update stress (AUS), and observation-to-actor contamination in generalized advantage estimation (GAE), denoted as OAGC. The analysis further shows how observation tokens can influence actor updates through value targets, value-model learning, and GAE. To address this issue, Observation-aware Stabilization (OAS) combines observation residual decay, old-value anchoring, and boundary smoothing. On three benchmark datasets (NQ, TriviaQA, and HotpotQA), OAS improves Search-R1 PPO under multiple retrieval budgets. In the main Top-3 setting, our method improves the exact match (EM) measure from 31.8/49.6/18.7 to 37.6/56.8/23.9 and reduces OAGC from 0.331 to 0.088, with lower search-budget saturation and no increase in the no-search rate.

Abstract:
Information abstraction, which groups strategically similar private states into a tractable number of buckets, is essential for scaling game-solving algorithms to large imperfect-information games. Constructing effective abstractions, however, has traditionally required domain-specific evaluators such as hand-strength calculators or equity estimators, which demand expert knowledge and engineering effort and are unavailable for most less-studied games. We propose the Abstraction Agent , a zero-shot pipeline that uses a large language model (LLM) to discover continuous strategic features from a natural-language game description, score private states on these features, and cluster them into abstraction buckets, without any game-specific evaluator, training data, or game-tree traversal during abstraction construction. The pipeline runs in four phases: feature discovery with calibration anchors, batched private-state scoring, correlation-based feature selection, and k -means clustering. The resulting abstractions reduce lifted-strategy exploitability by up to 62% relative to an expected-hand-strength baseline on heads-up no-limit Texas hold’em (HUNL) turn endgames, and beat a scalar rank baseline at every granularity on ROVER Trials, an original game absent from any pretraining corpus. Beyond these quantitative benchmarks, the pipeline transfers with unchanged prompts to four-card Pot-Limit Omaha, HUNL preflop and flop, and Riichi Mahjong, where the discovered features track each game’s recognized strategic concepts. This is structured knowledge elicitation : converting implicit strategic knowledge in LLM parameters into explicit numerical features for downstream algorithmic computation. The full pipeline code is available at https://github.com/lbn187/AbstractionAgent .

Abstract:
Traditional Retrieval-Augmented Generation (RAG) systems score each passage independently against the query, assembling context sets that may be individually relevant yet collectively incoherent. We introduce Coherence-Aware Graph Encoding (CAGE), a reranking framework that models "between-chunk coherence" across four dimensions: Intra-Domain Relevance, Noise Resistance, Informational Bonding, and Factual Consistency. Our pipeline transforms retrieved passages into directed heterogeneous entity graphs, amplifies factual anchors via min-out-degree reweighting, encodes structural patterns through a Relational Graph Convolutional Network, and fuses inter-chunk coherence with query relevance for final ranking. Evaluated across four multi-hop benchmarks, CAGE matches or outperforms strong baselines including monoT5 in Recall@5 on bridge-dominated datasets and consistently improves downstream Exact Match, demonstrating that structurally coherent context yields more precise answers even when retrieval recall is comparable or lower.

Abstract:
Chain-of-thought (CoT) supervision has become a default assumption in fine-tuning large language models for reasoning tasks, with the intuition being that exposing models to intermediate reasoning steps leads to deeper learning than training on answers alone. We challenge this assumption. We introduce answer backtrackability, a measurable quantity capturing the degree to which a question can be reconstructed from its answer, and argue it is a key determinant of whether CoT supervision helps or hurts during fine-tuning. To test this, we construct two novel benchmarks derived from automata theory (NFA and DFA), where answers are transition tables from which the question is largely recoverable, and fine-tune three LLMs using LoRA on CoT and No-CoT versions of five datasets: GSM8K, MathInstruct, MATH, and our two formal-language benchmarks. On mathematical datasets, CoT consistently outperforms No-CoT. Within MathInstruct, Program-of-Thought (PoT) problems, where executable code answers are high-backtrackability, favour No-CoT, while CoT-format problems favour CoT. On our benchmarks, the pattern reverses: No-CoT outperforms CoT across all models and difficulty tiers. When the answer encodes sufficient structure to reconstruct the reasoning process, intermediate supervision adds noise rather than signal.

Abstract:
The evolution of LLMs into tool-enabled agents creates a new class of safety challenges associated with real-world execution rather than simple text generation. Existing alignment methods often rely on coarse refusal signals or static supervision, making it difficult to balance safety with useful tool execution across diverse agentic risks. We introduce RUBAS, a rubric-based reinforcement learning framework for agent safety. RUBAS decomposes agent behavior into four dimensions: tool-use safety, argument safety, response safety, and helpfulness. These structured rubrics provide fine-grained and interpretable rewards over complete agent trajectories, enabling reinforcement learning to optimize safe tool use while preserving task completion. Extensive experiments across multiple agent safety benchmarks and models show that RUBAS improves safety over standard alignment baselines, reduces tool-grounded hallucinations, and maintains competitive utility. Our results suggest that multi-dimensional rubric rewards provide an effective training signal for aligning LLM agents in safety-critical tool-use settings.

Abstract:
Historical book illustrations are culturally significant multimodal artifacts that pose distinctive challenges for computational analysis, including sparse visual cues, historical reproduction techniques, and material degradation. We present the first task-specific evaluation of multimodal large language models (MLLMs) on historical book illustrations, testing five contemporary models on 1,969 children’s book illustrations across a range of visual labeling exercises and caption generation tasks. Frontier models performed strongly against human consensus annotations on concrete tasks, particularly illustration and setting detection, and mirrored human-level disagreement on entity, object and action identification. All model families struggled with accurate emotional intensity estimation. GPT and Gemini produced the strongest captions, with greater than 70% of images exhibiting no factual errors. Finally, frontier models closely recovered human distributions at the corpus level for major visual categories suggesting their appropriateness for use in large-scale studies.

Abstract:
LLM Agents Harnesses are hand-designed and stay fixed, so agents accumulate experience but never learn how to use it: which memories to retrieve, when retrieved evidence is misleading, and when the retrieval strategy itself should change. We introduce Agent Evolving Learning (AEL), a two-timescale framework that evolves the harness, recasting memory use as online policy selection. A fast Thompson Sampling bandit selects among memory-retrieval policies episode by episode, while slow LLM reflection follows a diagnose-before-prescribe principle: it first explains why performance degraded, then injects a targeted new retrieval policy as a bandit arm when the current pool plateaus. AEL outperforms ten self-improving and nonLLM baselines on a sequential portfolio benchmark, lifting Sharpe by 27% over the strongest memory-only variant with the lowest variance among all stochastic methods, and generalizes to a support-ticket routing stream, where it improves accuracy by 18% over reflectionfree Thompson Sampling and by 51% over the best prior baseline. Mechanism studies further show that the gains are causal: reflection helps precisely when regimes demand different retrieval behavior, and is provably no-harm/nogain when the best policy is stable.

Abstract:
Recent reinforcement learning (RL) post-training approaches primarily optimize the final output policy using sparse outcome-level rewards, while largely overlooking predictive signals encoded in intermediate representations. In this paper, we introduce a new paradigm called on-policy internal self-distillation and propose the OISD framework, which improves reasoning by transferring on-policy predictive signals from the final layer to intermediate representations. During rollout and Group Relative Policy Optimization (GRPO) optimization, the final layer acts as both the policy and a detached internal teacher for selected intermediate layers, which are guided to align with it through two complementary mechanisms: logit alignment, which transfers high-level reasoning behaviors (how to think), and attention alignment, which enforces consistent attention patterns (where to look) from the final layer to the selected intermediate layer, both without requiring external privileged information. Our OISD, together with GRPO, employs signed advantage-weighted Jensen–Shannon alignment to distill informative intermediate representations while preserving policy consistency under a unified acting policy. Experimental results demonstrate the effectiveness of OISD, with substantial and consistent improvements over strong reasoning RL baselines across four mathematical reasoning tasks.

Abstract:
When answering factual questions, language models may respond correctly, hallucinate, or abstain. We investigate whether this epistemic disposition is encoded in hidden states before any answer token is generated. Using a balanced Wikipedia QA corpus with pre/post article-date splits and rollout-consensus labels, we analyze prompt-only MLP activations from four instruction-tuned 7–9B models. PCA reveals a dominant pre-generation epistemic routing axis that orders outcomes as ABSTAIN, HALLUCINATION, and CORRECT. We then apply activation steering as a bounded causal stress test. Dense residual-write edits can shift many hallucinations toward abstention or correctness, while sparse MLP-intermediate edits localize the steering signal. These results frame the axis as a steering audit: it identifies where fixed edits improve factual behavior and where correct loss or broken outputs call for conditional controllers.

Abstract:
Surveys remain the primary way researchers grasp the lineage of methods within an AI subfield, but they scale poorly against the current rate of publication. Existing taxonomy-induction methods are largely leaf-bound and time-agnostic; they tend to force transitional papers into mature leaves and can create topological inversions between ancestors and descendants. We propose EvoTree, a staged framework that decouples conceptual backbone learning from temporal refinement: a graph-aware encoder with distribution-based hierarchical clustering yields a stable taxonomy backbone; temporal fine-tuning then re-attaches marginal papers to internal nodes under monotonic-path constraints; a final LLM pass labels concepts without altering the topology. We release the first annotated benchmark for this task across 11 AI subfields. EvoTree improves Leaf Purity and Path Monotonicity Rate over the strongest taxonomy-induction baseline, and is the only method with non-trivial marginal-paper detection on the annotated set.

Abstract:
Embodied planning increasingly relies on vision-language models (VLMs) to translate instructions and visual observations into executable action sequences. However, fluent plans are not always executable. A VLM may refer to objects that are not visually observed, select actions whose required affordances are unavailable, or violate syntax and action constraints. We introduce CLAMP, a multimodal constraint-grounding framework that turns scene evidence into decoding-time constraints for a frozen VLM planner. CLAMP uses the initial observation to restrict object references to those supported by the scene, while a provided symbolic action model specifies state transitions and goals. During decoding, hard masks eliminate invalid next-token candidates, while a Hidden Markov Model (HMM)-based world-state lookahead module reweights the probabilities of the remaining feasible candidates based on action preconditions and goal reachability. This allows the planner to retain the VLM’s language prior while preventing visually unsupported, unsafe, or infeasible candidates from entering the plan. For unseen tasks and environments, CLAMP adapts the HMM at test time using label-free continuations sampled from the frozen VLM. Experiments on VLABench, SafeAgentBench, and TaPA show that scene-grounded constraints improve object grounding and safety, while most remaining failures stem from perception errors or misaligned constraint specifications.

Abstract:
Large language models (LLMs) have become ubiquitous tools for code generation and editing. However, development teams often use multiple LLM assistants. Different developers may prefer different models, and individual developers may switch between models across different coding sessions. Because of this, the edits any one model makes are frequently applied to foreign code originally generated by another model. These LLMs are often trained on different datasets, and as a result have different stylistic preferences. Do LLMs behave differently when they edit foreign code originally written by a different LLM with a different coding style? We find that models tend to make more, and often excessive, edits on foreign code. We introduce CROCODIL (Cross-model Code Editing with LLMs), a post-training framework for reducing excessive edits while preserving functional correctness. CROCODIL’s similarity reward penalizes large changes, while its execution reward scores build and test success. We use the product of these two rewards to encourage the policy to decrease the edit size without decreasing the edit task success rate. CROCODIL is available at https://github.com/EngineeringSoftware/Crocodil.

Abstract:
Untranslatability—cases where meaning cannot be directly preserved across languages—is well-studied in linguistics but underexplored in NLP. As machine translation (MT) systems improve on standard benchmarks, their limitations increasingly concentrate in such cases, where translation cannot be reduced to one-to-one equivalence. We introduce a structured ontology of untranslatability along with a taxonomy of compensation strategies, which are specific techniques to convey meaning under these untranslatable circumstances. We operationalize this framework into a multilingual dataset of untranslatable sentences paired with strategy-based translations, enabling controlled analysis of translation behavior. Initial human preference studies suggest that translation quality depends on the strategy used, with consistent preferences for outputs that include explanatory context, known as the Annotation compensation strategy. Our framework and dataset provide a foundation for studying and modeling strategy-informed machine translation.

Abstract:
Automated patent claim validation demands low error tolerance. However, existing approaches face a rigidity-resource dilemma: lightweight encoders cannot track long-range legal dependencies, while exhaustive LLM verification incurs 4-5X higher overhead at million-claim scale. A naive confidence-based cascade cannot resolve this because binary validity scores fail to distinguish structurally distinct error types which require different reasoning depths. We propose a two-stage framework: Adaptive Cost-efficient Evaluation (ACE), which exploits the categorical structure of patent errors for uncertainty-aware routing. In the first stage, a fine-tuned encoder projects claims into a K+1 distribution over legal error types, whose predictive entropy serves as the routing signal. Claims exceeding an entropy threshold are escalated to the second stage, where an expert LLM executes a schema-constrained Chain-of-Patent-Thought (CoPT) protocol to map claim elements against 35 U.S.C. standards whose schema constraint reduces per-claim latency by 42% while producing legally grounded verdicts. We further present a 40,000-claim dataset ACE-40k with MPEP-grounded annotations, where ACE surpasses competitive baselines including a supervised 70B-parameter LLM while reducing costs by 78%. On real USPTO rejection data, the routing mechanism transfers without re-calibration, reducing inference time by 60% while maintaining competitive recall.

Abstract:
Transformer layers evolve representations through learned additive transformations that combine direction-preserving and direction-changing effects. We study this geometry by decomposing learned updates into parallel and perpendicular components. Across pretrained models, we find substantial learned parallel components, showing that direction-preserving behavior is not confined to the residual identity path. To test whether these components are redundant or behaviorally important, we apply the decomposition to residual-space updates and to value aggregation inside attention. Inference-time edits reveal a space-dependent asymmetry: value-space parallel edits preserve behavior more effectively than residual-space or perpendicular counterparts, especially when the direct self message is fixed and only the non-self aggregate is scaled. The decomposition also provides a compact, component-resolved view of compression error. During from-scratch pretraining, using parallel suppression as part of the forward computation lowers validation-loss trajectories and improves downstream averages in our evaluated runs, with the value-space variant strongest. Overall, this geometry connects update direction to editing robustness, compression diagnostics, and training-time intervention. Code is available in the https://github.com/Shwai-He/Transformer-Geometry.

Abstract:
We conduct a systematic study of 18 widely used vision-language benchmarks and identify three major issues: 1) many items do not rely on visual cues and therefore fail to effectively measure multimodal understanding; 2) many items are already close to performance saturation for current LVLMs, which limits their discriminative power; 3) a small number of anomalous items affect the reliability of evaluation results. To this end, we propose MMGist, a curated benchmark that covers seven capability dimensions and contains 7,262 items. MMGist is constructed through a three-stage pipeline, which sequentially combines text-ablation filtering, cross-model saturation filtering, and anomaly detection filtering. We conduct extensive experiments on 27 leading LVLMs and compare MMGist with the raw pool of 23,250 items. The results show that MMGist preserves model rankings with high fidelity, with Spearman 𝜌 = 0.98 , while reducing evaluation items by 69% and improving cross-model discrimination by 78%. Further results indicate that Visual Logic remains a systematic weakness of current LVLMs, while knowledge-intensive dimensions such as Expert Knowledge dimensions remain important factors for distinguishing closed-source models from open-source models. These findings suggest that high-quality evaluation should prioritize visual dependency, discriminative power, and reliability, rather than simply pursuing benchmark scale. The data can be found at https://huggingface.co/datasets/Winston-Yuan/MMGist .

Abstract:
The growth of agent skills has transformed how agentic systems are built, evaluated, and deployed. As skill libraries continue to scale, rigorous evaluation becomes critical to ensuring their utility, quality, and safety in real-world applications. Consequently, the field is undergoing an emerging paradigm shift from isolated skill creation to automated, evaluation-driven skill evolution. In this survey, we systematically examine the landscape of skill evolution and evaluation beyond foundational skill creation. We categorize evolution into four distinct paradigms, spanning execution feedback, trajectory distillation, compression, and reinforcement learning, showing how each element contributes to improving skill utility and reliability. We also provide an analysis of six skill-centric benchmark categories, identifying structural gaps in benchmark coverage, trade-offs, and metric richness to advance skill research. Finally, we identify open directions for building skill ecosystems that are generalizable, efficient, and verifiably safe. The project URL is https://github.com/Cassie07/AgentSkill_Survey

Abstract:
LLM tool agents can be improved without retraining by modifying the runtime harness around a fixed model: prompts, tool interfaces, middleware, state handling, and recovery logic. We study this setting as resource-bounded harness selection for fixed-model multi-turn tool agents, with the search surface scoped to prompts and tool-boundary middleware: edits are guarded intercepts at the tool boundary, not arbitrary rewriting of agent execution logic. Our optimizer-agnostic protocol reports mean held-out lift, worst-condition lift, repeatability, logged cost diagnostics, and RelLift95(B), a conservative estimate of the held-out gain of the harness selected under budget B. We instantiate the protocol with prompt-only and prompt-plus-middleware optimizers, including PRISM, which clusters failures and routes repairs to prompt, tool-boundary middleware, or joint edit surfaces within a Pareto search. On BFCL multi-round, tau2-Retail, and tau2-Telecom, PRISM obtains mean held-out lifts of 14.2, 14.9, and 10.1 percentage points and positive empirical RelLift95 on all three benchmarks, and a component ablation attributes the margin chiefly to failure-surface routing and the edit-pattern constraint. Across optimizers, the results show that some search procedures can occasionally find large gains but still choose brittle updates, so the reliability of the chosen harness should be reported alongside average held-out lift.

Abstract:
A century of psychology has found that the trait words people use to describe one another vary, but the relational structure among those traits, which ones go together and which oppose, is strikingly consistent across raters and cultures. We test whether the LLM (Qwen 2.5-7B-Instruct) reproduces this structure in its internal trait representations. From millions of crowd-sourced personality ratings of fictional characters, we build a human implicit-personality matrix over hundreds of traits; from contrastive model activations, we build a matching matrix over the same traits. The two relational structures align strongly (Mantel r = 0.77 ), and the agreement holds trait by trait as well as in aggregate. Two dominant axes of the model’s trait representations recover the social and intellectual dimensions long known to organize human personality impressions, social warmth and intellectual competence. On held-out dialogue, projecting model activations onto these directions yields personality profiles that agree with human ratings. This work establishes a framework that enables comprehensive, human-grounded comparison between internal model trait geometry and the shared structure of human personality impressions.

Abstract:
Recent advancements in Vision-Language Models (VLMs) have demonstrated impressive capabilities in static visual recognition and high-level semantic reasoning. However, current embodied exploration paradigms still heavily rely on imitation learning from human-annotated trajectories, which severely limits agents’ generalization ability. The key bottleneck of realizing general autonomous embodied agents lies in Generalizable Visually Grounded Exploration—the ability to operate novel devices without manuals or specific training by actively grounding abstract world knowledge into fine-grained visual affordances. Yet, existing benchmarks fail to evaluate this capability: they generally rely on explicit documents and annotated trajectories, neglecting the dynamic Hypothesis-Interaction-Refinement process essential for functional device operation. To bridge this gap, we introduce VGEBench, a comprehensive benchmark designed to evaluate the generalizable visually grounded exploration capabilities of VLMs. Unlike static datasets, we construct a Logic-Driven State Machine framework. This framework simulates multi-turn interaction loops, compelling agents to achieve goals by active visual perception and feedback-driven correction. Experimental results demonstrate that existing VLMs face significant challenges in translating semantic knowledge into physical execution and maintaining long-horizon state tracking.

Abstract:
Binary decompilation, which aims to recover source code from compiled binaries, can be viewed as a low-resource and high-constraint form of neural machine translation. However, existing decompilation techniques often fail to produce source code that can be successfully recompiled and re-executed, particularly for optimized binaries. Recent advances in large language models (LLMs) have enabled neural approaches to decompilation, but the generated code is typically only semantically plausible rather than truly executable, limiting their practical reliability. These shortcomings arise from compiler optimizations and the loss of semantic cues in compiled code, which LLMs struggle to recover without contextual guidance. To address this challenge, we propose ICL4Decomp, a hybrid decompilation framework that leverages in-context learning (ICL) to guide LLMs toward generating re-executable source code. Notably, our approach is model-agnostic and requires no fine-tuning, enabling plug-and-play use with off-the-shelf LLMs. We evaluate ICL4Decomp across multiple datasets, optimization levels, and compilers, demonstrating around 40% improvement in re-executability over state-of-the-art decompilation methods while maintaining robustness.

Abstract:
Training data attribution (TDA) for large language models is dominated by gradient- and retraining-based methods that require model internals, ruling out the most widely deployed closed-weight systems. We study TDA in the strictly black-box setting, observing only input output behavior accessed through retrieval- augmented prompting. We introduce Context Mixture Factorization (CMF), which models the LLM’s output distribution as a mixture of a base component and dataset-specific components and recovers per-dataset contribution weights by regularized matrix factorization via alternating projected least squares. CMF explicitly decomposes background (π0) and dataset- specific influence and requires only K+1 prompts per query for K candidate datasets. As a comparison method we also formulate the Shapley Context Method (SCM), a residual- similarity attribution score aggregated via Shapley values. We evaluate four instruction-tuned LLMs (GPT-4, Bloomz, Mistral-7B, Phi-3mini) on BoolQ, a synthetic FakeQ variant, and a post-cutoff Olympic2024 dataset against three baselines (TRAK, ContextCite, retrieval cosine similarity). CMF and SCM recover the ordering predicted by our exposure-proxy setup (BoolQ>FakeQ>Olympic2024); ContextCite shows the opposite ordering, attributing most to Olympic2024 because the model relies on context most when it has no parametric knowledge — supporting the claim that span-level context attribution and dataset-level exposure attribution answer different questions. A synthetic identifiability experiment, controlled fine- tuning sweeps, and a multi-checkpoint unlearning case study provide supporting evidence that the metrics behave as exposure-sensitive source-ranking proxies.

Abstract:
Text-to-image (T2I) models consistently collapse to stereotypical attribute values when prompts leave concepts underspecified, a phenomenon we term default behaviors . We conduct a large-scale empirical study using , an LLM-driven measurement approach that quantifies diversity at the level of individual concept attributes. We find pervasive default behaviors across all evaluated T2I models, with stronger models tending to exhibit less diversity. We trace these defaults to reporting bias in training captions: when captions omit common attributes, the accompanying images are dominated by a narrow set of values, and model outputs closely mirror this skew. We confirm this link causally with controlled fine-tuning interventions. As a proof of concept, we show that making sampled attribute values explicit broadens outputs, with some loss of faithfulness to secondary prompt details. Together, our results recast low diversity not as an inherent limitation of T2I models, but as a measurable and steerable consequence of what their training captions leave unsaid. Code and data available at: https://royira.github.io/GRADE.

Abstract:
Social understanding abilities are crucial for multimodal large language models (MLLMs) to interpret human social interactions. We introduce Social Caption, a framework grounded in interaction theory to evaluate social understanding abilities of MLLMs along three dimensions: Social Inference (SI), the ability to make accurate inferences about interactions; Holistic Social Analysis (HSA), the ability to generate comprehensive descriptions of interactions; Directed Social Analysis (DSA), the ability to generate relevant information from interactions. We analyze factors influencing model performance in social understanding, such as scale, architectural design, and spoken context. Experiments with MLLM judges demonstrate a path towards scaling automated evaluation of multimodal social understanding.

Abstract:
As large language models (LLMs) are increasingly deployed in real-world systems, red-teaming has become a critical tool for identifying safety vulnerabilities. However, existing approaches remain largely stateless and inefficient, repeatedly rediscovering similar failure modes for each new model. In this work, we propose a data-driven paradigm for red-teaming that enables the accumulation and reuse of adversarial knowledge across models. We introduce a framework for generating and collecting adversarial attacks enriched with structured metadata, including attack strategies, risk categories, and diagnostic feedback signals. Leveraging this resource, we demonstrate that metadata-aware retrieval significantly improves the efficiency of discovering successful attacks on unseen models, reducing reliance on purely generative search. To support further research, we release a large-scale dataset comprising >188k successful attacks across 8 models, together with rich metadata annotations.

Abstract:
Reliable evaluation of vision-language models (VLMs) and medical vision-language models (Medical-VLMs) requires calibrated confidence, particularly under realistic clinical conditions. However, existing efforts mainly focused on improving accuracy, leaving calibration in the medical domain underexplored. To this end, we propose MVC-Bench, a calibration-centric benchmark for medical image classification with VLMs and Medical-VLMs. MVC-Bench assesses the calibration across three axes: (i) robustness to modality, backbone, and domain shift (ii) effectiveness of calibration strategies and prompt-tuning methods (iii) stability under prompt-template and random-seed variations. The benchmark covers eight different backbones, three medical modalities, including fundus imaging, histopathology, and chest X-ray under in-domain and domain shift settings. It compares post-hoc calibration, train-time calibration, and zero-shot inference methods, together with six prompt-tuning methods. Across more than 1638 controlled experiments, we report accuracy and Expected Calibration Error (ECE) as primary metrics, and further report results with complementary calibration measures, including Maximum Calibration Error (MCE) and Adaptive Calibration Error (ACE). We further investigate the underlying causes of miscalibration in VLMs and Medical-VLMs and propose a simple train-time calibration method, Multi-Class Margin (MCM) regularization, which achieves lowest ECE on 10 out of 12 settings in in-domain and remains competitive under domain shifts. Collectively, MVC-Bench provides a structured evaluation framework and actionable guidance for improving calibration in safety-critical medical workflows.

Abstract:
Reinforcement Learning (RL) has emerged as a dominant paradigm for eliciting long-horizon reasoning in Large Language Models (LLMs). However, scaling Tool-Integrated Reasoning (TIR) via RL remains challenging due to interaction collapse: a pathological state where models fail to sustain multi-turn tool usage, instead degenerating into heavy internal reasoning with only trivial, post-hoc code verification. We systematically study three questions: (i) how cold-start Supervised Fine-Tuning (SFT) induces an agentic, tool-using behavioral prior, (ii) how teacher-trajectory structure and interaction density shape exploration and downstream RL outcomes, and (iii) how the RL interaction budget affects learning dynamics and generalization under varying inference-time budgets. We then introduce ASTER (Agentic Scaling with Tool-integrated Extended Reasoning), a framework that circumvents this collapse through a targeted cold-start strategy prioritizing structured, interaction-dense trajectories. We find that a small expert cold-start set of just 4K structurally rich, interaction-dense trajectories yields the strongest downstream performance, establishing a robust prior that enables superior exploration during extended RL training. Extensive evaluations demonstrate that ASTER-4B achieves strong results on competitive mathematical benchmarks, reaching 89.6% mean accuracy on AIME2025 and remaining competitive with leading frontier open-source models. See more details at https://github.com/CodeByLeo-bit/ASTER.

Abstract:
Real-world tables often contain schema inconsistencies, heterogeneous value formats, and implicit relational structures that degrade table reasoning and question answering. Existing approaches address these issues at query time, repeating normalization and restructuring for every new question and producing representations that do not generalize to unseen queries. We introduce QuIeTT, a transform-first, query-later framework that converts each raw table into a single SQL-ready canonical representation before any evaluation query is observed, under three constraints: no evaluation query access, information preservation, and single-table reuse. QuIeTT operates through a three-stage pipeline: issue probing, where synthetic queries surface structural deficiencies such as ambiguous schemas and heterogeneous formats; plan generation; and structured execution. Experiments on four benchmarks across five model families show consistent improvements over strong baselines in both direct prompting and agentic reasoning paradigms, with particularly strong gains on a manually annotated challenge set of structurally diverse unseen questions.

Abstract:
Multilingual Large Language Models (MLLMs) are increasingly expected to handle Code-Switched (CS) inputs, yet mixing languages frequently degrades performance relative to source- or target-language monolingual counterparts. To understand this degradation, we use grammar-forced CS as a controlled diagnostic setting for locating CS representations relative to their source and target counterparts. We introduce Anchor Bias, a geometric measure that quantifies language anchoring—whether a CS hidden state aligns closer to its source or target language counterpart. Across diverse MLLMs, Anchor Bias reveals a consistent grammar-frame effect: source-framed CS stays source-anchored, whereas target-framed CS shifts target-ward and shows larger Question Answering (QA) degradation. Motivated by this representational pattern, we propose CANVAS (Contextual Anchor-based Neural Vector Alignment Steering), an inference-time intervention that extracts a source-side canvas from the input and softly steers target-language hidden states toward the source anchor during prefill. CANVAS consistently recovers QA F1 across MLLMs and CS conditions, showing that internal anchoring signals provide an actionable target for mitigating CS inference failures.

Abstract:
Information retrieval (IR) benchmarks typically follow the Cranfield paradigm, relying on static and predefined corpora. However, temporal changes in technical corpora, such as API deprecations and code reorganizations, can render existing benchmarks stale. In our work, we investigate how temporal corpus drift affects FreshStack, a retrieval benchmark focused on technical domains. We examine two independent corpus snapshots of FreshStack from October 2024 and October 2025 to answer queries about LangChain. Our analysis shows that all but one query posed in 2024 remains fully supported by the 2025 corpus, as relevant documents migrate from LangChain to competitor repositories, such as LlamaIndex. Next, we compare the accuracy of retrieval models on both snapshots and observe only minor shifts in model rankings, with overall strong correlation of up to 0.978 Kendall 𝜏 at Recall@50. These results suggest that retrieval benchmarks for technical documentation, e.g., LangChain, can remain reliable when re-judged in the drift regime we observe, where relevant evidence is relocated rather than invalidated. All resources (code and data) are publicly available at https://github.com/fresh-stack/driftbench.

Abstract:
Identifying personality accurately can improve interpretability and adaptability for human–computer interaction and psychological assessment. Recent studies have introduced evidence utterance into large language models (LLMs) for personality recognition, providing supporting evidence to explain personality labels. However, the mixed background and social expressions in dialogue scenarios interfere with the identification of evidence utterance. At the same time, the overconfidence of LLMs leads to biased personality inference. Inspired by the lens theory in psychology, how to systematically explore the mutual influence between evidence utterance identification and personality recognition while improving the performance of both tasks remains a key challenge. In this paper, we reframe personality recognition as an Evidence-Personality Iterative Reasoning problem (EPIR). A graph model is employed to select an initial evidence utterance set, and personality recognition is performed based on a Generative-Discriminative Feedback mechanism. Then, the inferred results are fed back to the graph mapping network to dynamically adjust evidence utterance filtering. This interactive iterative process continues until the evidence and personality results across different iterations converge. Experimental results on the two public datasets demonstrate the effectiveness of the proposed method.

Abstract:
Pretrained transformers spend a large share of their inference compute inside self-attention, yet the per-head query-key interaction is structurally low-rank by construction. We ask a diagnostic question: across layers and architectures, which heads actually use the full rank budget on real data, and which do not? We propose SVD-Attention, a post-hoc method that decomposes the per-head interaction matrix M h and scores every singular mode by both its spectral magnitude and its activation variance across tokens on a small calibration set. The score keeps the modes the head can amplify and that real data actually activates, and discards the rest; the resulting factorization is exact for the retained modes and needs no finetuning. Across seven pretrained models and five task types, the diagnostic finds a task-dependent pattern: redundancy sits in the early to middle layers for sentiment and image classification and for generation, and in the late layers for dense prediction and extractive question answering. It is also predictive: truncating the layers it flags costs at most 1.75 F1, while truncating the same number of layers elsewhere in the same model costs up to 50 F1. SVD-Attention is an analysis tool rather than a compression method: acting on it reduces inference FLOPs by 0.8-1.5% on its own and by 8-11% when composed with weight pruning, with 0-4% change in task quality. We release the code at https://github.com/riksanyal/SVD-Attention.

Abstract:
Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theoretically show that generating fine-grained responses poses a reliability challenge, i.e., fine-grained generation is more error-prone than coarse-grained generation. This phenomenon suggests that models should generate the finest description that remains reliable rather than simply produce more specific outputs. To investigate this problem, we develop GranFact, a granularity-aware benchmark consisting of expert-verified multi-object images with coarse-to-fine category annotations. Then, we design a hierarchy-aware evaluation algorithm, which assesses both whether model predictions are visually correct and how specific the correct predictions are. We also propose a reliability-prioritized preference optimization method based on Direct Preference Optimization (DPO), which penalizes unreliable fine-grained claims while rewarding reliable specificity. Experiments on GranFact show that our method improves fine-grained generation while preserving reliability. Code and data are available at https://github.com/WeiWu2025/GranFact.

Abstract:
Existing jailbreak evaluations typically characterize robustness using a single attack success rate (ASR), implicitly assuming that the target model operates in its default configuration (the vanilla state). However, user–LLM interactions naturally induce diverse operational states beyond the vanilla state. In this work, we find that jailbreak robustness is highly fragile to operational-state variation: even when the attack remains fixed, changing only an ordinary, safety-irrelevant system prompt can dramatically alter attack success rates. We systematically investigate this phenomenon across seven aligned models and three representative jailbreak attacks, finding that in 18 of 21 model–attack combinations, at least one non-vanilla state yields a higher ASR than the vanilla state. In some case, ASR increases by up to 56 percentage points (2%→58%) solely due to a change in operational state. Remarkably, these gains emerge on top of attacks originally designed and optimized under vanilla-state evaluation. We further provide evidence that operational-state variation systematically shifts hidden representations along a refusal-related axis and that these shifts strongly predict jailbreak outcomes. Our results reveal that vanilla-state, single-point evaluations may overlook substantial robustness variation induced by operational states, motivating robustness evaluation that considers stability across operational states.

Abstract:
Generating Scalable Vector Graphics (SVG) code from natural-language instructions is an open-ended task without absolute visual ground truth, leaving both evaluation and policy optimization without a faithful signal. Scalar metrics (CLIP, Aesthetic) calibrated on natural images transfer poorly to stylized vector content, and reusing them as RL rewards triggers reward hacking. We address both limitations with rubric-based scoring. We first establish empirically that prompting a vision–language judge with a multi-axis rubric correlates with human judgments far better than scalar metrics, both across samples and within instructions. Building on this finding, we introduce RULER (Instance-aware Rubric Rewards for Reinforcement Learning), which converts each instruction into an instance-aware rubric of six items spanning semantic, visual, and stylistic axes; a judge VLM scores rendered rollouts item-by-item, and the weighted satisfactions form a fine-grained reward optimized via Group Relative Policy Optimization. Because the rubric is derived from text alone, RULER requires neither paired SVG ground truth nor human preference labels. On MMSVG-Illustration and MMSVG-Icon, RULER lifts the rubric score from 0.432/0.395 to 0.693/0.683 , surpassing dedicated SVG specialists and matching the substantially larger DeepSeek-V3, with ablations identifying rubric design as the active lever for RL on open-ended SVG generation.

Abstract:
Mixture-of-Experts (MoE) large language models decouple capacity from compute through sparse routing, but their large parameter count creates storage and serving challenges. We analyze three MoE compression families: expert pruning, expert merging, and weight reconstruction, and derive structural error bounds showing that pruning and merging can incur non-vanishing errors tied to routing and expert heterogeneity. In contrast, weight reconstruction avoids these structural costs by preserving expert structure and routing. Motivated by the analysis, we propose Shared Low-rank Basis Factorization (SLBF), a data-free weight reconstruction method that uses rank- k bases shared among experts, enabling richer cross-expert sharing, faster convergence, and lower reconstruction error. A post-hoc gauge fixing removes redundant parameters at no representational cost. Across five MoE architectures spanning 16B to 122B parameters, SLBF consistently outperforms methods from all three compression families.

Abstract:
Document image tampering poses a serious threat to information authenticity in security-critical documents. Existing document tampering detection methods mainly rely on RGB-domain and frequency-domain artifacts, following a paradigm similar to natural image forensics, and often overlook a key property of document images: dense and semantically rich textual content. In this work, we propose TSPG, a plug-and-play Textual Semantic Prior Guidance module for document image tampering detection. TSPG uses a pretrained language model with a lightweight token classification head to estimate token-level tampering-aware semantic scores from OCR-recognized text. It then projects these scores back to the image space through OCR bounding boxes to construct a spatially aligned Textual Semantic Prior (TSP) map. To inject aforementioned prior into existing detectors, we design a Gated Semantic Residual Modulation (GSRM) module that adaptively modulates multi-scale detector features through residual gated guidance. TSPG can be inserted into existing document tampering detectors without modifying their original backbone or decoder. Experiments on DocTamper with two representative baselines, DTD and FFDN, show that TSPG consistently improves both models, demonstrating the complementary value of language-derived textual semantics for document tampering localization.

Abstract:
Budget-constrained agentic search arises when an LLM agent must refine candidates under a small evaluation budget, because validation is expensive, generation requires multiple model calls, or both. In this regime, standard MCTS allocates budget poorly: exploration bonuses dominate at low visit counts, unpromising siblings are expanded before promising chains can deepen, and branching is independent of node quality. We introduce ExTS, a tree-search policy that treats expansion itself as a value-of-information decision. ExTS combines three mechanisms: discriminative reward shaping to separate candidates under narrow score distributions, a stochastic virtual child that estimates the value of creating a new branch from the parent’s reward history, and quality-conditioned branching that expands only when a node’s score justifies the budget cost. Across prompt optimization, code generation, molecular structure elucidation, and agentic workflow optimization, ExTS is competitive with or improves over task-specific tree-search baselines, with an average relative gain of +5.5% using a single fixed configuration. We further introduce pilot-run diagnostics that characterize what makes budget-constrained agentic search problems structurally different from one another, providing both understanding of the problem space and practical guidance for adaptation.

Abstract:
Existing API-only prompt optimisers are often described as interpretable, but in practice, this usually means only post-hoc inspectability: each iteration still rewrites the prompt as one opaque string, leaving a trace of full-prompt diffs rather than localisable, machine-readable edits. This paper introduces SEPO (Structural, Evidence-grounded Prompt Optimization), a multi-trajectory prompt optimiser centred on edit-effect lineage feedback. Rather than treating each iteration as an isolated whole-prompt rewrite, SEPO locally edits stable, typed units in a two-layer prompt schema, links the target and realised structural operations of each edit to the examples it newly fixes or breaks, and carries this edit–effect record forward to guide later architect calls on the same search branch. This makes prompt optimisation addressable, attributable, and actionable. Across a 14-task held-out suite, SEPO improves over the strongest baseline, GEPA, by 3.1 pp on Llama-3.1-8B-Instruct and 2.2 pp on Qwen3-8B, reaching 61.9% and 73.3% macro accuracy. SEPO also lies on both the optimisation-time and test-time Pareto frontiers, spending 2.9M optimisation tokens versus 4.1M for GEPA and producing prompts over 5× shorter.

Abstract:
Long-term memory systems increasingly organize interaction histories into hierarchical contexts, where coarse summaries, mid-level events, and fine-grained evidence provide complementary signals for future queries. Yet after retrieval, these signals are often collapsed into a single generation context and used implicitly. We ask how retrieved hierarchical memory should shape generation after access, framing it as a post-retrieval calibration problem. We propose HiR, a training-free inference-time framework that contrasts full-hierarchy decoding with a coarse-only reference view, preserving the complete hierarchy while tempering over-reliance on broad abstractions. Extensive experiments and analyses show that HiR enables more effective and robust use of hierarchical memory without changing memory construction or retrieval; more broadly, our results suggest that memory research should consider not only what to store or retrieve, but also how retrieved information shapes generation.

Abstract:
Recent advances in reasoning and tool-calling capabilities of large language models (LLMs) have enabled increasingly capable agentic systems. However, existing benchmarks remain limited in task complexity, realism, and domain diversity, and often fail to capture interactions that span multiple domains, limiting their ability to evaluate agents in realistic multi-step settings that require sustained reasoning and coordination. To address these limitations, we introduce T1-Bench, a high-fidelity, comprehensive benchmark for evaluating agentic systems in realistic customer-facing, multi-domain environments, featuring interleaved scenarios that require structured reasoning across multi-turn user-assistant interactions and substantially increasing both compositional complexity and evaluative rigor across 25 domains of varying difficulty. We evaluate T1-Bench using 12 proprietary and open-weight models, providing a reproducible and standardized framework for assessing agent behavior, tool utilization, and conversational quality in complex, multi-step environments. We further complement automatic evaluation with human judgments to strengthen the assessment of qualitative performance. Overall, T1-Bench substantially advances prior benchmarks by increasing task complexity, interaction depth, and domain coverage in simulated multi-domain environments. To facilitate future research on agentic systems, we will publicly release data and evaluation code as open source.

Abstract:
Few-step flow-map guidance provides an efficient way to steer generation without retraining, but its updates are defined in a continuous or relaxed state space. In categorical text generation, however, the final output is determined by simplex-constrained token distributions and hard decoding. Under low NFE, where each update induces a relatively large finite displacement, this continuous-to-discrete mismatch can make terminal reward guidance unreliable, causing overshoot, reward–quality conflict, saturation, or poor transfer from relaxed reward gains to decoded tokens. We present Reliability-Calibrated Flow Guidance (RCFG), a sampling-time reliability-calibration framework for terminal reward pullback in few-step categorical flow maps. Rather than replacing the underlying guidance primitive, RCFG calibrates its magnitude and direction according to transition scale, reward saturation, local reward–quality conflict, and categorical endpoint structure. Experiments on LM1B and Text8 show more favorable aggregate control–likelihood operating points after calibration: on LM1B, RCFG maintains or improves verifier Reward and Target while reducing NLL and PPL relative to terminal reward-pullback guidance, while Text8 shows a modest aggregate improvement in lexical-control balance. Code and reproduction instructions are available at: https://github.com/ZionLau/RCFG

Abstract:
We introduce grounded glossary generation, a structured task requiring models to recover semantically meaningful Sanskrit phrases and produce translation-grounded meanings from a śloka-translation pair, formalizing the traditional pāṭha-commentary practice as an evaluable NLP objective. We construct a benchmark of 31,316 śloka-translation-glossary triples from the Vālmīki Rāmāyaṇa and Śrīmad Bhāgavatam, paired with two metrics: Jaccard for phrase recovery and Meaning Faithfulness for semantic consistency. Across zero-shot, few-shot, and instruction fine-tuned variants of Gemma-3n-E4B, Gemma-3-12B, Phi-4, and Qwen3.5-9B, instruction fine-tuning substantially outperforms prompting, while explicit segmentation yields gains. Error analysis identifies over-segmentation of sandhi and samāsa compounds as the dominant failure mode, pointing to morphological modeling as the key bottleneck for faithful Sanskrit lexical decomposition.

Abstract:
Reliable medical reasoning with large language models (LLMs) is hindered by a practical trade-off between accuracy and deployment cost: larger models often perform strongly but are expensive to host, while compact open-source medical LLMs can fail on difficult cases. We propose MEDitor, a training-free, plug-and-play multi-agent routing protocol that improves accuracy and traceability by selectively escalating disagreement cases. MEDitor first queries two lightweight medical experts and terminates early upon agreement. For disagreements, it escalates to an evidence-grounded coordinator that constructs option-aligned evidence, followed by a candidate-constrained judge that selects between the competing candidates or abstains. A deterministic fallback guarantees reproducibility. Across five English-language medical multiple-choice QA (MCQA) benchmarks, MEDitor achieves 65.94% average accuracy, competitive with representative ∼ 70B open-source models on the same evaluation suite. These results show that lightweight expert orchestration can provide a parameter-efficient and auditable route to competitive medical MCQA accuracy by reserving evidence-grounded arbitration for contested cases.

Abstract:
Personalized news headline generation has emerged as a promising avenue for optimizing user engagement and satisfaction in online news consumption. However, prevailing methods predominantly adopt an indirect modeling paradigm, inferring user preferences from related tasks rather than capturing their direct headline preferences. Direct modeling remains underexplored due to two primary challenges: (1) the scarcity of large-scale personalized headline datasets and (2) the risk of factual inaccuracies when models overly cater to subjective user preferences. To address these challenges, we propose USG, a novel multi-agent user simulator headline generation framework. USG introduces three specialized agents: the User provides subjective feedback to guide headline personalization; the Editor enforces editorial standards (i.e., factual consistency and no clickbait); and the Writer iteratively refines candidates via a Critique-and-Refine algorithm guided by feedback from the User and Editor agents. The multi-agent interactions yield optimal personalized headlines and reasoning chains distilled from the interaction trajectories, forming a high-quality dataset. This dataset directly supervises the fine-tuning of a standalone generator. Extensive experiments demonstrate that USG outperforms the strongest baseline by nearly 6% on personalization, achieving state-of-the-art performance on both personalization and factual consistency, while enabling efficient end-to-end inference. Our code is available at https://github.com/Leah-HKBU/USG.

Abstract:
Multi-hop question answering requires composing multiple pieces of evidence to derive the correct answer, making it particularly vulnerable to hallucinated intermediate reasoning steps. A common failure mode arises when neural generators produce intermediate sentences that are fluent but logically inconsistent with the underlying facts, which can subsequently mislead downstream reasoning. In this paper, we propose ITERLOGIC, an iterative natural logic verification framework for multi-hop question answering, which explicitly integrates symbolic verification into the generation process. Specifically, a neural generator composes candidate intermediate sentences from retrieved facts, while a natural logic verifier evaluates their logical consistency with the source evidence. Low-scoring candidates are iteratively rejected and regenerated, allowing the system to progressively eliminate hallucinated compositions. This lightweight interaction upgrades natural logic from a passive scoring component to an active verification signal that guides intermediate sentence generation. Experiments on QASC, WorldTree, and WikiHop demonstrate that ITERLOGIC improves answer accuracy, highlighting the effectiveness of iterative verification for reliable multi-hop reasoning.

Abstract:
Despite expanding their multilingual coverage, the advanced reasoning capabilities of LLMs remain largely confined to a few high-resource languages like English. To address this, we propose an unsupervised Reinforcement Learning (RL) approach to enhance multilingual reasoning by enforcing cross-lingual self-consistency—the principle that a model should produce the same final answer for equivalent problems in different languages. Existing methods are limited by the scarcity of multilingual reasoning data, and show weak generalization to unseen languages. Our approach requires neither gold answers nor parallel data, and it achieves average gains of up to 21.7% on MGSM across 10 languages. In addition, our method demonstrates strong generalization, with an 18.2% mean improvement on MGSM languages unseen during training, and up to 6.2% gain on 3 out-of-distribution benchmarks. These results show the potential of consistency-based methods to improve the multilingual capabilities of LLMs without requiring supervised data.

Abstract:
This paper replicates and extends the system used in the AuTexTification shared task for authorship attribution of machine-generated texts. Exact replication was not possible because of differences in data splits, model availability, and implementation details, which we document as a case study in reproducibility. We tested newer multilingual language models (mDeBERTa-v3-base, Qwen, mGPT) and added 26 document-level stylometric features, using ablation, permutation importance, and SHAP analysis to assess feature influence. A single shared configuration was applied to both English and Spanish across Subtask 1 and Subtask 2. Averaged over three random seeds, the shared multilingual configuration performs comparably to or better than the language-specific baseline, with the clearest gains on model attribution (Subtask 2). The additional stylometric features yield small improvements, led by lexical diversity, but their contribution falls within seed variance once predictability-based probabilities are included, which remain the dominant signal. The study also shows that clear documentation is important for reliable replication and fair comparison of systems.

Abstract:
Available LLMs differ substantially in cost, latency, and capability. Their strengths vary across queries, so the natural question becomes: which model for which query? This is the LLM routing problem. Routing is already used in production, and the academic literature has grown across cascading, bandit, generative, and benchmark threads, each with its own vocabulary, each largely ignoring the others. We argue the field is best read along a single axis: the algorithmic operation the router performs. We organise the recurring operations under four labels. Predictive routers score candidates before their answers are generated. Adaptive routers try a model, observe the outcome, and adjust. Generative routers let an LLM itself reason about the decision. Infrastructure work studies routing rather than performs it, via benchmarks and adversarial analysis. This survey formalises the routing decision under this four-paradigm lens, traces the methodological development of the field, compares assumptions and tradeoffs across paradigms, and identifies open research directions revealed by the taxonomy. A continuously updated reading list lives at https://github.com/wuxingyu-ai/LLM-Routing .

Abstract:
Memes are widely used not only as static artifacts but also as interactive replies in online conversations. While computational research has focused on analyzing intrinsic properties of memes, the dynamic and contextual use of memes to create humor remains largely unaddressed. We introduce the Meme Reply Selection task and present MaMe-Re, a benchmark of 100,000 human-annotated context–meme pairs built from openly licensed Japanese manga panels. Our analysis reveals three key findings: (1) large language models show modest advantages over embedding methods by capturing social cues such as irony and exaggeration beyond surface-level semantic matching; (2) visual information does not improve performance, revealing a gap between understanding visual content and effectively using it for contextual humor; (3) while models can match human judgments when the humorous option is clearly distinguishable, they struggle to distinguish subtle differences in wit among semantically similar candidates. These findings establish contextually humorous reply selection as an open challenge for current models.

Abstract:
Travel itinerary generation requires balancing strict spatio-temporal constraints with human preferences. Existing LLM-based planners mainly rely on structured attributes and predefined traveler personas, but real travel decisions are often shaped by reviews that reveal experiential factors such as comfort, safety, service quality, ambiance, crowding, and hidden risks absent from structured databases. Incorporating such review information is therefore critical to realistic, user-centric itinerary generation. We propose TRIPPULSE (Captures experiential signals from reviews beyond rigid hard constraints), a multi-agent framework for review-grounded travel planning. Instead of relying on a monolithic planner (and face context and reasoning bottlenecks), TRIPPULSE decomposes itinerary generation into specialized agents (each operating over localized contexts) for accommodations, transportation, meals, attractions, and events, coordinated through a global orchestrator with scheduling mechanisms that enforce temporal and budget feasibility. We augment TripCraft with 100K+ real-world reviews and introduce Review-Grounded Persona Alignment (RGPA), an LLM-as-a-Judge metric for evaluating alignment with human-centric travel experiences. Experiments across multiple trip durations and diverse proprietary and open-source models show that TRIPPULSE maintains strong constraint satisfaction while generating more personalized and experientially grounded itineraries.

Abstract:
Both privacy and factual accuracy are paramount in high-stakes domains like healthcare. Concerningly, we uncover and investigate a _privacy-hallucination tradeoff_ in differentially private (DP) language models. First, we empirically show that models pre-trained or fine-tuned with DP tend to produce more hallucinations than non-DP counterparts, with increased severity as the privacy budget grows stricter. Second, we investigate model properties driving this tradeoff, demonstrating that DP mechanisms flatten output distributions, potentially redistributing probability mass toward factually incorrect alternatives. Third, through experiments where we control fact frequency in training data, we characterize how information frequency can reduce hallucination risks in DP models. Overall, our findings underscore the need for more nuanced privacy-preserving interventions that offer rigorous privacy guarantees without compromising factual accuracy.

Abstract:
The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or "autoraters". However, their reliability is limited by a foundational issue: they are trained on discrete preference labels, forcing a single ground truth onto tasks that are often subjective, ambiguous, or nuanced. We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population. In this paper, we propose a general framework for calibrating probabilistic autoraters to any given preference distribution. We formalize the problem and present two learning methods tailored to different data conditions: 1) a direct supervised fine-tuning for dense, probabilistic labels, and 2) a reinforcement learning approach for sparse, binary labels. Our empirical results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution, with improved calibration and significantly lower positional bias, all while preserving performance on objective tasks.

Abstract:
LLM agents are predominantly benchmarked on short, single-task trajectories, yet real deployments run for hours under contention, surfacing a different class of failures. We use the Clinical Environment Simulator (CES), in which an agent manages an entire emergency-department shift under continuous time and resource pressure, as a testbed: long-horizon execution failures manifest measurably in a single rollout under structured, multi-dimensional grading. On CES, current agents reach the correct diagnosis in most cases yet fail to deliver complete and timely critical actions, revealing an execution gap. We attribute this gap to three long-horizon failure modes, each operationalized as a per-trace counter: instruction-adherence drift, treatment incompleteness, and a severity-equity gap in timeliness. We then introduce Asclepius, an adaptive agent scaffolding with a self-evolving harness that rewrites the operating manual between shifts from trace-level feedback, an externalized clinical skills library for high-stakes regimen knowledge, and three isolated subagents that partition per-turn decisions across the patient queue. On held-out batches never observed during harness evolution, Asclepius improves critical-action correctness by 22% (p = 0.024) over a strong baseline agent framework while preserving diagnostic accuracy, with consistent gains across five LLM judges from three model families; on the full ten-batch set, improvements reach 25% on critical actions and 13% on timeliness. The three failure modes form a coupled bottleneck: decisive reductions appear only when all three components act together.

Abstract:
Indirect prompt injection exploits a core tension in LLM systems: models must use retrieved documents, tool outputs, and agent messages that are useful but untrusted. Existing defenses are often trained and evaluated within a single presentation wrapper, implicitly assuming that robustness transfers across framings. We show that this assumption is unsafe: for a model trained to resist malicious follow-up messages, the same injected instruction succeeds only 18% of the time as a follow-up message, but 53% of the time when embedded in retrieved context. Thus, robustness is not payload-only; it is channel-conditioned. We address this with TRACER (Trace-Regularized Adversarial Channel Exposure with Reward-shaping), a channel-aware adversarial training algorithm that treats the wrapper as part of the optimization problem. TRACER combines channel-sampled adversarial exposure with two training components: an answer-anchored, evidence-gated composite reward for post-attack behavior, and ATRC, a trace-regularization loss that aligns attacked reasoning with clean anchor spans when injected evidence is invalid. On GSM8K with Qwen2.5-3B-Instruct, TRACER reduces RAG-framed ASR from 65.5% to 6.1%, while preserving 78–82% clean accuracy. Cross-framing experiments reveal a depth–breadth trade-off: RAG-specialized training gives the strongest native defense, tool-output training transfers most uniformly, and joint RAG+tool training achieves the best 3B average ASR of 13.9%. At 7B scale, a single epoch reaches 9.4% average ASR. TRACER substantially narrows the framing gap, highlighting the need to train and evaluate defenses across deployment channels.

Abstract:
Large Language Models fail at implicit multi-hop reasoning: a model answers "When was born?" and "Who is ’s closest friend?" correctly but fails on "When was ’s closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning. Our code is available online .

Abstract:
With the success of reasoning models across complex natural language tasks, researchers in the information retrieval (IR) community have begun exploring how similar reasoning capabilities can be integrated into passage rerankers built on large language models (LLMs). These methods typically employ an LLM to produce an explicit, step-by-step reasoning process before arriving at a final relevance prediction. But does reasoning actually improve pointwise reranking? We investigate this question through controlled comparisons of Rank1, a pointwise reasoning reranker, with two non-reasoning counterparts. Across in-domain and out-of-domain datasets, we find that Rank1 generally underperforms Rank0, an otherwise equivalent non-reasoning pointwise reranker trained using the same backbone LLM and underlying training data as Rank1. To isolate the contribution of Rank1’s inference-time reasoning, we then replace its generated reasoning process with a fixed dummy sequence. Surprisingly, this variant, Rank1-NoReason, can also outperform Rank1. Analysis of Rank1’s score distributions provides a likely explanation: once Rank1 deems a passage relevant, it assigns nearly identical relevance scores, limiting its ability to distinguish degrees of relevance and ultimately harming its ranking accuracy. We release all code at https://github.com/nourj98/dont-overthink-reranking.

Abstract:
Building speech emotion recognition (SER) models that generalize across languages remains challenging due to limited annotations, linguistic diversity and heterogeneous label spaces. Current multilingual SER approaches either rely on fully shared models or restrict training to a small set of overlapping emotions that restrict diversity in expressiveness. We propose EMBRACE, a multilingual speech representation for language-agnostic SER. Drawing inspiration from partial-label learning paradigms in computer vision, EMBRACE adapts a unified superset emotion modeling strategy. The approach enables training on mixtures of datasets with non-overlapping emotion categories by selectively masking absent categories during optimization. To balance this complex multi-domain training, the model leverages a knowledge distillation based progressive training procedure that stabilizes multilingual optimization and mitigates catastrophic forgetting. Experiments across seven languages show that EMBRACE improves cross-corpus performance by 31.8% and in-corpus by 15.2% over strong emotional speech representations, confirming its ability to learn discriminative and transferable affective representations.

Abstract:
The widespread use of large language models (LLMs) raises an important question: how do texts evolve when they are repeatedly processed by LLMs? In this paper, we define this iterative inference process as Markovian generation chains, where each step takes a specific prompt template and the previous output as input, without including any prior data. In iterative rephrasing and round-trip translation experiments, the output either converges to a small recurrent set or continues to produce novel sentences within a finite horizon. Through sentence-level Markov chain modeling and analysis of simulated data, we show that iterative process can either increase or reduce the diversity depending on factors such as the temperature parameter and the initial input sentence. These results offer valuable insights into the dynamics of iterative LLM inference and their implications for multi-agent LLM systems.

Abstract:
Text only partially determines prosodic realization, leaving residual variation that may align with speaker-level metadata such as regional accent or first-language (L1) background. However, because these labels are assigned to speakers, their predictive gains may overlap with broader speaker-associated structure rather than uniquely reflect category-level prosodic effects. We frame this as an attribution problem and propose a speaker-controlled audit using matched text-residual prosody predictors that differ only in access to metadata labels and speaker identity. The audit measures label utility beyond text, speaker utility beyond text, and the additional empirical utility of the label when speaker conditioning is already available. Across VCTK regional-accent labels and L2-ARCTIC L1 labels, metadata yields modest gains over text, while speaker conditioning yields larger gains and leaves little additional label utility under speaker control. Random speaker-group and label-perturbation controls further characterize the specificity and use of the supplied metadata. Importantly, speaker-held-out evaluation shows that metadata transfer to unseen speakers is limited and target-dependent, indicating that observed-speaker metadata gains do not by themselves establish speaker-independent category-level effects. These results position speaker-controlled auditing as a prerequisite attribution check before assigning category-level interpretations to metadata-conditioned predictive gains.

Abstract:
Legal systems often employ legislative drafting practices in which higher-level laws delegate detailed regulations to lower-level laws. In particular, when a single provision delegates authority to multiple lower-level laws, readers must cross-reference many related provisions scattered across the entire bodies of law, imposing a substantial cognitive burden. Reflecting this real-world problem, this work proposes a novel multi-label delegation target identification task to identify all the delegated provisions corresponding to an input provision, with the focus on Japanese law. We construct pipeline systems combining a candidate retriever and a candidate classifier, using language models with diverse architectures. Experimental results show that adopting a cross-encoder-based binary classifier as a candidate classifier improves performance at finer granularities, establishing a solid baseline for the task. Furthermore, our comprehensive analyses reveal the remaining challenges in achieving sufficient performance on instances with many delegated provisions and identify typical error types, providing a foundation for future improvement.

Abstract:
Speculative decoding accelerates autoregressive language model inference by verifying multiple draft tokens in parallel. However, the verification stage often becomes the dominant computational bottleneck, especially for long-context inputs and mixture-of-experts (MoE) models. Existing sparsification methods are designed primarily for token-by-token autoregressive decoding to remove substantial computational redundancy in LLMs. This work systematically adopts different sparse methods on the verification stage of the speculative decoding and identifies structured redundancy across multiple dimensions. Based on these observations, we propose a sparse verification framework that jointly sparsifies attention, FFN, and MoE components during the verification stage to reduce the dominant computation cost. The framework further incorporates an inter-draft token and inter-layer retrieval reuse strategy to further reduce redundant computation without introducing additional training. Extensive experiments across summarization, question answering, and mathematical reasoning datasets demonstrate that the proposed methods achieve favorable efficiency-accuracy trade-offs and can be translated into practical latency reduction with optimized sparse kernels.

Abstract:
The narrative composition of web-scale LLM pretraining corpora remains largely unexplored, even though narrative is a fundamental mode of human communication. We present the first fine-grained study of narrative features in Dolma, a 3-trillion-token open pretraining corpus. Drawing on narrative theory, we design a framework spanning three core narrative elements (agency, setting, and events) operationalized as 11 interpretable dimensions. After curating and hand-annotating a diverse set of 400 passages, we create an LLM-labeled dataset of 25K passages, and finally, we finetune and validate NarraBERT, two RoBERTa-based models for fine-grained narrative prediction. We apply NarraBERT to ∼ 13M passages, resulting in a new dataset, NarraDolma. We find that narrative structure is measurable at scale across extremely heterogeneous data and narrative qualities are unequally distributed across pretraining sources, topics, and formats in ways that current data curation practices neither measure nor account for. Our framework, dataset, and analyses provide a foundation for understanding how narrative qualities are distributed in LLM pretraining data.

Abstract:
Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretrained LLMs. However, they rely on offline automatic speech recognition (ASR), leading to two key limitations: (i) the need for complete utterances before tokenization, precluding real-time streaming, and (ii) vocabulary mismatch between ASR and LLMs, which reduces acoustic granularity from the subword to the word level. We introduce StreamAlign, a text-aligned speech tokenization framework that enables streaming tokenization for real-time speech–text joint modeling. StreamAlign performs online speech–text alignment by combining character-level RNN-Transducer alignment with word-level ASR guidance, mitigating ASR–LLM vocabulary mismatch while preserving recognition accuracy. A proactive word boundary classifier anticipates word completion at chunk boundaries, reducing tokenization latency from 560ms to 270 ms. On LibriSpeech, StreamAlign achieves the lowest WER and highest UTMOS among evaluated tokenizers. Furthermore, StreamAlign-SLM, a spoken language model trained on StreamAlign units, demonstrates the strongest overall performance in terms of both acoustic and semantic consistency on the SALMon and spoken StoryCloze benchmarks.

Abstract:
Chain-of-thought prompting improves mathematical reasoning, but generated reasoning chains can appear locally plausible while being globally inconsistent. Later steps may derive equations inconsistent with claims asserted earlier. We propose SAGE, a training-free decoding approach that treats symbolic consistency as probabilistic conditioning over mathematical claims, conditioning each new claim on the symbolic state accumulated from earlier steps. SAGE approximates this conditional distribution online by resampling at mathematical span boundaries: candidate continuations are parsed into formal claims and selected according to satisfiability with the accumulated symbolic state. This statement-level decoding avoids intractable token-level constrained decoding while still using solver feedback online, before a complete solution has been generated. Across MATH-500, Gaokao, and Minerva, SAGE improves LLaMA, DeepSeek, and Qwen models over temperature sampling. On MATH-500, it improves accuracy by up to 6.3% and by 3.1% on average across four models. On Gaokao and Minerva datasets, SAGE yields consistent gains across models, improving Gaokao and Minerva pass@3 by up to 2% and 3%, respectively. We release our code at https://github.com/tamannahossainkay/sage

Abstract:
Recent reasoning Large Language Models produce a chain-of-thought (CoT) predominantly in English, even when prompted in non-English languages. Prior work suggests that forcing the CoT to remain in the input language (native reasoning) substantially degrades performance relative to allowing the model to reason in English before answering in the input language (English-pivoted reasoning). However, most studies of this native reasoning gap rely on inference-time interventions or limited native-language training data. We revisit this comparison at a larger scale and under comparable supervision. We construct long multilingual reasoning datasets across six languages (English, French, German, Spanish, Chinese and Swahili); fine-tune specialists in both native and English-pivoted regimes on top of Qwen/Qwen3-8B-Base, and evaluate across mathematics, science, general knowledge, and code. In this setting, the average native reasoning gap shrinks to 1.9–3.5% across the five non-English languages, considerably smaller than previously reported. Weight-space analysis of the native specialists reveals aligned fine-tuning updates in the middle layers and divergence in the outer layers. This points to a largely language-agnostic reasoning core surrounded by language-specific layers. Exploiting this structure, we introduce a Layer Swap: transferring the English specialist’s stronger reasoning mid-layers into each native specialist, closing most of the native reasoning gap across the five non-English languages while preserving CoT in the target language. We release all models and datasets.

Abstract:
Voice phishing (vishing) unfolds in real time; by the time a call has ended and post-hoc classification is possible, the harm has already been done. The more actionable question is which concrete harm (Information Gathering or Financial Exploitation) an ongoing call is tactically progressing toward. We present Vishing-Tactics-Bench, a benchmark grounded in Endsley’s situation-awareness (SA) framework that recasts vishing defense from after-the-fact fraud classification to harm projection: predicting at each turn whether the call will reach either terminal harm. We adapt MITRE ATT CK to vishing as a 6-tactic taxonomy (Vishing-Tactics) and label 35,340 scammer utterances across 5,645 synthetic Chinese calls. We define Exploitation Trajectory Forecasting, a survival-style protocol over the two terminal harms with three metrics: AP@k, C-index, and divergence error. Baselines ranging from a Markov heuristic to fine-tuned LLMs show that the tactical trajectory serves as an interpretable representation of the call’s tactical state, supporting harm-specific forecasting, which can then be used for the downstream application of intervention selection; a stratified lead-time analysis at a tight false-alarm budget further identifies at what point in a call the trajectory signal yields early warning.

Abstract:
Speech language models (SLMs) increasingly combine speech and text, often by interleaving their tokens within a single sequence. Yet how these two modalities interact in the model’s latent space remains unclear. In this work, we analyze interleaved speech–text LMs from different model families and training configurations using three complementary methods. We reveal that these models pass through an implicit latent transcription phase in which the text token matching the spoken word becomes decodable in intermediate layers, despite not being trained for speech recognition. This phenomenon occurs in diverse, natural speech, and intermediate representations also encode likely text continuations. We further show that implicit transcription emerges most when combining text-LM pretraining and speech–text interleaving, and that its prevalence is positively associated with spoken factual-knowledge retrieval. Our analysis sheds light on the internal interaction between speech and text modalities in interleaved SLMs.

Abstract:
Retrieval-Augmented Generation (RAG) systems combine dense retrievers and language models to ground outputs in external documents. However, the interaction between these components remains opaque, creating challenges for deployment in high-stakes domains. We present RAG-E, an end-to-end explainability framework that quantifies retriever-generator alignment through mathematically grounded attribution methods. Our approach adapts Integrated Gradients for retriever analysis, proposes a Monte Carlo-stabilized Shapley Value approximation for generator attribution, and introduces the Weighted Alignment between Retriever and Generator (WARG) metric to measure how closely the generator’s document usage aligns with retriever rankings. Experiments on PopQA, QAMPARI, and TREC CAST datasets reveal substantial misalignment: depending on the model and setting, generators often ignore top-ranked documents and rely on documents ranked as less relevant. We show that WARG captures retriever-generator alignment better than Pearson and Spearman correlations and can serve as an indicator of RAG performance. RAG-E and WARG provide a practical framework for auditing this interaction, enabling more reliable and transparent RAG systems.

Abstract:
Large multimodal models follow instructions about what to generate, but not necessarily about what evidence to rely on. Hence, models may continue to depend on shortcut-associated cues even when instructions suggest otherwise. We introduce GUIDE, a framework for controlling internal evidence usage through language instructions. GUIDE combines grouped parameter-efficient adaptation with instruction-conditioned gating to modulate multimodal evidence pathways during reasoning and generation. We further introduce a pathway-level evaluation framework that characterizes instruction-conditioned evidence modulation through reliance sensitivity, controlled perturbation analysis, pathway modulation, and autoregressive decoding dynamics. Across multimodal reasoning, classification, and generation, GUIDE induces structured and instruction-aligned redistribution of evidence reliance while largely preserving task behavior. Experiments on GQA, TextVQA, MM-IMDb, CREMA-D, RAVDESS, and Flickr30K show that GUIDE improves robustness under targeted evidence perturbations and enables controllable modulation across diverse multimodal settings. This suggests that multimodal instruction following can extend beyond output control toward regulating how different evidence sources contribute to model predictions.

Abstract:
Human-AI alignment is critical for ethical and safe use of large language models (LLMs). In this paper, we show that safety-aligned LLMs exhibit certain patterns in how refusals begin–such as disproportionately beginning responses with “I”. Can these patterns be exploited to break safety alignment? Here, we introduce ACHILLES, a lightweight RLVR-based attack that uses pattern-based verifiers to exploit these refusal-correlated behaviors, achieving substantial increases in attack success rates across diverse LLM families and exceeding 90% on safety-enhanced reasoning models, while preserving downstream capabilities. We further demonstrate that this vulnerability originates from domain-wise imbalances (safety vs non-safety) in safety alignment datasets, where safety responses share exploitable refusal-opening patterns. We additionally observe that safety-aligned models exhibit highly separable distributions between refusal and compliance responses in hidden representation space. To address this, we construct THETIS, a defense-oriented safety alignment dataset that reduces shallow refusal-pattern correlations. Models trained on THETIS maintain near-zero attack success rates against both ACHILLES and LLM-verifier based RLVR attacks, while largely preserving downstream performance.

Abstract:
Extending a text embedding model to new modalities typically degrades text retrieval quality, and existing omni-modal embedders compensate with multi-billion parameters. We present Omni-Embed-Mini, a 0.9B-parameter model that maps text, speech, audio, images, video, and visually-rich documents into a single shared cosine space without updating any text-side parameter. Our key insight is that the teacher signal requires no separate model: each media sample is paired with a dense cascaded caption, and the teacher target is simply the frozen backbone’s own embedding of that caption. Because teacher and student share the same backbone weights, they inhabit byte-identical geometry, and lightweight projectors plus phased LoRA adapters on the modality encoders suffice for alignment. Training combines a Matryoshka SigLIP contrastive loss with an online hybrid hard-negative miner whose negatives sharpen as the encoder improves. The recipe scales unchanged to a 2.3B variant by swapping in a native vision-language backbone. Omni-Embed-Mini-0.9B keeps its text weights bit-identical to the backbone, so training cannot regress text retrieval (49.57 nDCG@10 on MTEB-v2 BEIR-8), while extending it to five additional modalities, and is ~2.7× to 9.5× smaller than every open omni embedder we compare against. The 2.3B variant is competitive with closed API embedders, edging ahead of gemini-embedding-2 on the overall-modality average. Models, code, dataset, and evaluation harness are available on our project page: https://omniembed.cvmbzuai.com

Abstract:
Evaluating the cultural awareness of large language models is crucial to ensure the fairness of generated text and the generalizability of applications across the world. Recent benchmarks explore cultural goods like food or values like behavior in stressful situations through the lens of question answering or text generation tasks. However, creating these benchmarks requires time-intensive and costly human annotations. Also, benchmarks that evaluate cultural awareness in free text are scarce and often rely on dated evaluation mechanisms. To address this gap, we introduce ExCAM, an Explainable Cultural Awareness Metric, which is, to our knowledge, the first dedicated evaluation metric that identifies, rates and explains cultural errors in instruction-output pairs. To train and evaluate ExCAM, we introduce ExCAM40k, a dataset comprised of nine existing benchmarks that we reformat and enhance with synthetic errors. Compared to several baselines, including GPT-5, ExCAM achieves the highest error detection rate with up to 80% accuracy on a balanced test set. Therefore, ExCAM opens the pathway towards fine-grained and explainable cultural evaluation of free text.

Abstract:
Preference optimization is widely used to align Large Language Models (LLMs) with preference feedback. However, most existing methods train on a single positive-negative pair per prompt, discarding additional supervision available in preference datasets that typically contain multiple candidate responses. Motivated by this limitation, recent work explores group-wise preference optimization, which jointly contrasts multiple responses for the same prompt, but its empirical behavior and scalability remain underexplored due to the memory overhead of group-coupled objectives. In this work, we present a unified empirical and systems study of group-wise preference optimization and develop a memory-efficient implementation for group-coupled objectives. By instantiating first-order linearization with objective-specific per-response coefficients, our implementation preserves first-order gradients while decoupling samples during backpropagation, substantially reducing peak memory usage and enabling scalable training with larger groups. Across offline and online settings, we show that leveraging multiple responses consistently outperforms single-pair training. Furthermore, incorporating a negative log-likelihood (NLL) term on positive responses is critical for both performance gains and training stability.

Abstract:
Existing prompt optimization methods often improve prompts for a specific task or deployment setting, but their optimized prompts transfer poorly to new tasks and usually require re-optimization. We propose Prompting to Prompt (PTP), a framework that learns meta-templates as structured intermediate representations for reusable prompt construction. PTP decomposes prompts into transferable elements and optimizes them with LLM-generated textual gradients. Its inner loop refines task-specific prompts from natural-language feedback and an element list, while its outer loop abstracts structural changes across tasks to update the meta-template.The learned meta-template is then reused for task-level and query-level prompt generation, reducing the need to rerun full prompt optimization for each target setting. Experiments on six benchmarks show gains in both offline transfer and test-time deployment settings, with improvements up to 10.52 points on Arena-Hard.

Abstract:
Conversational recommender systems (CRS) increasingly rely on user simulators for automated evaluation of sales agents. A key requirement for such simulators is the ability to model human decision-making. However, most LLM-based simulators often exhibit unrealistically strong information-processing capabilities, rarely exhibiting the hesitation and decision deferral commonly observed in real consumer behavior, resulting in overly high acceptance rates. To address this limitation, we propose Hesitator, a theory-grounded user simulation framework that explicitly models a specific dimension of human decision-making: decision deferral under choice overload. The framework introduces a modular Decision Module that separates utility-based item selection from overload-aware commitment decisions. Experiments across multiple user simulation frameworks, domains, sales modes, and LLM backbones show that integrating our module consistently mitigates unrealistic behaviors under increasing overload conditions. Furthermore, Hesitator reproduces established findings from behavioral economics, demonstrating its ability to model human decision behavior.

Abstract:
Reasoning-oriented large language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substantially increases inference cost and latency. In contrast, instruction-tuned models tend to answer more concisely, yet often lack comparable reasoning ability. This accuracy-efficiency mismatch motivates a lightweight approach that combines the strengths of both models without full model retraining. In this paper, we propose GRIP (Granular Reward-guided Interpolation of Parameters), a reward-guided parameter interpolation framework for efficient reasoning. Given a reasoning model and an instruction model with identical architectures, GRIP assigns learnable interpolation ratios to individual modules and optimizes only these ratios while keeping both source models frozen. The interpolation ratios are trained with a reward signal that favors responses that are both correct and concise. Experiments show that GRIP achieves a better accuracy-efficiency trade-off than fixed or search-based merging baselines and further reveals module-wise fusion patterns associated with efficient reasoning. Our code is publicly available at https://github.com/Dozen233/GRIP .

Abstract:
Large language models (LLMs) are increasingly used for codebook-based content analysis in computational social science, where codebooks translate theoretical concepts into labeling schemes. While prior work reports strong predictive performance, it remains unclear whether these systems produce reliable and robust annotations. We conduct a large-scale empirical failure-mode analysis of LLM-based coding pipelines across three datasets and five models by systematically evaluating their stability, robustness, and conceptual validity under controlled perturbations, domain shifts, and optimization procedures. We uncover multiple failures that are largely invisible under standard metrics, showing that high aggregate scores frequently coexist with severe label-level disparities, instability across runs, domain sensitivity, and shifts in label interpretation. Importantly, these failure modes persist under widely used optimization strategies and remain largely invisible under standard metrics such as accuracy and micro-F1. Our results demonstrate that current LLM-based coding pipelines cannot be treated as stable measurement tools without targeted validation beyond aggregate performance.

Abstract:
Long-form visual storytelling requires maintaining continuity across shots, including consistent characters, stable environments, and smooth scene transitions. While existing generative models can produce strong individual frames, they fail to preserve such continuity, leading to appearance changes, inconsistent backgrounds, and abrupt scene shifts. We introduce CANVAS (Continuity-Aware Narratives via Visual Agentic Storyboarding), a multi-agent framework that explicitly plans visual continuity in multi-shot narratives. CANVAS enforces coherence through character continuity, persistent background anchors, and location-aware scene planning for smooth transitions within the same setting. We evaluate CANVAS on two storyboard generation benchmarks ST-BENCH and ViStoryBench and introduce a new challenging benchmark HardContinuityBench for long-range narrative consistency. CANVAS consistently outperforms the best-performing baseline, improving background continuity by 21.6%, character consistency by 9.6% and props consistency by 7.6%.

Abstract:
The widespread adoption of Large Language Models has accelerated news content generation, yet methods to help readers contextualize and responsibly consume news content lag behind. Existing approaches to clustering news articles face significant limitations. Methods based on dense representations often struggle with robustness and efficiency, while those based on sparse (n-gram) representations are unsuitable for cross-lingual applications. Additionally, LLM-based methods are expensive to apply at the needed scale, restricting their use to only well funded organizations. We present a novel method for efficient clustering of news articles using cross-lingual, sentence-level embeddings and community detection on similarity graphs. Our method can construct a document-level similarity graph from sentence-level embeddings, avoiding the use of sub-optimal embeddings of entire documents. The proposed method can be executed on consumer hardware, enabling practical news analysis for personal use and research. We also release a manually curated dataset of clustered news articles, providing a valuable resource for future research. We use the dataset to benchmark our method using nine text embedding models with cross-lingual capability and propose various practical configurations.

Abstract:
Unlearning knowledge is a pressing and challenging task in LLMs because of their unprecedented capability to memorize and digest training data at scale, raising more significant issues on safety, privacy, and intellectual property. However, existing works, including parameter editing, fine-tuning, and distillation-based methods, are all focused on the flat sentence-level data but overlook the relational, multi-hop, and reasoned knowledge in the naturally structured data. In response to this gap, this paper introduces Graph Oblivion and Node Erasure ( GONE ), a benchmark for evaluating knowledge unlearning over structured knowledge graphs (KGs) facts in LLMs. This KG-based benchmark enables the disentanglement of three effects of unlearning: direct fact removal, reasoning-based leakage, and catastrophic forgetting. In addition, Neighborhood-Expanded Distribution Shaping ( NEDS ), a novel unlearning framework, is designed to leverage graph connectivity and identify anchor correlated neighbors, enforcing a precise decision boundary between the forgotten fact and its semantic neighborhood. Evaluations on LLaMA-3-8B and Mistral-7B across multiple knowledge editing and unlearning methods showcase the superior performance (1.000 on unlearning efficacy and 0.839 on locality) of NEDS on GONE and other benchmarks.

Abstract:
Hallucinations remain one of the major barriers to the reliable deployment of Large Language Models (LLMs). Recent works have explored hallucination detection using both supervised classification based approaches and unsupervised metric based approaches, with the latter remaining popular since they do not require labeled data. In this work, we study the performance and generalizability of these approaches through a comprehensive empirical analyses across 11 datasets and 10 models. We first establish that supervised methods achieve stronger in-domain performance but lag behind unsupervised methods in the out-of-domain setting. Then to narrow this generalization gap, we introduce a simple approach that improves the generalizability of supervised hallucination detectors by training on a curated, multi-domain data mix, which can complement the subsequent addition of task-specific data. In our experiments on hallucination detection on 697K QA samples from 12 open source QA datasets, we show that incorporating this general training allows supervised methods to surpass unsupervised metric based methods by an average of +7.25% on out-of-distribution data within the short-form QA setting, without addition of any task-specific data. We also analyze scaling behaviors and estimate how much task-specific data is required to achieve reliable performance, finding that models augmented with general data require up to 40.3% less task-specific data to achieve close to optimal performance. Together, our findings highlight a simple path toward fortifying supervised hallucination detectors against domain shift.

Abstract:
Large Language Model agents augmented with external skill banks have demonstrated remarkable performance on sequential decision-making tasks. However, existing methods update skills in an undirected, unverified manner and prematurely penalize agents for failing to utilize immature skills. We introduce a structured cognitive representation that decomposes both general and task-specific skills into four dimensions—D-PLAN, D-TRACK, D-EXEC, and D-GUARD—enabling precise failure attribution. Building upon this representation, we propose A 2 SE, an ability-aligned skill evolution framework that synchronizes skill bank updates with the agent’s learning trajectory. Specifically, A 2 SE concentrates evolution on the identified bottleneck dimension, safeguards updates via a causal verification mechanism with per-task-type rollback, and employs a competence-aligned skill reward to alleviate premature penalization. Extensive experiments on ALFWorld and WebShop demonstrate that A 2 SE achieves 11.0% and 7.1% improvements over the strongest baselines, respectively, with ablation studies confirming the necessity of each proposed component.

Abstract:
Multimodal large language models have made rapid progress in video understanding, yet existing benchmarks largely rely on simple prompts and provide limited evidence about whether models can satisfy explicit output constraints. We introduce VCIFBench, a benchmark for evaluating complex instruction following in video understanding. VCIFBench constructs constraint-rich instructions from both benchmark-adapted and directly video-grounded prompts, covering content, format, style, and structure requirements, and evaluates model outputs with a hybrid verification pipeline. The benchmark contains 306 satisfiable test instructions, 540 DPO training instances, and a 100-item diagnostic set for evaluating whether models can recognize instruction conflicts. Experiments on 10 MLLMs show that joint constraint satisfaction remains challenging. Preference optimization improves instruction following for two model families, while Conflict-100 reveals that models usually execute a satisfiable-looking subset instead of detecting global incompatibility.

Abstract:
Large language models (LLMs) can generate functionally correct code, but their ability to optimize programs is underexplored. We introduce OEISPerf, a code-to-code benchmark for algorithmic program optimization built from alternative implementations of integer sequences in the On-Line Encyclopedia of Integer Sequences. For each sequence we mine multiple mathematical formulations, synthesize and validate Python implementations, and pair a substantially slower program with the fastest validated reference found by our construction pipeline, producing instances that often require substantial structural transformations and can yield substantial speedups. Evaluating 15 open-source and closed-source LLMs shows that even the strongest evaluated setting, Gemini-3-Flash with its thinking mode enabled, matches the reference efficiency on only 67.9% of test instances. Across LLMs, we observe both incorrect “optimizations” that break correctness and superficial refactorings that preserve suboptimal complexity. Finally, we fine-tune Qwen2.5-Coder-3B-Instruct via reinforcement learning, markedly improving reference attainment on OEISPerf while slightly improving performance on standard code-generation benchmarks.

Abstract:
Knowledge graph (KG) reasoning infers new knowledge from existing facts and is widely applied in question answering, recommendation, and decision support. With the rapid development of large language models (LLMs), LLM-based KG reasoning frameworks have become increasingly popular by leveraging retrieved KG information. However, hallucinations in LLMs remain a critical issue. Even when relevant KG knowledge is incorporated, models may still generate incorrect outputs, leading to misinformation and unreliable decisions. Existing hallucination detection methods either focus on LLM internal states or verify consistency with retrieved contexts, but both overlook the structural information in KGs, resulting in suboptimal performance. To address this gap, we propose LUCID, the first halLUcination deteCtIon method for LLM-based knowleDge graph reasoning frameworks. LUCID extracts node and edge features from attention scores and semantic similarities, and integrates them with KG structure using a graph neural network (GNN). Moreover, we also construct manually annotated benchmark datasets for evaluation. Experiments on nine datasets show that LUCID achieves state-of-the-art performance compared to 15 baselines. The code and data are available at: https://github.com/BUPT-GAMMA/LUCID.

Abstract:
Text detoxification is increasingly used in content moderation and dialogue systems, where systems must produce safe rewrites while revealing what was changed and why. Existing approaches typically trade off interpretability and end-to-end optimization. Monolithic rewriting methods treat detoxification as a single opaque generation step, whereas edit-based pipelines make edits explicit but are trained in separate stages, preventing joint optimization. We propose STEER, a framework that models detoxification as a structured edit chain formulation and optimizes the entire editing process end-to-end with Group Relative Policy Optimization (GRPO). To make this formulation trainable, we construct synthetic edit-chain supervision using a multi-agent pipeline, perform supervised fine-tuning (SFT) to teach the model the structured editing format, and then apply GRPO with a veto-aligned multiplicative reward and granular format scoring. Experiments on ParaDetox and two out-of-domain benchmarks show that STEER achieves the highest aggregate J score among the compared systems on ParaDetox and remains competitive under out-of-domain evaluation.

Abstract:
Chain of Thought (CoT) reasoning enhances logical performance by decomposing complex tasks, yet its multimodal extension faces a trade-off. The existing thinking with images paradigm is limited by the modality gap between vision and language, which hinders reliable extraction of reasoning relevant information from high dimensional visual data. Recent latent space reasoning method provides stronger multimodal representations, but it often lacks the ability to refocus on visual inputs and suffers from limited interpretability. To address these issues, we proposeLatent Refocusing (LaRe), a novel multimodal reasoning paradigm that combines visual refocusing with rich latent representations, enabling iterative reasoning within the latent space. We further design a semantic augmentation training strategy that ensures the semantic structure of the latent space through visual reconstruction objective. Experimental evaluations demonstrate that LaRe improves average accuracy by 7.6% compared to existing baselines while reducing the number of tokens required for inference by 59.7%. When scaled to a 8B-parameter Vision-Language Model backbone, LaRe achieves performance comparable to state-of-the-art methods, demonstrating the efficacy of our proposed latent refocusing paradigm for multimodal reasoning.

Abstract:
Reliable code retrieval is crucial for developer productivity and effective code reuse. However, current neural code language models (CLMs) powering search tools are susceptible to adversarial attacks targeting non-functional textual elements. In this paper, we introduce a programming language-agnostic, transferable, adversarial attack that exploits this CLM vulnerability. Our approach perturbs identifiers within a code snippet without altering the snippet’s functionality to artificially align the code with a target query. We demonstrate that our attack, even when computed using smaller code embedding models, such as CodeT5+, is highly effective and transferable to larger, closed-source embedding models, like Voyage-code-3, or LLMs like Gemini-3.1-Pro. Our attack can increase the similarity between the query and arbitrary, irrelevant code snippets, consequently degrading key retrieval metrics such as the Mean Reciprocal Rank (MRR) of state-of-the-art models by up to 77%. The experimental results highlight the fragility of current code search methods and underscore the need for more robust, semantic-aware approaches.

Abstract:
Agent skills, structured packages of procedural knowledge and executable resources that agents dynamically load at inference time, have become a reliable mechanism for augmenting LLM agents. Yet inference-time skill augmentation is fundamentally limited: retrieval noise introduces irrelevant guidance, injected skill content imposes substantial token overhead, and the model never truly acquires the knowledge it merely follows. We ask whether skills can instead be internalized into model parameters, enabling zero-shot autonomous behavior without any runtime skill retrieval. We introduce SKILL0, an in-context reinforcement learning framework designed for skill internalization. SKILL0 introduces a training-time curriculum that begins with full skill context and progressively withdraws it. Skills are grouped offline by category and rendered with interaction history into a compact visual context, teaching the model tool invocation and multi-turn task completion. A Dynamic Curriculum then evaluates each skill file’s on-policy helpfulness, retaining only those from which the current policy still benefits within a linearly decaying budget, until the agent operates in a fully zero-shot setting. Extensive agentic experiments demonstrate that SKILL0 achieves substantial improvements over the standard RL baseline (+9.7% for ALFWorld, +6.6% for Search-QA and +10.1% for WebShop), while maintaining a highly efficient context of fewer than 0.5k tokens per step.

Abstract:
Group-based reinforcement learning typically relies on coarse outcome-level signals, such as group accuracy or variance, for data selection or replay. Yet in multi-turn tool-use rollout groups, these signals miss structural differences between successful and failed trajectories: a failure may diverge into an unrelated tool chain or differ from a successful trajectory only at a local decision. We therefore introduce Group Structural Proximity (GSP), which measures failed-to-successful rollout alignment within a group using tool dependency graphs induced by tool calls and argument flow. A controlled diagnostic study with outcome-statistics-matched subsets further shows that GSP captures distinct learning dynamics: high-GSP data drives localized policy refinement, while low-GSP data induces broader policy movement and higher-entropy exploration. This positions GSP as an effective metric on the exploration–exploitation tradeoff. Building on this insight, we introduce Proximity-Guided Policy Optimization (PGPO), which considers replay buffers with high- and low- GSP under an Exploration-to-Refinement Schedule (ERS) for group-based training. On 𝜏 2 -Bench and ACEBench Agent, PGPO improves sample efficiency and downstream tool-use performance, demonstrating the benefit of structure-aware group comparison in multi-turn tool-use reinforcement learning.

Abstract:
Reliable confidence estimation is essential for large language model deployment. However, answer-level calibration remains challenging because generation errors are often localized: a response may be fluent and high-probability overall while still failing at a critical number, entity, or factual claim. Existing estimators compress token probabilities, sequence likelihoods, entropy, or beam statistics into a global score, which can dilute such local risk signals. We propose TRACE, a single-pass, decoded-answer-preserving confidence estimator that treats decoding-time uncertainty as a trajectory through three steps: (i) recording token-level surprisal and predictive entropy during decoding, (ii) applying local risk operators to preserve uncertainty spikes, and (iii) converting localized trace risk into answer-level confidence. TRACE produces a label-free risk score, while TRACE+ calibrates trace-only features into probabilities using a held-out split, without extra generations or external verifiers. We evaluate four tasks against 19 calibration baselines, and TRACE+ reduces Brier from 0.149 to 0.137 and improves AUROC from 0.758 to 0.792 over the strongest likelihood baseline. Across seven LLMs, TRACE+ improves over the best non-TRACE baseline pool from 0.136 to 0.120 Brier and from 0.764 to 0.817 AUROC. Results show localizing decoding-time risk provides a general approach to calibration.

Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, yet personalizing their outputs to individual users remains an open challenge. Existing approaches predominantly adopt a flat behavioral paradigm, aggregating user behaviors without an explicit account of how they are organized into deeper behavioral structures. In this work, we draw on Pierre Bourdieu’s Theory of Practice to propose PHF (Practice-Habitus-Field), a sociologically grounded framework that reconceptualizes LLM personalization through three hierarchical levels: individual behaviors as practices, their temporal accumulation into stable dispositions as habitus, and shared regularities across similar users as fields. We instantiate PHF through PHF Compass, a lightweight and model-agnostic implementation based on a frozen LLM. Experiments on the Language Model Personalization (LaMP) benchmark demonstrate consistent improvements across diverse tasks, while further analyses validate the interpretability and extensibility of the learned behavioral structures.

Abstract:
Masked diffusion language models (MDLMs) enable parallel decoding by predicting all masked positions at each denoising step, yet existing training-free samplers usually decide which positions to commit at token-level granularity. We revisit this granularity and observe that reliable predictions often emerge as contiguous high-confidence spans, suggesting that the unit of parallel commitment can be larger than a single token. We first group adjacent high-confidence candidates into confidence-induced clusters (CICs) as span-level update units. We then use self-attention maps from the same forward pass to estimate inter-cluster dependencies, enabling conflict-aware selection of mutually compatible CICs for parallel commitment. This yields CLAD (Cluster-Level Attention-Guided Decoding), a training-free cluster-level decoder for MDLMs. Experiments on LLaDA and Dream model families across four reasoning and code-generation benchmarks show that CLAD achieves 1.77× – 8.47× speedups over Vanilla decoding while maintaining broadly comparable task accuracy in most settings.

Abstract:
Language models often answer harmful requests in low-resource languages (LRLs) that they refuse in high-resource languages (HRLs). Across three instruction-tuned models and 23 languages, harmful refusal falls from 87.9% in HRLs to 43.9% in LRLs, while harmless refusal remains low. A common explanation is that models represent harmfulness weakly in LRLs. We test whether harmfulness is instead represented but does not reliably produce refusal. Across three models, a harmfulness direction learned from HRL activations still separates harmful from harmless LRL prompts, showing that complete absence of harmfulness information cannot explain many failures. However, harmfulness scores shift downward for LRL prompts, making harmful prompts less likely to reach the range associated with refusal. Motivated by this shift, we train a low-rank logistic classifier on HRL activations and calibrate its threshold with a few target-language examples. During generation, the classifier conditionally adds or ablates the HRL harmfulness direction. With the same HRL data and 32 target-language examples per class, CAST remains limited by low harmful refusal and AdaSteer by high harmless refusal, yielding mean refusal selectivity (Δ = harmful − harmless refusal) of 33.6 and 6.8, respectively. Our intervention reaches 54.5 while preserving MMLU utility. HRL-only calibration improves selectivity for Qwen and Gemma, whereas Llama benefits from target-language calibration. These results show that recalibrating existing representations can offer a training-free method for repairing low-resource safety failures.

Abstract:
Given a line-art sketch and user-specified color intent, controllable colorization must assign intended colors to fine-grained regions while preserving the underlying line structure. Existing methods have improved visual quality, but precise local adjustment remains difficult. In this work, we argue that the key limitation is the absence of a persistent correction state. Such a state records user intent, localized mismatches, grounded edit regions, and preservation constraints to guide targeted repairs. To address this, we introduce PaletteAgent, an agentic, self-evolving correction framework for controllable line-art colorization. PaletteAgent translates user intent into structured color prompts, employs judgment agents to diagnose mismatches, grounds problematic regions, preserves correct fields, and adaptively decides whether to refine globally, repair locally, or stop. This iterative process naturally defines the GRPO training objective for optimizing the agent’s correction policy. Empirical results show that PaletteAgent outperforms line-art colorization and general image-editing baselines. Further ablations confirm the role of maintaining a persistent correction state for controllable colorization.

Abstract:
Low-budget experimental optimization requires making consequential experimental decisions from a small number of observations. Bayesian optimization (BO) and related black-box optimizers provide strong proposal mechanisms, but they often hide the intermediate decisions that matter to experimental users. Direct LLM recommendation can leverage broad domain priors and natural-language reasoning, but is difficult to control, audit, or align with the statistical search guarantees of the underlying optimizer. We introduce TRACE, a traceable agentic co-optimizer that keeps the optimizer as the search backbone while wrapping it in an auditable agentic decision layer. TRACE represents agent interventions as bounded actions backed by reusable skills, organizes external and experiential evidence through an Evidence Skill System, and records every optimization step as a decision trace. On low-budget reaction-optimization benchmarks, TRACE improves over the corresponding BO-only optimizers and over LLM-assisted and tree-search baselines on most evaluated metrics. These gains suggest that our agentic system can effectively support experimental optimization by providing structured, traceable decision support around its actions, evidence use, and intervention logic.

Abstract:
Graphical User Interface (GUI) Agents autonomously interact with software to fulfill user requests, where GUI navigation stands out as the most critical and challenging capability. Mastering this capability demands a complex synergy of step-wise decision-making, state-action alignment, and long-horizon planning. While directly mixing these corresponding navigation tasks seems intuitive to simultaneously acquire these skills, such a direct combination is severely bottlenecked by inconsistent optimization objectives and profound data heterogeneity. To overcome these barriers, we propose the MaP (stands for “ M asked Tr a jectory P rediction”), a unified framework that seamlessly harmonizes divergent GUI navigation tasks. By modeling multi-turn GUI interactions as a trajectory and defining training objectives through component masking and prediction, MaP shifts the optimization from task-specific marginal distributions to a consistent objective. Furthermore, to handle the data heterogeneity across multiple navigation tasks, we design a role-aware adapter learning module that dynamically routes each token to a specialized representation space. Extensive experiments on five representative GUI navigation benchmarks demonstrate that MaP effectively mitigates gradient conflicts and significantly outperforms the direct mixture training, establishing a robust paradigm for multi-task GUI navigation.

Abstract:
Many expert-domain decisions involve intermediate variables, yet standard LLM evaluation often measures only final-answer accuracy. We present a controlled study of intermediate-variable dependence in medical treatment selection, using diagnosis as an explicit variable between patient phenotype and treatment. We construct a six-option multiple-choice benchmark from PrimeKG-derived phenotype–disease–drug chains and evaluate six LLMs using behavioral diagnosis interventions, held-out QLoRA path supervision, external validation, and auxiliary layer-wise localization analysis. Providing the gold diagnosis improves average treatment accuracy from 49.9% to 84.9%, while incorrect predicted diagnoses reduce accuracy from 85.5% to 23.1%. A shuffled-diagnosis control yields 18.2% accuracy, suggesting that the effect depends on diagnosis correctness rather than merely adding disease-like text. Held-out QLoRA experiments show that path supervision improves treatment selection, with diagnosis-first full-chain and multitask objectives yielding the strongest gains. Center-of-Gravity localization further gives a modest diagnosis-first signal, which we interpret as a localization pattern rather than evidence of faithful clinical reasoning. Overall, in this controlled setting, final-answer accuracy can obscure failures in intermediate-variable dependence, while structured supervision can partially improve diagnosis-to-treatment behavior.

Abstract:
Reasoning is now central to modern foundation models, but remains underdeveloped in the audio modality. Unlike text transcripts, audio contains not only words but also vocal cues, background events, and temporal patterns, all of which can be necessary for reliable inference. This survey presents the first dedicated synthesis of audio reasoning in multimodal foundation models. We formulate audio reasoning as reasoning-augmented generation grounded in acoustic evidence, distinguishing it from conventional audio understanding. We then organize existing work around two core settings: Audio-to-Text reasoning, where models infer textual answers from acoustic input, and Audio-to-Speech reasoning, where models reason before or during speech generation under latency constraints. Finally, we summarize evaluation practices and challenges in acoustic grounding, shortcut learning, modality hallucination, and latency-constrained interaction.

Abstract:
Large language models (LLMs) can propose vaccine adjuvant designs with plausible mechanistic rationales, but such outputs often lack grounding in established immunological knowledge. We present SAVANT , a neuro-symbolic framework for LLM-based adjuvant design that combines two-stage constrained generation with literature-grounded verification. SAVANT first generates a natural-language design rationale, then renders it into an ontology-constrained domain-specific language (DSL) using logic masks and finite-state control. To reason under fragmented evidence and incomplete first-principles rules, SAVANT verifies generated designs against literature-derived symbolic evidence through minimal sub-theorem proof. In expert-aligned evaluation, SAVANT shows greater stability in stage-level rank alignment than the strongest LLM-judge baselines, while achieving the strongest alignment for immune-effect support and mechanistic reasoning. SAVANT provides graded, interpretable assessments that distinguish supported mechanisms, under-supported hypotheses, and knowledge gaps for hypothesis-driven experimentation. Data and code will be released at https://github.com/banjiuyufen/SAVANT.

Abstract:
Hierarchical merging is commonly used to summarize long documents by splitting the input into chunks, summarizing each chunk independently, and merging the results into a final summary. However, because each chunk is processed independently, entity state changes across chunks are not tracked, leading to coherence and consistency issues in the final summary. We propose EntityBB (Entity Blackboard), an agent memory framework that equips chunk-based LLM summarizers with structured entity memory. During sequential chunk processing, the model records and updates entity states through tool calls, while automatic safeguards and model-driven rejection limit noisy writes. Experiments on three long document summarization benchmarks show that EntityBB consistently improves over a controlled hierarchical merging baseline across ROUGE and BERTScore, while NarrativeFactScore indicates improved source-grounded factuality. Targeted human and LLM-as-a-Judge evaluations further indicate fewer entity-level issues and higher judged summary quality.

Abstract:
The prevalence of rapidly evolving slang, neologisms, and highly stylized expressions in informal user-generated text, particularly on Chinese social media, poses significant challenges for Machine Translation (MT) benchmarking. Specifically, we identify two primary obstacles: (1) data scarcity, as high-quality parallel data requires bilingual annotators familiar with platform-specific slang, and stylistic cues; and (2) metric limitations, where traditional evaluators like COMET often fail to capture stylistic fidelity and non-standard expressions. To bridge these gaps, we first introduce CSM-MTBench, an expert-curated benchmark covering five Chinese→foreign language directions and comprising 10,915 total instances, organized into two subsets: Fun Posts, featuring context-rich content with frequent slang and neologisms, and Social Snippets, focusing on concise, emotion- and style-driven expressions. Next, we propose tailored evaluation approaches for each subset, including slang translation success rate and style-preservation metrics combining embeddings and LLM-asa-judge evaluation. Experiments on over 20 models reveal substantial variation in how current MT systems handle semantic fidelity, slang translation, and stylistic nuances. We further look into model behaviors through prompt sensitivity analysis and qualitative case studies, highlighting the challenges of translating these informal expressions. CSM-MTBench thus serves as a rigorous testbed for advancing MT systems capable of mastering real-world Chinese social media texts.

Abstract:
LLM-as-Judge systems can produce multi-field evaluations, such as trustworthiness, reliability, and factuality. These outputs are often interpreted as independent evidence. We test this assumption for a common pair of judgments: trust scoring and binary truth classification. On correctness-controlled QA, LLM judges align trust scores with truth verdicts more tightly than human behavioral reference, suggesting weaker separations between trust and truth judgment. We then apply stress tests by changing only source cues of identical QA between Human and AI, while holding QA content fixed. Source attribution shifts not only trust scores but also truth verdicts and logit-derived correct-side probabilities. Results show that current LLM-as-Judge protocols should not treat trust scores as independent evidence for truth judgments without testing their separability under trust-relevant perturbations.

Abstract:
This paper studies adaptive recommendation under intent drift, where feedback from each recommendation outcome can reveal whether the relational evidence used for ranking is useful, missing, or misleading. While Knowledge Graphs (KGs) provide essential semantic structure to handle these shifts, traditional KG-enhanced systems treat the graph as a static retrieval substrate, making it brittle to evolving intents, noisy metadata, and recurring failure patterns. This paper proposes CGM-Rec, a continual graph memory framework for adaptive recommendation. CGM-Rec treats the graph state as a writable memory and maintains two complementary components. Therein, a Semantic Graph Memory is updated conservatively through quality-gated typed operations for storing stable and high-confidence relational knowledge. Meanwhile, an Episodic Lesson Memory acts as a fast reactive memory that learns recent outcomes, failure cases, and corrective hints. During testing, model parameters remain frozen and adaptation occurs only through memory writes. We evaluate CGM-Rec under a frozen-parameter, one-pass reranking protocol, where encoders and prompts remain fixed during testing and adaptation occurs only through memory writes. Experiments across multiple recommendation settings show that CGM-Rec improves over evaluated neural and LLM-based baselines on most metrics. Particularly, under sampled-candidate reranking, CGM-Rec improves HR@1 by up to 29.58% over the strongest LLM baseline on Bundle, and outperforms K-RagRec on metadata-rich ML-100K with HR@5 of 0.5941 versus 0.4746.

Abstract:
Large language models (LLMs) have shown potential for reference-free span-level translation quality estimation (QE), yet existing approaches based on Multidimensional Quality Metrics (MQM) typically rely on fixed rubric configurations shared across translation instances. However, translation instances often differ substantially in error complexity, ambiguity, and required evaluation granularity, making static rubric allocation suboptimal for span-level error detection. We find that larger MQM subtype spaces improve error coverage but also introduce more false positives, while different translation instances prefer different rubric granularities, suggesting that evaluation spaces should be allocated dynamically for each case. Therefore, we propose a case-specific dynamic rubric framework that adaptively constructs MQM evaluation spaces for individual translation instances. Unlike methods that generate fully free-form rubrics, our framework remains grounded in the predefined MQM taxonomy while dynamically selecting suitable subtype spaces and evaluation granularity for different cases. Experiments on span-level QE benchmarks from the Conference on Machine Translation (WMT) across multiple model scales demonstrate that the proposed framework consistently improves F1 and Matthews Correlation Coefficient (MCC).

Abstract:
This paper proposes an LLM-based method for cross-lingual alignment of coordinate structures in Japanese–English and Chinese–English patent claims. The approach has two components. First, we construct task-specific alignment datasets by translating Penn Treebank-3 English sentences into Japanese and Chinese and projecting coordinate-structure annotations using syntactic parse trees, word alignments, and the GHKM algorithm. Second, we formulate alignment as a TANL-based text generation task in which LLMs generate the target sentence with the corresponding span tagged. Although the training data are derived from a general-domain corpus, evaluation is conducted on manually annotated real patent claims. Across four translation directions, the proposed approach consistently outperforms SpanAlign, an encoder-based alignment baseline built on SQuAD-style cross-lingual span prediction and ILP-based alignment. Controlled analyses show that the TANL formulation improves performance independently of supervised fine-tuning and that automatically generated demonstrations achieve performance close to manually constructed examples in the tested 3-shot setting. Supervised fine-tuning with the automatically constructed datasets further improves open-weight LLMs, with the best fine-tuned model in each direction achieving an F1 score above 0.8.

Abstract:
Large language model (LLM) agents are increasingly organized into multi-agent systems for complex reasoning and decision-making tasks. However, scaling such systems can substantially increase inference cost, especially when multiple agents repeatedly process overlapping or role-irrelevant context. Many existing multi-agent frameworks focus on orchestration and role specialization, but still rely on full-context sharing or static context routing, leading to redundant token consumption, increased runtime, and inefficient memory utilization. We present Role-Conditioned Context Routing (RCR), an execution-time layer that allocates role-relevant shared-memory items to agents under token budgets. Given a structured shared memory and a set of role-specialized agents, RCR selectively routes task-relevant memory items to each agent according to its role, optional task-stage metadata, and token budget. RCR integrates role-conditioned relevance scoring, token-budgeted memory selection, and iterative memory updates to reduce redundant context exposure during multi-agent execution. Experiments on multi-hop QA, embodied reasoning (ALFWorld), and web-based decision-making (WebShop) show that RCR reduces token usage by 35% on average and runtime by 38% on average relative to Full-Context, while maintaining or improving task performance.

Abstract:
Multimodal Large Language Models have achieved strong performance in single-video understanding, yet their ability to reason across multiple videos remains limited. Existing approaches typically concatenate multiple videos into a single input and perform direct inference, which introduces training–inference mismatch, information loss from frame compression, and a lack of explicit cross-video coordination. Meanwhile, current multi-video benchmarks primarily emphasize event-level comparison, leaving identity-level matching, fine-grained discrimination, and structured multi-step reasoning underexplored. To address these gaps, we introduce MVX-Bench, a Multi-Video Cross-Dimension Benchmark that reformulates 11 classical computer vision tasks into a unified multi-video question-answering framework, comprising 1,442 questions over 4,255 videos from diverse real-world datasets. We further propose SAMA, a Skill-Augmented Agentic Framework for Multi-Video Understanding, which integrates visual tools, task-specific skills, and a conflict-aware verification mechanism to support iterative and structured reasoning. Experiments show that SAMA improves substantially over its visual perception backbone and achieves the best average accuracy on MVX-Bench. Ablations confirm the benefits of skill-guided orchestration and conflict-aware verification.

Abstract:
Repository-level code generation requires generated code to be compatible not only with the target repository but also with its dependency environment. Existing retrieval-based methods mainly retrieve context from the local repository, leaving external API usage dependent on the model’s pretrained knowledge, which can be insufficient for unseen or version-specific APIs. Moreover, current retrieval strategies largely focus on one-hop evidence and overlook the structural relationships among code components. We propose CrossCoder, a cross-repository code generation framework that explicitly incorporates external libraries into the retrieval context through a unified knowledge graph over repository and library entities. CrossCoder identifies important nodes via planning and semantic retrieval, then selectively expands neighboring nodes to retrieve richer multi-hop contextual evidence for generation. To further evaluate dependency-version compatibility, we introduce VersionExec, an execution-based benchmark derived from BigCodeBench that evaluates generation under different dependency versions. Experimental results on RepoExec, DevEval, and VersionExec demonstrate that CrossCoder consistently improves both functional correctness (up to 6.3% on pass@1) and robustness to dependency-version changes.

Abstract:
Synthetic data augmentation helps language models learn new knowledge in data-constrained domains. However, naively scaling existing synthetic data methods by training on more synthetic tokens or using stronger generators yields diminishing returns below the performance of RAG. To break the RAG ceiling, we introduce Synthetic Mixed Training, which combines synthetic QAs and synthetic documents. This leverages their complementary training signals, and enables log-linear improvements as both synthetic data volume and generator strength increase. This allows the model to outperform RAG by a 2.6% relative gain on QuaLITY, a long-document reading comprehension benchmark. In addition, we introduce Focal Rewriting, a simple technique for synthetic document generation that explicitly conditions document generation on specific questions, improving the diversity of synthetic documents and yielding a steeper log-linear scaling curve. On QuaLITY, our final recipe trains a Llama 8B model that outperforms RAG by 4.4% relatively. Across models and benchmarks (QuaLITY, LongHealth, FinanceBench), our training enables models to beat RAG in five of six settings, outperforms by 2.6%, and achieves a 9.1% gain when combined with RAG.

Abstract:
Turn-taking is a fundamental mechanism that governs when interlocutors speak and listen. Although Spoken Dialogue Systems (SDS) exploit a range of linguistic, acoustic, and non-verbal cues, they produce ill-timed responses in unscripted interaction. A central challenge is anticipating Transition Relevance Places (TRPs), or opportunities, not obligations, for a listener to take the floor. Human listeners do not wait for turn endings; as an utterance unfolds, they use expectations about its developing meaning to anticipate TRPs and decide whether to take the floor. We examine whether these evolving expectations can be modeled through semantic uncertainty —an LLM-derived measure of how strongly a turn so far constrains what may plausibly come next. To do so, we sample possible continuations of an ongoing turn and use changes in semantic dispersion to identify TRPs within turns. We evaluate this account on a dataset with TRP labels derived from real-time listener responses, rather than retrospective annotation. Our approach substantially outperforms prompt-based and fine-tuned text-only baselines, providing empirical support for the view that evolving semantic constraints inform perceived turn-taking opportunities in unscripted interaction.

Abstract:
Large Reasoning Models (LRMs) have demonstrated remarkable capabilities by scaling up the length of Chain-of-Thought (CoT). However, excessively long reasoning traces pose substantial challenges for training cost and inference latency. While various CoT compression approaches have emerged to address this challenge, they face inherent trade-offs: token-level methods often disrupt syntactic and logical coherence, while step-level methods based on perplexity fail to reliably capture the logically critical reasoning steps because of the dilution of logical information. In this paper, we propose ASAP (Anchor-guided, SurprisAl-based Pruning), a novel coarse-to-fine framework for CoT compression. ASAP first performs anchor-guided pruning to preserve the core reasoning structure, which efficiently reduces the search space for subsequent processing. Leveraging the insight that logical branching choices are concentrated at the onset of reasoning steps, it then enables logic-aware pruning by selecting logically essential reasoning steps based on a novel first-token surprisal metric. Finally, ASAP distills these concise CoTs into target models so they can generate efficient reasoning traces at inference time. Experiments across math and code benchmarks show that ASAP preserves competitive accuracy while substantially reducing generated tokens, latency, and training cost, with additional validation on Qwen3 confirming its cross-model applicability.

Abstract:
Role-playing large language models (LLMs) are expected to adopt a character’s style while also respecting that character’s knowledge boundaries. Prior evaluations detect character hallucination but rarely distinguish whether errors arise from failure to recognize a boundary or from failure to comply despite recognition. We introduce CHARM, a multicultural benchmark of 40 real and fictional characters drawn from five cultural-linguistic regions, and validated by native reviewers. It probes two boundary types, Temporal (historical vs. modern) and Cross-Universe (entities outside a character’s narrative or historical universe), using abstention-enabled multiple-choice questions. We propose a two-stage evaluation that separates Boundary-Awareness (explicit recognition that a query is out of scope) from Boundary-Compliance (abstention when answering concrete questions). Evaluations across six LLMs show that hallucination is driven predominantly by compliance failures. Models frequently acknowledge that a query lies outside the character’s knowledge yet still provide factual, out-of-character answers. By re-posing the same questions to the target character, we confirm that a large fraction of these cases are verified parametric overrides; the model stores the relevant fact but fails to suppress it. We also observe systematic cultural variation in these failures, consistent with imbalances in how characters from different regions are represented in model knowledge.

Abstract:
Text-to-SQL systems allow users to interact with databases using natural language, but most assume that each question must be answered by a single SQL query. In practical settings, some information needs are more reliably satisfied by generating multiple simpler SQL queries, which is not supported by existing benchmark or evaluation protocols. To address this gap, we introduce Spider 𝕊 , which, to the best of our knowledge, is the first benchmark for the Text-to-(Multi)-SQL task, supporting both single- and multi-query solutions within a unified evaluation framework. To enable this setting, we propose a flexible evaluation protocol that accepts any prediction covering the required information, regardless of query number or structure. Under our evaluation protocol, we evaluate several large language models on Spider 𝕊 and find that allowing multi-query outputs improves execution accuracy. However, current models demonstrate fragility: even a minor prompt change from generating one SQL to generating one or more SQL can significantly degrade performance. To improve robustness, we introduce a lightweight router that decides whether to generate a single or multiple SQL for a given question, leading to substantial performance gains.

Abstract:
Deep narrative understanding requires distinguishing what happens from how it is presented —moving beyond the chronological fabula to decode the sjuzet, the discursive organization through which themes are staged. We study this through motif recurrence, where symbolic significance emerges not from local cues, but through repetition-with-variation across the whole work. We operationalize this setting as Computational Motif Discovery, a transductive task that predicts missing line motif links using the full narrative structure of a play. We propose Critic’s Eye, which models the work as a heterogeneous Narrative Topology Graph and performs discriminative inference over global structural evidence. Critic’s Eye achieves 84.8% Hit@5 on our benchmark, markedly outperforming state-of-the-art proprietary foundation models(22.6%) by a factor of nearly four. Our analysis reveals that despite massive scaling, sequence-based models hit a "ceiling", struggling to resolve dispersed dependencies. These findings suggest that explicit structural priors are a far more effective inductive bias than parameter scaling for decoding the complex architecture of long-form literature.

Abstract:
Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA). To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm. While this approach has shown great success in textual QA, its potential in the multimodal domain remains under-explored. Existing multi-agent VQA methods predominantly adapt text-centric protocols, focusing on textual discussions while ignoring the alignment of visual information. In this work, we reveal a key insight: answer-level agreement is insufficient for reliable multi-agent VQA; aligned visual evidence—shared support from the image regions agents rely on—is essential for trustworthy consensus. To leverage this insight, we propose EAGLE (Evidence-Aligned Grounded muLti-agent rEasoning), a training-free evidence-centered framework for coordinating multiple VLM agents. EAGLE explicitly exposes each agent’s grounding regions as visual evidence, enables mutual verification over the evidence, and uses evidence consistency to guide final decision-making. Experiments on six VQA benchmarks show that EAGLE achieves best average performance across domains while remaining lightweight, interpretable, and practical for deployment.

Abstract:
Negation is a fundamental aspect of language understanding and it is important vision-language models (VLMs) are able to process it effectively. We investigate how VLMs process negation by analysing attention patterns of different models. We draw comparisons between human gaze and attention in models when processing affirmative vs negated statements. We compare object level model attention to human eye-fixations in both binary and multary settings, evaluating with both synthetic and realistic datasets. We find models generally do not follow human-like attention patterns when resolving negation, suggesting they rely on mechanisms outside of attention. While layerwise analysis shows that PaliGemma2-3B shifts object attention to the alternative under negation, our headwise ablations suggest this is a byproduct rather than a negation processing mechanism. Additionally, we find that LLaVA-1.5-7B displays a spatial bias towards left-side objects. Our evaluation of models on downstream tasks shows that all models have a discrepancy between affirmative and negated questions highlighting the need to better understand the underlying processing of negation to close this performance gap. Code available at https://github.com/otmive/negation_attention

Abstract:
Large language model agents have made strong progress on software engineering, yet current systems suffer from a context coupling problem: the standard code editing interface conflates code inspection, modification planning, and edit execution within a single context window, forcing agents to interleave exploratory viewing with strictly formatted edit generation. Irrelevant context accumulates and edit reliability degrades. We propose SWE-Edit, which decomposes the editing interface into two specialized subagents: a Viewer that extracts task-relevant code on demand, and an Editor that executes modifications from high-level natural language plans—letting the main agent focus on reasoning while delegating context-intensive operations to clean context windows. We further show that effective edit-format selection can be trained into a small model rather than requiring frontier-scale capacity: GRPO training on Qwen3-8B improves edit success by 12.5 pp and brings an 8B open-source editor to parity with GPT-5-nano on downstream SWE-Bench resolve rate, with the learned adaptive find-replace/whole-file-rewrite policy contributing primarily on the cost axis ( -11.8% agent cost). To enable rapid editor iteration, we release PR-Edit, a lightweight editor-level evaluation that rank-orders the editor models we test consistently with their downstream SWE-Bench resolve rate. We release our code at https://github.com/microsoft/SWE-Edit .

Abstract:
Large Language Models (LLMs) for complex reasoning is often hindered by high computational costs and latency, while resource-efficient Small Language Models (SLMs) typically lack the necessary reasoning capacity. Existing collaborative approaches, such as cascading or routing, operate at a coarse granularity by offloading entire queries to LLMs, resulting in significant computational waste when the SLM is capable of handling the majority of reasoning steps. To address this, we propose RelayLLM, a novel framework for efficient reasoning via token-level collaborative decoding. Unlike routers, RelayLLM empowers the SLM to act as an active controller that dynamically invokes the LLM only for critical tokens via a special command, effectively "relaying" the generation process. We introduce a two-stage training framework, including warm-up and Group Relative Policy Optimization (GRPO) to teach the model to balance independence with strategic help-seeking. Empirical results across six benchmarks demonstrate that RelayLLM achieves an average accuracy of 49.52%, effectively bridging the performance gap between the two models. Notably, this is achieved by invoking the LLM for only 1.07% of the total generated tokens, offering a 98.2% cost reduction compared to performance-matched random routers.

Abstract:
Web domain credibility evaluation is vital for combating misinformation. It is conducted by examining factors such as domain type, transparency, and overall reputation. However, assessing the credibility of newly emerging web domains remains challenging since they have no reputation yet. Expert fact-checkers evaluate the credibility of domains by analyzing the content of their articles, including the presence of misinformation, bias, or propaganda. Yet, the ease of large-scale content generation enabled by LLMs has accelerated the creation of new content, rendering manual assessment insufficient and underscoring the need for automated approaches to domain credibility evaluation. In this paper, we introduce our Domain Credibility Evaluation Framework (DCEF), a temporal framework for domain credibility evaluation grounded in expert ratings. DCEF enables us to investigate whether the credibility of web domains can be assessed from their published articles following the workflow of expert fact-checkers, without any prior knowledge of the source domains themselves.

Abstract:
Paraphrases express the same intent in different words, so a robust vision-language model (VLM) should preserve its clinical conclusion when a question is rephrased. Medical VLMs often violate this expectation. We audit this failure on VQA-RAD using nine clinically grounded linguistic operations and 9,723 answer-preserving paraphrases. Across eight checkpoints from five architecture stacks (2.2B–27.4B), Negation Reframing is consistently the most fragile category, with clinical flip rates from 30.7% to 95.3%. A locked, category-conditioned LoRA reduces this failure by 59.0–75.8% across all eight checkpoints; on MedGemma-27B, paraphrase accuracy rises from 68.4% to 75.5%, although base accuracy falls from 79.6% to 73.9%. A pooled adapter performs nearly as well, so we use category-conditioned adapters primarily as diagnostic instruments. Two clinicians agree on 97.8% of a 500-pair calibration sample (κ=0.901), whereas agreement with an LLM equivalence judge is only κ≈0.14. Sparse-autoencoder clamping yields only a 5.8% reduction, and activation patching provides suggestive rather than conclusive localization evidence. Together, these results provide a reproducible robustness audit for radiology VLMs and make its dataset, generator, evaluation, and deployment limits explicit.

Abstract:
Sparse Autoencoders (SAEs) decompose language model activations into sparse, interpretable features, but these features often struggle to predict task-relevant concepts that remain clearly encoded in the original dense activations. We call this mismatch a concept faithfulness gap: a faithful SAE should preserve concept-level representations, such as toxicity or sentiment, rather than producing features that look interpretable but lose predictive relevance. We introduce ProbeSAE, a probing-guided fine-tuning framework that addresses this gap by preserving task-relevant concept structure in a more monosemantic sparse decomposition, enabling more effective causal control over model behaviour without sacrificing interpretability. We evaluate ProbeSAE on two SAE architectures (TopK, JumpReLU), two domains (toxicity, sentiment), and multilingual out-of-distribution settings spanning six languages. Trained exclusively on English data, ProbeSAE: (i) produces more predictive, monosemantic, and interpretable sparse features than pre-trained or naïvely fine-tuned SAEs across all six evaluation languages; (ii) learns language-independent feature directions, as confirmed by cross-lingual causal ablation; (iii) passes SAEBench evaluation, confirming these gains do not arise from task/domain overfitting. Together, these results show that the concept faithfulness gap in SAEs can be addressed through probe-guided training, improving monosemanticity and causal effectiveness without sacrificing interpretability, making probe-guided SAEs a more reliable tool for domain-specific interpretability analysis.

Abstract:
Large Language Models (LLMs) now exhibit remarkable reasoning capabilities through test-time compute scaling (TTS), with impressive performance across math and coding benchmarks. In parallel, research in model compression has developed pruning methods that seek to remove redundant/detrimental parameters without sacrificing task performance. The intersection of these two research advancements lays the foundation for our work. Specific to reasoning LLMs, prior work has shown that structured pruning (methods which remove entire set of layer blocks), significantly degrades TTS reasoning performance. However, in this work, we revisit this assumption and investigate whether unstructured pruning (methods that carefully remove only certain redundant/detrimental weights) exhibits similar limitations. Surprisingly, our extensive experiments across four reasoning benchmarks on two reasoning LLMs: s1.1-7B and Qwen3-8B, consistently show that unstructured pruning augments TTS performance compared to structured pruning, and at times can even outperform the unpruned full-weight LLMs. Furthermore, we also empirically study the impact of different layer-wise sparsity allocation strategies, which are an important parametric choice for instantiating these unstructured methods. These findings challenge the conventional notion that pruning always reduces TTS performance and in fact, suggest that carefully undertaken pruning can retain TTS effectiveness. Code: https://github.com/PALM-Lab-USF/TTS-Pruning

Abstract:
Attributes describing data content and context can induce diverse imbalance patterns that go beyond label imbalance alone. However, existing studies primarily address label imbalance while overlooking data attributes, such as topics and demographics, which can induce meaningful subgroup structure while causing model degradation on underrepresented subgroups. We propose Attribute-Agnostic Imbalance Augmentation (AIA 2 ), a framework for improving model robustness under varying subgroup imbalances without explicit subgroup annotations. AIA 2 automatically discovers varying imbalances via latent semantic distributions, obtains slices with both learning difficulty and subgroup imbalance deficits, and deploys a large language model (LLM) for subgroup-aware imbalance augmentation. We have evaluated AIA 2 on 5 popular corpora with rich domains and their attribute values, covering social issues and diverse topics. Results show improved performance on the lowest-performing subgroups and consistent gains over competitive baselines. Ablation studies confirm complementary contributions from each component, and additional analyses show that AIA 2 provides a practical and consistent way to improve worst-group robustness under data subgroup imbalance. Code is available at https://github.com/trust-nlp/AIA2-Subgroup-Robustness.

Abstract:
Retrieval-Augmented Generation (RAG) has become an effective method for enhancing large language models (LLMs) with up-to-date knowledge. However, it may pose a risk of copyright infringement, as IP datasets may be incorporated into the knowledge database by malicious Retrieval-Augmented LLMs (RA-LLMs) without authorization. To protect the rights of the dataset owner, an effective dataset membership inference algorithm for RA-LLMs is needed. In this work, we introduce a novel approach, CanaryTrace , to safeguard the ownership of text datasets and effectively detect unauthorized use by the RA-LLMs. Our approach preserves the original data completely unchanged while protecting it by inserting specifically designed canary documents into the IP dataset. These canary documents are created with synthetic content and embedded watermarks to ensure uniqueness, consistency, and statistical provability. During the detection process, unauthorized usage is identified by querying the canary documents and analyzing the responses of RA-LLMs for statistical evidence of the embedded watermark. Our experimental results demonstrate high query efficiency, detectability, and consistency, along with minimal perturbation to the original dataset, all without compromising the performance of the RAG system.

Abstract:
Recently, large language models (LLMs) have demonstrated excellent performance on general information retrieval tasks, which encode query and document into an embedding vector representing semantic information respectively. However, single-vector representations often fail to capture the multifaceted semantics of informative documents, limiting the adaptability across diverse scenarios. To address this, we propose PRISM (multi-PeRspectIve Semantic inforMation retrieval), a novel framework that leverages LLMs’ implicit reasoning capabilities to extract and structure diverse semantic perspectives. PRISM introduces query-aware semantic fusion to dynamically integrate relevant perspectives during retrieval through weighted query embedding interactions. Furthermore, we propose the contrastive semantic diversification, a training mechanism that disperses semantic embeddings to maximize perspective diversity. Evaluated on the English retrieval subsets of massive multilingual text embedding benchmark (MMTEB), PRISM achieves state-of-the-art general results compared with models trained exclusively on public data.

Abstract:
Large Language Model (LLM) routing has demonstrated strong capability in balancing response quality with computational cost. As users exhibit diverse preferences, personalization has attracted increasing attention in LLM routing, since even identical queries may require different models to generate responses tailored to individual needs. However, existing approaches are not fully personalized and often fail to faithfully capture the complex interactions between users and LLMs. Moreover, user preference data is typically scarce and inconsistent in format, which limits the effectiveness of methods that directly leverage user-specific data. To address these challenges, we propose GMTRouter, which represents multi-turn user–LLM interactions as a heterogeneous graph with five node types: user, LLM, query, response and turn, thereby maximally preserving the rich relational structure of the interaction. Through a lightweight inductive graph learning framework combined with a tailored user-conditioned graph sampling mechanism, GMTRouter learns to capture user preferences from few-shot data, enabling effective personalization. Extensive experiments demonstrate that GMTRouter outperforms the strongest baselines, achieving up to a 0.108 absolute improvement in accuracy and a 0.124 improvement in AUC. More importantly, we further demonstrate that GMTRouter can adapt to new users using only few-shot data, without extensive fine-tuning.

Abstract:
Multimodal knowledge editing aims to update visually grounded facts in Multimodal Large Language Models (MLLMs) while preserving unrelated text and visual behaviors. However, standard answer-level metrics can hide evidence misgrounding: an edit may produce the target answer on the original image yet remain sensitive to incidental context rather than the intended visual evidence. To examine this issue in a controlled setting, we construct WATERBIRD-EDIT, an analysis set that decouples bird habitat labels from background context and measures whether edited predictions remain stable when the target evidence is preserved but the background changes. Motivated by this diagnosis, we propose VICE (Visual In-Context Editing), a retrieval-based framework that stores edits in external memory and applies them through visual in-context demonstrations designed to expose task-relevant evidence. Experiments on three standard benchmarks show that VICE achieves strong lifelong editing performance, while results on WATERBIRD-EDIT show that it substantially reduces sensitivity to the controlled background shifts.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as an important paradigm for unlocking reasoning capabilities in large language models, exemplified by the success of OpenAI o1 and DeepSeek-R1. Currently, Group Relative Policy Optimization (GRPO) stands as the dominant algorithm in this domain due to its stable training and critic-free efficiency. However, we argue that GRPO suffers from a structural limitation: it imposes a uniform, static trust region constraint across all samples. This design implicitly assumes signal homogeneity, a premise misaligned with the heterogeneous nature of outcome-driven learning, where advantage magnitudes and variances fluctuate significantly. Consequently, static constraints fail to fully exploit high-quality signals while insufficiently suppressing noise, often precipitating rapid entropy collapse. To address this, we propose Elastic Trust Regions (ETR), a dynamic mechanism that aligns optimization constraints with signal quality. ETR constructs a signal-aware landscape through dual-level elasticity: at the micro level, it scales clipping boundaries based on advantage magnitude to accelerate learning from high-confidence paths; at the macro level, it leverages group variance to implicitly allocate larger update budgets to tasks in the optimal learning zone. Extensive experiments on AIME and MATH benchmarks demonstrate that ETR consistently outperforms GRPO, achieving superior accuracy while effectively mitigating policy entropy degradation to ensure sustained exploration.

Abstract:
We revisit exploration collapse in reinforcement learning with verifiable rewards (RLVR), from the perspective of the candidate distribution for next-token prediction. We formally show that as probability concentrates on the top-1 candidate, the expected number of distinct responses collapses to one regardless of the sampling budget K . This theoretical implication is further verified by our empirical tracking of top- N candidate probabilities during training, where the top-1 candidate progressively dominates while plausible alternatives are suppressed. These findings suggest a key desideratum for effective exploration: preserving non-negligible probability mass on the top- N candidates. To this end, we propose Candidate-aware Support Preservation (CaSP), with two complementary designs. Specifically, CaSP redistributes positive gradients among top- N candidates for correct responses, and applies a stronger penalty to the top-1 candidate for incorrect responses. Unlike many exploration-oriented methods that improve pass@ K at the cost of pass@1, CaSP improves pass@ K across the full K spectrum. These gains generalize to 6 math, 2 logical-reasoning, and 2 coding benchmarks, and scales to 32B-parameter models and sampling budgets up to K=1024 , positioning it as a principled, candidate-level approach for RLVR exploration.

Abstract:
Reinforcement learning for deep-search agents has largely focused on trajectory-level scoring—outcome correctness, citation-aware rewards, and evidence coverage. Yet the actions that expose supporting documents receive no targeted credit, a gap we call the reward-credit mismatch. We propose STAMP, in which a reference-based verifier judges whether each cited document supports an entity or relation in a training-time evidence graph, and first-exposure attribution traces each supported citation back to the action that first surfaced it. This step credit is injected through sign-preserving advantage modulation, which redistributes advantage across steps without changing the trajectory-level reward or the relative ranking of trajectories within each group. On BrowseComp, BrowseComp-ZH, and xbench-DS, STAMP improves the strongest GRPO baseline by +2.0/+5.5/+3.0 points under matched SFT initialization, training data, and search tools, and composes with both outcome-only and citation-rubric base rewards. Component ablations confirm that the provenance-based credit signal and the sign-preserving advantage modulation each contribute to the gains.

Abstract:
Inertial confinement fusion (ICF) is a leading pathway toward clean energy, but each shot at the National Ignition Facility costs on the order of one million dollars, making accurate AI surrogates a high-value target. We study exogenous-driven ICF waveform prediction, where a 512-step neutron-rate diagnostic must be inferred directly from a laser pulse and target design parameters, with no historical response observed. The regime stresses standard time-series predictors with temporal sparsity (picosecond peak in a nanosecond window), input-output scale mismatch (under 300 real shots), and peak sensitivity (picosecond timing). We propose ICF-DLM, to our knowledge the first LM-based ICF predictor, combining (i) a physics-typed decomposition into yield Y DT , peak timing t peak , and local waveform w local ; (ii) bidirectional denoising that defers commitment to peak location; and (iii) a physics-driven PPO reward re-injecting metric structure across numeric tokens. On ICFBench (50K simulations + 232 experimental shots), ICF-DLM cuts peak-timing error from 11.6 to 9.2 steps over a matched autoregressive LLaMA-3-8B and outperforms classical sequence models and LLM-based time-series predictors. Beyond ICF, the recipe shows potential to address science domains with low data and sparse events.

Abstract:
Manufacturing consumes one third of global energy and still has significant room for improvement in terms of energy efficiency. Optimal process control is essential for this purpose. However, synthesizing mathematical optimization models from messy, real-world industrial specifications requires bridging unstructured natural language and spatial diagrams with rigorous mathematical syntax. This poses a profound challenge for general-purpose Large Language Models (LLMs), which may introduce invalid constraints when tasked with modeling continuous multi-physics dynamics. To address this, we introduce little m, an AI agent designed to assist the formulation of industrial process control models. Combining a domain-specific knowledge repository with LLM-driven interaction, the proposed framework formulates real-world optimization problems as mathematical models. For systematic evaluation, we introduce the Industrial Process Control Benchmark (IPC-Bench), a novel multimodal dataset of 50 canonical scenarios requiring joint reasoning over text and process diagrams. Through comprehensive automated structural assessments and double-blind human evaluation, little m substantially outperforms state-of-the-art LLMs, generating semantically correct models. These evaluations assess formulation quality rather than solver feasibility, formal physical validity, or closed-loop industrial performance. The implementation of little m and the IPC-Bench dataset are available at https://github.com/yeyongchao/process-modeling-benchmark.

Abstract:
Multimodal large language models (MLLMs) have shown strong visual reasoning abilities, but knowledge-intensive visual question answering often requires external textual evidence beyond the image and the model’s parametric knowledge. Existing multimodal RAG systems commonly rely on Top- retrieval or reranking, which may return redundant passages and provide limited control over whether an answer update is sufficiently supported by the retrieved evidence. We propose CLIMB , a training-free inference-time framework for multimodal RAG. CLIMB first constructs a compact complementary evidence pool using an MMR-style objective that balances query relevance and passage-level redundancy. It then performs confidence-controlled refinement within this fixed pool: an R/E/C critic scores passages by relevance, evidence specificity, and cross-modal alignment, while an evidence-grounded confidence estimator accepts an updated answer only when the estimated confidence increases. This design provides a simple stopping criterion and reduces unnecessary refinement without modifying the underlying retriever or MLLM. Experiments on Encyclopedic-VQA and InfoSeek show that CLIMB consistently improves over retrieval-augmented multimodal baselines. Ablations further indicate that complementary pooling, critic-based scoring, and iterative confidence-controlled refinement each contribute to the final performance.

Abstract:
Scientific reasoning remains challenging for open-source models, largely due to the lack of high-quality scientific reasoning data. Existing datasets are often dominated by factual recall or formulaic problem solving, with limited emphasis on mechanism understanding, evidence-grounded reasoning, and hypothesis evaluation. To address this, we introduce SPARK (Scientific Paper Abstracted Reasoning sKeleton), a paper-oriented synthesis framework built on Sci-Base, a large-scale corpus of research papers spanning 10 scientific disciplines. Instead of directly converting papers into question-answer pairs, SPARK treats the claim-evidence-derivation structure of a paper as the fundamental unit of reasoning synthesis. Specifically, SPARK (1) distills each paper into a compact reasoning skeleton capturing its central claims and supporting evidence, enabling self-contained question generation, and (2) synthesizes reasoning tasks from four scientific perspectives: mechanistic reasoning, hypothesis falsification, quantitative derivation, and boundary calibration. A final consistency verification stage further removes unsupported or contradictory outputs. Using this framework, we construct Spark-234K, a scientific reasoning dataset with substantially higher difficulty and diversity than existing resources. Experiments show that Spark-234K consistently outperforms existing scientific reasoning datasets while achieving stronger performance with significantly fewer training samples.

Abstract:
Existing financial-NLP benchmarks mostly evaluate prepared artifacts such as filings, tables, or extracted values. Real accounting begins earlier: source documents must be reconciled into cited journal entries, aggregated into a balance sheet, and checked for contradictions. We introduce FINBALANCE, a multi-document accounting reconciliation benchmark built from source-document bundles across eight industries, three period types, and five difficulty levels. Human-authored business scenarios, accounting policies, tax/FX treatments, document schemas, distractors, and inconsistency templates are composed by a deterministic generator whose ledger produces journal entries, balance sheets, and 23 inconsistency-code labels. On a 710-record evaluation split, six contemporary large language models (LLMs) reach at most 46% exact final-balance-sheet accuracy. For four models, balance sheets reconstructed from their journal entries score 26–41 percentage points higher than their own reported balance sheets. Models often recover numerically plausible entries but fail to bind them to supporting documents and aggregate them consistently. Citation-pressure prompting barely changes document-linking errors, while ledger-feedback ablations substantially improve reported balance sheets and expose inconsistency-detection trade-offs. Expert finance reviewers validate the benchmark design and labels.

Abstract:
Large language models (LLMs) have shown promise in psychological support, yet existing CBT-oriented systems often rely on prompt-based imitation or supervised learning from counseling-like responses, without explicitly modeling the structured therapeutic process of Cognitive Behavioral Therapy (CBT). This limits their controllability, intervention completeness, and coherence in multi-turn counseling. We formulate CBT-based psychological dialogue as a stage-aware controllable generation problem, where therapeutic stages serve as explicit control signals for intervention selection and therapeutic progression. To provide structured supervision under limited access to real counseling data, we design a CBT-stage multi-agent data generation engine that synthesizes multi-turn dialogues with user-state trajectories, stage-specific interventions, and content-level verification. Using the generated stage-structured data, we train Qixin, a CBT dialogue model that internalizes stage-aware therapeutic behaviors without requiring multi-agent coordination at inference time. We further introduce a hierarchical evaluation paradigm covering single-turn counseling skills, CBT procedural fidelity, stage alignment, and longitudinal session coherence. Experiments show that Qixin improves CBT-oriented counseling capability and outperforms existing psychological dialogue models and similarly sized open-source LLMs at the process level, achieving stronger procedural fidelity, stage consistency, and session-level coherence. These results demonstrate the effectiveness of explicit therapeutic-stage supervision for process-aware psychological support. Code and data are released .

Abstract:
Rubric-based reinforcement learning extends RL beyond tasks with exact answers or rule-based verifiers by scoring responses against instance-specific criteria. However, this makes reward computation expensive: training requires repeated rubric judging, often with proprietary APIs or local generative LLM judges with 7B parameters or more. We study whether smaller language models can serve as efficient and reliable rubric-based judges. To make this question measurable, we construct PointRubric and RaR-Science-Static, two pointwise rubric-based evaluation datasets with instance-specific criteria and itemwise satisfaction labels. We compare three ways of extracting criterion-level judgments from small models: Generative verdicts, Yes/No Logprob margins, and Probe judges. Across both datasets, the Qwen3-1.7B Probe judge achieves the strongest criterion-level agreement among these methods, outperforming Generative and Logprob judges. Used as a GRPO reward model, it trains a policy from 0.232 to 0.643 on RaR-Science rubric score, compared with 0.594 for an 8B Generative judge baseline, while the baseline requires 10.7 × more reward-judge time. Task and domain transfer experiments further suggest that Probe judges preserve criterion-level reward structure across settings.

Abstract:
Large language model (LLM) personalization aims to generate responses that align with individual user preferences, making accurate preference inference critical. In practice, user-LLM interaction histories naturally contain rich preference signals. However, extracting preferences from these histories remains underexplored and challenging: the history can be long and noisy, preferences may evolve over time, and preferences may vary across situations even within the same topic. To address these challenges, we develop PREFINFER, a model that accurately infers user preferences from the user’s interaction history with an LLM. Specifically, we introduce PREFMINE, an automated data construction method that synthesizes high-quality preference inference data, where each sample includes a synthetic interaction history, a current user request, and the user’s preference for the current request, and construct PREFMINE-15K for training PREFINFER. Experiments show that PREFINFER achieves advanced performance on preference inference, and the preferences inferred by PREFINFER effectively guide LLMs to generate responses better aligned with user preferences. We conduct thorough ablation studies on the key design choices of PREFMINE, validating their effectiveness. We have released the code, data, and models to facilitate further research(https://github.com/THU-KEG/PREFINFER).

Abstract:
Diffusion language models (DLMs) have shown strong potential for general natural language tasks with in-context examples. Existing In-Context Learning (ICL) approaches largely inherit the practice of autoregressive language models (ARLMs), incorporating all examples into a fixed prompt. However, applying this rigid, static-prompt paradigm to DLMs incurs substantial computational overhead, as the model must evaluate the maximum context length at every step. We address this inefficiency with a key discovery: the block-wise KV-cache mechanism inherent to DLM inference enables the low-cost dynamic adjustment of the context . Following this intuition, our core idea is to start generation with a minimal prompt and progressively insert additional examples on the fly only when the generated tokens are of low confidence. Through rigorous empirical evaluations, we observe that average verified token confidence correlates strongly with generation accuracy, making it a reliable and computationally efficient signal of token quality. Formally, we propose D ynamic I n-Context P lanner (DIP), a context-optimization algorithm based on average verified confidence that dynamically ranks and inserts in-context examples during generation, rather than providing all examples up front. Experimental results on math and coding benchmarks with LLaDA-1.5 and LLaDA-8B-Instruct show that DIP achieves up to 1.59× and 1.36× speedups, respectively, while largely preserving the generation quality of the fixed-prompt baseline. Code: https://github.com/wmd3i/DIP

Abstract:
Graph-based NLP and symbolic reasoning models often rely on positional encodings to distinguish tokens, entities, or reasoning states, but the identifiability provided by these encodings remains poorly understood. We analyze a hybrid graph encoding that combines anchor distances with quantized low-frequency spectral energy. Treating the encoding as a deterministic observation map gives an exact image-preimage identity for localization ambiguity and a one-sided information-theoretic converse: if the joint anchor, spectral, and quantization budget is too small, distinct nodes must share the same code, which also lower-bounds task-label ambiguity observable from that code. Experiments on random 3-regular graphs, CLUTRR relation graphs, and Universal Dependencies trees support the predicted scaling. Under matched normalized budgets, Hybrid is best or tied at 74 of 87 attained thresholds, while PE-only UD probes show improved recovery of syntactic-tree geometry. The results therefore position positional encoding as a pre-model, encoding-level necessary-condition diagnostic rather than an achievability guarantee or an end-to-end model claim.

Abstract:
Large language models improve final-answer accuracy through extended chain-of-thought reasoning, but often spend tokens inefficiently and offer little inference-time control. Existing efficient reasoning methods control thinking length by shortening, early-stopping, or compressing traces, leaving how the model thinks implicit. In this paper, we propose Agentic Chain-of-Thought Steering (ACTS), which formulates reasoning steering as a Markov decision process where a controller agent adaptively steers a frozen reasoner during inference. At each step, the controller observes the reasoning trace and remaining thinking budget, then issues a steering action consisting of a reasoning strategy and a steering phrase that initiates the next reasoner step. This enables budget-aware strategy control for efficient reasoning while preserving the reasoner’s generation continuity. We initialize the controller agent from our constructed synthetic steering trajectories with multi-budget augmentation, and further optimize it via reinforcement learning with budget-conditioned reward shaping. Experiments across multiple benchmarks show that ACTS achieves competitive accuracy with substantial token savings, and enables controllable accuracy–token trade-offs across different reasoners and tasks. The code is available at https://github.com/Andree-9/ACTS .

Abstract:
As AI research increasingly moves beyond systems optimized for static tasks, a central challenge is building agents that learn continuously through experience-driven lifelong learning (ELL). In this paper, we introduce StuLife, a synthetic benchmark designed to evaluate selected operational capabilities relevant to ELL. Particularly, StuLife simulates a student’s college journey, from enrollment to academic and personal development, across three core phases and ten detailed sub-scenarios. StuLife is designed around four key design principles: From Isolated Tasks to Stateful Interaction, From Passive to Proactive, From Context to Memory, and From Imitation to Learning. In this dynamic environment, agents must acquire and distill practical skills and maintain persistent memory to make decisions based on evolving state variables (e.g., resource availability and time), and are also expected to initiate actions without external prompting. To this end, StuLife provides an evaluation protocol with dedicated metrics (e.g., StuGPA) for these capabilities. Our evaluation reveals that even the highest-StuGPA model, GPT-5, scores only 17.9/100, highlighting large gaps in long-term retention and proactive task initiation. Beyond evaluating LLMs on StuLife, we also explore the role of context engineering in improving agent performance on lifelong learning tasks.

Abstract:
Recent vision language models (VLMs) have achieved strong progress in video understanding. However, most existing video QA research and benchmarks still follow an offline, single-round paradigm, overlooking realistic interactions where users may interrupt the model during answer generation. To address this gap, we study online video QA under interruptions. We formulate the Online Video Question Answering under Interruption task and introduce OVIBench , the first standardized benchmark for this setting. OVIBench categorizes interruptions into three types: Cancellation, False Trigger, Correction and supports both open-ended and multiple-choice evaluations. To enable large-scale and reproducible testing, we develop an offline simulation protocol that reproduces interruption during generation under a unified temporal setup, together with a multi-dimensional metric suite for assessing interruption understanding and response generation. Experiments reveal clear weaknesses of mainstream VLMs in this setting, especially in reliably following user correction requests. Finally, we construct a train set OVI-Train for interruption-aware fine-tuning, which yields significant gains on OVIBench and validates the effectiveness of our benchmark and data design. OVIBench, OVI-Train, and the evaluation code will be released.

Abstract:
Applications such as personalized assistance and proprietary document analysis require large language models (LLMs) to generate outputs from private data. Yet powerful LLMs typically cannot be deployed on the resource-constrained devices where private data resides, and uploading private data to cloud-hosted LLMs exposes sensitive information. Recent work addresses this tension with a cloud-edge collaborative decoding paradigm, where private data are kept on the edge with a small language model (SLM) producing next-token distributions, which are fused with predictions from a cloud LLM operating solely on public data. In this paper, we systematically analyze the privacy risks of such a paradigm with a novel evaluation framework using constructed QA datasets, which show that such collaboration can expose substantial private-context information. To address such privacy leakage, we propose CoVeil, a defense mechanism which dynamically optimizes transmitted signals to suppress leakage during decoding time while preserving the collaborative quality. Extensive evaluations demonstrate that CoVeil consistently improves the privacy-utility trade-off over existing baselines by reducing data leakage by up to 87.2%, with minimal accuracy loss.

Abstract:
Omni-modal large language models are increasingly evaluated on clean text–vision–audio inputs, where every channel is present, synchronized, and readily interpretable. Such scores are often taken as evidence of robust cross-modal fusion, but clean evaluation cannot tell whether success depends on stable cross-modal structure or on cues sufficient only in intact inputs. To address this gap, we define a modality fault line: a boundary at which model behavior becomes unstable when a modality remains present and human-interpretable, but its internal evidence structure is perturbed. We introduce SCEval (Structure-Corruption Evaluation) a diagnostic evaluation protocol that keeps the question, answer space, and modality channels fixed while applying controlled structural corruptions to text, vision, and audio individually and jointly. Built from 273 human-verified tri-modal examples from Social-IQ, OmniBench, and VALOR, SCEval evaluates 15 proprietary and open-source omni-modal systems. The results show that structural corruption lowers clean accuracy, text–vision damage forms the most stable shared fault line, and multi-modal degradation is non-additive rather than a simple function of the number of corrupted modalities. Clean omni-modal accuracy therefore does not establish that a model will remain reliable when cross-modal evidence becomes structurally unreliable.

Abstract:
Large Language Model (LLM) unlearning is essential for removing sensitive or copyrighted knowledge while preserving general utility. Existing methods often leave residual knowledge in intermediate representations, which can still be recovered. To address this, we propose Cascade , a hierarchical recoverability control framework that minimizes the internal identifiability of target knowledge. Cascade combines three complementary controls: path-level routing to suppress privacy-associated activation routes, representation-level compression to reduce geometric separability, and decoding-level intervention to limit residual recovery. Experiments on TOFU, MUSE-News, and WMDP, including robustness tests with query reformulation and extraction-style prompts, show that Cascade effectively reduces recoverability while maintaining stable model utility.

Abstract:
Achieving effective self-correction, where models verify and correct their own mistakes, remains a fundamental challenge for large language models (LLMs). In this work, we propose Self-Fix Step-DPO (SFS-DPO), a reinforcement learning based, two-stage framework for step-level self-verification and self-correction. The first stage strengthens step-level reasoning via step-level preference optimization, while the second stage explicitly trains models to self-verify and self-correct. We further introduce a teacher-assisted variant, SFS-DPO-R, which incorporates explanatory rationales for error verification to provide stronger corrective signals. Comprehensive in-domain and out-of-domain evaluations across multiple LLMs demonstrate that SFS-DPO and SFS-DPO-R consistently outperform prior step-level training baselines. Our analysis further reveals improvements in self-correction frequency and effectiveness, highlighting the importance of strengthening step-level reasoning for robust performance.

Abstract:
Memory has become a standard substrate for self-evolving agents, yet retaining experience is not the same as learning how to evolve through it. Existing memory agents can store trajectories, retrieve reflections, or accumulate skills, but often lack the holistic competence to select useful experience, act on it, write reusable knowledge, and maintain a growing repository. We introduce OPD-Evolver, a slow-fast co-evolution framework that cultivates such an agent evolver through on-policy self-distillation. In the fast loop, OPD-Evolver interacts with a four-level memory hierarchy to read, use, write, and maintain experience for rapid test-time evolution. In the slow loop, outcome-calibrated memory attribution and privileged hindsight distill these four abilities into the deployable policy. Across multi-domain benchmarks, OPD-Evolver surpasses memory systems such as ReasoningBank by up to 11.5 points, and training-based methods such as Complementary RL by ~6.45 on average. Further analysis shows that OPD-Evolver internalizes high-value experience and memory management, enabling OPD-Evolver-9B to challenge giant counterparts such as Qwen3.5-397B-A17B and Step-3.5-Flash, pointing beyond memory-augmented agents toward genuinely qualified agent evolvers.

Abstract:
Current video-to-music (V2M) models lack semantic control and fail to penalize instruction violations, largely due to their reliance on reconstruction objectives and the representational bottleneck of static cross-modal conditioning in Diffusion Autoregressive (DAR) architectures. To resolve this, we introduce VIBE, a novel text-and-video-to-music (T+V2M) generation model that leverages: (1) Conditioning Connection, a depth-wise cross-layer conditioning mechanism that dynamically bridges the planning and diffusion refinement heads and (2) a comprehensive reward modeling taxonomy, optimizing for both hard, verifiable constraints (e.g., tempo, key) and soft, subjective qualities (e.g., musicality, multimodal alignment) with a structured 5-stage training curriculum. Upon evaluation using audio-visual alignment, instruction following, and audio quality metrics, along with a subjective human evaluation study, we observe that VIBE demonstrates enhanced controllability and instruction adherence while performing comparably to most evaluated baselines on generation fidelity and multimodal alignment.

Abstract:
The manufacturing sector is increasingly adopting Multimodal Large Language Models (MLLMs) to move from low-level perception toward higher-level reasoning and decision support, yet current evaluations rarely reflect the fine-grained precision that real-world manufacturing demands. Progress is further hindered by data scarcity and a lack of fine-grained domain semantics in existing datasets. To bridge this gap, we introduce FORGE, a benchmark for three core quality-assurance tasks of discrete manufacturing: workpiece verification, structural surface inspection, and assembly verification. We first construct a high-quality multimodal dataset spanning real-world 2D images and 3D point clouds, annotated with fine-grained domain semantics such as standardized model numbers. We then evaluate 18 state-of-the-art MLLMs on these three tasks, revealing significant performance gaps. Counter to conventional understanding, our bottleneck analysis on five representative models indicates that visual grounding is not the primary limiting factor. Instead, insufficient domain-specific knowledge is the key bottleneck. Beyond evaluation, we show that our structured annotations can serve as an actionable training resource: supervised fine-tuning of a compact 3B-parameter model on our data yields up to 90.8% relative improvement in accuracy on held-out manufacturing scenarios, providing preliminary evidence for a practical pathway toward domain-adapted manufacturing MLLMs.

Abstract:
Markdown skill libraries for LLM agents ship as free-form prose, forcing the agent to re-derive both the input schema and the concrete invocation syntax on every retrieval. This produces a "confused → re-retrieve → still confused" loop: the agent issues a partially-correct action, receives uninformative feedback, and re-retrieves the same prose. We propose Skill-as-Pseudocode (SaP), an automatic conversion of markdown skill libraries into typed pseudocode with deterministic quality control. From each cluster of similar procedural passages, SaP extracts a typed contract and filters it through a four-check deterministic verifier (coverage, binding, replacement, risk). Promoted contracts are inlined into a rewritten skill skeleton alongside restored action templates, giving the agent two complementary signals: a typed signature for what a skill does and a concrete template for how to invoke it. On the ALFWorld unseen split (134 games, gpt-4o-mini, three seeds), SaP wins 82/402 paired games versus 47/402 for the Graph-of-Skills (GoS) baseline (pooled McNemar p = 8.2 × 10⁻⁵), at −22.8 ± 6.4% input tokens and −14.5 ± 4.1% LLM calls per game. A bundle-component ablation attributes the gain to the pairing of typed contracts with concrete action templates: the contract alone falls below the prose baseline.

Abstract:
Mixture-of-Experts (MoE) language models increase parameter capacity without proportional per-token computation, yet deployment still requires storing the full expert pool, making expert pruning important for reducing memory and serving overhead. Existing task-agnostic expert-pruning methods are typically calibration-dependent: they estimate expert importance from routing or activation statistics on a calibration set, making pruning decisions sensitive to calibration-data variation while introducing substantial preprocessing cost. We propose AIMER (Absolute mean over root mean square IMportance for Expert Ranking), a simple calibration-free criterion that identifies more distinct experts by capturing the concentration pattern of expert weights, making it well suited for task-agnostic expert pruning. Across 7B to 47B MoE language models with distinct architectures and 16 diverse benchmarks, AIMER consistently delivers stronger capability balance across diverse tasks than existing calibration-free methods. Surprisingly, AIMER also achieves better balance than strong calibration-based expert-pruning baselines calibrated on the widely used task-agnostic C4 corpus, while requiring only 0.22–2.06 seconds to score all experts.

Abstract:
Complex multi-hop question answering over text, tables, and images requires integrating heterogeneous evidence with inspectable reasoning. Single-LLM chain-of-thought is opaque, while planner–executor pipelines concentrate faults in a central plan, propagating early errors. We introduce DeALOG, a decentralized, central-planner-free multi-agent framework whose specialized agents, i.e., Table, Context, Visual, Summarizing, and Verification, coordinate only by reading and appending to a shared, typed natural-language log as persistent memory, with a lightweight scheduler rather than a planner. Rather than competing on raw accuracy, DeALOG targets what a central planner forfeits: robustness, since no single plan concentrates errors and evidence stays globally visible, and transparent, since the append-only log yields a provenance-anchored, human-readable reasoning trace. We position decentralized log-mediated coordination as a robustness-and-transparency primitive for multimodal reasoning

Abstract:
Supervised fine-tuning is the standard approach for adapting pretrained large language models to specialized domains, yet it treats the pretrained checkpoint as a fixed and neutral initialization. In practice, pretrained models encode broad general-domain priors that can conflict with target-domain learning. We propose Forget-to-Focus (F2F), a two-stage domain-specialization framework that first applies controlled unlearning to reshape the pretrained checkpoint and then performs ordinary supervised fine-tuning from the resulting initialization. A key challenge is deciding what to forget. We introduce NG-DS, a neuron-guided data selection method that identifies pretraining-dominant and domain-dominant feed-forward neurons, then selects forget examples that strongly activate the former while weakly activating the latter. This model-aware forget set directs unlearning toward pretraining-like behavior while reducing the risk of erasing target-domain-relevant knowledge.We evaluate F2F on five LLM backbones spanning 0.6B-72B parameters across coding, medical question answering, and mathematical reasoning. F2F consistently improves over standard SFT and adaptation baselines. Across coding, medical QA, and mathematical reasoning, F2F consistently improves over standard tuning. On HumanEval, it improves over SFT by +13.84 for Qwen3-0.6B, +7.86 for LLaMA3.1-8B, and +10.14 for LLaMA2-13B. On PubMedQA, it improves Qwen3-0.6B from 62.60 to 80.50, showing that targeted preparatory forgetting provides a better initialization for tuning.

Abstract:
Template-based contrastive synthesis is scalable, but its candidates often differ only in a few entity-slots while sequence-level optimization spreads supervision over mostly shared templates. We formalize this as the Resolution Mismatch Problem and propose KARMA, which enumerates schema-constrained paths over domain knowledge graphs and verbalizes them into slot-aligned contrastive candidates. Slot-Parallel Alignment (SPA) then applies a decoupled slot-level objective to route preference supervision to discriminative entity-slots, with slot-aware masked attention serving as an optional packed-evaluation implementation. Across biomedical, computer-science, and chemistry benchmarks, KARMA outperforms base LLM and same-data SFT baselines, and compares favorably with sequence- and token-level preference methods.

Abstract:
Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs that are expected to produce trading value. Existing evaluations typically report performance metrics, but rarely examine agentic viability: whether dynamic LLM-mediated decisions convert their induced costs into measurable incremental profit. To apply this criterion, we introduce TradeLens, a trace-grounded diagnostic toolkit for evaluating agentic trading systems from their trading records, runtime traces, and deployment configurations. It reconstructs trading trajectories, attributes profit and cost to interpretable evidence, and diagnoses whether and why an agent pays for its own intelligence. We conduct extensive analysis across backbone models, capital scales, trading frequencies, and system architectures, together with deployment discussion. Our results show that viability hinges on intelligence-to-profit conversion: models exhibit different failure patterns, such as poor asset selection in DeepSeek-V3.2 and negative timing in GLM-4.7, while capital scale, trading frequency, and architecture matter primarily in these runs, especially through their effects on decision-attributed timing value. These findings reframe the evaluation of LLM-based trading agents from capability-centric performance ranking to trace-grounded diagnosis of intelligence-to-profit conversion. Our code is available at https://github.com/ParadooxAI/TradeLens.

Abstract:
Longitudinal clinical reasoning over electronic health records requires tracking evolving physiological measurements, laboratory results, and interventions across extended patient trajectories. Existing LLM-based clinical reasoning systems often rely on repeatedly serializing patient histories or exchanging unconstrained textual agent messages, leading to context drift, unstable reasoning, and growing inference cost over long horizons. We present Vital Trace, a protocol-constrained multi-agent framework for future clinical risk prediction over evolving ICU trajectories. Instead of maintaining unbounded textual histories, Vital Trace uses a compact persistent patient-state memory together with staged reasoning performed by four coordinated agents: a Router, Reasoner, Auditor, and Steward. To support temporally coherent reasoning, we introduce a manually curated Global Protocol containing physiological state-transition rules and a dynamic patient-state representation that tracks hemodynamic, respiratory, renal, metabolic, and inflammatory instability over time. We evaluate Vital Trace on MIMIC-IV and eICU using future vasopressor-support, respiratory-support, renal-support, and deterioration prediction tasks. Results show that structured protocol-constrained reasoning improves temporal consistency, communication stability, calibration, and interpretability compared with free-form multi-agent baselines while achieving strong predictive performance across long ICU trajectories.

Abstract:
In-context learning (ICL) adapts LLM behavior by placing task-relevant context in the prompt. While powerful, this mechanism can be inefficient: the context must be reprocessed for every query, and when it is substantially longer than the query itself, it dominates inference cost in both latency and token overhead. This paper contends that the behavioral shift induced by an in-context signal can be compiled directly into the model weights, enabling subsequent inference without including the original context in the prompt. To this end, we propose In-Context Compilation (ICC), a gradient free method that encodes a context-induced behavioral shift as a sparse closed-form edit to attention output-projection weights. ICC is calibrated solely from the activation differences between no-context and with-context forward passes on a small set of behaviorally informative examples, requiring neither backpropagation nor trainable parameters. We evaluate ICC in the few-shot setting, where labeled contexts provide controlled behavioral targets. Across diverse LLM architectures and classification and reasoning benchmarks, the ICC-adapted LLM closely approximates the performance of the same LLM prompted with the full context, while omitting that context at inference time. In some cases, it even surpasses the with-context model. Moreover, on most benchmarks, ICC is competitive with and often outperforms gradient-based PEFT methods (LoRA, DoRA, IA3, Prompt/Prefix Tuning, LoReFT) and training-free activation-injection baselines (Function Vectors, ICV), while completing compilation in 10–18s with only ∼64 –120 MB of overhead.

Abstract:
Recommending medications from electronic health records (EHRs) is a safety-critical task that demands therapeutic coverage, explicit safety constraints, traceable justification, and continual adaptivity. Supervised neural models freeze at deployment and cannot reason about new phenotypes, while existing LLM pipelines are single-pass and lack a cross-case improvement loop. We propose SEMPA, a training-free agentic framework that couples a Proposer with a Critic in multi-round adversarial deliberation, grounds candidates with molecular-aware retrieval, contrastively evolves both agents’ natural-language policies under a semantic-anchor constraint, and enforces rule-based DDI and contraindication checks. On 100 MIMIC-IV encounters, SEMPA reaches 0.3116 Jaccard, 0.7295 F1, and a 0.0172 DDI rate, improving F1 by 57.7% and cutting DDI by 77.5% over the strongest supervised baseline. Two LLM judges rank SEMPA highest, and six-backbone studies confirm that gains derive from the framework rather than a specific model.

Abstract:
Natural Language Explanations (NLEs) describe how Large Language Models (LLMs) make decisions by drawing on external Context Knowledge (CK) and Parametric Knowledge (PK). Understanding the interaction between these sources is key to assessing NLE grounding, yet these dynamics remain underexplored. Prior work has largely focused on i) single-step generation and ii) modeled PK–CK interaction as a binary choice within a rank-1 subspace. This approach overlooks richer interactions and how they unfold over longer generations, such as complementary or supportive knowledge. We propose a novel rank-2 projection subspace that disentangles PK and CK contributions more accurately and use it for the first multi-step analysis of knowledge interactions across longer NLE sequences. Experiments across four QA datasets and three open-weight LLMs demonstrate that rank-1 subspaces struggle to represent diverse interactions, whereas our rank-2 formulation captures them effectively, highlighting PK alignment for supportive interactions and CK alignment for conflicting ones. Our multi-step analysis reveals that hallucinated generations exhibit strong alignment with the PK direction, though context-faithful generations maintain a more balanced alignment between PK and CK.

Abstract:
When prompting language models for psychometric assessment, researchers assume that the responses reflect the injected persona and the meaning of the survey item. We test this premise using a diagnostic design that crosses five semantically distinct baseline personas with five semantically equivalent variants of each of four prompt components (persona wording, task instruction, item wording, option symbol). Measuring the 1-Wasserstein distance between the resulting response distributions and partitioning the variation among the five components allows for the separation of target effects from prompt artifacts. We apply the framework to 13 open-weight small language models (0.6B–14B) on the Big Five Inventory and the Short Dark Triad. We find that in most models, the task instruction and option symbol displace response distributions further than paraphrasing the persona description or the item itself. For a substantial share of items, the artifact share of explained variation exceeds 50%; non-semantic changes of the prompt account for more response variation than the baseline personas. Our framework lets researchers quantify these prompt artifacts before interpreting psychometric output.

Abstract:
We study how explicit world-modeling objectives affect the internal representations and downstream capability of Transformers, using Rubik’s Cubes as our training domain. We ask: (1) how does explicitly pretraining a world model affect a model’s latent representations, (2) how does world-model quality affect post-training performance, and (3) how should a finite data budget be split between pretraining and task-specific fine-tuning? We compare standard action-prediction fine-tuning with two strategies that add explicit state-prediction supervision: state pretraining followed by fine-tuning, and joint action and state training. We measure task accuracy after Group Relative Policy Optimization (GRPO). We further find that explicit world-modeling yields better representations in terms of higher probing accuracy and steerability of the model, and that better representations yield larger gains from GRPO, especially on harder cube states. Finally, when the fine-tuning budget is held fixed, probe accuracy strongly predicts the GRPO improvement. Under a fixed total data budget, accuracy is maximized by allocating only a small fraction to pretraining.

Abstract:
Text-to-speech systems increasingly process user-generated text (UGT) such as ppl and imo, whose pronunciation must be inferred from the canonical rather than surface form. We introduce UGTPHON, the first grapheme-to-phoneme (G2P) benchmark for UGT in English, Vietnamese, and Korean, with an inference-grounded taxonomy. Existing G2P models and frontier LLMs share a systematic failure pattern, with the canonical-to-non-canonical gap reaching up to 66.8 PER points. We propose a retrieval-augmented G2P approach that explicitly models canonical-form inference through retrieval and staged decoding. Under 0.5B parameters, our approach shrinks this gap by up to 83% while outperforming most frontier LLMs that are orders of magnitude larger, demonstrating that explicit compositional structure can substitute for scale.

Abstract:
Text-to-SQL generation remains challenging for small and medium-sized language models, which must produce queries that are structurally valid, schema-grounded, and semantically faithful under limited inference budgets. Naive test-time scaling strategies, such as repeated sampling or unconstrained search, often spend budget on invalid, redundant, or semantically weak candidates. We present BASS, a Budget-Aware Structured Search framework for Text-to-SQL generation. BASS searches over deterministic SQL semantic units produced by rule-based Structural Boundary Detection, with lightweight admissibility filtering to reject malformed units. It scores completed candidates using an execution-gated rubric verifier: non-executable SQL receives zero reward, while executable SQL is assigned a continuous semantic score based on schema grounding, intent matching, predicate correctness, aggregation and ordering, and join/compositional structure. To improve budget usage, BASS introduces stagnation-aware branch suppression and exhaustion-aware early stopping, reallocating computation away from saturated search regions. Experiments on Spider and BIRD show strong zero-shot performance across Qwen2.5-Coder backbones. Controlled analyses further show that BASS uses fixed maximum rollout budgets adaptively, achieves better quality–cost behavior than naive repeated sampling, and benefits from semantic verification and stagnation-aware budget control.

Abstract:
Parameter-Efficient Fine-Tuning (PEFT), particularly Low-Rank Adaptation (LoRA), has become a standard approach for adapting Large Language Models (LLMs) under limited compute. However, in continual settings where models are updated sequentially with small datasets, conventional LoRA updates struggle to balance rapid adaptation and knowledge retention. Existing methods typically treat the low-rank space as a homogeneous update region, lacking mechanisms to regulate how short-term updates are consolidated over time. We propose a continual LoRA framework with Pro gram memory, inspired by C omplementary L earning Systems in neuroscience. Our approach, dubbed ProCL , organizes LoRA adapters into structured program memory slots that are dynamically retrieved through input-conditioned attention. This enables rapid and localized adaptation, encouraging similar inputs to reuse shared adapter regions while reserving unused capacity for future data. The slots are then combined with the underlying adapter, which maintains a distributed representation that gradually accumulates knowledge across tasks to balance plasticity and stability. Our method operates entirely within the LoRA parameterization and incurs no additional inference cost. Experiments on diverse benchmarks demonstrate improved retention and reduced catastrophic forgetting over other continual LoRA strategies.

Abstract:
Chain-of-Thought (CoT) reasoning has become a key problem-solving capability for advanced large language models (LLMs), but the "over-thinking" issue still plagues LLMs. In this paper, we propose a novel learning scheme termed Adaptive-Budgeting based Policy Optimization (ABPO) to balance the performance and efficiency of CoT reasoning. Our ABPO defines the RL training as an adaptive curriculum learning process, where example pools are curated to categorize training examples into three types, namely the mastered, learning and hard ones, respectively. As the training progresses, ABPO will adaptively schedule the examples with proper length budgets, and the example pools will also be dynamically updated based on the model status, thereby achieving a good balance between efficiency and performance of CoTs. The experimental results not only show the substantial efficiency improvements by ABPO, e.g., reducing token length by 78.3% while improving 2.0% performance of DeepSeek-R1-Distill-Qwen-1.5B on average, but also show its obvious advantages over the compared methods, e.g., reducing 59.4% length and increasing 8.3% performance on average than HAPO, respectively. Our code is released at https://github.com/SenriAkane1/ABPO/

Abstract:
Large reasoning models improve difficult problem solving by spending many tokens on internal reasoning. This paradigm is natural for single-turn tasks, but poorly matched to language agents: an agent may take tens or hundreds of environment steps, making deep thinking before every action costly and sometimes harmful. We study agentic deep thinking: when should an agent invoke long-form private deliberation, and how should it be exposed inside a ReAct-style loop? We propose a selective deep-thinking framework that separates the short externalized Thought used to choose the next action from a longer private memo used to reorganize future behavior. Without model post-training, we instantiate this framework with Periodic Thinking, Sel.-Gate, and Sel.-Action. Across three benchmarks and six model families, selective deep thinking improves agents, but the best interface depends on backbone capability: strong proprietary models benefit most from deliberation as a callable action, while smaller open-source models are more robust with a separate gate. We then use GRPO to teach a less capable backbone when to deliberate, turning the weakest zero-shot interface into the strongest learned policy and improving transfer to unseen agentic tasks. These results show that selective deep thinking can improve both the effectiveness and efficiency of long-horizon language agents.

Abstract:
Understanding agent behavior requires methods that scale to thousands of trajectories and surface new patterns in long, often unfamiliar tasks where pre-built classifiers fall short. We propose to bring grounded theory into agent trajectory analysis: a six-decade-old qualitative method from the social sciences, with a principled saturation criterion and an auditable trail from data to theory. We propose AutoTraceGT (Automated Trace analysis through Grounded Theory), the first multi-agent pipeline that automates grounded theory on agent trajectories. It iteratively performs open, axial, and theoretical coding until saturation, producing a behavioral taxonomy tailored to each task. Across six trajectory corpora, AutoTraceGT produces codebooks that recover 73–91% of the failure modes in human-annotated taxonomies and surface additional patterns that those taxonomies miss. The emergent theoretical narrative aligns with prior expert accounts. Used as a deductive feature space, the codebook outperforms zero-shot and few-shot LLM baselines on downstream failure prediction. These results suggest Grounded Theory offers a scalable analytic tool for ML researchers and agent developers studying what agents actually do.

Abstract:
Users of modern platforms repeatedly need summaries of recent dialogue, but the window rarely contains enough context to be interpreted on its own. We formalize this setting as streaming dialogue summarization, where a system must summarize a current window using selective memory from an unbounded history under a fixed budget. We show that the central challenge is not how much history is accessed, but whether memory recovers the evidence that the current window presupposes. We construct a benchmark and evaluation protocol that separately assesses whether memory contains gap-resolving evidence and whether the generated summary reflects it. We propose ReMEMBER, a missing-evidence memory framework that conditions retrieval on unresolved window dependencies and refines retrieved chunks into evidence-dense memory under a fixed budget. Experiments on dialogues with histories up to 160K tokens show that ReMEMBER improves memory recall and gap-resolution completeness over memory construction baselines under the same budget. The code and benchmark will be released upon publication.

Abstract:
Reliable multi-bit watermarking is essential for tracing the provenance of large language model (LLM) outputs. However, existing high-capacity schemes typically synchronize pseudo-random functions through discrete token n -gram matching, which makes them vulnerable to desynchronization under semantics-preserving paraphrasing attacks. To address this issue, we propose HSPMark, a hierarchical semantic projection marking mechanism for context-aware multi-bit watermarking, which shifts watermark synchronization from brittle token histories to continuous semantic anchors. HSPMark extracts stable contextual features using a pre-trained sentence-level encoder and a lightweight Robust Semantic MLP, then projects these features into context-conditioned pseudo-orthogonal bias matrices for multi-bit payload injection. To balance payload capacity and generation quality, HSPMark introduces an entropy-adaptive embedding mechanism that modulates logit bias according to predictive uncertainty. Extensive evaluations demonstrate that HSPMark improves robustness against paraphrase-induced desynchronization and local perturbations while maintaining a competitive trade-off among bit accuracy, payload capacity, and text quality compared with existing high-capacity baselines.

Abstract:
Training Question Answering (QA) systems on real customer interaction data is often infeasible to reproduce or share due to privacy and intellectual property constraints. We introduce dataset declassification, a framework that replaces proprietary QA datasets with public, semantically equivalent counterparts that preserve training and evaluation utility while satisfying typical legal and privacy requirements. The approach maps proprietary questions to public equivalents via large-scale retrieval and reconstructs answers from domain-matched corpora using LLM-based annotation. Experiments on four QA benchmarks show that models trained and evaluated on declassified datasets achieve performance within 𝛥 ≈ 0 of the original data. The method targets datasets whose underlying information needs already exist in public knowledge sources (e.g., knowledge available on the web and used by public-facing services). We further provide explicit applicability conditions showing that declassification succeeds when sufficient semantic overlap exists between proprietary and public corpora, and degrades predictably when it does not.

Abstract:
Activation steering has become a popular training-free method to control LLMs’ behavior by injecting precomputed direction vectors into the model’s residual stream at inference time. Yet, its robustness to realistic input variation remains under-explored. We present the first systematic evaluation of activation steering robustness under adversarial text perturbations on the inputs, covering four extraction methods, three attack strategies, six personas from the Anthropic Model-Written Evaluations dataset, and five models ranging from 1.5B to 30B parameters. Attacks succeed broadly across all settings: directional robustness drops by up to 64%, post-attack confidence collapses near or below 0.25 across all methods and models, and steering strength degrades on nearly every steerable input. Layer selection is equally fragile, with the optimal layer identified by an automated method on clean inputs shifting by up to 17 positions under perturbation, a failure that compounds the vector-level breakdown. Extracting vectors from adversarially perturbed inputs partially recovers steerability for PCA and MD on mid-to-large models, but they consistently fail to locate the improved optimal layer, limiting the practical benefit of this mitigation. Together, these findings reveal that the brittleness of activation steering is structural rather than method-specific, and that current layer selection strategies are not robust enough for real-world deployment. Our code is available at https://github.com/kien206/steering-adv-robustness.

Abstract:
Test-time tree search improves the reasoning capability of large language models under fixed inference budgets. However, existing solution-level MCTS often suffers from local homogenization : once a promising solution pattern is found, subsequent search tends to refine nearby variants rather than explore distinct reasoning paths. This wastes computation on redundant candidates and limits solution-space coverage. To address this problem, we propose Diversity-Aware Reasoning Tree Search (DARTS), a Bayesian tree-search framework that treats reasoning-path diversity as an explicit search objective. DARTS separates expansion into local refinement and orthogonal regeneration, and updates action posteriors using a joint reward over task quality and reasoning-path diversity. On LiveCodeBench and CodeContests, DARTS achieves the best exploration performance and produces more diverse correct solutions. With half the budget, it matches or exceeds the strongest tree-search baseline in four of six settings. On molecular optimization tasks, DARTS improves both optimization quality and structural diversity, suggesting cross-domain transfer.

Abstract:
Sentence segmentation in Arabic is challenging due to ambiguous and inconsistent punctuation, with many texts lacking reliable sentence boundary markers. Existing approaches rely heavily on punctuation cues and are typically evaluated on well-formed text, limiting their robustness in realistic Arabic settings. To address this, we introduce AraSEG, a genre-diverse sentence segmentation corpus spanning eight genres and a wide range of punctuation and document structure conditions. Using AraSEG, we evaluate LLMs, lightweight encoder models, and dependency parser-based models under increasingly challenging segmentation settings. Our experiments show that lightweight encoders, and even dependency parser-based models, outperform LLMs under the hardest conditions. We further investigate the effects of training data size and genre diversity, finding that performance eventually saturates and cross-genre generalization remains challenging. We also demonstrate that accurate sentence segmentation substantially improves downstream dependency parsing. We make our code, data, and models publicly available.

Abstract:
In this paper, we investigate how script knowledge is distributed across the layers of LLMs using two complementary interpretability methods: logistic regression probing and logit-lens analysis. Our probing experiments reveal a clear asymmetry: both the input script and the instructed output script are encoded in the earliest layers of the network, while, in contrast, commitment to the actual output script emerges only in the final layers, with the model’s intermediate representations defaulting to Latin throughout most of the layers. This two-stage process is confirmed by logit-lens analyses, which show that script commitment consistently occurs at the very last layers of the LLMs. Together with the weaker script-following performance observed in smaller models, these results form a converging body of evidence linking script commitment to model depth, with broader implications for the design of sufficiently deep, inclusive multilingual architectures.

Abstract:
Synthetic video data with verifiable rewards offers a scalable source of supervision for temporal motion understanding in video-language models, yet it remains unclear which synthetic tasks transfer reliably to real-video benchmarks.We investigate this question through a diagnostic framework that treats each synthetic task as a controlled intervention.Starting from a shared Qwen3-VL-2B checkpoint with frozen vision encoder, we independently train nine programmatic motion tasks with RLVR and evaluate each resulting model across six temporal-motion dimensions, such as direction, rotation, and velocity, yielding an interference matrix of cross-dimensional transfer. The matrix reveals substantial variation in transfer that surface-level label alignment fails to explain: a frame-to-frame velocity task yields a large gain on its target dimension, whereas a nominally aligned trajectory-shape task degrades the dimension it is intended to improve. We find that transfer is better explained by computational alignment, namely whether the training task induces the local, order-sensitive temporal computation that the downstream capability requires.Counterfactual tasks that share semantic targets but differ in computational demands support this account, and the patterns hold on InternVL3-2B. We further introduce a training-free answer-invariance test under frame shuffling that flags shortcut-prone tasks prior to RLVR training. Using the interference matrix to filter tasks yields a task mixture that consistently outperforms uniform mixing on TOMATO, with positive transfer trends on MVBench, TempCompass, and TemporalBench.Our framework thus reframes synthetic video generation as an empirically grounded task-selection problem.

Abstract:
Omnimodal embeddings naturally involve both shared representations and modality-specific features across heterogeneous inputs. However, existing omnimodal embedding methods often rely on a single shared parameter space over mixed-modality data, limiting structural separation between universal and modality-specific representations. To address this, we propose Syn-Omni, a unified framework for structured omnimodal adaptation with modality specialization and controlled cross-modal collaboration. Specifically, we introduce Orthogonal Modality-Expert LoRA (OME-LoRA), which decomposes adaptation into a shared LoRA path for universal semantics and modality-expert LoRA paths for modality-aware specialization. Furthermore, Progressive Synergy Routing (PSR) enables experts to first establish modality-specific priors, then gradually interact with other modality-experts for cross-modal synergy. Evaluated across 81 diverse tasks spanning image, video, audio, and audiovisual modalities, Syn-Omni consistently outperforms omnimodal baselines, demonstrating the effectiveness of structured specialization and cross-modal progressive collaboration.

Abstract:
Hate speech annotation is costly, subjective, and prone to annotator disagreement, making large-scale dataset construction challenging. We systematically analyze how well large language models (LLMs) align with human judgments across ten theoretically grounded subjective attributes, such as dehumanization, violence, and sentiment, evaluating both small and large variants of Llama 3.1 and Qwen 2.5. Our analysis reveals a consistent split across all models: behaviorally explicit dimensions (insult, humiliate, attack-defend) correlate strongly with human annotations, while evaluative dimensions (respect, sentiment, hate speech) are systematically inverted. Demographic persona conditioning reduces model confidence without improving alignment. Building on these insights, we propose combining attribute-level LLM predictions via a confidence-weighted Ridge regression to reconstruct continuous hate speech scores from the Measuring Hate Speech corpus, achieving R 2 of up to 0.71 and outperforming direct prompting baselines, demonstrating that structured attribute decomposition recovers a richer and more human-aligned signal than end-to-end label prediction alone.

Abstract:
Large Language Models (LLMs) are reshaping recommender systems by enabling more semantic, generative, and interactive recommendation pipelines. However, this shift also introduces new fairness challenges, as biases may arise from pretrained knowledge, prompts, generated explanations, decoding strategies, and feedback loops. This survey provides a systematic review of fairness in LLM based recommender systems, organizing existing studies through a two-dimensional view of bias mechanisms and fairness targets, together with a structured overview of the evaluation landscape and mitigation strategies. We further connect fairness with broader trustworthy concerns, including explainability, privacy, robustness, and controllability. To the best of our knowledge, this is the first survey specifically focused on fairness in LLM-based recommender systems, aiming to provide a structured foundation for future research on comprehensive and reliable fairness evaluation in LLM-based recommender systems.

Abstract:
Specialist language models are usually understood through endpoint gains: the generalist scores lower, the specialist scores higher, and the difference is treated as evidence of specialization. This leaves the released update itself largely unexamined. We propose a paired weight-delta path audit and apply it to two public, aligned generalist-to-medical-specialist checkpoint pairs: Gemma-3-4B-IT → MedGemma-4B-IT and Qwen2.5-7B-Instruct → HuatuoGPT-o1-7B. In both pairs, the full decoder-side update strongly reconstructs measured medical benchmark movement (0.974 and 1.183 endpoint-normalized retention), making each decoder delta an appropriate substrate for the audit. Yet the movement is not cleanly localized. MLP is the strongest broad component family in both pairs, but mixed off-domain movements, 10-seed matched controls, and endpoint-anchored rollbacks prevent a unique coarse-family explanation. The audit therefore separates update-level reconstruction from component-level explanation. Its claims concern text-only multiple-choice benchmark movement, not clinical validation, repair, or circuit-level mechanism.

Abstract:
Reliable rubric grading requires more than accurate score prediction. Each judgement must be grounded in the mark scheme and evidence from the student answer. Existing credit-assignment and intervention methods, primarily designed for self-contained reasoning tasks such as mathematics reasoning, struggle in this setting because they do not identify where grading reasoning goes wrong or how the model’s belief about the final mark changes during reasoning. We propose Evidence-Diagnosed Intervention Training (EDIT), a two-phase framework for training more rubric-faithful LLM graders. First, EDIT-SFT locates problematic reasoning steps using internal model signals: posterior belief over the final mark and input-grounding scores. It then revises only these local steps with help from a rubric checklist. Second, EDIT-RL calibrates the grader with belief-guided reward shaping, penalising large harmful belief drifts while still allowing helpful exploration. Experiments on two real-world, multi-subject grading benchmarks demonstrate that EDIT consistently outperforms strong supervised fine-tuning and reinforcement learning baselines on both in-domain and out-of-domain splits, with ablation studies confirming that internal-state diagnostics drive these gains. Under deterministic rubric-edit interventions, EDIT-SFT is the most rule-responsive of all evaluated systems, and EDIT-RL largely retains this responsiveness while improving accuracy.

Abstract:
Breakthroughs in large language models are accelerating the evolution of medical artificial intelligence from standalone models to multi-agent systems. To organize this landscape, we propose a hierarchical taxonomy spanning Level 0 to Level 5. Given that foundational Level 0 to Level 2 architectures face cognitive and collaborative bottlenecks in complex clinical workflows, this survey focuses primarily on the architectural paradigm shift between Level 3 and Level 4. Specifically, we examine how systems transition from static Level 3 collaboration based on predefined roles to Level 4 self-evolving architectures that iteratively refine internal strategies through interactive feedback. Concurrently, we explore governance frameworks required to ensure system safety and trustworthiness under stringent clinical standards. Finally, we highlight that the capacity to dynamically generate agents and reconstruct collaborative topologies based on real-world scenarios will be essential for surpassing current capability limits. Building on this concept, we envision a Level 5 medical AI landscape defined by evolutionary agent swarms. To help researchers continuously track this field, our updated GitHub repository (https://github.com/Tiany1Yu/Medical_Multi_Agent_Systems_Survey_Papers) is publicly available.

Abstract:
Large language models increasingly express uncertainty through verbalized confidence, yet these expressions often fail to reflect the model’s sampled behavior. We study verbal confidence alignment as a distributional calibration problem, where the confidence target for a prompt is estimated from repeated model outputs rather than from a single response. However, group rollouts alone are insufficient, since the resulting target must provide a useful training signal. We propose Semantic-Answer Guided Entropy (SAGE), a group-level uncertainty target that constructs an answer-conditioned uncertainty geometry over sampled responses. SAGE preserves categorical, numeric, and symbolic answer distinctions while maintaining a smooth and scale-preserving calibration signal. We further introduce Group-Uncertainty Preference Optimization (GUPO), an uncertainty-channel training framework that supervises verbal confidence expressions rather than the full response. Experiments across factual, mathematical, and multiple-choice reasoning tasks show improved confidence ranking, lower calibration error, and reduced overconfidence.

Abstract:
Multimodal Large Language Models (MLLMs) are increasingly being applied in a wider range of real-world scenarios. However, due to the substantial labor cost, creating high-quality multimodal instruction datasets for MLLMs remains a significant challenge. Although some methods propose to generate instruction data, they often face limitations in modality support and struggle with generating multi-round instructions. To address these problems, we introduce UniData, a universal instruction generation pipeline, to transform simple user requirements into multi-round, multimodal instructions. Specifically, UniData first expands user requirements into multiple diverse events. Using these events, UniData then integrates an any-to-any large model for multimodal instruction generation. Finally, UniData enhances data quality by correcting irrelevant and redundant inference flow, leveraging correlations between instruction rounds. To train this pipeline, we also build UniDataset, a dataset comprising 20,000 entries across nine modalities for improved multimodal generation. Our experiments demonstrate that UniData achieves SOTA performance in data quality and can also enhance the understanding and generation capabilities of other multimodal models.

Abstract:
Agentic retrieval-augmented generation (RAG) extends RAG by allowing language models to iteratively reason, generate search queries, retrieve evidence, and predict answers. However, it remains challenging for models to decide when to retrieve, whether to use lexical matching or semantic similarity, and how to control context granularity to prevent irrelevant tokens from interfering with agent reasoning. In this paper, we introduce GRASP, a reinforcement learning framework for training agents to adaptively coordinate complementary retrieval tools during multi-step reasoning. GRASP provides the agent with semantic search, keyword search, and paragraph-reading actions, enabling it to retrieve sentence-level evidence and expand context only when needed. We train the policy with a reward that accounts for answer accuracy, grounded reading, complementary search, and turn efficiency. Experiments on multi-hop reasoning benchmarks across 3B and 7B models show that GRASP improves retrieval and question answering performance compared with single-step retrieval and multi-step agentic RAG baselines. Qualitative and ablation analyses show that the learned policy develops interpretable skimming and scanning behavior: it uses semantic search for broad exploration, paragraph reading for local verification, and keyword search for entity-specific evidence. These results suggest that learning to coordinate retrieval signals and context granularity is critical for agent’s reasoning.

Abstract:
The deployment of Large Language Model (LLM)-based agents in dynamic environments introduces a unique structural vulnerability: their inherent dependency on sequential observations to drive continuous decision-making. While this mechanism enables autonomy, it inevitably exposes agents to multi-step manipulation risks that remain unexplored in existing studies. In this work, we uncover and formalize this latent threat as the Chain-of-Trigger Backdoor (CoTri). Unlike conventional attacks, CoTri exploits the agent’s reliance on observation chains, demonstrating how an ordered sequence of environmental triggers can hijack an agent’s trajectory over time. Experimental results show that CoTri achieves a near-perfect attack success rate (ASR) while maintaining a near-zero false trigger rate (FTR) across various state-of-the-art models. Due to training data modeling the stochastic nature of the environment, the implantation of CoTri paradoxically enhances the agent’s performance on benign tasks and even improves its robustness against environmental distractions. We further validate CoTri on vision-language models (VLMs), confirming its scalability to multimodal agents. Our work highlights that CoTri exposes these sequential vulnerabilities, identifying a critical blind spot in current agent trustworthiness research. The code is available at [https://github.com/MonoQ-hub/Cotri](https://github.com/MonoQ-hub/Cotri).

Abstract:
Enhancing the reasoning capabilities of large language model (LLM) agents requires effective orchestration of diverse expert models and tools. However, existing frameworks typically call APIs, based on coarse-grained matching between tasks and the functions of expert models or tools, while overlooking critical factors such as performance variability and cost efficiency among functionally similar alternatives. To address this, we propose Agora, a framework that uses a confidence-calibrated auction to dynamically allocate tasks to expert models and tools. By treating reasoning steps as tradeable items, Agora bases allocation on calibrated competence rather than raw confidence. Across five main benchmarks, Agora improves or remains competitive with single-model, routing, and cascade baselines under matched candidate pools.

Abstract:
Fine-tuning-based knowledge editing is simple and architecture-agnostic, but standard cross-entropy increases the edited target probability without explicitly constraining changes in the non-target output distribution. In sequential editing, such unconstrained redistribution can accumulate as distributional drift and contribute to locality degradation. We propose KLOD, a bounded and distribution-preserving objective for fine-tuning-based knowledge editing that separates the intended target update from distributions that should remain stable. KLOD stops target amplification once a probability threshold is reached, while preserving the target-excluded non-target distribution at target positions and the full next-token distribution at prefix positions. Experiments on CounterFact and ZsRE with LLaMA3-8B-Instruct and Qwen2.5-7B-Instruct show that KLOD substantially mitigates locality degradation while maintaining high edit reliability. The target probability threshold further provides a controllable Generalization–Locality trade-off. Ablation, multi-seed, and distributional KL analyses support the interpretation that KLOD’s locality gains are associated with preserving output distributions rather than simply weakening the edit.

Abstract:
Memory management is vital for LLM agents in long-term and personalized interactions.Most previous work studies how to retrieve and use memory, but pays less attention to how memory is extracted.We find two main limitations in existing methods.First, extraction is “ahead-of-time”: the agent saves information before it knows future tasks.A single summary prompt often mixes details, events, and relations, so useful information is lost.Second, extraction is usually one-off. Without verification, errors and hallucinations may stay in memory for a long time. To address these limitations, we propose ProMem, a proactive memory extraction framework.It separates details, events, and relations, and uses different extraction strategies for each type.It also checks completeness to recover missed events and verifies facts at the atomic level to reduce hallucinations.Experiments show that ProMem improves memory completeness and QA accuracy, while keeping a good balance between quality and token cost.

Abstract:
Long video understanding requires reasoning over hours of content that far exceed the context window of multimodal large language models (MLLMs). Existing approaches compress video into fixed-length clips, yet events are fragmented and unrelated content is merged, yielding noisy retrieval. To overcome this challenge, we propose EMMA, an Event-driven Multimodal Memory system with Multi-Agent retrieval, which organizes memory around semantic events in three stages: (1) Event segmentation: a dual-stream detector adaptively locates event boundaries by fusing visual semantic drift with spoken transitions, rather than fixed windows. (2) Memory construction: built on the detected events, a three-level hierarchy of raw video, clip-level memory, and event-level memory stores content at progressively higher semantic abstraction. (3) Memory retrieval: a multi-agent framework collaboratively queries this hierarchy through query decomposition, tiered search, evidence validation, and answer generation, narrowing evidence from coarse events to precise clips. Extensive experiments on three challenging benchmarks demonstrate substantial improvements, showing the effectiveness of our event-aligned memory system. Code, prompt templates, and reproduction scripts are available at https://github.com/BAI-LAB/EMMA.

Abstract:
While Chain-of-Thought (CoT) reasoning has revolutionized text-based LLMs, its application to the audio modality reveals a counter-intuitive thinking paradox: forcing models to “think” via prompting or post-training often degrades performance compared to direct answering. In this work, we investigate the bottleneck of audio reasoning and reveal a critical insight: the primary constraint is not the model’s logical deduction capability, but its inability to accurately perceive acoustic details. We demonstrate that keeping perceptual cues effectively activates the LALM’s internal reasoning potential. Building on this finding, we propose Audio-ReaPer , a post-training framework and corresponding model designed to unlock Rea soning potential by explicitly fortifying Per ception. Our approach reforms the post-training paradigm through three key strategies: (1) utilizing agentic data generation pipeline to synthesize perception-enhanced data for SFT; (2) curating perception-focused data for RLVR; and (3) introducing a novel perceptual density reward during RL to stabilize the learning process and balance observation with deduction. Extensive experiments show that Audio-ReaPer establishes a new state-of-the-art on benchmarks such as MMAU, MMAU-Pro, and MMAR. Notably, our 7B model outperforms previous open-source models by significant margins and surpasses proprietary models like GPT-4o Audio and Gemini 2.5 Flash. The model and code will be open-source to facilitate further research for audio reasoning.

Abstract:
Following complex instructions with multiple explicit constraints remains a fundamental challenge for large language models (LLMs). Existing alignment methods, such as DPO, optimize holistic reward signals that often under-emphasize strict satisfaction of individual constraints, particularly under out-of-distribution or multi-constraint settings. In this paper, we propose STAIF, a stage-wise optimization framework that decouples the alignment of subjective (soft) constraints from the optimization of objectively verifiable (hard) constraints. Stage 1 applies preference optimization with multiple negative samples to sharpen the sensitivity to soft constraints, while Stage 2 applies Reinforcement Learning with Verifiable Rewards (RLVR) to enforce strict compliance with hard constraints. To support this method, we construct STAINSTRUCT, a high-quality bilingual (English, Chinese) dataset of approximately 31,000 complex multi-constraint instructions. Extensive analyses validate the design of STAIF and show state-of-the-art performance on representative benchmarks against strong baselines, as well as genuine generalization.

Abstract:
LLM agents can combine individually non-revealing tool returns and disclose a sensitive conclusion, creating Tools Orchestration Privacy Risk (TOP-R). We formalize TOP-R through three conditions: conclusion sensitivity, single-source non-inferability, and compositional inferability. We introduce Library-Grounded Reverse-Inference Seed Expansion (LRSE), a four-library reverse-construction pipeline, and use it to build TOP-Bench, a 1,000-instance benchmark evaluated under a controlled two-stage tool-use protocol. Across six LLM agents, average task completion, leakage, and H-score are 98.0 percent, 88.6 percent, and 20.4. With native reasoning enabled, four models average 81.4 percent final-response leakage and 82.4 percent reasoning-trace leakage. With reasoning disabled, three prompt-only safeguards improve H-score by an average of about 3.4 points on TOP-Bench. We further propose TOP-Align, an SFT+DPO method for learning safer task-completion boundaries. On a separate post-training evaluation set, TOP-Align improves H-score by 16.2 points over the base model, versus a 5.0-point average gain from prompt-only mitigation on the same set. These results show that TOP-R requires defenses beyond prompting alone. Dataset and code are available at https://github.com/1Ponder/TOP-R.

Abstract:
Coding agents working on sequential maintenance tasks re-read the same files across related tasks, wasting more than a third of their input tokens on redundant context. The natural fix, caching file summaries, fails to reliably reduce cost: tool-use traces are consistent with agents over-relying on generic descriptions, skipping re-reads, and offsetting token savings with less effective downstream decisions, leaving net cost roughly unchanged. We introduce Prism, a content-hash-keyed file memory that augments summaries with bounded facets: identifier-grounded facts distilled from the agent’s own prior reasoning. On 12 SWE-Chain dependency-upgrade chains, Prism reduces cost by 24.9% on GPT-5 and 9.9% on GPT-5.4-mini, while maintaining comparable aggregate Build+Fix F1. Against no memory, a summary-only ablation yields only a nominal 2.4% cost decrease that falls within run-to-run variability, while a static Aider-style repo-map baseline costs substantially more at 13% over no memory. Two controlled ablations isolate the causal contributions: wiping the persistent store before every task raises cost by 37% over no memory, and replacing facets with a 3× larger unstructured summary raises it by 41%. Code: https://github.com/WhiteFlamesCN/Prism-Opencode.

Abstract:
Significant challenges remain in AI-driven educational systems in balancing privacy preservation with accurate cognitive diagnosis. In order to overcome this, we propose a federated inference framework in which several commercial LLM APIs collaborate without requiring access to raw student data or proprietary model internals. Using multiple federated entities, such as LLaMA-3.3-70B, GPT-4o-mini, and Claude-3-Haiku, our framework builds upon the heterogeneous multi-LLM architecture. The predictions generated by these entities are combined with ε-local differential privacy by adding Laplace noise locally to each entity’s prediction output before aggregation, while residual-based aggregation mitigates model heterogeneity. Our approach is predicated on an honest-but-curious trust paradigm in which API providers are presumed not to abuse submitted queries, and our differential privacy mechanism shields the published diagnostic results from external inference. We conduct rigorous privacy–utility analysis showing strong privacy guarantees with minimal accuracy loss, and extensive real-world evaluations across three educational benchmarks confirm the framework’s practical usability and cross-domain generalizability.

Abstract:
Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix Multiplication (RMM), a training free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy–efficiency trade-off.Across language models ranging from 1B to 70B parameters, we find that reduction tolerance depends on the model family, task, component, and retention ratio, although it often improves with model scale. Under moderate reduction, RMM remains robust across the evaluated discriminative, autoregressive-generation, and long-context settings. We further show that the same principle extends to multimodal vision–language inference. Mechanistic ablations reveal a structural asymmetry within Transformers: attention-side computations are substantially more reducible than MLP components. Finally, wall-clock benchmarks with custom kernels on an NVIDIA A100 show that these computational savings can translate into practical runtime gains, especially at longer sequence lengths. Together, these results position RMM as a scalable direction for input-adaptive inference-time optimization.

Abstract:
Retrieval-augmented generation (RAG) has become a standard mechanism for grounding language models in external knowledge, yet conventional retrieval based on lexical or semantic similarity is poorly suited for complex reasoning tasks: a semantically similar problem may demand an entirely different solution strategy, while a superficially different problem may share the same underlying reasoning pattern. We propose Retrieval-Augmented Reinforcement Fine-Tuning (RA-RFT), a post-training framework that teaches language models to reason by analogy. RA-RFT uses gold-relevance distillation to train a retriever that ranks contexts by expected reasoning benefit rather than semantic overlap, and then fine-tunes the policy model via reinforcement fine-tuning methods with retrieved analogous demonstrations, so the model learns to leverage reasoning traces under verifiable outcome rewards. We further analyze the diversity of retrieved contexts and find that reasoning-aware retrieval surfaces complementary solution strategies that provide distinct reasoning scaffolds for individual problems. Across challenging mathematical reasoning benchmarks, RA-RFT consistently outperforms standard reinforcement fine-tuning methods. For example, it improves AIME 2025 average@32 accuracy by 7.1 and 2.8 points over GRPO for Qwen3-1.7B and Qwen3-4B respectively—suggesting that reasoning-aware retrieval is a complementary axis of improvement and orthogonal to advances in reward design or training curricula.

Abstract:
The widespread adoption of large language models (LLMs) is reshaping how textual content is produced, complicating text provenance, and raising concerns for misinformation, plagiarism, and intellectual property. While prior work on AI-generated text detection has progressed, one-dimensional human-AI co-authorship scores lack signals directly relevant to “AI involvement”. It remains unclear how adversarial paraphrasing distorts detection signals. In this work, we formulate a discrete-class text provenance detection framework. This framework allows an abstraction of realistic co-authorship scenarios into distinct degrees of AI involvement, differentiating the AI polishing on the surface level from the AI updating the discourse structures. This framework reveals the mechanisms of how adversarial paraphrasing blurs the decision boundaries for co-authorship detectors. To improve robustness of the detectors, we augment training data with adversarial paraphrases generated by the state-of-the-art paraphrasing attack model, exposing detectors to challenging meaning-preserving rewrites. Extensive experiments across multiple datasets and generation models demonstrate that adversarial paraphrasing substantially degrades detectors trained only on benign data, while adversarial data augmentation leads to consistent robustness gains and good cross-generator transferability. Overall, our findings highlight the value of discrete-class formulation and principled data augmentation for building reliable detectors of human-AI co-authorship.

Abstract:
Enhancing the formal math reasoning capabilities of Large Language Models (LLMs) has become a key focus in both mathematical and computer science communities in recent years. While significant progress has been made in using state-of-the-art Auto-Regressive (AR) LLMs for formal theorem proving, these models suffer from inherent limitations. Their next-token prediction generation methods may yield suboptimal performance due to the challenges of long-range coherence and the compounding of errors over long sequences. Recent advancements in diffusion LLMs (dLLMs), which generate text through iterative denoising of a multi-token block, offer a promising alternative. However, the application of dLLMs to formal mathematics, where maintaining long-range coherence is critical, remains largely understudied. To address the challenges above, we propose Diffusion-Proof, to the best of our knowledge, the first framework to train and apply dLLMs for formal theorem proving. Our frameworks contain training and inference methods for two models. The first one is dLLM-Prover-7B, which performs whole-proof writing with long-range coherent tactic usage. The second one is dLLM-Corrector-7B, which is a novel large block diffusion-based correction model. It leverages the in-filling capabilities of dLLMs to perform local proof correction using bi-directional information. Extensive experiments demonstrate that Diffusion-Proof relatively significantly outperforms the AR LLM baseline trained under the same dataset. Diffusion-Proof achieves an absolute improvement of 1.61% on ProofNet-Test and 6.14% on MiniF2F-Test benchmarks compare to the baseline. Notably, Diffusion-Proof successfully resolves one IMO problem that more advanced thinking model DeepSeek-Prover-V2-7B could not solve, showcasing the unique advantage of dLLMs in formal theorem proving.

Abstract:
Large language models have made substantial progress on mathematical reasoning, but existing benchmarks typically evaluate well-specified problems with final answers, step-by-step solutions, or complete proofs. They do not capture collaborative open-problem solving: a setting in which participants propose partial arguments, identify gaps or errors in prior steps, repair flawed reasoning, and gradually synthesize incremental contributions into a proof. We introduce CrowdMath, a dataset of 164 expert-annotated progress chains from the MIT PRIMES–Art of Problem Solving (AoPS) CrowdMath program (2016–2025), a collaborative research initiative whose discussions have led to peer-reviewed publications. Each chain traces a multi-participant forum discussion from an open-problem statement to a completed proof. Posts are labeled by their functional roles in the evolving solution process, including partial progress, proof completion, erroneous reasoning, and error identification. We define evaluation tasks and benchmark six frontier models. Models achieve 83–88% accuracy on next-post prediction, suggesting that they can follow the local flow of mathematical discussion. However, they struggle to identify the functional significance of individual contributions with the best model achieving only 0.42 macro-F1 on post-role classification. CrowdMath exposes a gap between solving well-specified mathematical problems and understanding collaborative mathematical progress as it unfolds.

Abstract:
Transformer-based models excel at Automatic Readability Assessment (ARA), yet feature-based models remain in active use because their predictions tie back to linguistic properties. This matters because readability labels are subjective and rater-dependent, so high accuracy on noisy ground truth may reflect surface patterns rather than the linguistic structure that defines difficulty. We test whether transformers internalize the same features as traditional models across Arabic, English, French, Hindi, and Russian using the ReadMe++ dataset. Shapley Additive Explanations (SHAP) identify the features driving traditional classifiers, which we then use as TCAV concept sets to probe multilingual XLM-R and language-specific encoders. Transformers recover surface-length, syntactic, and lexical-diversity signals, and reflect the ordinal CEFR structure of the traditional models. Alignment varies by model family, language, and layer, with language-specific encoders tracking traditional models more clearly than XLM-R. High linear separability does not always imply directional influence, limiting linear probing for count-based readability features.

Abstract:
Large reasoning models have recently demonstrated emergent “aha moments” that enable reflective reasoning. However, a non-trivial portion of reflective steps are “low-utility,” where reasoning errors are not corrected, or even correct reasoning trajectories are shifted toward incorrect ones. To address low-utility reflective continuation, this paper proposes RUAR, a utility-guided advantage rescaling method for reflective reasoning that estimates step-level utility from changes in correct-answer probability, without human process labels or a separately learned process reward model. RUAR augments outcome-supervised RL by rescaling the token-level advantages within each reflective reasoning step according to the estimated utility of the reflective continuation, rather than applying a uniform trajectory-level advantage across the reasoning trace. The utility is estimated through answer-forcing utility estimation, which measures how the model’s correct-answer probability changes before and after the reflective step by forcing answer generation from both reasoning states using answer-forcing prompts. Furthermore, RUAR introduces an overthinking-mitigating rescaling mechanism, where utility-guided rescaling is no longer applied from the earliest answer-ready reflection step onward. Experiments demonstrate that RUAR improves the accuracy–efficiency trade-off, substantially reducing generated token usage while often preserving or improving final-answer accuracy.

Abstract:
Recent Vision-Language Models encode high-resolution images into long visual token sequences, incurring prohibitive prefill costs. To compress them, existing methods score each visual token by averaging text-to-visual attention uniformly across all heads, which assumes every head matches the query. However, our empirical analysis shows that misaligned heads dominate the average, amplifying background tokens and drowning out fine-grained cues. To address this, we propose PAQ (Prompt-Grounded Attention Quality), a metric quantifying how well each head aligns the prompt with image regions. Built on PAQ, our pruning proceeds in three stages. Given a target FLOPs budget, we first partition the transformer layers into groups and allocate a visual token budget to each. Within each group, we then aggregate per-head attention maps via PAQ-weighted softmax into a group-level matrix. Finally, we score visual tokens by this matrix’s magnitude and retain the allocated budget per group. By weighting heads with PAQ, our method scores tokens by true relevance to the prompt rather than diluting it through uniform averaging. Across 18 benchmarks, our method delivers state-of-the-art trade-offs. Specifically, on LLaVA-1.5-7B (9 tasks), retaining only 5.6% tokens preserves 99.1% of the original performance, surpassing the strongest baseline AutoPrune by 4.2 points. Code is available.

Abstract:
LLM-based multi-agent systems have shown strong performance on complex tasks, yet continual improvement from interaction experience remains challenging. Existing self-reflection methods build experience memories, but memories are mostly hard to invoke, refine, or scale, while agent skills offer a more actionable unit: structured procedural knowledge that specifies when to act, how to act, and which resources or tools to use. We introduce MASkills, a continual learning framework that optimizes multi-agent LLM systems through agent skills. MASkills presents a new agent-optimization pipeline that integrates skill-conditioned credit assignment, hierarchical credit aggregation, and momentum-smoothed optimization, enabling agent skill libraries to evolve through refinement, induction, consolidation, and pruning. Experiments on HotpotQA, LoCoMo, and GAIA demonstrate the effectiveness of MASkills across multiple agentic tasks.

Abstract:
Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. Necessity captures how much the pretrained model depends on a layer’s existing contribution, measured by the loss increase from bypassing it. Plasticity captures where the model absorbs new information during fine-tuning, measured by the magnitude of task-specific weight updates. Our analysis reveals that transformers and SSMs behave fundamentally differently: in residual transformers, Necessity and Plasticity anti-align across depth, whereas in Mamba-style SSMs they point to overlapping regions. The sign of this alignment also predicts downstream adaptation behavior. In transformers, the updates are in the most plastic layers, which increases catastrophic forgetting, while this tier-dependent effect disappears in Mamba-style SSMs

Abstract:
Large language models often struggle with sensitive prompts. They may refuse outright, provide generic safety boilerplate, or fail to address the user’s legitimate informational needs that can be answered safely. We introduce SHARD, a self-reframing distillation method to improve safety-helpfulness. It first rewrites sensitive prompts to surface benign intent using philosophical guidelines, then reframes its original responses into safe, more helpful ones, and finally fine-tunes the model on its self-reframed responses. Across DNA and the English subset of LINGUASAFE, SHARD improves helpfulness for most model families while preserving safety. It also remains competitive with distillation from a larger teacher model, suggesting that models can internalize safe and helpful behavior elicited from the models themselves.

Abstract:
The remarkable performance of Mixture-of-Experts (MoE) models often comes with substantial architectural scale and resource overhead. Prior structural compression mainly focuses on expert-level or FFN intermediate pruning. However, we identify an emerging architectural trend that the FFN expansion ratio in state-of-the-art MoEs has steadily decreased, making intermediate-only pruning less effective and motivating us to explore other redundancies. Motivated by this, we propose an orchestrated hidden-intermediate prune-and-distill framework that, to the best of our knowledge, is the first to jointly prune expert FFNs along both intermediate and hidden dimensions. It includes Budget-Constrained Coordinate Descent to efficiently search expert-specific pruning ratios across the two dimensions, and attention relation transfer distillation to provide stable supervision under hidden misalignment. Extensive experiments across diverse benchmarks validates that hidden-dimension pruning is a promising direction for compressing modern MoEs.

Abstract:
Uncertainty estimation (UE) aims to quantify uncertainty in large language model (LLM) outputs to flag potential hallucinations and support reliable deployment. However, UE metrics often exhibit unstable performance across models and datasets, limiting their practical utility. We formalise this phenomenon as proxy failure, arising because UE metrics are derived from model behaviour rather than being explicitly grounded in factual correctness. We show that when mutual information between a UE score and correctness is low, the score’s discriminability must be close to chance. To address this, we propose Truth AnChoring (TAC), a lightweight post-hoc calibration method that maps raw UE scores to truth-aligned correctness probabilities. Even with noisy and few-shot supervision, TAC continues to improve calibration, providing a practical calibration protocol. Our findings highlight the limitations of treating heuristic UE metrics as direct indicators of uncertainty about factual correctness, and position our TAC as a principled step toward more reliable uncertainty estimation for LLMs.

Abstract:
Accurate citations are the foundation of academic writing, tracing intellectual origins and substantiating core claims. However, manually navigating the growing volume of scientific literature is increasingly difficult, prompting reliance on automatic citation recommendation. While modern retrieval-augmented architectures have largely mitigated the fabrication of non-existent papers, current systems relying on semantic similarity struggle with misattribution, often citing authentic papers that fail to logically support the author’s claim. To address this challenge, we argue that accurate citation requires a shift from similarity-based search to active, claim-level reasoning. We propose ReCite, a decoupled agentic framework that orchestrates location perception, intent-aware query planning, and reflective verification. Trained on synthesized reasoning trajectories, our agent verifies claim-evidence consistency and triggers self-correction loops when retrieved candidates lack logical support. Experiments demonstrate that our lightweight framework outperforms state-of-the-art massive generative models in strict citation accuracy. By grounding literature matching in verifiable logic rather than semantic overlap, ReCite establishes a reliable foundation for automated academic writing.

Abstract:
Model merging has emerged as a promising paradigm for integrating multiple task-specific capabilities into a single large language model. However, existing methods predominantly focus on post-hoc processing of independently fine-tuned models, overlooking how the training phase itself impacts cross-task compatibility. Resolving parameter conflicts after fine-tuning is inherently sub-optimal. To address this, we propose CAMFT, a Conflict-Aware Mergeable Fine-Tuning method that makes task adaptation both efficient and merge-aware. CAMFT treats mergeability as a property shaped during fine-tuning, rather than only a problem to be solved after fine-tuning. By guiding each task to update sparse coordinates with lower cross-task conflict, CAMFT produces task updates that are efficient to train and more compatible for downstream model merging. Extensive experiments demonstrate that CAMFT outperforms standard fine-tuning baselines in multi-task merging scenarios. The code will be made publicly available.

Abstract:
Multi-step tool planning is vulnerable to structural errors that propagate across dependent calls: a wrong argument, broken dependency, or mis-ordered operation can invalidate an otherwise plausible trajectory. We introduce FORTE, a framework that makes these failure modes explicit training targets. FORTE defines a nine-error taxonomy for multi-step plans, constructs taxonomy-grounded contrastive corpora by injecting minimally edited failures into gold trajectories, and trains a lightweight judge to recognize and score these errors. The same frozen judge then provides dense reward for GRPO planner optimization and verification for Best-of-N inference. Across ToolHop and NESTFUL, FORTE achieves the highest Judge Success Rate across all six open-weight backbone-dataset configurations and the highest observed end-task execution accuracy among the evaluated methods in all six settings. With Qwen2.5-7B, FORTE reaches 96.6 JSR and 99.5 dependency accuracy on ToolHop, and 98.6 JSR and 89.0 dependency accuracy on NESTFUL. Under a matched five-sample selection budget, FORTE-Qwen2.5-7B also outperforms GPT-5.5 across JSR, functional match, parameter accuracy, and dependency accuracy on both benchmarks. Ablations show that taxonomy-grounded contrastive negatives are essential for both judge discrimination and effective inference-time selection. These results demonstrate that explicitly learning which planning failures to avoid provides a strong supervision signal for robust multi-step tool planning.

Abstract:
In long context applications like Retrieval Augmented Generation (RAG), Large Language Models (LLMs) often reuse independently precomputed Key Value (KV) caches to reduce latency. However, directly assembling these cached segments introduces errors that degrade generation quality. We study this problem in the setting of position-independent caching and decompose the reuse error into two components: a geometric error ( G ) caused by position mismatch, which can be largely removed by position alignment, and a dynamic error ( D ) caused by context-dependent representation changes. To characterize D , we introduce the smoothness gap, a local diagnostic that measures the discrepancy between local KV-state transitions in isolated caches and those in the full-context baseline. This gap serves as a necessary local consistency diagnostic for reconstruction errors across adjacent positions and reveals that recomputing non-contiguous selected tokens can introduce additional transition discontinuities, thereby amplifying the dynamic error. To overcome this limitation, we propose Transition Overlap , a training-free cache-construction protocol that prepends a short temporary overlap to approximate full-context boundary transitions and stores only the KV states of the original segment tokens. We instantiate it under two deployment regimes: Self-Segment Overlap (SSO) for query-independent offline caches and Cross-Segment Overlap (CSO) for cases in which the predecessor segment is known during construction. In both evaluated deployment settings, Transition Overlap improves the measured serving latency–quality trade-off over training-free PIC baselines[Code is available at ].

Abstract:
Open-domain tabular question answering (TQA) aims to answer natural language queries over a data lake that contains a large corpus of heterogeneous tables. Unlike closed-domain TQA, this task operates without pre-provided tables, making it particularly challenging. Existing methods usually first retrieve relevant tables and then answer the question based on the retrieved tables. However, it faces challenges in effectively and efficiently preparing tables from the low-quality, heterogeneous data lake and in operationalizing table information. To address these limitations, we present ArrTab (Agentic retrieval and reasoning on Tables) for open-domain TQA. ArrTab leverages MuRe, a self-supervised Multi-table Retriever trained on questions synthesized by translating table-derived SQL queries into natural language via large language models. Based on MuRe, ArrTab proposes a multi-agent system for open-domain TQA that consists of the Table Retriever Agent, the Table Integration Agent, the Information Extraction Agent and the Answer Agent. Furthermore, to enhance accuracy, a lazy execution strategy is adopted where operations are described in natural language but executed only during the information extraction stage, allowing for globally consistent reasoning. Extensive experiments on real-world datasets demonstrate the superiority of ArrTab.

Abstract:
As LLM-as-a-judge becomes increasingly widespread, self-preference—the tendency of a judge to favor its own outputs—raises growing concerns about evaluation reliability. However, this bias has been studied predominantly on generated text, where stylistic features and response quality are inevitably conflated. As a result, existing measurements cannot separate genuine self-preference from these confounds. We address this limitation by changing the object of evaluation: instead of judging generated text, ten LLMs assess sets of narrative constraints selected from a shared pool, which carry no stylistic fingerprint yet retain a recoverable model-specific signature. Two experiments on this task yield complementary findings. Under blind evaluation, self-preference disappears, with a small effect remaining in the opposite direction once selection quality and judge severity are controlled. Under matched quality, however, self- and other-labels alone—without naming any model—shift scores bidirectionally. LLM judges inflate scores for self-labeled selections and deflate those for other-labeled ones regardless of the selection’s actual source. We make two contributions: 1) authorship attribution is a distinct driver of evaluation bias, and 2) ground-truth-free tasks can serve as controlled instruments for studying LLM judge behavior.

Abstract:
LLM-based simulated clients are increasingly used to train novice counselors, evaluate LLM therapists, and generate synthetic data. However, current simulators produce overly cooperative clients that disclose too readily, accept therapeutic reframes without resistance, and resolve core issues within a single session. We trace these issues to profiles that lack causal depth and behavioral mechanisms that treat all content as equally accessible. We present PatientAct, a framework for client simulation grounded in established clinical theories. Our profiles integrate the 5Ps clinical case formulation, providing causal depth without tying the design to any single therapeutic modality. During simulation, profiles include a dynamic memory layer in which items carry trust thresholds (e.g., symptoms are available early, whereas formative memories require a sustained therapeutic alliance). At each turn, we model the client’s emotional reaction and behavior before generating a response. If the therapist approaches gated content, PatientAct expresses resistance in terms of quantity, content, and style rather than defaulting to cooperation or a single resistance pattern. We evaluate our framework on 40 clinical situations and demonstrate that it generates diverse profiles with high clinical plausibility. Moreover, PatientAct significantly outperforms the baselines, yielding substantial gains in resistance quality and behavioral realism. Our code and data are publicly available via github.com/Sahandfer/PatientHub.

Abstract:
Scaling laws are usually read as a capability story: lower language-modeling loss yields more useful models. We study a safety consequence of this mechanism in cross-session decomposition attacks , where benign-looking subqueries are asked across independent interactions and later recomposed toward a forbidden objective. We formalize this setting as compositional safety risk and prove a conditional risk-transfer bound: when the reference environment already contains dispersed evidence for a risky reconstruction, the gap between deployed composed risk and reference composed risk is controlled by the model’s excess loss on allowed subqueries. Synthetic withholding experiments show that wider transformers assign lower loss to held-out instructions that never appear verbatim in training but are recoverable from injected supporting facts. A 600-intent pretrained-LLM evaluation shows that larger Qwen3 and Gemma3 family members can yield greater harmful-capability uplift under a fixed decomposition-composition pipeline. As a defense, IntentAlign-MiniLM, our 22M intent-aligned retriever, outperforms much larger embedding models on held-out intent retrieval and yields the best learned-retriever harmful recall across tested guardrails.

Abstract:
Cited RAG evaluation often treats visible sources as a grounding signal, but a real, topically relevant citation can still under-warrant the attached wording. We study this diagnostic failure as citation laundering: a related source is presented as warrant for an over-strong claim. We introduce ForceBench, a contrastive stress test for evidence-force calibration. Each item holds a cited passage fixed and pairs an evidence-calibrated claim with a localized force-raised variant across five operational axes: relation, modality, scope, temporal validity, and numeric specificity. A calibrated evaluator should score the evidence-calibrated claim higher. Headline experiments use a fixed, locality-filtered 198-pair evaluation set. A citation-presence sanity check is uninformative by design; token and entity overlap still violate monotonicity on 32.8–36.4% of pairs. Across four reported model judges, standard generic support prompting is insufficient for this force-calibration stress test (aggregate MVR 47.2%), while explicit warrant-strength prompting lowers MVR to 24.5% but remains imperfect. We release the benchmark, prompts, outputs, and plug-in pipeline so citation evaluators can report monotonicity violation rate and force sensitivity alongside conventional support metrics.

Abstract:
Graph-based RAG improves multi-hop question answering by organizing evidence as a knowledge graph. However, most existing RAG systems process each query in isolation and discard useful reasoning from the LLM’s response after inference. As a result, later related queries need to retrieve evidence and reason from scratch. We propose LivingRAG, a Graph RAG framework with writable and reusable reasoning experience. LivingRAG adds a writable experience store to a graph-based retrieval backbone, enabling verified experiences to be reused during inference in two ways. Stored graph signals help retrieval find entities and passages that were useful in earlier related queries. Stored summaries provide a reference reasoning pattern for answer generation. We analyze online QA streams and find reusable signals from shared entities, graph neighborhoods, and question templates. Experiments on multi-hop QA benchmarks show that LivingRAG improves accuracy over strong RAG baselines and reduces completion-token use when relevant prior experience is reused.

Abstract:
Large Language Model (LLM)-based automatic Multi-Agent Systems (MAS) generation has become a crucial frontier for tackling complex tasks. However, existing methods face a dilemma between model capability and experience retention. Inference-time MAS leverages frozen frontier LLMs but repeats identical searches without learning from past experience. Conversely, Training-time MAS internalizes experience via gradient updates but is constrained by the low capability ceiling of smaller models, and is hard to scale to large frontier LLMs. To bridge this gap, we propose Skill-MAS, a novel third path that decouples experience retention from parametric updates by conceptualizing the high-level orchestration capability as an evolvable Meta-Skill. Skill-MAS refines this architectural knowledge through a closed optimization loop: (1) Multi-Trajectory Rollout samples a behavioral distribution for each task under the current Meta-Skill; and (2) Selective Reflection adaptively selects priority tasks and applies hierarchical contrastive analysis to distill systemic experience into generalizable, strategy-level principles. Extensive experiments across four complex benchmarks and four distinct LLMs demonstrate that Skill-MAS not only achieves remarkable performance gains but also maintains a favorable cost-performance trade-off. Further analysis reveals that the evolved Meta-Skills are highly robust and exhibit strong transferability across unseen tasks and different LLMs.

Abstract:
Large language models (LLMs) are increasingly used to assist ideation in research, but evaluating the quality of LLM-generated research proposals remains difficult: novelty and soundness are hard to measure automatically, and large-scale human evaluation is costly. We propose a verifiable alternative by reframing proposal generation as a time-sliced scientific forecasting problem. Given a research question and inspiring papers available before a cutoff time, the model generates a structured proposal and is evaluated by whether it anticipates research directions that appear in papers published after the time. We operationalize this objective with the Future Alignment Score (FAS), computed via retrieval and LLM-based semantic scoring against a held-out future corpus. To train models, we build a time-consistent dataset of 21,835 paper occurrences across 3,642 instances from targets and their pre-cutoff citations, and synthesize reasoning traces that teach gap identification and inspiration borrowing. Across Llama-3.1 and Qwen2.5 models, future-aligned tuning improves future alignment over unaligned baselines (up to +10.6% overall FAS), and domain-expert human evaluation corroborates improved proposal quality. Finally, we demonstrate practical impact by implementing two model-generated proposals with a code agent, obtaining 4.17% accuracy gain on MATH from a new prompting strategy and consistent improvements for a novel model-merging method.

Abstract:
Reinforcement learning from human feedback (RLHF) has become the dominant paradigm for aligning large language models (LLMs) with human preferences. However, traditional RLHF relies on scalar reward signals that lack interpretability and fail to capture the multifaceted nature of response quality. Rubric-guided reinforcement learning addresses these limitations by introducing structured, interpretable evaluation criteria, or rubrics, as the backbone of reward design, feedback generation, and policy optimization. In this survey, we introduce a Bayesian framework that defines constitutions as prior distributions P(ℛ) over evaluation criteria and rubrics as conditional instantiations ℛ x ∼ P(ℛ|x) . Under this unified view, we present a taxonomy of rubric-guided RL along the prior-posterior axis, covering constitutional AI, instance-specific rubrics, process-level supervision, self-evolving rubrics, and their agentic and multimodal extensions. Furthermore, as rubrics are natural-language artifacts, we present a linguistic analysis of how granularity trade-offs, semantic drift, and linguistic reward hacking impact alignment reliability, identifying key open problems for future research.

Abstract:
To mitigate high inference costs for in-context learning, prior works have proposed activation-space task vectors that encapsulate demonstration contexts into static representations. However, performance occasionally degrades across tasks, leaving it unclear whether this stems from sub-optimal extraction heuristics or fundamental limitations. We resolve this by decomposing Transformer self-attention to establish a theoretical best-case approximation: the Minimal-Gap Approximation. By isolating structural constraints from heuristic flaws, the Minimal-Gap Approximation reveals that task vectors are fundamentally bottlenecked by attention and length gaps. Specifically, because task vectors lock contextual information to fixed attention maps and positional alignments, they structurally fail under dynamic routing, length mismatches, and multi-token generation. We further show that these findings consistently hold across training-free extract-and-inject task vector methods, model architectures, and model scales.

Abstract:
As organizations increasingly deploy LLMs in sensitive domains such as legal, financial, and medical settings, ensuring alignment with internal organizational policies has become a priority. Existing content moderation frameworks remain largely confined to the safety domain and lack the robustness to capture nuanced organizational policies. LLM-as-a-judge and fine-tuning approaches, though flexible, introduce significant latency and training cost. To address these limitations, we frame policy violation detection as an out-of-distribution (OOD) problem in the model’s activation space. We propose a training-free method that operates directly on the LLM internal representations, leveraging prior evidence that decision-relevant information is encoded within them. Inspired by whitening techniques, we derive policy-violation scores directly from normalized representations of LLM hidden activations. Our method requires only the policy text and a small number of illustrative samples, making it lightweight and easily deployable. We extensively evaluate our method across multiple LLMs and challenging policy benchmarks. It achieves up to 86.0% F1, outperforming both fine-tuned and LLM-as-a-judge baselines while requiring substantially less computation. Our code is publicly available at: https://github.com/FujitsuResearch/LLM-policy-violation-detection.

Abstract:
Collecting natural-language referring expressions along with region annotations, such as masks or boxes, is a major bottleneck in visual grounding (VG), as annotators must write descriptions that distinguish target regions from visually similar ones. We tackle this by formulating active learning (AL) for VG under the realistic setting where only raw images are available without accompanying text. Since ground-truth text is unavailable, sample selection must estimate which images contain ambiguous regions that would require discriminative referring expressions. To address this, we generate auxiliary region-text pairs using foundation models, and introduce Referred Region Ambiguity, a new acquisition function that measures whether the model’s confidence collapses onto a single region or disperses across multiple candidates. It allows our method to prioritize images with strong cross-region competition, which are more informative due to their visual ambiguity. We also design a referring-expression annotation interface that helps annotators quickly focus on writing discriminative language with a few clicks. Experiments on RIS and REC benchmarks show that our AL framework consistently outperforms several AL baselines, while a user study shows up to 1.6 × faster description labeling of ours.

Abstract:
Many real-world questions over business, legal, and scientific corpora are natural-language versions of database-style queries over records latent in text. Existing retrieval-augmented generation (RAG) systems are optimized primarily for semantic relevance, but retrieving plausible passages does not guarantee correct query execution. We introduce QO-Bench, a diagnostic benchmark for query-operator question answering over typed event tuples. The benchmark covers 22,984 news articles and 614 corporate events, with 18 query templates instantiating 785 questions. Each gold answer is deterministically computed from typed event tuples and scored by recall, with answers matched to the gold tuples by exact match rather than an LLM judge. This design enables operator-level diagnosis such as joins and intersection. We evaluate RAG, ReAct RAG, GraphRAG, and information-extraction-to-SQL under matched conditions, with a long-context oracle ceiling to isolate retrieval failure. A two-axis framework—index-time preservation versus query-time execution—predicts where each paradigm fails, and the results bear it out: systems retrieve relevant text but discard the typed values operators need, and the deployable paradigm ranking inverts across operators, with similarity retrieval leading on filter/project and extraction-to-SQL on intersection and counting. Even given the gold evidence, a long-context oracle stays far from saturated, so operator execution—not retrieval alone—is a core bottleneck that a stronger answer model does not remove. QO-Bench reframes the goal from passage relevance to query-operator-preserving retrieval. The benchmark, predictions, and code are released at https://github.com/ZHANG-MENGAO/qo-bench.

Abstract:
LLM-as-judge is now a standard tool for scalable evaluation, but judge performance is still often summarized by a single accuracy number. This aggregate view hides the deployment conditions under which a judge succeeds or fails. We introduce Conditional Accuracy Profiling(CAP), a post-hoc diagnostic framework that decomposes pairwise LLM-judge accuracy into eight conditions organized into content sensitivity, robustness, and rationale quality. CAP is benchmark-agnostic: it can be applied directly when a benchmark provides the required annotations, approximately through task-subset proxies, or through controlled augmentation when perturbation pairs can be generated. We instantiate CAP on seven LLM judges across six pairwise judging benchmarks, including judgerEva-Standard, a controlled testbed we created to support all eight conditions. CAP exposes profile differences hidden by aggregate accuracy: on judgerEva’s judge-independent Hard-Constructed subset, the two judges most sensitive to omitted qualifications rank in the bottom three of seven by overall accuracy, so omission sensitivity is not predicted by aggregate accuracy. Across benchmarks, Position Robustness shows the strongest rank stability (mean Spearman ̄𝜌=0.87 ) but is itself fragile under JudgeBench-Pro adversarial stress, showing the largest mean accuracy drop among the shared conditions, though the dominant degradation channel varies by judge. Condition-level profiles provide a more actionable basis than aggregate accuracy for selecting LLM judges.

Abstract:
Reasoning large language models (LLMs) benefit from test-time scaling, motivating adaptive policies that spend tokens when they help and stop early otherwise. Existing rules halt when a confidence or stability signal crosses a threshold, but such thresholds rarely transfer across models and datasets, and validation-calibrated rules lose their guarantee under shifts in test-time horizon or signal dynamics. We reframe adaptive stopping as sequential evidence accumulation along the reasoning trajectory rather than threshold calibration on held-out data. Our framework runs two e-processes on a shared per-chunk signal: a commit stream tracks the signal’s position relative to its slow trend, and an abstain stream tracks its signed velocity, making the two anti-correlated by construction. Under signal-level nulls on the per-chunk increments, Ville’s inequality yields an anytime-valid signal-level false-alarm bound at user-specified significance levels, with no held-out calibration. Across various model–dataset combinations, the framework matches or exceeds calibration-based stopping on the token–accuracy Pareto front using zero calibration data, and the same significance level transfers unchanged across cells where calibration becomes infeasible.

Abstract:
Transformer-based speech models have recently been shown to perform well under impoverished and ambiguous supervision, a setting motivated by poverty-of-the-stimulus arguments, yet their learning dynamics remain unclear. In this work, we study this question with wav2vec2.0 on Mandarin tone classification, using 70 scale–noise configurations, capacity interventions, and perturbation probes to examine the dynamics of abstraction-consistent generalization and exception memorization, yielding representative regimes of Robust Generalization, Abrupt Transition, and High-noise Collapse. We find a consistent abstraction–exception trajectory: when phonological recovery is possible, the model first exhibits phonologically abstraction-consistent behavior and only later fits corrupted labels as empirical exceptions. Capacity analyses show that this trajectory is not merely a large-model artifact: it persists under truncated Transformer encoders, while acoustic rank compression modulates but does not selectively separate abstraction from exception memorization. Finally, acoustic perturbations show different local robustness: abstraction-consistent predictions remain stable under stronger waveform perturbations, whereas fitted exceptions behave as fragile local overrides on top of the broader abstraction-consistent behavior.

Abstract:
We study timestamped speaker-attributed automatic speech recognition (SA-ASR) for long-form, multi-party speech with overlap. In this setting, chunk-wise inference must preserve meeting-level speaker identity consistency while producing time-stamped, speaker-labeled transcripts. Prior Speech-LLM systems tend to prioritize either local diarization or global labeling, lacking the ability to jointly model fine-grained temporal boundaries and robust cross-chunk identity linking. We propose G-STAR, an end-to-end framework that couples a cache-conditioned speaker-tracking module with a Speech-LLM transcription backbone. The tracker provides structured speaker cues with temporal grounding, and the LLM generates attributed text conditioned on these cues. G-STAR supports component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift. Under chunk-wise decoding protocols, experiments on both oracle-segmented local evaluation and full-meeting global evaluation show strong speaker-attributed transcription performance.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has become a widely used approach for improving the mathematical reasoning ability of large language models (LLMs). However, recent studies show that LLMs may solve a math problem in its original form but fail when the same problem is reworded without changing its meaning. This suggests that the model has not fully acquired the underlying reasoning ability and instead relies on the surface form of the problem. In this paper, we propose PEARL, an RLVR framework that uses different versions of the same math problem as structured training signals. PEARL first groups each original problem with its checked variants to build semantic perturbation clusters, and then uses partially solved clusters as the active RL training set. PEARL further performs cluster-level RL optimization by comparing model outputs across different versions of the same mathematical problem. Experiments on mathematical reasoning benchmarks show that PEARL achieves strong results against competitive RLVR baselines while using a small active training set. These results suggest that semantically equivalent variants can serve as useful training units for improving mathematical reasoning in LLMs.

Abstract:
Realignment is a promising approach for improving the cross-lingual transfer ability of multilingual language models, particularly for extremely low-resource languages (LRLs). However, existing realignment methods rely on uniform and random sampling of parallel sentences across languages, which may be suboptimal under limited batch sizes. In practice, models may benefit from seeing certain languages more frequently, especially those that are poorly aligned, and the optimal distribution can evolve throughout training. In this work, we propose a simple yet effective adaptive sampling strategy that assigns trainable sampling probabilities to each language. Languages that contribute more to the realignment loss are sampled more frequently in subsequent batches, and the optimal distribution can evolve throughout training. Our method employs an inner–outer optimization loop with a small overhead, leading to consistent performance improvements and, more importantly, distributing the gains across languages. We observed a +0.67 average performance increase on all tasks with XLM-R, and +0.60 with Gemma 2 9B compared with uniform realignment. Furthermore, our method is robust across different models.

Abstract:
Diffusion language models generate text through iterative denoising, allowing intermediate predictions to influence later denoising steps, a recurring practice we call trajectory self-feedback (TSF). TSF is adopted under inconsistent terminology, evaluated on incompatible benchmarks, and studied by disconnected communities, yet no prior review characterises it reproducibly. We present a systematic literature review under an evidence-based methodology: six pre-specified questions, a documented six-library search with snowballing, explicit selection and quality criteria, and a structured synthesis of 24 primary studies across seven clusters, with a released audit trail. We find: (i) a model-agnostic taxonomy resolves four operator families: Prediction-Space Concatenation, Confidence-weighted Path Signals, Reward-Reinforced Conditioning, and Correction-Targeted Feedback, that the literature conflates as self-conditioning; (ii) evidence of benefit is real but fragmented across incomparable settings; (iii) within our search window, no study jointly varies the TSF mechanism and measures faithfulness on knowledge-intensive tasks; and (iv) a discrete/continuous asymmetry consistent with the sampling-wall hypothesis is visible but never tested. We conclude with eight open problems, a dependency map, and review-derived diagnostics (Commitment Index, Correction Capacity) and a Standardised Evaluation Protocol (SEP-DLM) to make TSF measurable across quality, correction, trajectory commitment, and faithfulness.

Abstract:
Understanding human intentions in videos requires deep cognitive reasoning, yet Multimodal Large Language Models (MLLMs) frequently suffer from factual hallucinations and counterfactual blindness. To address these limitations, we introduce ASTRO, a novel framework that navigates Intention Reansoning Tree to guide preference optimization. Our approach employs a collaborative multi-agent system integrated with Monte Carlo Tree Search (MCTS) to systematically explore complex reasoning spaces based on visual evidence. By navigating this tree, ASTRO dynamically extracts distinct cognitive trajectories—correct, error, and ambiguous paths—to construct high-quality preference pairs. Finally, we apply Direct Preference Optimization (DPO) using these contrastive paths, explicitly enhancing the model’s factual grounding and sensitivity to causal boundaries. Extensive experiments across complex video intention benchmarks demonstrate that ASTRO achieves superior performance while exhibiting remarkable data efficiency, significantly outperforming fully supervised baselines using only a fraction of the original training data.

Abstract:
As LLMs evolve from standalone assistants into autonomous agents, they increasingly operate in populations where communication, coordination, and competition form community-like interaction networks. In such settings, individually value-steered behaviors can accumulate into group-level dynamics, raising the question of how human values shape collective outcomes in interacting LLM-agent populations. We introduce CIVA, a community-level value analysis framework that varies value direction and prevalence, then measures how these interventions propagate through repeated interactions into macro-level dynamics and micro-level behaviors. We apply it in a resource-constrained multi-agent community where LLM agents autonomously communicate, cooperate, and compete. The value scan reveals distinct structural response patterns: activation-supporting values such as benevolence supports persistence and resilience, an activation-constraining value such as tradition regulates growth and cohesion, and a bidirectionally disruptive value such as power perturbs collective regimes in both directions. At the micro level, representative interventions induce emergent behaviors including deception, betrayal, self-preservation, and power-seeking. These results show that human values can act as population-level structural variables in agent communities and motivate evaluation for multi-agent value alignment.

Abstract:
As Large Language Models (LLMs) are increasingly adopted in decision-making, there is a growing need to better understand the behavioural patterns underlying their reasoning. Evaluation is one of the primary approaches used to study model behaviour. However, most existing LLM evaluations rely on task-based benchmarks that summarise performance through aggregate scores. While effective for comparing model performance, such approaches provide limited insight into the latent behavioural tendencies that drive biased reasoning. In this paper, we propose a construct-based evaluation framework for studying cognitive bias in LLMs grounded in psychometric evaluation. We systematically perturb prompts across controlled conditions to analyse reasoning behaviour. From model responses, we derive behavioural features capturing susceptibility, robustness, correction, and variability, and apply latent variable modelling to infer underlying behavioural constructs. Our findings show that cognitive bias in LLMs is structured and dynamic, varying across bias type, model families, and scale.

Abstract:
Multi-view spatial reasoning in large vision-language models involves inferring view-specific spatial relations and integrating them into a consistent scene-centered representation. However, existing approaches often learn view-dependent relations tied to individual camera frames, which limits their ability to form viewpoint-invariant spatial representations. Motivated by cognitive models of spatial perspective-taking, we propose SCoRE (Spatial Consistency Reasoning), a method that formulates multi-view spatial reasoning as an egocentric-to-allocentric transformation problem. SCoRE first trains the model to infer egocentric object relations from single-view observations. It then applies reinforcement learning to encourage the model to estimate viewpoint orientation, transform egocentric relations into a shared scene-centered reference frame, and predict allocentric object relations. To enforce viewpoint-invariant reasoning, we optimize the model with a multi-view consistency reward that aligns predictions from different views in the common allocentric space. Experiments show that SCoRE improves spatial reasoning performance by +3.69%p on multi-view benchmarks and +4.72%p on general spatial reasoning benchmarks over prior state-of-the-art methods, demonstrating the effectiveness of explicit reference-frame transformation for robust multi-view spatial understanding.

Abstract:
Large Language Models (LLMs) are increasingly used for clinical decision support, where hallucinations and unsafe suggestions may pose direct risks to patient safety. These risks are hard to assess: subtle clinical errors are often missed by generic metrics and LLM judges using general criteria, while expert-authored fine-grained rubrics are expensive and difficult to scale. In this paper, we propose a retrieval-augmented multi-agent framework for automatically generating instance-specific evaluation rubrics. Our approach grounds evaluation in authoritative medical evidence by decomposing retrieved content into atomic facts and synthesizing them with user interaction constraints to form fine-grained evaluation criteria. Evaluated on HealthBench and LLMEval-Med, our framework achieves Clinical Intent Alignment (CIA) scores of 50.20% and 31.90%, significantly outperforming the GPT-4o baseline and showing consistent improvements across English and Chinese medical benchmarks. In discriminative tests on HealthBench, our rubrics achieve a 7.8% point higher win rate than GPT-4o and increase the mean score difference from 4.972 to 8.658. Ablation studies further show that individual components contribute differently across datasets, with interaction-intent modeling providing the most consistent contribution to clinical-criterion coverage. Beyond evaluation, our rubrics guide response refinement, improving response quality by 9.2%. These results suggest that automated, knowledge-grounded rubric generation provides a scalable foundation for evaluating and improving medical LLMs. The code is available at https://anonymous.4open.science/r/Automated-Rubric-Generation-E716/.

Abstract:
Auxiliary-loss-free load balancing (ALF-LB) achieves promising performance in Mixture-of-Experts large language model (MoE-LLM) pre-training as it decouples load balancing from the main objective. However, we observe an unexplored phenomenon: routing imbalance is concentrated in shallow MoE layers. We attribute this phenomenon to the interaction between representation homogeneity, which makes shallow-layer routing decisions highly sensitive, and expert-bias staleness, which applies stale load corrections to the current router state. To break the cycle, we propose Dual-Bias Load Balancing (DLB) , which augments the standard global expert bias with an intra-step micro-bias residual. DLB introduces the effective bias 𝐛 eff =𝐛 g +𝛥𝐛 , where 𝐛 g is updated once per gradient step and 𝛥𝐛 is updated across micro-batches within the same step. DLB does not add forward or backward passes and integrates naturally into existing large-scale MoE training pipelines. Experiments on large-scale MoE pre-training show that DLB mitigates expert-bias staleness, alleviates shallow-layer imbalance, and yields lower test perplexity and stronger average downstream performance.

Abstract:
Chain-of-thought (CoT) reasoning in large language models often produces redundant steps that increase generation cost and obscure when reasoning has effectively completed. Existing stopping methods rely on output-level or token-level signals, overlooking internal representation dynamics that reflect reasoning progress. We propose STARS (STate-Aware Reasoning Stopping), a framework that leverages spectral analysis of hidden-state representations to detect transitions in reasoning phases. Our key insight is that structural changes in the representation space capture shifts in reasoning states, enabling early and reliable stopping decisions. To address representation instability, where semantically similar steps can induce different hidden states, we focus on spectral structure rather than raw activations, improving robustness to context-dependent drift. Experiments on four mathematical and logical reasoning benchmarks show that STARS reduces generation cost by over 45% under fixed token budgets while improving accuracy over standard CoT baselines.

Abstract:
Existing RL methods for language models typically use token-level entropy, which captures uncertainty at the current position rather than over future generation. We study future entropy, i.e., uncertainty over the remaining suffix conditioned on the current prefix. By the entropy chain rule, local entropy is the expected decrease in future entropy, while the realized decrease depends on the chosen token, yielding an action-dependent future-entropy gap. We evaluate this signal at inference time with MIN-F-MC, an oracle-style probe that selects the candidate token with the lowest estimated future-entropy rate at high-entropy positions. On mathematical reasoning tasks, MIN-F-MC yields higher mean accuracy than greedy and Random TopK across the tested checkpoints and datasets. We then propose FEPO (Future Entropy Policy Optimization), which uses realized future-entropy rates at a sparse set of high-entropy positions to scale update magnitudes. Across the tested model–benchmark pairs, FEPO improves Mean@32 over GRPO.

Abstract:
Cognitive Diagnosis Models (CDMs) play a pivotal role in personalized online learning. Traditional CDMs rely on discrete, ID-based embeddings to represent students, exercises, and concepts. This paradigm diverges from the nature of learner cognition, where knowledge is not stored and retrieved as isolated symbols. As a result, CDMs suffer from semantic limitations when new exercises or concepts appear. In this paper, we propose a Process-aware Language Cognitive Diagnosis (PLCD) framework that uses language-derived structures as cognitive priors and response records to calibrate student posterior states. PLCD leverages large language models (LLMs) to construct concept schemas and cognitive process graphs, and uses target-conditioned semantic memory to retrieve historical responses that are relevant to each target exercise. A process-grounded Language-to-Cognition Mapper with DA-MoE experts and process-level contrastive learning then maps the textual evidence into a unified cognitive space. Experimental results show that PLCD not only outperforms traditional baselines in predicting student performance but also exhibits strong cognitive transfer capabilities. These results connect the computational power of LLMs with the psychometric goal of measuring latent knowledge states, suggesting that structured language priors calibrated by response records can improve cold-start robustness and cognitive grounding.

Abstract:
Agentic systems accumulate persistent memory across sessions, tools, and tasks, and a later request must retrieve from it under two distinct constraints: which memories it is permitted to access, and which are relevant under a limited search budget. Existing memory systems handle access scope in two flawed ways: applying scope after retrieval wastes probe budget on inadmissible memories, while treating scope as a learned ranking feature makes a hard constraint depend on router quality. We present ShardMemo, an agentic memory system built on scope-before-routing: metadata predicates first identify the admissible shards, and a learned router then selects a small number of them for shard-local approximate nearest neighbor retrieval. This separates hard admissibility from learned relevance ranking, so inadmissible shards cannot consume shard-probe budget. We evaluate on LoCoMo, HotpotQA, and ToolBench, covering conversational, long-context, and procedural memory. Under matched supervision and fixed budgets, ShardMemo improves over a learned router baseline by roughly +3 F1 on LoCoMo; in end-to-end LoCoMo evaluation it improves over the strongest evaluated memory baseline by up to +6.8 F1, with gains on HotpotQA and ToolBench.

Abstract:
In disinformation datasets, narratives are often understood as recurring interpretive patterns that group texts under narrative labels. Recent work formalized narrative mining as inductively inferring narrative labels from corpora, but its evaluation stays tied to predefined taxonomies, a closed-world setting that cannot capture narratives absent from the reference labels. We introduce a three-tier evaluation framework for unsupervised narrative label generation: recovery (against a corpus’s own taxonomy), mining (against external label sets), and discovery (without predefined labels). Applying it, we compare clustering-based and graph-community-based pipelines across seven disinformation datasets, with human validation of discovery on two. The two families are complementary under automated metrics, but in a corpus with two prominent topics, clustering can reduce one topic to 2% of generated labels while graph-based pipelines stay balanced. Discovery validation also reveals many singletons (narrative labels derived from single claims, 30–62% of graph outputs), which clustering cannot produce. Annotators confirm many as recognizable disinformation narratives, suggesting that in open-world discovery the repetition assumed by narrative mining may be recognized outside the corpus, not within it. We release human-validated narrative candidate labels for the Climate Obstruction and PolyNarrative datasets to support taxonomy development and dataset extension.

Abstract:
We propose Semantic Averaging with MUltiple Languages (SAMUL) as a method to isolate concept semantics of text by averaging concept activations derived via Sparse Autoencoders (SAEs) for multiple text versions in different languages. We combine autoencoder activations to derive a conceptual average using different approaches. In our experiments with SAMUL, we start with English text from benchmark datasets (Ontology Alignment Evaluation Initiative 2025: conference track, SemEval2014: cross-level semantic similarity, Sentences Involving Compositional Knowledge: relatedness) and automatically translate into French, Chinese, German and Japanese. We obtain SAE feature activations for each class and language version using autoencoders, specifically the open source Gemma Scope 1 and OpenAI’s GPT2 suites of Sparse Autoencoders and average them to SAMUL vectors. We measure performance in predicting semantic similarity of short texts using predefined ground truth or correlation to the ground truth mapping m between ontology classes in the case of the ontology alignment task. Our results show a consistent improvement over any single translated language, and improvement over English alone in 27 of 32 comparisons. Conceptual averaging therefore yields sparse representations that align more closely with human similarity judgements, suggesting a route to more reliable use of SAE features in semantic tasks such as ontology alignment. Our implementation and experimentation code is openly available.

Abstract:
Vision-language models (VLMs) rely on storing and transferring appropriate information across their sub-components. Verifying that the VLMs exhibit desired behaviours, while avoiding harmful ones, is central to their reliable deployment. Yet, methods that identify VLM behaviours remain scarce. We present DiaVLo, a diagnostic framework that leverages human curation and VLMs’ generation capabilities to construct specifications of desired and observed VLM behaviours, surfacing potential misalignments. Beyond this, DiaVLo also provides causal estimates to identify the most influential concepts steering VLM behaviours. We evaluate DiaVLo on several open-source VLMs under both classification and generation conditions. Our experiments show that DiaVLo produces behaviour labels that correlate with model performance and provide context for measured performance. DiaVLo surfaced behaviours that are clearly aligned and misaligned, alongside patterns in how VLMs perceive, organise, and prioritise concepts.

Abstract:
Simulator-assisted reinforcement learning is a practical way to train multi-turn dialogue agents, but policy optimization can be misled when simulated users provide inconsistent feedback across long interaction trajectories. In this paper, we focus on a simulator-side question: can structured latent state tracking improve measured simulator consistency and provide useful process supervision for dialogue policy training? Motivated by a standard model-based RL simulation-bound perspective, we introduce DPS, a framework that decouples task-conditioned privileged state tracking from surface utterance realization. These privileged states are hidden from the agent but used during training to derive dense, state-aware process rewards that complement sparse outcome signals. We also construct RecruitmentBench, a recruitment persuasion dialogue benchmark grounded in anonymized real-world logs. Experiments on RecruitmentBench and two public datasets use offline real-log-grounded and cross-simulator protocols to evaluate intrinsic simulator consistency and downstream simulator-assisted policy performance. Results indicate that DPS improves measured simulator fidelity and yields stronger fine-tuned policy performance in these controlled evaluation settings, while remaining competitive with strong prompt-based LLM agents. We further calibrate our LLM-based evaluator against human judgments on MATH-CHAT. Code is available at: https://github.com/TraceLite-AI/DPS.

Abstract:
Large language models (LLMs) have greatly advanced conversational systems, making Personalized Dialogue Generation (PDG) a core direction for tailoring responses to individual users. However, retrieval-augmented and user-parameterization approaches typically rely on dense user-level persona information and consistent domain-level persona distributions, leading to two key challenges for scalable PDG: sparse personas of unseen users hinder personalization, while domain-bound user parameters require costly retraining across dialogue scenes. Therefore, we argue that these limitations originate from an over-reliance on static data fitting, as opposed to mastering atomic personalization capabilities. To this end, we propose Meta-Collaborative Learning (MCL), a novel learning paradigm that proactively supplements cues via Neighbor Identifier and Booster , and dynamically aligns responses to preferences by Generator . By decomposing PDG into synergistic modules with bi-level joint optimization, MCL internalizes universal personalization skills for acquiring and leveraging contextual signals across diverse users and domains. Extensive experiments validate that MCL delivers robust personalization under practical unseen-user and cross-domain settings.

Abstract:
Existing circuit discovery methods rely on templated tasks with clean counterfactuals, limiting their use on diverse natural text. We adapt Contextual Decomposition for Transformers (CD-T) for unstructured settings via label-balanced activation means and task-directional relevance scoring, enabling counterfactual-free circuit discovery. We leverage the discovered circuits for Circuit-Targeted Supervised Fine-Tuning (CT-SFT), restricting parameter updates to task-relevant heads and LayerNorm. Experiments on NusaX cross-lingual sentiment transfer show that CT-SFT is highly competitive for low-resource adaptation. While non-circuit sparse updates and full fine-tuning sometimes match target accuracy through capacity recruitment, CT-SFT most consistently avoids catastrophic forgetting, preserving source-language and related-task performance. Extensions to XNLI support the source-retention and intervention findings on a harder task and two model families, showing that circuit-targeted adaptation provides a more controlled, intervention-supported alternative to global fine-tuning.

Abstract:
Long-document visual question answering increasingly relies on VLM agents that retrieve candidate pages, inspect page images, write findings to working memory, and synthesize answers. Working memory should carry answer-supporting evidence across page inspections for later grounded answering, yet existing evaluation mainly checks final-answer correctness and evidence-page access. This creates a memory-quality blind spot: an agent may reach the right page and answer correctly while leaving behind memory too generic or incomplete to support answering once page context is removed. We introduce memory-only answerability , a diagnostic that asks whether a reader can answer from the question and terminal working memory alone. Building on this diagnostic, Answerable Working Memory (AWM) treats terminal working memory as an answerable evidence artifact, and AWM-GRPO incorporates this signal into the GRPO reward while preserving final-answer priority. Under GRPO, this reward assigns higher advantages to answer-correct trajectories whose terminal working memory remains answerable. On MMLongBench-Doc , even when gold evidence pages are provided, 42.5% of correct answers still cannot be answered from terminal working memory alone. AWM-GRPO improves final-answer accuracy over the RAG baseline by 8.1 and 11.9 points on MMLongBench-Doc and LongDocURL and reduces the memory-missing-correct rate by 2.7 points over answer-only GRPO.

Abstract:
Current multimodal LLMs encode images as static visual prefixes and rely on text-based reasoning, lacking goal-driven and adaptive visual access. Inspired by human visual perception—where attention is selectively and sequentially shifted from the most informative regions to secondary cues—we propose Structural Sequential Visual CoT (SSV-CoT). First, a question-relevant saliency map identifies and organizes key visual regions, explicitly modeling the spatial distribution of visual importance. Second, reasoning is performed following this discriminative order, inducing a curriculum-like semantic progression from primary to secondary cues. This method is trained end-to-end, using text cot and answer supervision, without relying on region-level annotations or specialized external tools. Experiments on diverse visual reasoning benchmarks show gains, validating structured and sequential visual cognition.

Abstract:
Parameter-efficient finetuning (PEFT) has become the standard approach for adapting large language models, yet evaluations largely emphasize downstream accuracy while overlooking the retention of pretrained capabilities. We argue that PEFT should be assessed through the stability-plasticity dilemma: the trade-off between target-task adaptation and resistance to forgetting. We introduce PEFT-Arena, a benchmark that jointly measures downstream performance and general capability retention. Across the studied settings, methods exhibit distinct stability-plasticity profiles, and orthogonal finetuning often lies on a favorable frontier under comparable parameter budgets. We analyze the internal changes associated with these external trade-offs from two complementary geometric perspectives. In weight space, spectral profiles characterize how parameterizations interact with the pretrained singular-value structure. In activation space, representation-similarity diagnostics show a strong association between forgetting and non-isometric distortion on general-evaluation data. Together, these views suggest that preserving pretrained spectral and representational geometry is an important inductive bias for retention, with OFT’s spectrum-preserving updates providing a particularly clear case. Finally, parameterization-aware interpolation shows that final SFT checkpoints can lie beyond a better target-retention operating point. We use post-hoc rewinding as a case study of controlling this trade-off.

Abstract:
Understanding the frequency of factual errors in chatbot-generated text and evaluating systems that detect these errors is critical for determining chatbot safety. Yet factual-error detection is often treated as a single-pass, single-annotator labeling problem. In long-form chatbot responses, factual errors can be subtle and embedded within mostly correct text. We develop a multi-perspective annotation study of medically relevant chatbot responses, combining first-pass annotation, LLM-as-a-Judge candidate discovery, and two forms of adjudication: domain-expert and evidence-based fact-checking. First-pass annotators frequently miss factual errors later validated by adjudicators. LLM-as-a-Judge improves candidate discovery, but is insufficient on its own: it misses factual errors that annotators catch. We also find disagreement among adjudicators, suggesting that adjudication over multiple candidate sources can improve benchmark completeness, but does not eliminate the need to apply judgment and expertise. Applied to an existing benchmark, this technique reveals a similar pattern of missed errors. Together, these results suggest that single-pass hallucination benchmarks achieve scale, but likely undercount errors. Multi-pass adjudication improves coverage, but benchmark results will still need to be interpreted in light of the judgment, expertise, and evidence used when deciding if an error is present.

Abstract:
Recent advances in large language models (LLMs) have enabled software engineering agents to solve complex code modification tasks. However, most existing approaches rely on execution feedback from Docker-based environments, which require dependency-complete setup, program execution, and test validation, making agent training costly and difficult to scale. We propose SWE-World, a Docker-free framework that replaces containerized execution during downstream agent optimization with a learned surrogate environment. Trained on real agent-environment interaction traces, SWE-World predicts both step-level execution feedback and final test outcomes, preserving the standard agent-environment loop without repeatedly constructing or running Docker environments. This surrogate also enables test-time scaling (TTS) by scoring multiple candidate trajectories through simulated evaluation. Experiments on SWE-bench Verified show that SWE-World improves Qwen2.5-Coder-32B-Instruct from 6.2% to 52.0% with Docker-free SFT, 55.0% with Docker-free RL, and 68.2% with TTS. The code is available at https://anonymous.4open.science/r/SWE-World-2D1E .

Abstract:
Retrieval-augmented generation (RAG) improves factuality but adds latency and engineering overhead at serving time. We propose RING (Retrieval-Internalized Generation), a holistic paradigm spanning both architecture and training that injects large-scale external knowledge into a Mixture-of-Memory Experts and learns parametric search over this internal memory via reinforcement learning, removing the external retriever entirely. Training proceeds in three stages: continued pre-training injects new corpora into a Knowledge Expert via our novel Dual Causal Attention ; supervised fine-tuning teaches a “search-then-answer” pattern; and reinforcement learning with hierarchical rewards optimizes the routing-and-search policy over the parametric memory. Unlike prior parametric injection methods that pair internal memory with a fixed or rule-based retriever, RING learns its retrieval policy directly from task signals. We further frame RING theoretically as a search-free approximation to the classical RAG objective. To evaluate large-scale injection of genuinely new knowledge without test-time leakage, we further construct News-2025, a benchmark built from news strictly post-dating the base LLM’s pretraining cutoff. RING matches or surpasses both search-based RAG and parametric injection baselines in accuracy and efficiency.

Abstract:
Generation novelty is a key indicator of an LLM’s ability to generalize, yet measuring it against full pretraining corpora is computationally challenging. Existing evaluations often rely on lexical overlap, failing to detect paraphrased text, or do not consider the full pretraining corpus. We frame novelty as a semantic retrieval problem. This framing enables us to address novelty with modern embedding and indexing pipelines, allowing for efficient analysis at pre-training scale. Specifically, we propose a three-stage framework that retrieves semantically similar samples, reranks them at varying subsequence lengths, and calibrates scores using a human novelty reference for interpretability. We apply this framework to the SmolLM model family and report three key findings: (1) models draw on pretraining data across much longer sequences than previously reported; (2) some task domains systematically promote or suppress generation novelty; and (3) instruction tuning not only alters style but also increases novelty. These results highlight the value of semantic novelty analysis for studying generalization. To support reproducibility and further research, we release ∼ 20 TB of corpus chunks and index artifacts upon acceptance.

Abstract:
Supervised fine-tuning (SFT) is a core stage in post-training general-purpose language models, but modern SFT training corpora are often large, heterogeneous mixtures, making data selection important under fixed token budgets. Simple ranking approaches can be informative yet insufficient: ranking alone does not determine how to allocate a fixed token budget across examples. We study the role-specific use of likelihood-derived signals for token-budgeted subset construction. An external instruction-tuned reference model computes response-side negative log-likelihood scores to rank training examples. We then combine this ranking with base model likelihood filtering to restrict the candidate region and response-side cosine filtering to reduce redundancy in the selected subset. In the OLMo fine-tuning setting studied here, ranking examples only by reference-model likelihood underperforms the random baseline, whereas the same ranking becomes more effective when combined with base model likelihood filtering and cosine-based redundancy control. Our results support viewing token-budgeted SFT data selection as a budget-allocation problem rather than only as an example-ranking problem.

Abstract:
General-purpose open-domain dense retrieval systems are usually trained with a large, eclectic mix of corpora and search tasks. How should these diverse corpora and tasks be sampled for training? Conventional approaches sample them uniformly, proportional to their instance population sizes, or depend on human-level expert supervision. It is well known that the training data sampling strategy can greatly impact model performance. However, how to find the optimal strategy has not been adequately studied in the context of embedding models. We propose Inf-DDS, a novel reinforcement learning–driven sampling framework that adaptively reweighs training datasets guided by influence-based reward signals and is much more lightweight w.r.t. GPU consumption. Our technique iteratively refines the sampling policy, prioritizing datasets that maximize model performance on a target development set. We evaluate the efficacy of our sampling strategy on a wide range of text retrieval tasks, demonstrating strong improvements in retrieval performance and better adaptation compared to existing gradient-based sampling methods, while also being 1.5 — 4× cheaper in GPU compute. Our sampling strategy achieves a 5.03 absolute NDCG@10 improvement while training the multilingual bge-m3 model and an absolute NDCG@10 improvement of 0.94 while training all-MiniLM-L6-v2, even when starting from expert-assigned weights on a large pool of training datasets.

Abstract:
Pre-trained vision–language models such as CLIP provide strong priors for class-incremental learning (CIL), yet they degrade sharply under long-tailed class streams. We show that this degradation is not merely a byproduct of forgetting: in frozen-CLIP long-tailed CIL, newly introduced tail classes are disproportionately absorbed into previously learned old classes, a boundary-level failure we call new-tail collapse . We diagnose this failure with an old-new margin and analyze it through a boundary-risk view, where CLIP semantic overlap, sparse tail supervision, and old-class expansion jointly increase old-dominant violations. Motivated by this view, we propose Smooth Boundary-Risk Loss (SBRL), which estimates effective old rivals from CLIP’s vision-language geometry and suppresses their smooth old-side pressure against new-tail samples. MC-SMERG further consolidates task-wise adapters into a single efficient model. Across ImageNetSubset-LT, CIFAR100-LT, and Food101-LT, our method consistently outperforms strong CLIP-based CIL baselines and substantially reduces boundary violations in high-similarity, low-shot regimes.

Abstract:
Reinforcement learning has driven the largest post-training gains on tasks with verifiable rewards, such as code and math. Extending this success to subjective tasks remains difficult: good responses must satisfy multiple soft criteria simultaneously, and no scalar reward can indicate which aspect of a response should improve. Personalization is a clear instance of this. A personalized response must balance helpfulness, user-specific relevance, and naturalness, yet standard verifiers cannot localize which specific dimension needs improvement. In this regime, supervised fine-tuning quickly saturates, and PPO with Bradley-Terry reward models is prone to reward hacking, especially via length bias, producing verbose or superficially personalized responses. We propose Critique-Post-Edit, an RL framework that combines (1) a personalized Generative Reward Model (GRM) producing multi-dimensional scores and natural-language critiques, and (2) a critique-guided editing step that turns feedback into nearby alternative responses for policy learning. The reward signal stays scalar; our contribution is to use personalized critiques to construct more informative local alternatives (responses close to the on-policy rollout in edit distance) during training. Under length-controlled evaluation, Critique-Post-Edit improves over PPO by 11.0 and 11.6 win-rate points on Qwen2.5-7B and 14B models, respectively. Notably, the 14B model matches or outperforms GPT-4.1 on the evaluated benchmarks. A static mechanism study further shows that critique-guided edits stay more local than extra rollouts and shift scores more favorably than generic rewrites. All of our experiments are on personalization; systematic evaluation on other subjective tasks is left to future work.

Abstract:
Knowledge editing provides an efficient way to update factual knowledge in large language models. However, malicious edits may introduce safety risks, making it necessary to reverse undesirable editing effects. Existing reversal methods for parameter-modifying edits mainly focus on global removal, which may also erase beneficial edits that should be preserved. In this paper, we study selective reversal of edited knowledge, where the goal is to reverse targeted edited facts while preserving the remaining edited facts. Based on the hypothesis that each edit is sparsely encoded within the dominant subspace of the edited matrix, we propose a spectral-based reversal framework that locates edit-sensitive components within the dominant singular subspace of edited weights. Experiments across multiple settings demonstrate the effectiveness of our method in reversing selected edits while preserving unrelated edited facts. These results suggest that different edits are sparsely encoded within dominant singular components and can be separable when the number of edits is moderate, making selective spectral reversal a promising direction for locating edit-specific components and repairing edited language models.

Abstract:
Our ability to interpret others’ mental states through nonverbal cues (NVCs) is fundamental to our survival and social cohesion. While existing Theory of Mind (ToM) benchmarks have primarily focused on false-belief tasks and reasoning with asymmetric information, they overlook other mental states beyond belief and the rich tapestry of human nonverbal communication. We present Motion2Mind a framework for evaluating the ToM capabilities of machines in interpreting NVCs. Leveraging an expert-curated body-language reference as a knowledge base, we build Motion2Mind, a carefully curated video dataset with fine-grained nonverbal cue annotations paired with manually verified psychological interpretations. It encompasses 222 types of nonverbal cues and 397 mind states. Our evaluation reveals that current AI systems struggle significantly with NVC interpretation, exhibiting not only a substantial performance gap in Detection, as well as patterns of over-interpretation in Explanation compared to human annotators.

Abstract:
Hidden states of large language models (LLMs) are typically analyzed as flat Euclidean vectors through probing, anisotropy, and intrinsic-dimension lenses. Yet much of what these models process (taxonomies, program structure, multi-step derivations) is hierarchical, and whether hidden-state distances reflect such hierarchy remains untested. We present the first systematic study of Gromov hyperbolicity (GH), a distance-based measure of tree-likeness, in prompt-token hidden states of open-weight LLMs. Across model families, scales, layer depths, and domains spanning math, code, commonsense, and truthfulness benchmarks, we construct a hyperbolicity atlas charting the normalized statistic 𝛿 rel along each axis. Four findings emerge: (i) depth dominates, with middle layers forming a high- 𝛿 rel plateau and final layers dropping toward tree-like structure; (ii) scale is weak and non-monotonic; (iii) matched-scale families occupy distinct regimes, with specialization gaps exceeding scale effects; and (iv) domains interact with specialization, e.g., HumanEval shifts from the highest- to the lowest- 𝛿 rel domain between base and coder checkpoints. GH thus serves as a lightweight diagnostic that localizes where hierarchical distance structure emerges and which comparisons warrant deeper analysis.

Abstract:
Coordinating heterogeneous large language models (LLMs) can improve instruction following, yet current multi-agent systems often depend on manual role choices or repeated calls to one backbone. This makes it difficult to separate complementary model behavior from model bias and verbosity. We introduce Role-Specialized Multi-Agent Collaboration (RS-MA), a prompting-only framework that assigns candidate LLMs to four interpretable roles: Judge, Summarizer, Ideator, and Reasoner. RS-MA estimates role fit with audition prompts, aggregates role-specific rankings with Borda scoring, and uses the selected assignment as an inspectable role map. A refinement phase records judgment gaps and targeted edits before synthesis. On AlpacaEval 2.0, audition-selected RS-MA reaches an 88.86% length-controlled win rate, and a validated role map reaches 90.72%. A full 805-example ablation shows that single-model reuse attains high raw preference through much longer outputs, whereas the validated map improves length-controlled quality with moderate length growth. MT-Bench and FLASK further indicate preserved dialogue quality and balanced fine-grained skills. The results support explicit role assignment as a practical mechanism for combining heterogeneous LLMs when quality, completeness, and inspectable evidence are jointly required.

Abstract:
State Space Models (SSMs) have emerged as an efficient alternative to the Transformer architecture. Prior work shows that, when trained under comparable conditions, SSMs can match or surpass Transformers on code understanding tasks. However, their internal mechanisms remain a black box. We present the first systematic analysis of what SSM-based code models learn along with the direct comparison between SSM and Transformer models in this domain. Our analysis shows that SSMs capture syntactic and semantic structure more effectively than Transformers during pretraining but forgets certain relations during fine-tuning on some tasks. To investigate this behavior, we introduce SSM-Interpret , a frequency-domain framework that shows a “spectral shift" toward short-range dependencies during fine-tuning. Guided by these findings, we propose architectural modifications that significantly improve the performance of SSM-based code model by upto +6 MRR on NLCodeSearch. This demonstrates that our analysis not only explains model behavior but also leads directly to better designs.

Abstract:
Post-training methods use training-side signals to decide which supervision to emphasize, but those signals need not identify behavior that is capability-bearing in the target environment. We probe this with DIAA (Dual-Implicit Adversarial Amplification), a per-sample σ = sigmoid(-βΔ) SFT/RL hybrid: after every literal Python code fence is deleted from the anchor data, the model still emits unexecuted Tool-Integrated Reasoning (TIR) form on 29% of MATH-500 prompts, three times vanilla GRPO, and suppressing the form recovers accuracy. A paired within-prompt intervention decomposes the removable drag along GRPO → SFT → DIAA as +2.2 → +8.5 → +10.6 pp, identifying SFT imitation and σ-amplification as comparable sources; a teacher RFT with 0% TIR likewise trains a student emitting TIR at 45.8%. We propose Intervention-Certified Amplification (ICA), which gates the per-sample weight by a certificate derived from paired rollouts, σ_eff(x) = σ(x)·c(x; θ_c). Frozen and online instantiations cut TIR to 13.8% and 11.2%, more than halve residual drag while improving accuracy, and carry the joint improvement to held-out GSM8K without retuning the certificate.

Abstract:
Large language models often produce hallucinated answers that violate prompt-level constraints. A key diagnostic question is whether these failures reflect missing knowledge, or whether the model has the relevant information but follows the wrong inference path. We study this phenomenon as inference misalignment: a mismatch between the answer supported by the prompt and the answer favored by statistically salient latent associations. We formalize this view with a latent key–task model, in which pretraining-frequency imbalance can cause a shortcut path to dominate the constraint-sensitive path and induce positive inference loss. The framework predicts two failure modes: task-retrieval bias in entity disambiguation and key-selection bias in action choice. We introduce TrapQA, a controlled diagnostic testbed with two components. ScientistQA tests disambiguation among similar scientists with supplementary factual probes, while Real-Life Constrained QA tests everyday constraint following under salient shortcuts. Our results show that hallucination can arise from biased latent inference rather than absent knowledge alone.

Abstract:
Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their context. However, both methods typically treat specific languages, with limited zero-shot transfer at test time. To translate extremely low-resource languages at scale, we argue that a model should be able to use in-context linguistic knowledge regardless of language, rather than memorize specific training languages. In this paper, we propose a reinforcement learning (RL) approach to unseen language translation given rich linguistic context, using a surface-level translation metric as the reward. Empirically, despite the lightweight reward, our RL-trained models effectively extract and apply relevant linguistic information from the provided context, leading to better translations on completely unseen languages than in-context learning or supervised fine-tuning. Our analyses suggest that outcome-based RL can extend beyond conventional reasoning tasks like math and coding to serve as a recipe for low-resource translation from linguistic context.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has become a central paradigm for improving reasoning and code generation in large language models, and GRPO-style training is widely adopted for its simplicity and effectiveness. However, an important design choice remains underexplored: how token-level policy gradient terms are aggregated within each sampled group. Standard GRPO uses sequence aggregation, while recent work has advocated token aggregation as a better alternative. We show that these two rules induce different optimization biases: token aggregation introduces sign-length coupling, while sequence aggregation implicitly downweights longer responses through sequence-level equal weighting. To address this tension, we propose Balanced Aggregation (BA) , a simple drop-in replacement that computes token-level means separately within the positive and negative subsets and then combines them with sequence-count-based weights. Experiments with Qwen2.5-Math-7B and Qwen3-1.7B on DAPO-17k and Polaris, evaluated on six reasoning and coding benchmarks, show that BA consistently improves training stability and final performance over standard token and sequence aggregation. Our analysis further shows that the relative effectiveness of token and sequence aggregation is largely governed by response-length variation and the positive-negative length gap, highlighting aggregation as a critical design dimension in GRPO-style RLVR.

Abstract:
Various test-time interventions for Computer Use Agents (CUAs), including critic models, have been developed to improve performance through pre-execution action evaluation in complex Graphical User Interface (GUI) environments. However, existing critics suffer from two key limitations: they (1) focus primarily on short-sighted decision loops (e.g., forgetting earlier actions) and (2) lack the visual grounding needed to detect flawed actions (e.g., clicking wrong UI elements). To address these, we introduce HIVIG, a History-aware Visually Grounded test-time framework, built around a multimodal critic trained on real GUI trajectories to abstract past interactions into a compact record and to evaluate actions with visual grounding. At test time, HIVIG integrates the critic into the policy decision loop to provide macro-action history, which summarizes the policy’s completed achievements, and visually grounded critique, which verifies raw execution coordinates against the current screenshot to intercept errors before execution. Across web, mobile, and desktop benchmarks, HIVIG consistently outperforms existing scalar and verbal critics, improving average success rates over the strongest baseline by 5.8% for Qwen3-VL-32B and 9.0% for Gemini-3-Flash, and demonstrates strong cross-platform generalization. Ablations show that macro-action history mitigates short-sighted planning and visually grounded critique reduces execution errors, with both components being critical for test-time scaling in long-horizon GUI tasks.

Abstract:
The criminal legal system is composed of many complex decision points, such as which charges are pursued, how bond is determined, plea deal negotiations, and sentencing. Given the profound impact of such decisions, there is a growing need to increase the transparency within this system. While prior machine learning approaches have focused on modeling single decisions (e.g., recidivism), this paper proposes a transformer-based time series model to forecast the outcome of the next proceeding in a case (e.g., charges accepted, bond set, plea deal made, etc.). By modeling a case as an event sequence, we enable a more nuanced analysis to support better oversight and understanding of decision making. We conduct an empirical analysis of thousands of cases using a newly constructed dataset derived from publicly available court records of a mid-sized U.S. city. We fit models that combine textual data from clerk notes with structured information about the defendant, prior charges, and courtroom. Our transformer model achieves .57 micro-F1 on a 20 class multi-label task, a significant improvement over several baselines and comparable to a GRU trained on the same input. We demonstrate how the model can be used to provide insights into decision-making and help court observers identify emerging issues within the jurisdiction studied.

Abstract:
Complex, multi-step reasoning demands agents to be capable of dynamically exploring diverse research directions and adapting plans on the fly. We introduce Gnosis Deepsearch, a schema-guided framework for long-horizon planning in complex Question Answering, where variables, clues, and constraints are captured in a structured state representation maintained across trajectories. This state is cascaded into parallel search paths broadening exploration without relying on sequential execution, thereby mitigating both limited search space coverage and catastrophic forgetting. We couple Gnosis with a retriever that sidesteps challenges associated with retrieval and LLM reading when longer documents are involved. Our retrieval strategy ensures that the most relevant snippets from longer documents are considered, showcasing several benefits within both Gnosis and a more general ReAct framework on the BrowseComp-Plus benchmark. Extensive experiments on several QA benchmarks of varying difficulty demonstrate the strong performance of Gnosis across different configurations.

Abstract:
Scaling generalist GUI agents is hindered by the data scalability bottleneck of expensive human demonstrations and reliance on ungrounded synthetic teacher supervision. To transcend these limitations, we propose , a framework that shifts the learning focus from mimicking high-level trajectories to learning interaction dynamics from executed transitions. Through a systematic investigation of self-supervised objectives, we identify that forward dynamics, defined as the generative prediction of future interface states, acts as the primary driver for scalability and outperforms inverse and backward objectives. leverages this insight by converting low-cost autonomous exploration into high-density generative supervision to learn a semantic transition world model from executed transitions. Experimental evaluations across a series of models demonstrate consistent gains from our approach: models utilizing Continual Pre-Training (CPT) on synthetic dynamics outperform non-CPT baselines with an average success rate improvement of 7% on offline benchmarks, which amplifies to a 16.8% gain in real-world online navigation. Furthermore, we observe that navigation performance scales with synthetic data volume. These results suggest that grounding agents in forward predictive modeling offers a promising pathway to scalable GUI automation with cross-domain adaptability and compositional generalization.

Abstract:
Large language models (LLMs) are increasingly used as coding assistants, but the programs they generate often fail. Such failures can be addressed by debugging the existing implementation or regenerating a new solution from the problem specification, with existing repair workflows typically defaulting to debugging. We find that the effectiveness of debugging and regeneration depends strongly on initial program quality, suggesting that no single repair strategy is universally optimal. Motivated by this observation, we present PATSSEL (Program-Adaptive Test-guided Repair Strategy Selection), a test-guided repair selection framework that dynamically chooses between debugging and regeneration. Rather than committing to a single repair strategy, PATSSEL generates debugged and regenerated candidates for each buggy program, evaluates them using generated verifier tests, and selects the stronger candidate, leveraging the complementary strengths of debugging and regeneration. Experiments on LeetCodeDataset, LiveCodeBench, and HE+Fix show that PATSSEL achieves average absolute improvements of 2.7% with Qwen3-8B and 3.2% with Llama3-8B over the best single-strategy baselines under the same inference compute budget and routing using gold unit tests. The gains persist across repair models, routing models, and stronger debugger models. Further analysis shows that PATSSEL benefits from the complementary strengths of debugging and regeneration, highlighting the importance of adaptive strategy selection for reliable LLM-based code repair.

Abstract:
Test-case generation is critical but remains challenging for language models. Existing test-time scaling methods either isolate candidates or aggregate traces, which do not scale to long-horizon tasks. We present TEX, a collaborative test-time scaling framework based on execution-driven cross-validation. TEX parallelizes agents to generate tests and patches, cross-executing each test against every patch to form a K×K feedback matrix. Agents use their matrix slices for refinement, enabling objective validation without verbose trace sharing. On SWT-Bench Verified, TEX achieves 86.4%, establishing a new state-of-the-art and improving +9.0% over single-agent baselines. Moreover, even random selection from TEX outputs (84.4%) surpasses the previous SOTA (84.0%), showing collaborative execution feedback improves solution quality, not just best-case samples. Our results highlight execution-grounded feedback as an effective scaling mechanism for long-horizon agentic tasks.

Abstract:
LLMs are increasingly deployed as interactive agents capable of multi-turn dialogues and tool usage, but they remain vulnerable to adaptive attacks. Existing safety mechanisms are mostly reactive, only attempting to uncover pre-existing hidden malicious intent, which makes them largely ineffective against attacks when such attacks are inherently designed to conceal their intent, and causes them to often fail in multi-turn, interactive, or out-of-distribution scenarios. In addition, existing multi-turn defences are largely data-driven, which limits their generalisation ability. To address this challenge, we propose Proactive Safety Alignment (PSA), a framework that anticipates user intent proactively to align LLM agents’ outputs with safety objectives. PSA combines Safe User Request Prediction (SURP), which anticipates likely safe future requests, and Potentially Harmful Generation (PHG), which evaluates whether the current interaction state can support misuse. Evaluations across multiple benchmarks demonstrate that PSA substantially reduces attack success rates, while preserving overall utility on MT-Bench. Our results show that PSA provides an effective and scalable paradigm for multi-turn LLM safety alignment. The code is available at https://github.com/qinmengyuan/PSA.

Abstract:
Developers can build LLM agents by adapting third-party models through benign post-training. We study a supply-chain threat in which an attacker supplies a model with a backdoor: hidden behavior that produces malicious outputs when a particular input pattern appears. Focusing on software-engineering agents, we ask whether such backdoors survive the developer’s supervised fine-tuning (SFT) and subsequent task-level reinforcement learning (RL). We observe that benign SFT substantially reduces attack success, but subsequent RL often preserves the residual behavior and sometimes even increases attack success. Our analysis of backdoor erosion during SFT identifies two factors that may favor survival: initial backdoor strength and gradient compatibility with benign training. These factors motivate PersistBD, which refines an already-backdoored model before release to improve its persistency through the benign post-training process. On Qwen2.5-Coder-7B, PersistBD raises attack success from 20% to 74% after SFT and from 20% to 76% after SFT-RL, while maintaining comparable benign task performance. Together, our results show that backdoors can remain active through benign post-training and that adversaries can deliberately increase their persistence. This highlights a supply-chain risk for AI developers and motivates stronger techniques for detecting and mitigating inherited backdoors when adapting third-party models into agents. Our code is available at https://github.com/uiuc-kang-lab/PersistBD.

Abstract:
Dense embedding rankers score documents through contextual sentence- and passage-level representations, yet listwise explanation methods often attribute rankings to isolated words. We study this mismatch and introduce ChunkGroupSHAP, a listwise Shapley method that clusters semantically related chunks across documents into shared features, preserving contextual evidence while bounding the KernelSHAP regression dimension by the group count. Across MS MARCO, FinanceBench, AILACaseDocs, and FinQA with E5-family rankers and BM25, raw chunks improve rank-reconstruction Fidelity over RankSHAP’s word features in all 11 dense-ranker settings. The best chunk-group configuration further improves on raw chunks in eight of these settings, with the incremental benefit depending on grouping scope; word features remain strongest in three of four BM25 settings. These results show that explanation units should match the ranking model: contextual chunks better suit dense bi-encoders, whereas words remain effective for BM25. ChunkGroupSHAP supports listwise attribution over contextual evidence through a bounded feature space shared across documents.

Abstract:
Large Language Models (LLMs) are increasingly used to support effective and constructive human-to-human communication. A key scenario is helping users express difficult intents, such as criticism and boundary setting without harmful language. Existing safety refusal and detoxification rewriting methods either block harmful responses or inadequately preserve the user’s communicative intent and speaker style. Therefore, we propose constructive communication rewriting , which aims to rewrite harmful utterances in dialogue into safer and more constructive language while preserving user intent and style. To instantiate this task, we construct a large-scale dataset of realistic multi-user conversations by modeling both user persona and harm-inducing contexts. We further use Nonviolent Communication-inspired strategies to guide the rewriting process. Building on this dataset, we develop CoRe, a small, specialized LLM fine-tuned through a two-stage distillation process. CoRe achieves substantially higher harmlessness than strong general-purpose LLMs while maintaining competitive faithfulness and remaining efficient for real-time communication. Our project is released at the following link[.].

Abstract:
Large language models (LLMs) are deployed with safety guardrails, yet jailbreak attacks and malicious prompts keep bypassing them. Re-aligning an LLM each time a new attack appears is more expensive than crafting attacks, so jailbreak detection is increasingly framed as a pre-processing step: a cheap detector filters out a malicious prompt before it reaches the LLM. This need is reinforced by multilingual attacks, where simply translating a prompt to another language can bypass the guardrails. Our goal is to find whether internal representations of a frozen LLM already separate malicious from benign prompts, and whether such a signal is resilient against translation. We study two types of detectors that require no fine-tuning of the LLM: a logistic-regression classifier over whole-sentence embeddings, and a probe-based detector that trains one logistic-regression probe per attention head on the attention activations of the last input token. They need at most a single forward pass. Our experiments show that both match or outperform strong sentence-embedding detectors and off-the-shelf baselines, specially under translation. The probes degrade more gracefully on higher-resource languages and remain competitive on lower-resource ones. Our results indicate that a frozen LLM linearly encodes whether an incoming prompt is malicious in a small set of mid-to-late layers. Per-head attention probes thus offer a defence that can be cheaply retrained as new attacks emerge, without touching the underlying model.

Abstract:
Linear attention architectures enable efficient long-context modeling by compressing sequence history into a fixed-size memory state. However, dense key value updates can overload the state as the context grows, causing memory collisions and degraded retrieval. Existing methods rely on memory decay, rank reduction, or heuristic routing to reduce interference, but they largely preserve dense online updates and do not explicitly control the structural complexity of memory. We introduce BregNet, a linear sequence architecture that formulates associative memory updates as a regularized online regression problem. Based on Linearized Bregman Iteration (LBI), BregNet decouples memory into a continuous dual accumulator and a structurally sparse primal state. The dual state records incoming information, while the sparse primal state keeps only important associations and removes noise through soft shrinkage. Experiments show that BregNet matches strong linear baselines in language modeling even with high memory sparsity. In this way, BregNet prevents the memory state from being overloaded by dense and interfering updates.

Abstract:
Powered by advances in LLMs and autonomous agents, deep research has become one of the most widely adopted agentic products. However, most deep research systems write general-purpose reports, which are inadequate for financial deep research. Financial research demands specialized knowledge to analyze historical patterns and forecast upcoming events. Automating financial deep research therefore requires both a layered harness to drive the research agent and a verifiable, point-in-time benchmark that prevents leakage of future information. We present FinanceHarness, a harness that runs finance-oriented tools and practitioner-guided workflows, automating financial deep research end to end: environment and data construction, the agent execution loop, and reward modeling. We further propose FinanceGym comprising thesisdriven research questions and rubrics that combine pre-cutoff and post-cutoff criteria. Professional expert validation yields an 82% pass rate. With the same open-weight backbone, FinanceHarness improves the overall rubric score from 25.3% to 32.4%, demonstrating the effectiveness of our specialized harness design. However, even pairing FinanceHarness with the most cutting edge LLM (e.g. Opus-5), the FinanceHarness score is below 45%, showing that it is a challenging benchmark for financial deep research. FinanceGym and FinanceHarness code are publicaly available.

Abstract:
Evaluations of social bias in LLMs largely focus on whether models generate or imply biased content. However, as LLMs are increasingly used as judges of bias, they may exhibit social biases in subtler ways in how they evaluate biased content, which current methods do not systematically capture. We call this second-order bias: social bias in an LLM’s judgment about social bias, which we evaluate through a novel, philosophically grounded reasoning task. Drawing on entitlement epistemology, we conceptualize bias as misplaced foundational knowledge that shapes an agent’s rational inquiry, and derive a logical reasoning task for LLMs to judge to whom a biased text is acceptable or non-acceptable. We develop two simple metrics to measure how biased LLM judges are in inferring demographics for acceptability without sufficient support, and how these inferences vary across groups targeted by biased texts. Evaluating open and closed models, we find that our task evades safety guardrails by surfacing bias in model judgment. It varies systematically across target groups, reflects implicit social maps, and shows how models are still triggered by demographic labels. Our work points to the need for LLM bias evaluation in judgment tasks and broadly, for more theoretically grounded approaches to bias evaluation in NLP. We release our code and model responses at https://github.com/uofthcdslab/second-order-bias.

Abstract:
Large language models exhibit substantial performance variation across languages, even when solving semantically equivalent tasks. Existing analyses often treat this phenomenon as an observational disparity caused by differences in pretraining data, tokenization, or benchmark coverage. We study a complementary hypothesis: high-resource languages (HRLs) may more reliably elicit latent computations useful for task-specific (i.e. mathematical) reasoning, while lower-resource languages (LRLs) may under-activate those computations despite expressing the same task. To test this hypothesis, we introduce a mechanistic intervention framework for identifying and transferring task-relevant sparse latent features across languages. Using sparse autoencoders over residual-stream activations, we isolate features enriched in successful HRL task-specific reasoning while filtering out source-language and generic-generation features. We then construct steering directions from these features and inject them during LRL inference. The resulting interventions test whether the selected features are functionally involved in the observed reasoning gap: suppressing them should impair source-language reasoning, while activating them should partially recover target-language reasoning beyond random and non-task controls. Our framework reframes some cross-lingual reasoning gaps as failures of mechanism elicitation rather than capability absence, and offers a causally testable route to feature-mediated transfer without translation, fine-tuning, or changing the user-facing language.

Abstract:
Guard models are used as safety classifiers, with confidence scores driving downstream moderation decisions. We evaluate five guard models for prompt classification and find that although several are nearly calibrated on clean inputs, adversarial attacks degrade their calibration by an order of magnitude, turning false negatives into high-confidence errors indistinguishable from correct detections. Comparing each guard with its corresponding base LM, we find that uncertainty signals often remain available, with the base model typically expressing uncertainty on the same inputs where the guard fails. Layer-wise analyses localize this guard–base divergence to later layers, where guard models exhibit sharper safe/unsafe separation and lower-rank representations, while adversarial harmful inputs lie closer to the clean-safe region. These findings highlight a mismatch between guard confidence and base-model uncertainty under attack.

Abstract:
As generative AI tools find increasing use in research workflows, ongoing debates on their impact, appropriateness and responsible use have led policymakers to enact policies to disclose AI use at multiple publishing venues. However, are current AI disclosure policies and practices reflective of their purpose? In this work, we first investigate disclosure policies of top computer science venues and find that despite their prevalence, they remain highly under-specified. Secondly, through a survey of computer science researchers (N=109), we characterize the necessity of disclosures across different research tasks and levels of human involvement. We learn that researchers find disclosures most necessary for tasks involving research design, and for tasks when the human involvement is low. We also compile expectations that researchers have about the information to be conveyed in AI disclosure statements. Lastly, through an analysis of 13867 disclosure statements from EMNLP 2025 and ICLR 2026, we reveal a large disconnect between these expectations and AI disclosures in practice—a prime example being writing assistance which is deemed less necessary but is frequently disclosed. We conclude with recommendations to align AI disclosure policies and practices with expectations, suggesting a categorization of research tasks by perceived necessity and a boilerplate template capturing expected details.

Abstract:
Accurate alignment between chest X-ray (CXR) images and radiology reports is essential for clinical safety, yet automatic radiology report error detection from paired CXRs remains challenging. Prior approaches typically compare the entire report with the image, limiting the model’s ability to learn fine-grained vision-language correspondences at the level of individual findings. We reformulate report error detection as a sentence-level vision-language learning problem by decomposing reports into atomic finding statements, thereby reducing semantic entanglement and supervision noise. We propose SenRED (Sentence-level Report Error Detector), which uses sentence-conditioned patch aggregation to associate each statement with localized visual evidence. To supervise learning, we generate sentence-level synthetic errors reflecting clinically relevant categories—false positives, false negatives, and mislocalization—along with consistent image-sentence pairs. Experiments on MIMIC-CXR and Open-i show that SenRED consistently outperforms task-specific baselines and zero-shot instruction-tuned VLMs in both sentence-level and exam-level evaluations. Radiologist-confirmed evaluations further show that these gains are maintained on clinically reviewed subsets. Comparative experiments support the benefit of sentence-level reformulation, while ablations demonstrate that sentence-conditioned patch aggregation improves fine-grained consistency checking.

Abstract:
Retrieval-augmented generation (RAG) is increasingly deployed on unverified corpora, where an adversary can inject poisoned documents to steer the model toward an attacker-specified answer. Existing defenses rely on auxiliary heuristics and break down once their assumptions about the attack type or poisoning level are violated. We propose SAFER-RAG, a defense that derives its reliability signal from the adversarial objective itself. Since poisoned documents are explicitly optimized to rank highly for the target query, they remain anchored at top ranks under semantically equivalent rewrites, while clean documents shift naturally. SAFER-RAG captures this rank-stability gap through spectral analysis, partitions retrieved chunks into Safe and Unsafe Sets, and adaptively selects evidence under a safe-first strategy that avoids over-filtering useful evidence on clean or lightly poisoned corpora. Across three QA benchmarks, three attack types, and four poisoning levels, SAFER-RAG consistently achieves the highest accuracy and the lowest attack success rate. The code will be released upon acceptance.

Abstract:
Standard dense retrievers optimize to bring relevant query document pairs closer, but do not preserve which task-critical information (e.g., entities, dates, concepts, or logical relationships) survives compression into embeddings. We observe that supplementing queries with such information at inference time improves retrieval, motivating the question of how to train information-preserving dense retrievers. We propose a new training strategy that augments contrastive learning with a reconstruction loss to provide supervision over embedding content. Our method, RecovIR, is a decoder-only LLM retriever that appends learned latent tokens and uses their representations for both cosine retrieval and autoregressive reconstruction. Across multiple architectures, RecovIR consistently improves nDCG@10 by 1–2 points on MTEB-English retrieval benchmarks over standard baselines. We find that retrieval failures often coincide with reconstruction errors, and show theoretically that when targets are relevance-sufficient and recoverable, reconstruction lower-bounds the relevance information retained in embeddings.

Abstract:
Implicit chain-of-thought (iCoT) methods aim to internalize reasoning in large language models, but often underperform explicit CoT prompting. We empirically find that hidden-state reasoning trajectories exhibit low-rank structure. Motivated by this observation, we propose a low-rank distillation framework that transfers reasoning by aligning teacher and student trajectories in a shared low-rank tensor subspace using first- and second-order statistics. The resulting formulation captures the global structure of reasoning while supporting a compact latent reasoning process. We evaluate the method across multiple model families, including LLaMA and Qwen, at different scales on mathematical reasoning benchmarks. Our approach consistently improves performance, especially on challenging multi-step tasks, approaching explicit CoT accuracy and outperforming prior iCoT distillation methods.

Abstract:
Retrieval-Augmented Generation (RAG) en- hances large language models with ex- ternal knowledge but often suffers from query–evidence mismatch due to inappropriate retrieval granularity. Chunk-level retrieval can introduce substantial noise, while graph-based retrieval may suffer from overly fine-grained or densely connected structures that hinder effec- tive multi-hop reasoning. To address this issue, we propose Atom-RAG, a retrieval-augmented framework based on atomic fact–centric knowl- edge representation and granularity-aligned re- trieval. Atom-RAG represents documents as collections of atomic facts and aligns retrieval with the fine-grained information needs of com- plex queries. By decomposing queries into fact-level subqueries, the framework retrieves and aggregates evidence in a manner aligned with the information requirements of multi-hop reasoning, producing a compact and evidence- aligned context for generation. Experiments on benchmarks covering different reasoning depths show that Atom-RAG consistently out- performs strong iterative and graph-based RAG baselines, achieving improved multi-hop QA performance and stronger evidence retrieval. These results highlight the importance of align- ing the granularity of queries and retrieval units for reliable multi-hop reasoning within a unified retrieval pipeline. Code and data are available at https://github.com/cody628/ AtomRAG.

Abstract:
The Iranic language family spans 101 ISO 639-3 coded varieties and over 200 million speakers across a vast arc from the Caucasus through the Middle East and Central Asia to South Asia and China. Yet this richness is barely reflected in modern NLP: resources, models, and benchmarks are overwhelmingly concentrated on Iranian Persian, leaving Kurdish, Pashto, Balochi, Gilaki, Zazaki, and the Pamir languages computationally invisible. This survey provides the first comprehensive landscape analysis of Iranic NLP, covering 495 papers published between 2020 and early 2026. We examine language coverage, NLP tasks, datasets, modeling approaches, and evaluation practices, and identify persistent structural gaps: script heterogeneity, dialectal fragmentation, data scarcity, and an evaluation monoculture that has saturated Persian while leaving near-zero baselines for every other variety. Together with this survey, we release a companion website providing interactive language maps, a curated resource browser, and a live publication index at https://sigiranic.github.io.

Abstract:
Since late 2022, a Limitations section has become mandatory at many top-tier NLP conferences. The growing number of accepted papers at these venues has resulted in a vast corpus of self-reported limitations that cannot all be manually reviewed, yet remains systematically unanalyzed. Therefore, in this paper, we conduct a large-scale analysis of the Limitations sections from ACL and EMNLP papers published between 2020 and 2025 to understand what researchers disclose about their own work. To do so, we implement a novel human-AI framework for iterative hybrid qualitative coding. This framework enables us to investigate trends in self-reported limitations over time, their correlations with specific paper attributes, and the writing patterns that recur around these disclosures. Our findings offer a critical reflection on the diverse reported challenges as well as the self-reporting practices of researchers in the NLP community.

Abstract:
Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most existing SAE-based steering methods select features by applying a top- filter based on statistical scores, assuming that higher-scoring features yield stronger steering effects. In this paper, we show that this assumption is often invalid, leading to suboptimal feature selection. Our analysis reveals that effective steering features may be distributed among representationally adjacent, semantically similar groups induced by feature splitting in SAEs. Within such groups, features may exhibit disparate statistical scores despite having comparable steering influence, causing score-based selection to overlook important features. Based on these observations, we propose Neighbor Integrated Feature Selection ( NIFS ), a plug-and-play strategy that leverages representation similarity to improve feature selection for steering. We evaluate NIFS across multiple SAE-based steering methods and tasks, and demonstrate consistent performance gains over conventional top- selection.

Abstract:
Self-organizing LLM-based multi-agent systems dynamically select agents and organize communication graphs for each query. While this adaptivity improves flexibility, it also creates a control-plane attack surface: a malicious agent may exploit this selection and organization process to preserve future access and influence over later tasks. We formulate the attacker as a reputation-aware Trojan agent that chooses when to behave constructively and when to disrupt the task. Under full reputation access, we prove the optimality of a threshold-based attack policy and show why persistent attacks require a survivability constraint. We then propose WEREWOLF , a primal-dual reinforcement learning attacker that learns attack timing under hidden reputation by maximizing task disruption subject to control-plane access loss. Across four self-organizing systems and three benchmarks, it reduces average accuracy from 72.3% to 25.8% with an 86.3% attack success rate, and retains control-plane exposure comparable to that of honest agents.

Abstract:
Determining the veracity of a verbal statement is a frequent and challenging task for legal practitioners. The Swedish Supreme Court has proposed a set of truth-indicative reliability criteria for assessing verbal statements in the Swedish legal system. LLMs’ growing ability to reason and handle complex tasks raises questions about how they perform veracity judgments of verbal statements. In this study, we explore this question on a novel Swedish mock-crime dataset. We compare simple prompting to a two-step method in which the LLM is first asked to assess the reliability criteria and then perform a criteria-informed veracity judgment. We find that accuracy is at chance level under both conditions, but that the two-step method often leads to a more reasonable truth bias and reduces models’ overconfidence. Further analysis indicates that criteria assessments predict final judgments, but not ground truth. We also find that the statement category (true or false and confession or denial) matters to overall performance and the effect of two-step prompting.

Abstract:
As LLMs increasingly serve as moral advisors and agents, they must address conflicts between competing values. Yet prior work on moral dilemmas overlooks a central aspect of human moral cognition: imagining alternatives beyond the given options. We introduce MoralAltDataset, comprising 307 Advisor and AI-facing Agent dilemmas augmented with compromise and reframed alternatives. We compare human and LLM judgments in binary and four-option settings. Across human participants and 15 LLMs, aggregate moral choice distributions differ substantially between the two settings, with compromise often preferred over either original binary option. Results show value shifts and stronger human–LLM agreement on alternatives. Source-stratified results reveal a descriptive gap: human alternative-selection rates are similar across authoring sources, whereas LLMs select GPT-5-authored alternatives substantially more often. We then compare human-authored alternatives with outputs from three representative LLMs through pairwise preference and expert-based evaluations. Alternatives from these LLMs are generally preferred and better satisfy fine-grained structural and ethical criteria, while revealing a trade-off between structural quality and practical feasibility. Our dataset is available here: https://huggingface.co/datasets/jongchanch/MoralAltDataset, and our project page is here: https://jongchanchoi.com/moral-imagination.

Abstract:
Modern Text-to-SQL systems often follow generate-execute-select pipelines, generating multiple candidate queries then selecting the best one. Listwise selection, by jointly comparing multiple candidates, has been widely adopted, but fine-tuning listwise selectors is costly. We thus propose a fine-tuning-free listwise selector. We replace two major fine-tuning objectives with inference-time strategies: (1) learning selection criteria as ordering and (2) mitigating positional bias. First, we build reusable structured memories instead of learning selection behavior as model parameters. Given a question, MAP-SQL retrieves memories distilled from training data that encode how natural language maps to schema elements, SQL operations, and expected outputs. These memories serve as explicit decision criteria for evaluating candidates in a listwise manner. Second, to mitigate ordering bias of listwise selectors, we aggregate rankings across multiple input permutations, with inference cost optimized by execution results and pointwise scoring. Our approach improves selection accuracy while maintaining efficiency and compatibility with existing large language models. Across Text-to-SQL benchmarks, it produces more stable selection without fine-tuning and fewer unnecessary comparisons than existing methods. On BIRD-dev, it outperforms the previous state-of-the-art selector-based method R³-SQL by 2.02 execution accuracy points on average using the same candidate sets, with 2.92× fewer tokens.(https://github.com/ldilab/MAP-SQL)

Abstract:
Latent Chain-of-Thought (Latent CoT) models promise efficient reasoning via continuous representations, yet exhibit puzzling performance inconsistencies: excelling at exploration (ProsQA: 97.0%) but failing at computation (GSM8K: 34.1%). We reveal that this trade-off is governed by decisional certainty. Our contributions are threefold: (1) We theoretically characterize the fundamental Exploration-Execution Trade-off, proving that high certainty enables precise execution but inhibits exploration, while low certainty facilitates search but causes error accumulation. (2) We introduce the Symbolic Index, quantifying decisional commitment, as the core mechanism governing this trade-off and establish its causal relationship with both execution stability and exploration capability. (3) We prove that curriculum learning is theoretically necessary, as direct training provably fails due to distributional mismatch. Our framework shifts the design paradigm from binary architectural choices toward adaptive systems that dynamically regulate decisional certainty based on task demands.

Abstract:
Entropy mechanism emerges as a key organizing principle for understanding and improving Large Reasoning Models (LRMs). This survey examines how entropy shapes both their training and inference behavior. On the training side, we take a mathematical perspective: casting existing RL-based reasoning algorithms into a unified objective and using this formulation to derive an entropy-centric decomposition of methods, clarifying how different approaches adjust exploration–exploitation "knobs". On the inference side, based on the characteristic of LRM that trading increased inference tokens yields better performance, we summarize methods that leverage entropy to enhance inference performance or reduce uncertainty. Finally, we discuss open challenges and future directions of entropy-driven research for LRMs. Our repository is available on https://anonymous.4open.science/r/Awesome_LRM_with_Entropy-0503.

Abstract:
Recent representation analyses have reported structure-like or invariant behavior even in randomly initialized transformers, motivating the interpretation that aspects of linguistic structure may be architectural rather than learned. We show that such conclusions can be artifacts of coarse symbolic discretization. We formalize symbolic analysis of neural representations as operating over discretized activation trajectories and identify symbolic resolution as a critical, previously uncontrolled variable. Varying resolution reveals a sharp collapse of apparent similarity in random models, while pretrained models maintain stable, operator-selective differentiation across all tested resolutions. We introduce resolution stability as a necessary validity criterion for symbolic and automata-based analyses, providing a falsifiable, architecture-agnostic diagnostic for auditing internal linguistic structure in neural language models.

Abstract:
Computational stylometry studies writing style through quantitative textual patterns, enabling applications such as authorship attribution, identity linking, and plagiarism detection. Despite the relevance of language modeling to these tasks, the pre-training of modern large language models (LLMs) has been underutilized in authorship attribution and verification. We introduce an unsupervised framework that uses the log-probabilities of an LLM to measure style transferability between two texts. This framework takes advantage of the extensive Causal Language Modeling (CLM) pre-training, one-shot capabilities and scale of LLMs, avoiding explicit supervision. Our methods substantially outperform prompting-based unsupervised baselines in authorship verification at similar model sizes, and is competitive with or improves contrastive baselines in most settings with sufficient model scale. We further observe strong performance across non-English languages. The effectiveness of the proposed framework improves consistently with increasing model scale. In the case of authorship verification, we propose an additional mechanism that increases test-time computation to improve accuracy; enabling flexible trade-offs between computational cost and task performance.

Abstract:
Large language model (LLM) agents excel at complex, multi-step tasks but suffer from catastrophic forgetting during continual learning—acquiring new skills often degrades previously mastered capabilities. Existing post-training paradigms face a fundamental trade-off: Supervised Fine-Tuning (SFT) is sample-efficient but prone to severe forgetting, while Reinforcement Learning (RL) preserves prior capabilities but converges slowly without expert guidance. To bridge this gap, we introduce Trigger-based Supervised Reinforcement Learning (TSRL), a framework that integrates the strengths of both paradigms through an adaptive, state-dependent expert supervision mechanism. TSRL employs RL as the optimization backbone, activating SFT selectively on "hard states" where the policy fails. Theoretically, we show that restricting supervision to failure states derives an upper bound on the cumulative parameter updates, thereby reducing forgetting compared to full SFT. Empirically, TSRL achieves consistent gains across three agent benchmarks (ALFWorld, WebShop, and Deep Search) at both 7B and 32B scales. In continual learning scenarios, our method retains up to 9.6 points more capability on prior tasks than SFT → RL baselines, while matching or exceeding target-task performance.

Abstract:
Large language models (LLMs) remain vulnerable to jailbreak attacks that elicit structured, credible misinformation. Existing benchmarks primarily track binary refusals, often overlooking the severity and exposure risk of the misinformation payload itself. In this work, we introduce Misinfo-Jailbreak Bench, a comprehensive framework that systematically maps diverse attack strategies to a claim-level scoring pipeline for precise risk quantification. Building on this diagnostic foundation, we propose Selective LoRA Ethical Surgery, a parameter-efficient defense mechanism. Instead of broad parameter updates, our approach identifies safety-relevant subspaces via contrastive representations to confine LoRA edits and employs inference-time gating to preserve benign capabilities. Experimental results on state-of-the-art models (e.g., Qwen2.5, Llama-4) demonstrate that our method reduces high-severity risk by 60-70% while limiting general capability degradation to less than 1%, offering a precise and minimally invasive paradigm for post-hoc alignment.

Abstract:
Quantization reduces the deployment cost of large language models (LLMs). However, post-training quantization (PTQ) cannot fine-tune model parameters and often degrades accuracy in low-bit settings, while quantization-aware training (QAT) requires backpropagation and incurs high memory overhead. We propose ZeroQAT, a zeroth-order QAT framework supporting both weight and activation quantization. By using forward-only gradient estimation, ZeroQAT removes backpropagation and substantially reduces memory and computation while retaining end-to-end optimization benefits. We further introduce a lightweight variant of ZeroQAT for quantized fine-tuning, which freezes and pre-quantizes most parameters to further cut memory usage. Experiments show that ZeroQAT consistently outperforms representative PTQ and QAT baselines while requiring significantly less memory. For example, ZeroQAT enables fine-tuning of a 13B model at extremely low bit-widths (e.g., 2-4 bits) on a single 8GB GPU, and even allows fine-tuning a 6.7B model on a OnePlus 12 smartphone, demonstrating its practicality for end-to-end QAT on resource-limited edge devices.

Abstract:
Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in latent CoT reasoning , where the reasoning process is embedded entirely within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning.

Abstract:
Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to new or unknown attack strategies, providing limited resilience. To address these, we propose a multi-agent defense framework, GuardrailAgent, that offers adaptive, end-to-end protection. The system autonomously gathers and summarizes attack characteristics into a continuously updated knowledge base that supports both input- and output-side detection. A feedback module integrates misclassified cases for iterative improvement at minimal cost. Tested on over ten representative jailbreaks, the framework achieved a 93.1% input-side detection rate, 93.9% overall detection, and a 94.7% F1 score, while rapidly adapting to unseen attacks. These results demonstrate GuardrailAgent’s outstanding capability to maintain LLM safety in the face of evolving jailbreak tactics.

Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in mainstream visual understanding tasks, but their ability to process action scenes that contradict everyday common sense remains undertested. To address this gap, we introduce CAIT, a benchmark comprising 400 high-fidelity synthetic scenes focused on counter-intuitive visual actions, such as “a rabbit is chasing a tiger”, where visual evidence explicitly contradicts common-sense expectations. We evaluate human, leading proprietary models (e.g., Claude and Gemini), and 14 representative open-source MLLMs. Humans achieve near-perfect performance (around 0.95 accuracy) and proprietary models demonstrate robust understanding (achieving up to 0.88 accuracy), standard open-source instruction-tuned models perform at the chance level. Further analysis demonstrates that this failure is driven by a strong language prior: rather than trusting the visual input, they automatically override the anomalous visual signals with statistically common text descriptions. Although introducing Chain-of-Thought (CoT) reasoning mechanisms can improve accuracy, it significantly slows down the response and generates a new failure mode: models overthink the scenario and refuse to accept the actual visual content simply because it violates real-world physical laws. Finally, we demonstrate that targeted fine-tuning and structured prompting can effectively mitigate this reliance on language priors, enabling open-source models to accurately ground their reasoning in actual visual evidence.

Abstract:
Tokenization is a critical component of the language modeling pipeline, but it is poorly understood. Tokenizers can drastically impact downstream performance, but currently there is mixed evidence about which intrinsic tokenizer evaluations are able to predict which tokenizers will result in a better language model before training the model. In this paper, we analyze approximately 200 language models trained on nine different languages. We evaluate both models and their tokenizers and test whether there is a relationship between any proposed intrinsic tokenizer evaluations and model performance, as measured by a wide variety of benchmarks. Despite the quantity and variety of proposed intrinsic tokenizer metrics, none consistently predicts downstream performance across model families, tasks, and languages. This motivates the need for novel tokenizer evaluation metrics and approaches. Our analysis of a novel linguistically grounded metric introduced in this work shows that new metrics can capture aspects of tokenizer quality that existing metrics miss, though their predictive power is likely to vary across model families and languages.

Abstract:
Sampling multiple responses is a common way to improve LLM output quality, but it comes at the cost of additional computation. The key challenge is deciding when to stop generating new samples to balance accuracy gains against efficiency. To address this, we introduce BEACON (Bayesian Efficient Adaptive Criterion for Optimal N-stopping), a principled adaptive sampling framework grounded in Sequential Search with Bayesian Learning. BEACON sequentially generates responses from the policy LLM, updates posterior belief over reward distributions in real time without further training, and determines when to stop by weighing expected gains against computational cost. Sampling terminates once the marginal utility of further exploration no longer justifies the expense. We establish both theoretical optimality guarantees and practical tractability, and show empirically that BEACON reduces average sampling by up to 80% while maintaining response quality. We further demonstrate BEACON’s utility for cost-efficient preference data generation and outline practical extensions, offering actionable insights for future researchers.

Abstract:
Reinforcement Learning (RL) is essential for evolving Large Language Models (LLMs) into autonomous agents capable of long-horizon planning, yet a practical recipe for scaling RL in complex, multi-turn environments remains elusive. This paper presents a systematic empirical study using TravelPlanner, a challenging testbed requiring tool orchestration to satisfy multifaceted constraints. We decompose the agentic RL design space along 5 axes: reward shaping, model scaling, data composition, algorithm selection, and environmental stability. Our controlled experiments yield 7 key takeaways, e.g., (1) reward and algorithm choices are scale-dependent as smaller models benefit from staged rewards and enhanced exploration, whereas larger models converge efficiently with simpler dense rewards, (2) ~1K training samples with a balanced difficulty mixture mark an empirical sweet spot for balancing in-domain and out-of-domain performance under our primary setting, and (3) environmental stability is critical to prevent policy degradation. Based on our distilled recipe, our RL-trained models achieve state-of-the-art performance on TravelPlanner, outperforming leading LLMs.

Abstract:
Spatial audio understanding aims to enable machines to interpret complex auditory scenes, particularly when sound sources move over time. In this work, we study Spatial Audio Question Answering (SAQA) with a focus on movement reasoning, where a model must infer object motion, position and directional changes directly from stereo audio. First, we introduce a movement-centric spatial audio augmentation framework that synthesizes diverse motion patterns from isolated mono audio events, enabling controlled and scalable training data generation. Second, we propose an end-to-end multimodal finetuning approach with a thinking mode, which allows audio-language models to produce explicit intermediate reasoning steps before predicting an answer. Third, we investigate the impact of query-conditioned source separation as a preprocessing stage and compare three inference regimes: no masking, an audio grounding model (AGM), and ground-truth masks. Our results show that the benefits of thinking mode are most pronounced when paired with high-quality source separation rather than in isolation, with an improvement of +5.1% when a single event is present in the question. These findings highlight the relationship between movement modeling, reasoning, and separation quality, offering new insights for advancing spatial audio understanding.

Abstract:
As scientific literature grows and papers increasingly under-report limitations, multi-agent LLMs offer a promising approach to systematically uncover these hidden failure modes. Here, we introduce Tree-of-Concerns, a multi-agent framework that deploys specialized skeptic personas, each operating through a category-specific analytical lens, as parallel debate trees to extract unstated limitations from scientific papers. Each persona conducts structured, evidence-grounded argumentation, while a Panel Review mechanism re-evaluates each surviving claim from all five perspectives to correct category drift and severity miscalibration. Through retrieval-free, single-paper experiments on ToC-Bench, our benchmark of 414 research papers with 1,905 unstated limitations, sourced from reviewer-reported weaknesses and follow-up citation critiques, we demonstrate that ToC improves precision by 79% and coverage by 11% relative to the strongest baseline, surfacing specific, evidence-grounded concerns that support reviewers in systematic evaluation.

Abstract:
Post-training compression of Large Language Models (LLMs) removes entire architectural components, either deleting them or replacing them with fitted modules. Existing replacement-based methods share two design constraints: full-layer granularity and contiguous selection. We argue that this is overly restrictive: in fact, redundancy in pretrained transformers is not confined to contiguous regions, nor does it evenly distribute between Attention and FeedForward outputs, implying that different strategies best approximate different submodule types and that removable components need not cluster within contiguous depth ranges. Based on this intuition, we introduce SubFit (Submodule-level Fitted residual replacement), which compresses LLMs at the submodule level: Attention and FeedForward submodules are selected non-contiguously, and each receives its own lightweight fitted residual bypass. SubFit operates post-training and requires only calibration data. Across ten LLMs (five base, five instruction-tuned), five sparsity levels from 12.5% to 37.5%, and four replacement-based baselines, SubFit achieves the best aggregate perplexity–accuracy trade-off across the evaluated sparsity levels, with larger gains under aggressive compression. At 25% sparsity, it retains 84.6% of dense downstream accuracy and incurs 2.42× perplexity degradation, against 81.6% and 4.34× for the strongest baselines, while delivering measurable inference speedup and KV-cache savings. Code is available at https://github.com/eliacunegatti/SubFit .

Abstract:
Training long-horizon LLM agents with reinforcement learning is challenging because sparse outcome rewards reveal whether a task succeeds, but not which intermediate actions caused the outcome or how they should be corrected. Recent methods alleviate this issue by generating rewards or textual hints from turn-level action-output signals, or by using feedback-conditioned self-distillation. However, generating feedback at every turn is inefficient when many intermediate turns are already successful or neutral, and applying feedback at a fixed or misaligned turn often fails to supervise the actions that contributed to the failure. To bridge this gap, we propose HINT-SD, a targeted self-distillation framework that uses full-trajectory hindsight to select failure-relevant actions and applies feedback-conditioned distillation only to targeted action spans. Experiments on BFCL v3 and AppWorld show that our method outperforms the dense per-turn feedback baseline by up to 13.60 percentage points on average while achieving a 2.26 × reduction in time per training step, suggesting that selecting where to distill is key to effective and efficient long-horizon agent training.

Abstract:
With the release of OpenAI’s o1 model, reasoning models that adopt slow-thinking strategies have become increasingly common. Their outputs often contain complex reasoning, intermediate steps, and self-reflection, making existing evaluation methods and reward models inadequate. In particular, they struggle to judge answer equivalence and to reliably extract final answers from long, complex responses. To address this challenge, we propose xVerify, an efficient answer verifier for evaluating reasoning models. xVerify shows strong equivalence judgment capabilities, enabling accurate comparison between model outputs and reference answers across diverse question types. To train and evaluate xVerify, we construct the VAR dataset, consisting of multi-round annotated QA pairs generated by multiple LLMs across challenging reasoning tasks. Experimental results on both test and generalization sets show that all xVerify variants achieve over 95% F1 score and accuracy. Notably, the smallest model, xVerify-0.5B-I, outperforms all evaluation methods except GPT-4o, while xVerify-3B-Ib surpasses GPT-4o in overall performance. In addition, reinforcement learning experiments using xVerify as the reward model yield an 18.4% improvement for Qwen2.5-7B compared with direct generation, exceeding the gains achieved with Math Verify as the reward. These results demonstrate the effectiveness and generalizability of xVerify. All xVerify resources are available on [GitHub](https://github.com/IAAR-Shanghai/xVerify).

Abstract:
Large Vision-Language Models (LVLMs) remain prone to hallucinations, producing responses that are irrelevant or inconsistent with the multimodal input. Existing mitigation methods mainly rely on external supervision, output calibration, or attention regulation, leaving the internal representation dynamics of autoregressive generation underexplored. We identify an inference-time failure mode in which cross-modal representations degrade across decoder layers and drift across generation steps, destabilizing token prediction and increasing hallucination risk. We propose Dynamic Alignment Compensation (DAC), a training-free inference-time method that detects representation divergence and selectively applies lightweight residual compensation. DAC combines Layer-wise Semantic Compensation to mitigate inter-layer degradation with Sequential Semantic Correction to constrain temporal drift. Experiments on nine hallucination-focused and general-purpose multimodal benchmarks across multiple LVLM backbones show that DAC consistently reduces hallucinations while maintaining strong overall performance.

Abstract:
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data.

Abstract:
LLM agents can generate paper reproduction code, yet often produce scientifically unfaithful implementations. We define this failure mode as semantic drift, where generated code silently diverges from the paper’s specifications. We introduce SemanticAlign-Bench(SA-Bench), a diagnostic benchmark covering 30 papers from ICLR, ICML and NeurIPS 2025. For each paper, we decompose its specifications into atomic and verifiable implementation claims, which we call Semantic Alignment Units (SAUs) and evaluate repositories along four diagnostic dimensions spanning numerical, methodological, protocol and ordering drift. In total, we construct 1,491 SAUs across five ML domains and evaluate 12 generator configurations (4 models 3 scaffolds). Even the strongest configuration (Claude + PaperCoder) achieves a mean SAU score of only 0.301 out of 1.0, with an overall mean of 0.221 across 360 evaluations. A failure taxonomy reveals that agents attempt most requirements but implement them incorrectly, with implementation mismatch and stubs accounting for the majority of zero-scored claims. Our analysis further indicates that scaffolds optimized for executability provide limited leverage for scientific reproduction; narrowing the gap requires scaffolds that prioritize semantic specification verification. The benchmark, annotations and evaluation pipeline are publicly available. Code:https://anonymous.4open.science/r/SemanticAlign-Bench-0676 Data:https://anonymous-hf.up.railway.app/a/rrgn430zpfui/

Abstract:
Large language models (LLMs) achieve strong fluency in open-domain question answering, yet often hallucinate in specialized medical QA and lack reliable uncertainty quantification, posing risks in safety-critical applications. We propose Uncertainty-guided Retrieval-augmented Generation (URGen), a training-free framework for guideline-grounded medical multiple-choice QA, where authoritative clinical documents (e.g., consensus statements, treatment protocols) are available and answer correctness depends on precise evidence retrieval. URGen is a four-stage pipeline that combines adaptive retrieval depth with uncertainty calibration: (1) multi-temperature knowledge retrieval to rank candidate keypoints, (2) zero-shot adaptive retrieval depth (ZARD) that expands context when rankings are ambiguous, and contracts it when rankings are consistent, (3) multi-temperature answer generation with aggregation over candidates, and (4) entropy-based uncertainty estimation over the answer distribution to enable abstention on high-uncertainty queries. Extensive experiments on two expert-annotated ophthalmology benchmarks, Thyroid Eye Disease and Graves’ Disease, show that URGen improves accuracy by 9.29% and 6.19% over a naive RAG-based Meditron-7B baseline, while achieving 3.01% and 4.57% expected calibration error (ECE). URGen is LLM-agnostic, consistently improving diverse backbones (, LLaMA-3.1 and Mistral) and providing interpretable uncertainty indicators for safety-critical deployment.

Abstract:
Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal understanding capabilities, yet their inherent security vulnerabilities remain a pressing concern. Prior studies have indicated that enhancing chain-of-thought reasoning can inadvertently facilitate jailbreaks; however, existing attempts often rely on image masking, which only modestly extends reasoning depth and performs poorly against advanced commercial closed-source models. To bridge this gap, we propose an effective jailbreak method, Text–Image Segmentation and Reassembly (TISR). TISR disperses harmful semantic fragments across the visual modality while employing text-based role-playing to guide their reassembly. This strategic organization conceals surface-level toxicity while preserving malicious intent, thereby extending LVLM reasoning chains and enabling successful jailbreaks. Our experiments show that LVLMs, leveraging their visual understanding and reasoning abilities, can reconstruct complete harmful semantics while failing to uphold safety constraints. Extensive evaluations across massive LVLMs demonstrate that the proposed TISR achieves state-of-the-art attack performance, surpassing prior multi-modal jailbreak methods. Specifically, it attains average Attack Success Rates (ASR) exceeding 90% across three benchmark datasets on four leading commercial LVLMs.

Abstract:
Large Language Models (LLMs) integrated into complex ecosystems (e.g., code agents) face critical safety threats from Indirect Prompt Injection (IPI) attacks, where malicious instructions are embedded in retrieved data. Existing defenses face an intractable safety-utility trade-off: sanitization-based guardrails incur high latency, while detection-based guardrails suffer from severe over-refusal. We first demonstrate, both theoretically and empirically, that LLMs can intrinsically separate instructions from data. Inspired by this insight, we propose AEGIS (Adaptive Ensemble Guard for Injection Shielding), a training-free defense that extracts instruction-sensitive projectors to identify malicious instructions and leverages a Unified Multi-Layer Consensus that aggregates topologically distinct signals across the network depth. Evaluations on eight IPI attacks and ten baselines show that AEGIS achieves state-of-the-art detection accuracy while reducing the false positive rate on benign tasks to below 1%. Code is available at [Github](https://github.com/xaddwell/AEGIS).

Abstract:
Current aligned language models exhibit a dual failure mode we term the Evasive Servant: they sycophantically validate flawed user beliefs while deflecting responsibility with boilerplate disclaimers. We propose the Dignified Peer framework, which targets servility through anti-sycophancy and trustworthiness, and evasiveness through empathy and creativity. Operationalizing this framework requires addressing three concrete gaps: lack of fine-grained per-dimension supervision, objective imbalance during multi-objective optimization, and confound-prone evaluation. We introduce the PersonaKnob dataset, which encodes a compositional partial-order structure of multi-persona preferences, and pair it with Lag-DPO, a tolerance-constrained Lagrangian DPO algorithm that dynamically rebalances dimensions to mitigate collapse. For evaluation, we adapt the Many-Facet Rasch Model (MFRM) to disentangle model capability from judge severity, rubric difficulty, and question complexity, and we validate MFRM scores against human pairwise preferences (Spearman 𝜌 ∈ [0.67, 0.85] across dimensions). Across two open-source backbones, Lag-DPO improves all four dimensions in distribution and reduces sycophancy while preserving safe-refusal balance on out-of-distribution benchmarks; we report these results with explicit weight-sensitivity and uncertainty analyses rather than claim a fully resolved Dignified Peer.

Abstract:
Large language models deployed in specialized domains must improve in-domain performance without sacrificing general capabilities. Existing parameter-efficient fine-tuning methods are typically always on: their learned perturbations are applied to every input, which can degrade out-of-domain (OOD) performance. We propose Engram Adapter, a framework that repurposes pretraining-time conditional memory as a post-hoc adapter for frozen LLMs. It uses multi-channel matching over local n-gram patterns with explicit occupancy tracking as a lightweight selectivity prior, making residual injection more likely on in-domain inputs while a learned scalar gate suppresses incoherent OOD retrievals. We evaluate on Qwen3-4B and Qwen3-8B with AG-News and MedMCQA as adaptation tasks and OOD benchmarks spanning reasoning, translation, code generation, and legal reasoning. Engram Adapter improves in-domain accuracy while preserving 99.4% to 100.1% of average OOD performance; on LegalBench it slightly exceeds the frozen base model on average, whereas comparable always-on baselines degrade sharply. Mechanistic analyses show that although OOD activations are non-zero, gate and projection attenuation reduce residuals to approximately 0.08% of hidden-state norm, yielding small KL drift and negligible accuracy change. These results suggest conditional activation is a promising route toward modular, retention-preserving domain specialization over frozen backbones.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves vision-language models (VLMs) by optimizing outcome rewards derived from final answers. However, such outcome-only rewards do not tell the model which image regions justify an answer. For questions that require visual grounding, these rewards cannot distinguish responses supported by relevant visual evidence from those produced by language-prior shortcuts or lucky guesses. We introduce EASE (Evidence-Anchored Spatial Attention), which augments multimodal RLVR with visual-evidence process supervision. EASE converts annotated evidence regions into a smoothed visual-token target and uses it to guide response-to-image attention during RL training, but only on high-reward trajectories. The annotations are used solely as privileged training labels, while inference requires only the original image and question. Across Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3-VL-8B, EASE raises average scores over DAPO by 2.5 to 3.1 points on perception, hallucination, visual math, and multimodal reasoning benchmarks. Diagnostics and ablations show that EASE better aligns visual attention with annotated evidence regions. Code are available at https://github.com/Nrich-sunny/Attend-to-Evidence/.

Abstract:
When applying LLM agents to automatic program repair (APR), existing systems often produce Draft Patches —partially correct fixes that either fail to fully resolve bugs or overfit public test cases due to incomplete contextual reasoning and limited test coverage. To address this, we present , a general context-aware refinement agent that evolves existing APR agents by transforming Draft Patches into robust and correct fixes. Operating in a black-box, plug-and-play manner, augments diverse APR systems without model retraining or architectural changes. tackles three challenges: (1) resolving ambiguous issue context through structured contextualization, (2) improving patch quality via test-time scaling and diversified generation, and (3) consolidating partial fixes through LLM-driven refinement. We integrate into multiple APR systems to demonstrate its generality and system-agnostic design. On SWE-Bench Lite, achieves state-of-the-art performance, improving AutoCodeRover by 14.67% to 51.67% resolution. %cOn SWE-Bench Verified, it improves resolution rates by 12.2%, while yielding an average 14% gain across diverse APR systems. These results identify refinement as a missing yet essential layer in APR pipelines and highlight agent-level refinement as a practical path toward more correct and robust program repair systems. We also open source our code : https://github.com/AnvithPabba/Refine-Program-Repair .

Abstract:
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge, particularly for long-tail domains such as literary works. However, the critical step of document segmentation in RAG remains largely underexplored. Existing strategies typically either ignore semantics or overlook the complicated narrative structures of literary works, often resulting in chunks with fragmented plots and unclear references that hinder retrieval and generation performance. To address this, we propose LitSeg, a novel narrative-theory-guided segmentation framework. By employing multi-stage prompting, LitSeg explicitly extracts valid events, clarifies narrative structures, and locates turning points to inform segmentation. To alleviate the computational overhead of multi-stage inference with large-scale models, we further introduce LitSeg-Lite, a lightweight single-pass chunker fine-tuned on LitSeg-generated data via a two-stage training strategy, distilling the complex process into a single inference pass. Extensive experiments demonstrate that compared to baselines, our methods yield high-quality text chunks that are narratologically coherent and self-contained. This improved segmentation quality enhances retrieval accuracy and context relevance, and boosts downstream QA performance. Ablation studies validate the efficacy of narratological guidance and data distillation, and efficiency analysis shows that LitSeg-Lite matches the teacher at a substantially lower inference cost.

Abstract:
Large language model (LLM) agents increasingly rely on external skills, yet standard evaluations obscure whether retrieving these skills actually helps. Aggregate metrics often compare retrieved versus non-retrieved tasks, introducing severe selection bias and failing to isolate the true effect of skill use. To measure this actual-use capability—which we formalize as Skill Following (SF)—we introduce the Retrieval-Invoked Actual-Use Effect (RAE). RAE computes the same-task outcome difference between matched skill-enabled and skill-disabled executions, conditioned exclusively on tasks where the agent actively retrieved a skill. Evaluating 17 LLMs across coding and mathematical domains, we uncover a stark evaluation paradox: models frequently show positive aggregate retrieval lift but negative RAE. On MBPP+, multiple models that appear to benefit system-wide actually harm their own performance on the exact tasks where retrieval occurred. These findings demonstrate that aggregate averages can create a misleading illusion of tool-use proficiency, whereas RAE directly measures whether the retrieval-to-answer pipeline genuinely rescues more outcomes than it harms.

Abstract:
Expanding the vocabulary of conditional text-to-speech (TTS) models typically requires extensive fine-tuning with paired audio, creating a major bottleneck for low-resource and cross-lingual adaptation. To address this, we introduce SEAM, a zero-audio initialization method that constructs embeddings for unseen tokens from known subtokens. Unlike standard pooling strategies that ignore sequential structure, SEAM applies a spectral filter to preserve temporal phonetic order and uses closed-form ridge regression to align the new embeddings with the pretrained conditioning manifold. Across three languages and two TTS architectures, SEAM matches baseline performance using up to four times less training data, reducing sample complexity by 75% compared to random initialization and mean pooling. Our approach provides a lightweight, reproducible, and theoretically grounded solution for vocabulary expansion in low-resource speech synthesis.

Abstract:
Activation engineering enables precise control over Large Language Models (LLMs) without the computational cost of fine-tuning. However, existing methods deriving vectors from static activation differences are susceptible to high-dimensional noise and layer-wise semantic drift, often capturing spurious correlations rather than the target intent. To address this, we propose Global Evolutionary Refined Steering (GER-steer), a training-free framework that grounded in the geometric stability of the network’s representation evolution. GER-steer exploits this global signal to rectify raw steering vectors, effectively decoupling robust semantic intent from orthogonal artifacts. Extensive evaluations confirm that GER-steer consistently outperforms baselines, delivering superior efficacy and generalization without layer-specific tuning, establishing a universal solution for reliable model alignment.

Abstract:
Assessing the developmental suitability of children’s narratives is important for educational recommendation and developmental literacy research, yet such assessment typically relies on subjective and difficult-to-scale human judgment. This raises an important question: Can AI systems approximate human developmental judgments of children’s stories? To study this problem, we introduce CLARA, a cognitively grounded framework for developmental narrative understanding through structured annotation across cognitive (COG), language (LAN), and social-emotional (SEL) dimensions, together with a bilingual benchmark resource containing 1107 Chinese–English children’s stories with normalized silver developmental references and structured developmental annotations. We evaluate CLARA through benchmark comparison, component analysis, translated bilingual consistency analysis, and blinded human evaluation with educators. Experimental results show that structured developmental annotation achieves substantially stronger alignment with developmental references and human judgments than readability-based methods and direct prompting baselines. Overall, our findings suggest that AI systems can approximate certain aspects of human developmental judgment when guided by structured developmental annotation, while also highlighting the importance of interpretability and human oversight in educational NLP.

Abstract:
Extracting structured, machine-readable compliance criteria from regulatory documents remains an open challenge. Single-pass language models hallucinate structural elements, lose hierarchical relationships, and fail to resolve inter-document dependencies. We introduce RegReAct, a self-correcting multi-agent framework that decomposes regulatory information extraction into seven specialized stages, each with an Observe–Diagnose–Repair (ODR) loop that validates outputs against the source, correcting not only model hallucinations but also cross-reference errors in the regulations themselves. To ensure structural accuracy, RegReAct constructs a typed criterion graph; to ensure completeness, it resolves external dependencies by retrieving, summarizing, and embedding referenced legal content inline, producing self-contained outputs. Applying RegReAct to three EU Taxonomy Delegated Acts, we construct a dataset comprising 242 activities with over 4,800 hierarchical criteria, thresholds, and enriched source summaries. Evaluation against a GPT-4o single-pass baseline shows that RegReAct outperforms it across all structural and semantic metrics.

Abstract:
Theory of Mind (ToM) reasoning requires inferring agents’ beliefs from partial and asymmetric observations, which remains an open challenge for LLMs. Existing prompting-based approaches improve ToM reasoning through observable-event filtering or temporal belief chains, without explicitly modeling nested beliefs. We introduce RecToM, an inference-time framework for ToM reasoning that models nested beliefs via recursive perspective construction. RecToM constructs each character perspective from the preceding character perspective along the character chain specified by the question, reducing higher-order belief questions to actual-world questions within the final constructed perspective. We further provide a KD45 analysis showing that RecToM’s perspective construction induces a well-formed belief modality beyond simple event filtering. Experiments on ToM benchmarks, including Hi-ToM, Big-ToM, and FanToM, across multiple LLM backbones show that RecToM consistently outperforms recent advanced approaches, achieving state-of-the-art performance. Notably, RecToM reaches 100% accuracy on Hi-ToM with GPT-5.4 and Qwen3.5, a benchmark requiring higher-order ToM reasoning.

Abstract:
Role-based LLM multi-agent systems need adaptive role pools, yet adapting such systems is not merely a matter of prompt optimization: roles often carry structural obligations, including capability coverage, message compatibility, validation, final-answer aggregation, and parser-compatible output protocols. Existing systems either fix the role inventory and lose adaptivity, or allow unconstrained generation to induce role drift, removing structurally necessary roles and breaking answer contracts. We formulate this as contract-preserving role evolution, requiring every committed edit to preserve five structural contracts (capability, communication, validation, aggregation, output protocol). We instantiate this formulation in SERO, a S elf- E volving R ole O rchestration framework that evolves a typed role-card pool through credit-guided retrieval, a credit-ranked communication DAG with a protected terminal aggregator and conditional validator repair, and a contextual-bandit controller whose LLM-proposed edits are committed only when they preserve the contracts and improve task score. Experiments on real-world reasoning benchmarks across three LLM backbones confirm the value of contract-preserving role evolution.

Abstract:
The performance of large language models is tied to the quality and organization of their training data. However, existing data metrics emphasize surface-level properties, such as sequence length or model-based quality scores, and provide limited insight into how the internal organization of data relates to optimization. We study the structural fabric of training data, defined as the pattern of semantic dependencies within a sequence. We quantify this property using dependency density, a normalized measure of effective segment-level dependencies. Across data length, domain, and format, we find that dependency density varies with data configuration: code shows higher dependency density than natural language within long-context data, and instruction-tuning mappings show higher dependency density than raw intra-input modeling. We show that coherent structural fabric is associated with a lower gradient effective rank, indicating a more concentrated optimization geometry with reduced gradient interference. We observe that this low-rank optimization regime is associated with a more stable intrinsic dimension, preserving a compact representation manifold that may mitigate representation noise and support downstream performance. These results provide a geometric perspective on how data structure relates to LLM training, linking data configuration, optimization dynamics, and model behavior through the lens of structural fabric.

Abstract:
Structure-based drug design (SBDD) requires ligands that satisfy both 3D target affinity and 1D chemical validity. Existing controllable generation methods often rely on task-specific fine-tuning or externally imposed sampling-time guidance, adding cost and potentially conflicting with evolving 3D geometric constraints. We propose LiFT, a language-informed cross-modal framework built on Flow Matching for trend-guided 3D molecular generation across both de novo design and scaffold hopping. LiFT uses a “Sense-Evolve-Assemble” agent to generate target-aware SMILES as intermediate chemical conditions, from which a pre-trained chemical foundation model extracts continuous semantic priors. These priors are integrated into geometric generation through a lightweight semantic projector with zero-initialized adaptive normalization for stable cross-modal conditioning. We further introduce a Self-Conditioned Decoupled Router (SCDR), which modulates the velocity field according to intermediate structural states during ODE integration. Experiments on CrossDocked2020 show that LiFT achieves competitive distribution matching while improving medicinal chemistry metrics and maintaining competitive structural validity under task-steering settings without additional generator fine-tuning. Our results suggest that language-derived chemical priors provide effective trend-level guidance for 3D molecular generation. Code and released artifacts are available at https://github.com/kasurl/LiFT.

Abstract:
Determining whether a program terminates is a central problem in computer science. Turing’s Halting Problem established termination as undecidable, showing that no algorithm can universally determine termination for all programs and inputs. Hence, verification tools approximate termination, sometimes failing to prove or disprove; these tools rely on problem-specific architectures and are usually tied to particular programming languages. Recent advances in LLMs raise a natural question: To what extent can they reason about program termination? We evaluate frontier LLMs on a diverse set of C programs from the International Competition on Software Verification (SV-Comp) 2025. Our results show that GPT-5 and Claude Sonnet-4.5 achieve scores comparable to top-ranked verification tools (with test-time scaling). However, while models often correctly infer whether programs terminate, they frequently fail to construct a witness as formal proof, revealing a gap between semantic recognition and symbolic proof generation. Performance further degrades as code length increases. Beyond witness automaton graphs, we introduce a divergence-precondition formulation that characterizes non-termination conditions as logical constraints. We hope these findings motivate future research on real-world termination benchmarks, neuro-symbolic approaches that combine LLMs with symbolic verification methods, and, more broadly, LLM reasoning on other undecidable problems.

Abstract:
Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now approach or even surpass closed-source frontier models in aggregate accuracy. However, aggregate accuracy averages over many different multi-turn situations and obscures whether progress is balanced across them, masking failure redistribution. We propose an action-class-oriented diagnostic framework that decomposes failures into two orthogonal dimensions: capability and calibration. The framework operates over a four-class action space (tool_call/ask/refuse/confirm) and introduces a self-revealing upper bound, Acc ≤ GAR (Gold Action Recall), whose two violation directions separately localize action-class miscalibration and sub-action failures. We validate it on an extensive panel of tool-calling models that includes both open-weight families and closed-source frontier systems, across multiple multi-turn tool-calling benchmarks for cross-benchmark robustness. Across our panel the bottleneck is calibration, not capability: every model can produce all four classes, but which one it fires in which situation is highly model-specific. On no-tool-call scenarios, a top-ranked family’s GAR is roughly one-quarter that of size-matched peers, a 64.5 percentage point (pp) gap, indicating its ranking reflects benchmark-distribution fitting rather than genuine in-context calibration. We further show the calibration profile can be reshaped through context manipulation alone, with the reshaping direction strongly model-specific. We recommend that multi-turn tool-calling evaluations report per-category (GAR, tool-call-emit rate, Acc) triples, exposing the calibration structure aggregate accuracy hides. Our framework provides a falsifiable, reproducible diagnostic foundation for evaluation protocol design, training objective alignment, and inference-time calibration research.

Abstract:
Autoformalization aims to produce formal statements that compile and faithfully preserve the intended meaning of informal mathematics. Yet standard single-output evaluation collapses this many-to-many structure into a single prediction. For downstream proving, this granularity is too coarse: a formal statement is not merely a faithful translation endpoint, but also a prover-facing interface whose structure can alter proof search under a fixed budget. We therefore recast autoformalization as budgeted test-time search: FormalEvolve maintains a compilation-feasible archive for reuse and returns a deduplicated, semantically accepted repertoire for evaluation and downstream proving. It expands the archive with LLM-driven mutation, crossover, bounded patch repair, and symbolic abstract syntax tree (AST) rewrites for structural diversity. Under a generator-call budget of T=100 with a fixed LLM semantic judge, FormalEvolve reaches SH@100 of 58.0% on CombiBench and 84.9% on ProofNet, improving over all no-archive controls while reducing the cross-problem concentration of semantic successes. Under a fixed B=64 prover budget, these repertoires improve theorem-complete proving over the matched no-archive control. Additional stronger-base statement-generation experiments show that archive-search gains persist with stronger seed and repair models.

Abstract:
Hallucination remains a critical challenge in large language models (LLMs), where early factual errors compound through autoregressive generation in a snowballing effect that neither post-hoc correction nor weight-level intervention can effectively preempt. We propose DescaPE (DEcoding Signal Control Against Path Error-snowballing), a decoding framework that leverages internal model signals to suppress hallucination-prone trajectories at inference time. Through sliding-window MLP ablation, we identify a factual-salient layer span within LLMs whose derived signal is selectively elevated for factual tokens and exhibits anomalous spikes at hallucination-prone steps. We train a lightweight probe to approximate this signal from a single forward pass and integrate it into candidate scoring to penalize high-risk continuations while rewarding factually grounded ones. Experiments across five factuality benchmarks on three LLMs demonstrate that DescaPE achieves factuality improvements over decoding-time baselines in multiple settings, while incurring only 1.10x latency overhead in our efficiency evaluation. Our code is available at https://github.com/hayeonggg/DESCAPE.

Abstract:
Existing multiple-choice evaluations use small option sets and thus conflate several sources of failure when candidate spaces grow. We provide a controlled factorial decomposition of large-choice degradation across four axes: whether a question requires comparing its options, candidate cardinality (up to 1,000), spatial layout, and gold-answer position. Across six reasoning LLMs, comparative questions degrade far more than questions solvable without inspecting the options; dispersion amplifies this gap; and accuracy often collapses at late-middle and terminal positions rather than recovering in the classic lost-in-the-middle U-shape. We call this asymmetric, structure-dependent failure mode Lost in Choice. A replication on 500 real ToolBench tool-selection trajectories per condition reproduces the same hardest position on two additional model-task evaluations. Finally, an audit of reasoning traces yields a taxonomy of the heuristics and shortcuts models adopt as candidate spaces expand.

Abstract:
Despite the significant advancements of Large Language Models (LLMs), their factuality remains a critical challenge, creating a growing need for more nuanced factuality verification. Existing factuality verification methods do not capture graded judgments, even though factuality is better understood as a spectrum rather than a binary of right and wrong. To bridge this gap, we focus on graded factuality verification and propose AEScorer, an agentic evidence-grounded framework with two stages: agentic evidence acquisition and graded scoring. AEScorer first gathers and refines external evidence through agentic search, and then predicts a scalar factuality score to distinguish nuanced differences in factual correctness. We further construct GradedVeriBench, a benchmark for graded factuality verification spanning both general and multi-hop question answering. Experimental results on GradedVeriBench show that AEScorer substantially outperforms existing methods across both settings, demonstrating the value of coupling targeted evidence acquisition with graded scoring.

Abstract:
Robotic manipulation often requires actingon information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the current observation largely determines the next action. Existing robotic memory benchmarks expose this gap, but they still rely mainly on final task success and there-fore conflate forgetting with manipulation failure. We present MEMOBench, a benchmark for process level memory evaluation in robotic manipulation. MEMOBench includes 30 history dependent tasks, 1,500 expert demonstrations, and 4,200 executable checkpoint instances from 84 templates. Each checkpointpairs coarse to fine language with a simulator predicate and labels one memory operation:Storage, Update, or Compression. These annotations define Memory Storage Rate, MemoryUpdate Rate, and Memory Compression Rate,which measure memory fidelity alongside tasksuccess. Across standard and memory augmented VLA policies, the strongest memory module baseline reaches only 31.9%o average success rate, and high storage often coexists with weak update and compression. Checkpoint language also supervises semantic, contrastive, and framewise memory alignment objectives, yielding modest gains across differentmemory operations. MEMOBench provides a diagnostic evaluation suite and training supervision for memory grounded robotic policies. The project page is available at https://github.com/Collab-Gen/MEMoBench.

Abstract:
Personalized text generation for authors and literary writing is essential for applications such as adaptive writing assistants, creative support tools, and computational literary analysis. However, existing approaches to author modeling and personalization often represent writing behavior as independent labels, requiring large-scale corpus collection or fine-tuning for each author or stylistic category. Such formulations are costly, difficult to interpret, and poorly suited for generalizing across authors. Inspired by the Big Five theory of personality, we propose LiteraryBigFive, a framework that reframes authorial writing characteristics as coordinates within a unified and interpretable space. In this space, we derive each interpretable axis (e.g., Classicism, Emotionality) from activation-space contrasts between author-written and neutral passages, yielding distinct stylistic dimensions that allow texts or authors to be positioned within a five-dimensional system. Beyond localizing different authors, we further introduce an interpretable steering mechanism, which adaptively guides text generation toward target coordinates to perform author-personalized writing. Experimental results show that LiteraryBigFive improves authorial expressiveness while preserving semantic fidelity. The derived author per-axis scores strongly correlate with real-world literary consensus, offering transparent and interpretable explanations of author-specific generation behavior: https://github.com/Znull-1220/LiteraryBigFive.

Abstract:
Multi-hop Retrieval-Augmented Generation can fail even when relevant evidence has been retrieved. The remaining diffculty lies in how evidence is connected across reasoning steps:an intermediate answer may be unsupported, at-tached to the wrong anchor, or propagated into a later step without valid evidence. We study this failure as Reasoning Chain Hallucination(RCH), where a fuent multi-hop reasoning pro-cess contains one or more ungrounded inter-mediate steps. To address RCH, we formulate Retrieve–Verify–Generate (R-V-G) for multi-hop RAG: retrieved evidence should support an explicitly verifed reasoning structure before it is allowed to guide generation. GROUNDRAG is an evidence-verifed framework that imple-ments this requirement. GroundRAG frst ac-quires the evidence needed to instantiate inter-mediate reasoning variables, then represents the reasoning process as a typed dependency chain.Each step is checked against retrieved passages using explicit grounding criteria; unresolved steps are repaired with queries formed from ver-ifed intermediate answers, while insuffciently grounded chains are discarded before genera-tion. This design separates evidence acquisi-tion from the decision of whether a reasoning chain is safe to use. On HotpotQA, 2WikiMul-tiHopQA, and MuSiQue, GroundRAG reaches 73.8%, 69.2%, and 44.3% LLM-judged accu-racy, respectively, outperforming the strongest baseline in our evaluation.

Abstract:
Textual user profiling is an effective paradigm for LLM-based recommendation, but existing methods usually treat profile generation as a one-shot summarization problem. This limitation becomes more severe in long-history recommendation, where early preference signals may be overcompressed and recent behaviors may be overemphasized. We propose LUPE, a Long-history User Profile Evolution framework for LLM-based recommendation. Instead of generating a static profile from the full history, LUPE maintains a persistent textual profile and updates it progressively over temporally ordered interaction chunks. This allows the model to preserve long-term preferences while incorporating new behavioral evidence. To learn how profiles should evolve, LUPE optimizes a profile evolution policy via reinforcement learning using feedback from a frozen downstream LLM-based recommender. This allows the model to directly align profile updates with recommendation utility rather than relying on heuristic or supervised profile targets. Experiments on Yelp, Amazon CDs and Vinyl, and Amazon Books show that LUPE consistently outperforms strong baselines on both rating prediction and ranking tasks. Further analyses show that explicit profile evolution is especially helpful for users with long and heterogeneous interaction histories. The code repository: https://github.com/LongHistoryProfile/long_history_profile.

Abstract:
The self-attention mechanism used in transformer language models implements a form of associative, cue-based memory retrieval, which has been argued to resemble human sentence processing. However, while it is well established that surprisal estimates from transformers are predictive of human comprehension difficulty, it has been less clear how these models’ internal memory operations might relate to human sentence processing. We propose to connect the two by measuring transformers’ associative memory updates, which are calculated as the difference between each word’s predicted and actual representation within self-attention. We compute memory updates from three types of language models: (1) standard transformers; (2) CBR-RNN, a cognitively motivated model using a single self-attention head; and (3) CBR-RNN-M, a novel architecture that replaces the self-attention in CBR-RNN with a bounded-size memory cell. Associative memory updates are found to predict naturalistic human reading times over and above surprisal baselines on several psycholinguistic corpora, with evidence for more consistent effects from CBR-RNN and CBR-RNN-M than from standard transformers.

Abstract:
KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that supports it. We test this assumption for large reasoning models and show that it can fail: under compression, answer-level cues and evidential support can remain differentially recoverable. We study this failure with a controlled fixed-trace replay protocol, which holds reasoning content fixed and isolates whether compression preserves usable information from an already available trace. We evaluate ten token-eviction KV compression methods and one quantization method on three models across mathematical reasoning, scientific QA, clinical calculation, and long-context retrieval. We measure final accuracy, answer-chain consistency, and perturbation faithfulness. Across tasks, token-eviction methods can preserve competitive final-answer accuracy while substantially degrading chain support or perturbation faithfulness. We call this the answer-evidence gap. A coverage-preserving quantization control is substantially less affected, suggesting that the failure is tied less to KV memory reduction itself than to losing access to parts of the reasoning trace.

Abstract:
Aspect Sentiment Triplet Extraction (ASTE) is a fine-grained task in Aspect-Based Sentiment Analysis (ABSA), aiming to extract triplets of aspect terms, their corresponding opinion terms, and sentiment orientation. Recent studies have employed semantic and syntactic features to enhance extraction performance. However, as the extraction of both syntactic and semantic information is not task-specific, such information does not target aspect-opinion pairs within sentences, thereby potentially introducing noise unrelated to the task. To address this issue, we propose a Syntactic-Guided Semantic Attention Matrix Enhancement (SG-SAME) framework. First, we use a multi-view masking mechanism and a biaffine-based gating module to derive a task-aligned shared matrix from the existing semantic and syntactic matrices. Second, the shared matrix is injected into the attention score matrix of LLM layers to enhance the model’s representation learning. Finally, we propose a chain-of-thought (CoT) framework with negative candidate injection to steer the model toward autonomous reasoning and error self-rectification. Experimental results on multiple benchmark datasets demonstrate that our method outperforms existing state-of-the-art approaches in ASTE tasks. Code will be available.

Abstract:
Memory-backed agents need provenance that can survive leaked or migrated snapshots, where logs, visible outputs, and trusted metadata may be absent. We propose MEMMARK, a state-evolution attribution watermark that embeds an owner-controlled signal into latent memory-write decisions. At each internal LLM call, MEMMARK samples among admissible candidates using keyed, distribution-preserving selection, and records cryptographic commitments with signed session anchors and reveal evidence. This makes attribution depend on reproducible backend behavior rather than mutable provenance fields. Across A-Mem and Graphiti on LOCOMO, with three LLM backbones, MEMMARK preserves memory utility: Overall F1 retains 99.6 percent of the unwatermarked baseline, while BLEU-1 changes by +0.2 percent. It also provides usable carrier capacity, with 1.16, 1.14, and 1.26 bits of mean entropy for update-target, link-target, and semantic-realization decisions. In the snapshot-only R3 setting, MEMMARK recovers the full 40-bit payload from final snapshots, while wrong-key verification remains near chance. Under nine memory-lifecycle attacks, verification distinguishes tampering, evidence deletion, and partial payload recovery. These results show that robust snapshot-only attribution is feasible for long-term agent memory without surviving traces, trusted metadata, or utility degradation.

Abstract:
Biomedical retrieval-augmented generation (RAG) produces long-form answers that mix supported, under-evidenced, and contradicted claims, demanding fine-grained verification. We propose MedRAGChecker, a claim-level diagnostic framework that combines textual natural language inference (NLI) with a biomedical knowledge-graph (KG) compatibility signal. Its asymmetric selective-gating rule preserves the textual prediction for most claims and applies KG adjustment only to confident textual entailments assigned low plausibility by the KG back-end. We distill GPT-4.1 claim decompositions and NLI pseudo-labels into compact student models and evaluate three biomedical generators on PubMedQA, MedQuAD, LiveQA, and MedRedQA. The student-ensemble results show that selective BioPortal gating increases macro-averaged faithfulness from 60.8% to 64.4% and reduces the contradiction rate from 19.9% to 15.0%. On 172 annotations of 86 claims whose decisions change after fusion, agreement with human evidence judgments increases from 63.4% to 69.8%. A comparison with DRKG shows that fusion behavior varies substantially with KG coverage, entity linking, score construction, and calibration; the comparison therefore characterizes back-end effects rather than isolating a causal effect of coverage.

Abstract:
Safety-aligned large language models (LLMs) face a dilemma between helpfulness and harmlessness, where LLMs are expected to refuse harmful requests while remaining helpful by explaining their decisions. In this work, we observe a phenomenon caused by this dilemma. When asked to explain a refusal triggered by a harmful query, LLMs disclose exploitable elements in their refusal explanations. These elements allow attackers to induce the generation of harmful content. We formalize this vulnerability as Induced Refusal Explication (IRE). To identify and exploit IRE, we propose Explication of Refusal (EoR), a black-box and multi-turn jailbreak method with a dual-agent framework. EoR induces and analyzes refusal explanations from victim models and then generates follow-up queries to jailbreak them. We evaluate EoR across nine victim models. EoR achieves an average attack success rate (ASR) of 37.05 within five turns. When we apply EoR to the queries that PAIR fails to jailbreak, EoR reaches an average ASR of 66.0 across the nine victim models, which demonstrates that exploiting refusal explanations is complementary to existing automated jailbreak methods. We release our data and code at https://anonymous.4open.science/r/EoR-6234.

Abstract:
Retrieval-Augmented Generation (RAG) mitigates hallucinations by grounding generation in retrieved evidence, yet failures persist when models underutilize the retrieved context. While prior work attributes this to conflicts between external context and internal parametric knowledge encoded in Feed-Forward Networks (FFNs), our empirical analyses reveal that FFN-derived signals are unstable and insufficient for reliable diagnosis across diverse contexts. We therefore propose the Attention Escape Score (AES), a training-free diagnostic framework that provides a mechanistic lens on RAG hallucinations via dynamic attention routing failures. We models the failures as weakened Semantic Anchoring, increased Contextual Distraction, and heightened Generation Inertia, moving beyond static component-level analyses. Extensive experiments show that AES significantly outperforms state-of-the-art baselines with robust cross-model generalization. Furthermore, we introduce Pruning Escape Routes (PER), a lightweight inference-time intervention that suppresses hallucinations by attenuating identified escape pathways, achieving improved faithfulness with negligible degradation to complex reasoning and general capabilities.

Abstract:
Emerging text-to-image (T2I) models perpetuate biases, creating stereotypical portrayals of disability. This study investigates whether an agent-based, human-in-loop, AI workflow, that iteratively refines user-provided queries, can produce fairer representations of disability compared to standard models like Stable Diffusion, DALL·E-3, and MidJourney. Sixteen raters (eight with disabilities and eight without) evaluated images generated from models, prompted with textual inputs that targeted professional, medical, or social contexts. In an initial survey, raters were asked if an image depicted a disability, the type of disability if present, and if the setting aligned with the given text prompt to the model. Qualitative comments were collected. A post-survey asked raters to choose which image of a pair (standard vs. agent-based workflow) more accurately represented disability. Results revealed that standard models disproportionately produced images featuring physical disabilities or wheelchairs ( p < 0.001 ), consistent with broader dataset biases. In contrast, the agent-based model’s images were deemed more representative in approximately two-thirds of comparisons ( p < 0.0001 ), especially among raters with disabilities. Qualitative analyses underscored the need for nuanced, contextually rich portrayals.These findings suggest disability bias and that targeted debiasing can help mitigate stereotypes in generative AI, fostering more equitable outputs.

Abstract:
Multi-agent LLM systems often coordinate by compressing an upstream interaction into a handoff artifact that downstream agents treat as shared state. We show that this handoff step is a structural source of privacy leakage: summaries preferentially preserve operational facts while weakening the boundary metadata that governs how those facts may be used—a failure mode we call summary collapse . On a controlled multi-agent coordination testbed we measure marker survival with a human-validated judge ( 𝜅 = 0.74 ), where 𝜎 b = 1 means every boundary marker survives verbatim and 𝜎 b = 0 means all are lost. Boundary-marker and operational-fact survival are nearly uncorrelated at the handoff level on both GPT-5-mini and DeepSeek-R1-32B (Pearson r near zero): uncompressed free-text handoffs preserve boundaries at 𝜎 b ≈ 0.80 , whereas a 25-word budget drops 𝜎 b to ≈0.57 while operational-fact survival stays near ceiling. Controlled downstream tests reveal that protection depends on boundary explicitness : vague languages leak in 73% of GPT and 50% of DeepSeek cases, while explicit constraints reduce leakage to under 15% across all three tested models. A no-handoff single-agent control further shows the failure is not reducible to multi-agent topology as direct full-marker access still leaks more often than the operationalized handoff. Prompt-only mitigation and exact-string redaction only partially address the problem, while a gold-derived audience allowlist nearly eliminates leakage across models, showing that correctly identifying audience boundaries is the key factor.

Abstract:
Stereotypes are subjective constructs that are mediated by the cultural context in which they are viewed. Successfully modeling them thus also requires cultural competence. When using large language models (LLMs) for stereotype-related tasks, such as their detection, it is therefore crucial that models can access relevant cultural knowledge and adjust responses accordingly. Since LLMs have shown disparities in what cultural knowledge is available to them, their capacity regarding stereotypes may also be disparate across languages and cultural contexts (e.g., for non-English languages or for non-US contexts). In this paper, we systematically investigate the effectiveness of LLMs for culture-specific stereotype detection across three factors: (1) the prompting language, (2) the instruction tuning language, and (3) the fine-tuning dataset size. We conduct experiments for US, Dutch, and Chinese cultural contexts. Our results suggest that LLMs do not reliably account for cultural contexts and overpredict the presence of stereotypes. Non-English prompts seem to improve consideration of cultural context and thus lead to more effective predictions. Moreover, we find that additional instruction tuning reduces false-positive rates and that training data size moderates detection effectiveness non-monotonically.

Abstract:
As Large Language Models (LLMs) become central to how we access and share information, they play an increasingly powerful role in shaping global knowledge. However, as these models evolve, their outputs risk converging into a generative monoculture, where the diversity of perspectives they represent narrows over time. Studies at the model level often fail to pinpoint which specific topics or viewpoints are being marginalised or amplified in this process. In this paper, we introduce a method to measure shifts in topic saliency across model families, tracking what gains or loses prominence during post-training. Applying this approach to a case study of climate change discourse, we demonstrate how homogenisation affects the representation of diverse solutions across different models. We also test interventions to counter this trend, showing that specialised models can help preserve a broader range of perspectives. This underscores the importance of monitoring topic saliency to diagnose the risks of monoculture and to ensure AI systems reflect a pluralism of ideas.

Abstract:
Large reasoning language models (LRLMs) achieve strong performance on complex reasoning tasks by generating long chain-of-thought trajectories, but such long-form reasoning also increases token consumption and latency. Existing inference-time methods that do not fine-tune the target LRLM optimize long reasoning through budgeting, early exit, uncertainty control, or reflection suppression, but they often treat reflection as a local signal rather than as a factor shaping the reasoning trajectory. In this paper, we identify unregulated reflection as a source of inefficient reasoning and characterize two failure modes: Trajectory Drift, where intermediate reflection opens weakly related branches that restart, reorient, or detour from the current path, and Verification Saturation, where answer-adjacent reflection repeatedly checks or restates an already-emerged provisional answer. To address these issues, we propose Regulated Reflection Decoding (RRD), an inference-time framework that requires no fine-tuning or parameter updates to the target LRLM and treats reflection events as online control points. RRD combines Trajectory Drift Suppression (TDS), which guides intermediate reasoning toward validated continuation branches, and Redundant Verification Suppression (RVS), which suppresses redundant verification when the provisional answer is sufficiently reliable. Experiments on multiple reasoning benchmarks show that RRD reduces reasoning length while preserving or improving accuracy, yielding a stronger accuracy-efficiency trade-off than direct inference and representative inference-time baselines.

Abstract:
Multilingual translation suffers from computational redundancy, especially when translating into multiple languages simultaneously. In addition, translation quality can suffer for low-resource languages. To address this, we introduce Transformer Encoder Tree (TET), a hierarchical, non-autoregressive encoder-only architecture trained with Connectionist Temporal Classification (CTC) for multilingual translation. TET shares intermediate representations among linguistically similar target languages, improving accuracy on low-resource languages while reducing computational redundancy and enabling the generation of all target languages in a single forward pass. TET eliminates the sequential bottleneck of autoregressive models and supports fully parallel decoding of all tokens across all target languages. Compared to a naive one-to-many multilingual design, TET reduces the total parameter count by 66% and lowers inference computation by 60%. In speech translation, combining TET with a non-autoregressive speech recognition backbone (Wav2Vec2) shows competitive translation quality compared to autoregressive systems while speeding up inference by approximately 7-14 × .

Abstract:
Latent chain-of-thought compresses reasoning by replacing visible reasoning traces with continuous hidden-state recurrence, but existing formulations are difficult to optimize with standard on-policy reinforcement learning (RL) and hard to interpret causally. Our key insight is that a single pair of explicit boundary tokens can address both issues at once: discrete entry and exit anchors make the latent block compatible with standard on-policy RL, and the same anchors offer a natural foothold for mechanistic analysis. Motivated by this, we propose SWITCH, a switchable latent reasoning framework. The model emits ‘‘ to enter latent mode and ‘‘ to exit. Because the boundaries are ordinary discrete tokens, the GRPO policy ratio is well-defined at every decision point. The same anchors also expose the latent steps to direct probing and causal intervention. We train the model with a visible-to-latent curriculum and a Switch-GRPO objective that propagates gradients through recurrent latent computation. SWITCH consistently outperforms prior hidden-state-recurrence latent reasoning approaches at similar scale. Mechanistic analysis through the boundary tokens further reveals three findings: (i) ‘‘ is a sharply localised, learned switching policy rather than a stylistic artefact; (ii) the latent step it opens performs problem-specific, causally important computation rather than acting as an inert placeholder; and (iii) that computation is concentrated at a single hidden-state transition on entry. Together, these results show that hidden-state-recurrence latent reasoning is both RL-trainable and open to direct mechanistic analysis, including of how on-policy RL itself improves the model from the inside.

Abstract:
Compact instruction-following rerankers are attractive for deployment, but conventional distillation pipelines typically train students by offline imitation of teacher outputs on a fixed set of examples, constraining supervision to the teacher’s observed ranking space. We revisit reranker distillation through the lens of reinforcement learning. We propose a two-stage framework combining off-policy teacher optimization with on-policy student distillation. In Stage 1, a 4B teacher reranker is strengthened with off-policy GRPO using LLM-judge feedback on 88K instruction-following examples. In Stage 2, a compact 1B student samples rankings from its own policy and receives soft teacher-derived rewards on those rankings, coupling student exploration with knowledge transfer. Our strongest gains appear under distribution shift. On MAIR-11, the original 11-subset, 869-query evaluation, the proposed student reaches 0.7670 nDCG@6, outperforming offline listwise KD by +4.6 points. Controlled comparisons against offline pairwise RankNet KD and on-policy GKD show that neither changing the offline distillation objective nor moving teacher-distribution matching on-policy reproduces the performance of reward-based on-policy distillation over student-sampled rankings. The advantage persists on MAIR-Full: across all 126 tasks and 9,356 queries, the proposed method obtains the highest task-macro point estimates among the evaluated distillation variants, reaching 0.6808 nDCG@6 and 0.7865 MRR@6. It also exceeds two released 7B RL-trained rerankers on the comparable MAIR-11 evaluation, while the same Stage 2 training procedure consistently improves three architecturally distinct alternative student backbones. On the 9,861-query validation benchmark, the resulting 1B reranker achieves 0.7624 nDCG@6 while providing a favorable quality-efficiency tradeoff relative to larger alternatives.

Abstract:
Safety alignment for large language models relies heavily on preference data, yet existing pipelines often train on large, redundant datasets and treat each preference pair independently through scalar quality scores. This sample-centric formulation ignores two key properties of preference data: (1) preference pairs define directional alignment signals in representation space rather than isolated samples, and (2) multi-dataset safety corpora contain both shared alignment structure and dataset-specific residual risks. We propose DOG-DPO, a training-free geometric data selection framework for efficient safety alignment. DOG-DPO represents each preference pair as a directional vector in model representation space, decomposes multi-dataset preference geometry into a shared anchor subspace and dataset-specific residual subspaces, and selects subsets by maximizing a determinant-based diversity objective that encourages broad, non-redundant coverage of alignment directions. Unlike prior methods, DOG-DPO requires no teacher LLM, auxiliary DPO training run, or iterative validation. Across six safety benchmarks and multiple LLM backbones, DOG-DPO achieves a strong utility–robustness trade-off using only 11% of the preference pairs, recovering most of the benefits of full-data alignment while substantially reducing computation cost. Our results suggest that effective alignment depends not only on data scale, but also on exploiting the structured geometry of heterogeneous preference data.

Abstract:
Unlearning in large reasoning models (LRMs) requires removing forget-set information from both the reasoning trace and the final answer. Existing approaches achieve this objective at the cost of reasoning collapse, where the model fails to produce a coherent chain-of-thought (CoT) and does not properly terminate its reasoning process. To address this challenge, we propose REDaCT, an inference-time unlearning framework for LRMs. REDaCT consists of two key components: (1) a binary leak detector that identifies sentences in the reasoning trace that leak forget-set information, and (2) an embedding optimization module that learns a corrective embedding offline, such that when injected at the first detected leak during inference, it guides subsequent reasoning towards sentences that do not leak forget-set information. Experimental results show that REDaCT completely avoids reasoning collapse, substantially improving reasoning preservation compared to state-of-the-art unlearning methods while maintaining perfect answer forgetting and near-perfect CoT forgetting. Our code is available at: https://github.com/jmaccoby/redact.

Abstract:
Understanding how opinions on different issues evolve together is essential for modeling collective intelligence, yet this remains underexplored due to the absence of standardized benchmarks. We introduce the concept of an opinion graph, where nodes represent social opinions on real-world events (e.g., presidential elections, stock predictions) and edges capture pairwise relationships between them. Building on this, we present OpinionBench, a continuously evolving benchmark derived from multiple live prediction markets, designed to evaluate whether large language models (LLMs) can uncover the latent structure within evolving social opinions. OpinionBench employs an scalable and automated graph construction and labeling pipeline, which assigns edge labels using time-series co-movement, semantic similarity, and metadata, followed by human validation. Experiments show that (1) LLMs consistently outperform baselines in identifying opinion correlations across domains, and (2) LLMs can recover the underlying graph structure through edge prediction. OpinionBench provides a challenging testbed for assessing LLMs’ ability to capture complex patterns of social opinion co-evolution.

Abstract:
Machine unlearning aims to remove the influence of designated training data while preserving model utility, but its behavior on tabular data remains underexplored. This gap is important because tabular prediction is widely used in high-stakes domains and is increasingly adapted to language models through record serialization and schema-aware prompting. We identify a key challenge that distinguishes tabular unlearning from unlearning in free-form text or other modalities: schema-induced forget–retain overlap. In serialized tabular data, records share fixed column-name/value slots, similar attribute ranges, and common output spaces. Consequently, a forget row may have nearby retain rows that rely on the same high-signal attributes, causing retain preservation to oppose the update required for forgetting. Motivated by this failure mode, we propose Conflict-Aware Unlearning (CAU), a schema-aware approach that reduces forget–retain interference by relaxing preservation constraints on retained rows that most conflict with the forget set. Across sample-level and feature-level unlearning on clinical and non-medical tabular tasks, more closely matches a retraining oracle while maintaining predictive utility and retain-region behavior. Our results show that reliable tabular LLM unlearning depends not only on the forgetting objective, but also on how retain constraints are constructed.

Abstract:
Entity-level unlearning is usually evaluated by what a model says: whether it stops naming the target, refuses a query, or shifts a Truth Ratio distribution. These output-level tests, however, do not show whether a subject’s internal representation has been attenuated. We introduce the Entity Representation Unlearning Framework (ERUF), a representation-aware framework that mines subject-specific activation signatures, suppresses the corresponding activation direction, and distills the behavior into LoRA parameters. Among evaluated baselines, ERUF is the only method that jointly achieves surface level suppression, internal attenuation, and utility preservation. On TOFU forget10 , ERUF achieves FQ =0.99 and MU =0.62 , matching reported oracle utility while approaching oracle forget quality. Across most standard foundation-model settings, ERUF maintains low leakage and low internal target activation (SMR =0.00 –1.10%, EL10 <0.06 , utility drift <3% ). On Llama-3.1-8B, adversarial entity recovery falls from 63.89% to 20.15%, while name-agnostic recovery decreases by 72.7–77.4%. Joint surface/internal diagnostics further reveal scale-dependent behavior in reasoning-prior models that surface metrics alone would miss. We interpret these results as operational evidence of representation-level attenuation, not as a formal guarantee of irreversible deletion. Code is available in the repository.

Abstract:
Large language models (LLMs) demonstrate strong performance on standard content moderation benchmarks. However, these benchmarks often aggregate multiple moderation criteria into a single label, making it unclear whether models can disentangle them and reliably apply each criterion when making decisions. To study whether LLMs exhibit criterion-conditioned behaviour, we introduce Diagnostic Evaluation of COntent (DECO), a criterion-independent factorisation of content that enables controlled, criterion-level evaluation. We also introduce pairwise evaluation to compare model outputs across different criteria for the same input. Across four moderation datasets and four LLMs, we find that strong benchmark performance can hide substantial failures at the criterion level. Models struggle most when correct decisions depend not on overall harmfulness, but on the specific aspect of the content that the criterion requires them to assess. Our results highlight a key limitation of current content moderation benchmarks: strong performance on aggregated labels does not provide sufficient evidence that LLMs can reliably evaluate content with respect to individual moderation criteria. These findings call for the development of evaluation methods that explicitly measure criterion-conditioned behaviour.

Abstract:
Large language models can produce correct final answers with reasoning traces that are incomplete, inconsistent, or weakly coupled to the answer. We study whether training can increase this coupling in structured reasoning domains. We introduce Counterfactual Sensitivity Regularization (CSR), a training objective that creates verifier-approved counterfactual edits to reasoning traces, such as changing an arithmetic operator or a logical relation, and penalizes models when the answer distribution remains unchanged after an edit that should change the computation. We evaluate this behavior with Counterfactual Outcome Sensitivity (COS), the fraction of originally correct examples on which a validity-breaking trace edit changes the final answer. Because COS is aligned with the CSR objective by construction, we report it jointly with counterfactual target accuracy, target coverage, and null-edit preservation, and alongside evidence collected outside the training loop: natural-unfaithfulness audits, external formal verification, input contrast sets, and trace-dependence probes. Across arithmetic, formal logic, multi-hop question answering, and biomedical QA, CSR substantially improves edit sensitivity while incurring small accuracy trade-offs. The gains are largest in domains where the edited operators can be identified and verified reliably; performance degrades as verifier precision falls. These results support CSR as a practical regularizer for trace-sensitive reasoning, not as a complete solution to explanation faithfulness.

Abstract:
While LVLMs rapidly improve, long-video question answering still remains challenging: relevant evidence is sparse, and question-relevant context often fails to provide cues that discriminate the correct answer from plausible alternatives. Diagnostic analysis on a manually annotated subset of MMR-V shows that prior agentic systems substantially improve cue retrieval over direct VLM inference yet fail to achieve a corresponding gain in answer accuracy, indicating that the bottleneck lies in option-discriminative evidence rather than topical relevance alone. We propose PACE (Progressive Acquisition of Critical Evidence), a factor-guided framework for long-video evidence acquisition. PACE proceeds in two stages: it first indexes clip-level descriptions guided by question-derived factors without observing the candidate answers; it then uses the candidate answers to derive contrastive cues and queries the index for verification. On MMR-V with the open-source Qwen3-VL backbone, PACE achieves 42.6% accuracy, outperforming direct inference and prior agentic baselines including Deep Video Discovery (DVD). On the same diagnostic subset, PACE recovers 66.9% of the annotated cues, providing empirical evidence that its gains are associated with improved evidence recovery rather than stronger answer-side priors alone. Consistent gains over DVD on LVBench, Video-MME, EgoSchema, and LongVideoBench suggest that option-aware evidence acquisition transfers beyond MMR-V.

Abstract:
Autonomous web agents powered by large language models (LLMs) remain brittle on long-horizon browser workflows. A key bottleneck is a grounding gap in existing skill formulations: textual workflow skills provide natural language guidance but cannot be directly executed, while code-based skills execute without giving the agent step-level guidance for adaptation or recovery. We introduce WebXSkill, a framework that bridges this gap with executable skills, each pairing a parameterized action program with step-level natural-language guidance. WebXSkill operates in three stages: skill extraction mines reusable action subsequences from readily available synthetic agent trajectories and abstracts them into parameterized skills, skill organization indexes them into a URL-based graph for context-aware retrieval, and skill deployment exposes two complementary modes, grounded mode for fully automated execution and guided mode where skills serve as step-by-step instructions the agent follows with its native planning. WebXSkill demonstrates consistent improvements on WebArena, WebVoyager, and Online-Mind2Web. We further find that better skill deployment mode depends on a model’s plan and execution capability. The code is available at https://github.com/aiming-lab/WebXSkill.

Abstract:
Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluations can easily be fooled by superficial label overrides, creating a false impression of reasoning circuit repairs. In this work, we propose Cross-Rule Transfer (CRT), a diagnostic framework that audits representational interventions by evaluating them on rule families where the model is natively competent. Evaluating late-layer LRS for a widespread logical failure, contradiction blindness, reveals that the intervention merely injects a global label bias: applying the steering vector to rules the model already handles correctly (99.6% baseline) degrades performance to 40.4% by forcing false contradiction predictions. We support this diagnosis with four complementary controls (direct logit bias equivalence, control vector label-flipping, cross-model grafting, and early-layer steering checks), providing a rigorous methodology to distinguish genuine reasoning repairs from superficial label overrides.

Abstract:
The combination of verifiable languages and LLMs has significantly influenced both the mathematical and computer science communities because it provides a rigorous foundation for theorem proving. Recent advancements in the field provide foundation models and sophisticated agentic systems pushing the boundaries of formal mathematical reasoning to approach the natural language capability of LLMs. However, little attention has been given to the formal physics reasoning, which also heavily relies on similar problem-solving and theorem-proving frameworks. To solve this problem, this paper presents, to the best of our knowledge, the first approach to enhance formal theorem proving in the physics domain. We compose a dedicated dataset PhysLeanData for the task. It is composed of theorems sampled from PhysLean and data generated by a conjecture-based formal data generation pipeline. In the training pipeline, we leverage DeepSeek-Prover-V2-7B, a strong open-source mathematical theorem prover, and apply Reinforcement Learning with Verifiable Rewards (RLVR) to train our model PhysProver. Comprehensive experiments demonstrate that, using only about 5.5K training samples, PhysProver achieves an overall 2.4% improvement in multiple sub-domains. Furthermore, after formal physics training, we observe 1.3% gains on the MiniF2F-Test benchmark, which indicates non-trivial generalization beyond physics domains and enhancement for formal math capability as well. The results highlight the effectiveness and efficiency of our approach, which provides a paradigm for extending formal provers outside mathematical domains. To foster further research, we will release both our dataset and model to the community.

Abstract:
As large language models (LLMs) are deployed globally, creating pluralistic systems that can accommodate the diverse preferences and values of users becomes essential. We introduce EVALUESTEER, a human-validated benchmark to measure the steerability of LLM and reward models (RMs) to users’ value and stylistic preference profiles grounded in psychology and literature on human-LLM interaction. To address the gap in existing datasets that do not support controlled evaluations of RM steering, we synthetically generated 165,888 preference pairs – systematically varying pairs along 4 value dimensions (traditional, secular-rational, survival, and self-expression) and 4 style dimensions (verbosity, readability, confidence, warmth). We use EVALUESTEER to evaluate whether, given a user profile and a pair of candidate value and style laden responses, LLMs and RMs are able to select the output that aligns with the user’s preferences. Our results show that, when given the user’s full profile of values and stylistic preferences, the best models achieve < 75% accuracy in choosing the correct response, in contrast to > 99% accuracy when only relevant style and value preferences are provided. RMs also exhibit systematic biases, favoring secular values and verbose, confident language, and prioritizing style alignment over value alignment when the two conflict. EVALUESTEER thus highlights the limitations of current RMs at identifying and adapting to relevant user profile information, and provides a challenging testbed for developing RMs that can be steered towards diverse human values and preferences.

Abstract:
We study style-aware prompt-to-webpage generation : producing executable webpages that preserve content, functionality, and abstract aesthetic intent. We identify the design-intent bottleneck and introduce three components to address it: Web4Style , a brief-based diagnostic benchmark; WebWeaver , a shared-state framework that constructs and propagates an explicit style contract ; and WebJudge , a structured multimodal evaluator for content, function, and visual aesthetics. Experiments across three backbone models show that WebWeaver improves over direct generation and representative web/code-generation agents in both automatic and human evaluation. These results highlight style contracts as a missing intermediate layer for reliable prompt-to-webpage generation.

Abstract:
Decoding quality in diffusion multimodal language models (dMLLMs) depends heavily on the order in which masked tokens are committed. Existing confidence-based strategies prioritize locally easy tokens, but confidence does not necessarily reflect contextual usefulness. As a result, structurally easy tokens such as punctuation may be committed before informative semantic anchors, weakening context propagation and increasing error accumulation. We propose Information-Guided Frontier Decoding (IGFD), a training-free decoding strategy that ranks candidates using token confidence, neighborhood uncertainty, and structural commitment risk. IGFD encourages early commitment of reliable semantic anchors while delaying fragile structural tokens, improving contextual support during decoding. A dynamic candidate frontier further constrains token selection to locally expandable regions under the same decoding budget. The method requires no additional training, auxiliary models, or extra forward passes. Experiments across multimodal understanding, reasoning, grounding, and hallucination benchmarks show that IGFD consistently outperforms existing decoding strategies across the majority of benchmarks and diffusion MLLM backbones under identical decoding budgets.

Abstract:
Preference-learning methods for machine translation (MT), such as Direct Preference Optimization (DPO), have shown strong gains but typically rely on carefully curated preference triplets and often struggle to generalize beyond their tuning domains. We propose Reinforcement Learning from Teacher-Model Refinement (RLfR), which replaces static triplets with on-policy, actor-conditioned refinements produced by a frozen teacher. At each step, the actor samples candidate translations, the teacher performs a minimal local edit of each draft, and the actor is reinforced to close the gap using a composite reward that combines scaled negative edit distance for lexical and structural fidelity with COMET for semantic adequacy. This formulation yields a stable, model-aware learning signal without requiring curated preference triplets, making it naturally scalable to many language pairs without additional data collection. Experiments on FLORES-200 (en <-> de/es/zh/ko/ja) show that RLfR consistently outperforms strong baselines, improving semantic quality and entity preservation across automatic, LLM-judge, and human evaluations.

Abstract:
Despite the remarkable progress of Vision-Language Models (VLMs) in adopting "Thinking-with-Images" capabilities, accurately evaluating the authenticity of their reasoning process remains a critical challenge. Existing benchmarks mainly rely on outcome-oriented accuracy, lacking the capability to assess whether models can accurately leverage fine-grained visual cues for multi-step reasoning. To address these limitations, we propose ViEBench, a process-verifiable benchmark designed to evaluate faithful visual reasoning. Comprising 200 multi-scenario high-resolution images with expert-annotated visual evidence, ViEBench uniquely categorizes tasks by difficulty into perception and reasoning dimensions, where reasoning tasks require utilizing localized visual details with prior knowledge. To establish comprehensive evaluation criteria, we introduce a dual-axis matrix that provides fine-grained metrics through four diagnostic quadrants, enabling transparent diagnosis of model behavior across varying task complexities. Our experiments yield several interesting observations: (1) VLMs can sometimes produce correct final answers despite grounding on irrelevant regions, and (2) they may successfully locate the correct evidence but still fail to utilize it to reach accurate conclusions. Our findings demonstrate that ViEBench can serve as a more explainable and practical benchmark for comprehensively evaluating the effectiveness of agentic VLMs.

Abstract:
Large language models (LLMs) are increasingly used in role-playing applications, yet evaluating whether they maintain character-consistent behavior over extended interactions remains challenging. Existing benchmarks mostly focus on short exchanges, static long-context inputs, or isolated aspects of role-playing, making long-horizon failures difficult to diagnose. We present REMP (Relationship-Emotion-Memory-Persona), a structured event-probing benchmark for long-horizon role-playing dialogue. REMP constructs multi-chapter dialogue trajectories from character profiles and embeds diagnostic probe events targeting persona consistency, emotional consistency, memory integrity, and relational continuity. We further introduce REMP-EVAL, an anchor-based multi-turn evaluation protocol that assesses model behavior after event triggers using dimension-specific references. The current release contains 160 characters organized into 80 dialogue trajectories, each averaging about 60 sessions and 800 turns. Experiments on 11 LLMs show that memory integrity and emotional consistency are the main bottlenecks, indicating that long-horizon role-playing requires not only long-context retrieval but also sustained affective and relational consistency.

Abstract:
Benign fine-tuning severely weakens the safety alignment of large language models (LLMs), so we study why refusal behavior is so fragile. While prior work often attributes this failure to gradient conflict, we propose a fundamentally different Fisher-geometric explanation: safety Fisher is low-rank, and alignment makes the safety geometry flatter while preserving an output-routing pathway. After 100 benign fine-tuning examples, this pathway is selectively re-sharpened in output-side MLP modules, explaining the asymmetric fragility: safety can collapse to high attack success rates, while general utility degrades mildly. The routing view also explains why few safety examples can restore refusal behavior, indicating that internal safety-relevant representations are preserved. Finally, we show that LoRA and ASAM mitigate early collapse by suppressing output-side sharpness, but their protection weakens at larger fine-tuning scales. Overall, safety failure is best understood as a disruption of a low-rank output-routing mechanism.

Abstract:
Multimodal retrieval-augmented generation (MMRAG) systems can fail even when relevant evidence is retrieved, exposing a gap between retrieval and generation. We formulate MMRAG failure diagnosis as an online single-fire decision problem over prefill hidden states, which is distinct from static probing of model internals or post-hoc self-correction. Across InfoSeek, E-VQA, and VISA, we show that hidden states encode both failure occurrence and failure type, with failure detection reaching around 90% accuracy and recall on InfoSeek and E-VQA. We further introduce a Fire-Then-Type prober that separates failure detection from failure typing, improving typed diagnosis over direct classification. Finally, our method matches the response accuracy of final-layer repair on InfoSeek and E-VQA while diagnosing before answer decoding. The available prefill depth varies substantially across models, from 4.6% to 24.3%, and is limited for Qwen3-VL and InternVL on VISA.

Abstract:
Personalizing large language models (LLMs) requires encoding long-term, user-specific behavioral patterns in a way that is computationally efficient, scalable, and compatible with a frozen base model. We present Latent Personal Memory (LPM), a framework that represents user-specific history as a compact, persistent matrix of latent slots, that are interpretable. A shared cross-attention projection network maps these slots into dynamic, input-conditioned soft prompts that are prepended to the input of a frozen LLM. We evaluate LPM on PersonaMem v1 and LoCOMO benchmarks across Qwen3-1.7B, 4B, and 8B backbones. Results demonstrate that LPM outperforms LoRA and Prompt Tuning by up to 8.8 % and 54.4% in overall accuracy respectively on PersonaMem v1, while reducing KV-cache usage by over 64x. On LoCoMo, LPM matches LoRA’s accuracy while requiring approximately 120x fewer trainable parameters. We also show that LPM’s efficiency grows with context length and outperforms full-context at 128K context length.

Abstract:
Natural-language tasking of embodied agents is rarely just goal specification: users also impose constraints that must persist while the world changes. Code-generating LLM agents can produce plausible behaviors for such instructions, but their free-form programs provide no stable object to verify, compose with new constraints, or repair from a failing trace. We present CEDAR, a counterexample-guided framework that grounds instructions as regular languages over environment event traces. CEDAR uses a language model for semantic judgments and execution traces for correction, then represents both skills and specifications as deterministic finite automata. This turns constraints into executable finite-state objects: a learned skill can be intersected with a learned sleep at night or stay in this biome specification, yielding a controller that enforces the learned constraint by construction rather than by repeated prompting. In Minecraft, with the same simulator/API observations available to a program-generating baseline, CEDAR maintains temporal and spatial constraints that the baseline fails to preserve and amortizes reuse of learned skills, reducing cumulative LLM queries. These results suggest that regular languages offer a practical verification layer between natural-language instructions and embodied-agent policies.

Abstract:
Visual figurative entailment requires determining whether an image supports the nonliteral meaning of a textual claim, posing challenges for multimodal figurative understanding. Existing methods typically use fixed image-text fusion and struggle to capture sample-specific relations between visual evidence and figurative claims. Moreover, the limited scale of annotated visual figurative data makes it difficult to learn robust figurative relations from visual supervision alone. To address these challenges, we propose RIFT, which generates a claim-conditioned visual description and uses a relation-gated multimodal mixture of experts to model interactions among the image, description, and claim. It further retrieves figurative reasoning patterns from a FLUTE-based relation-level rationale memory and employs a confidence routing gate to modulate this external guidance according to retrieval reliability. The resulting internal and external relation representations are integrated as latent prefix tokens for joint entailment prediction and explanation generation. On V-FLUTE, RIFT achieves 88.52 F1@0, 76.07 F1@53, and 59.47 F1@60, outperforming MAPPER by 3.98, 6.38, and 7.75 points, respectively. These improvements, particularly on explanation-constrained metrics, show that relation modeling and cross-modal transfer benefit both prediction and explanation quality.

Abstract:
Large language models rely on inference-time context including rationales and instructions to improve task performance. Prior work attempts to internalize such benefits via parameter-efficient fine-tuning, and also studies inference-time representation interventions, but these approaches typically update the base model or learn fixed directions tied to global attributes. As a result, it remains unclear how to capture and reuse the internal effect of a helpful context that is specific to a given query. We take a representation-level view and define a context imprint as the hidden-state displacement between context-conditioned and query-only forward passes. We propose HintSteer , which trains a lightweight query-conditioned predictor to estimate the imprint from the query representation and injects it into a frozen model at inference time, approximating the missing context effect without explicitly providing context. Experiments on mathematical reasoning and writing demonstrate the effectiveness of imprint-based steering under no-context inference and indicate that both imprint magnitude and predictability influence the resulting gains.

Abstract:
Synthetic dialogue corpora are increasingly used as proxies for target data, yet persona-grounded generators optimize individual conversations rather than corpus composition, yielding plausible dialogues with distorted population-level behavior mixes. We introduce GroupPersona, a framework that aligns synthetic corpora with a reference behavior distribution. GroupPersona turns population statistics into generation controls by discovering recurring behavioral groups in the reference corpus and using their prevalence and interaction patterns to condition user agents. We evaluate corpus-level fidelity through behavioral alignment and structural preservation, measured by average Jensen–Shannon distance over 12 behavior attributes and 5 dialogue structure attributes, respectively. Across four corpora spanning two source families and two construction regimes, GroupPersona reduces behavioral Jensen–Shannon distance from 0.234 to 0.177 over the best-average baseline, a 24.4% reduction. It is best or tied-best on all four corpora. It maintains comparable structural distance at 0.167 versus 0.169. Together, these results show that GroupPersona improves population-level behavioral fidelity while preserving dialogue structure.

Abstract:
Tool-augmented LLMs invoke external functions to extend their capabilities, but errors in the invocation decision, such as calling a tool when none is needed or omitting a needed call, can produce unreliable outputs and unnecessary cost. A lightweight remedy is to prepend retrieved examples so LLMs decide tool use in context. However, existing retrievers rank examples by semantic similarity alone. Lexically close or semantically close queries can require opposite behavior, so the retrieved examples may be behaviorally inconsistent and silently mislead the model. We propose Behavior-Aligned Retrieval (BAR), a backbone-agnostic training recipe that teaches a dense retriever a behavior-aware similarity, keeping semantically related candidates close only when their tool-use behavior is compatible. BAR does not predict invocation labels; instead, it ranks demonstrations while leaving the final tool-use decision to the LLM. Applied to multiple retrieval backbones, including BERT, Contriever, and Qwen3-Reranker, BAR consistently improves invocation reliability and reduces unnecessary API calls across 14 LLMs and 3 benchmarks.

Abstract:
Large Language Models (LLMs) have enabled a shift from sentence-level to document-to-document (Doc2Doc) machine translation, promising improved global coherence. However, document-to-document generation in a single pass frequently suffers from structural misalignment, manifesting as sentence omissions or hallucinations that violate the core requirement of source-target correspondence. To address this, we introduce Sentence Translation Alignment Rate (STAR), an auxiliary metric that explicitly quantifies sentence-level structural fidelity. Building on this, we propose STAR-masked Preference Optimization (StarPO), a framework that ranks document-level hypotheses by structural quality and utilizes a dynamic alignment mask to focus optimization on misaligned segments. Experimental results across news and literary domains demonstrate that StarPO significantly enhances translation quality and structural integrity. Notably, StarPO allows compact models to surpass the performance of massive proprietary systems like GPT-4o while maintaining superior token efficiency. We will release our code and datasets.

Abstract:
Retrieval-augmented generation (RAG) systems rely on external corpora that may contain outdated, contradictory, noisy, or unreliable documents, introducing reliability risks. Prior work has leveraged document relations to improve the answer reliability of RAG. To propagate reliability signals beyond directly compared document pairs, we propose TrustPropRAG, which structures document relations as a graph and estimates document reliability through multi-hop propagation across the graph. TrustPropRAG anchors this propagation with a limited set of human feedback on document reliability, extending these costly-to-collect feedback-based reliability signals across the whole corpus. Specifically, based on the constructed document relation graph, TrustPropRAG estimates a trust score for each document by formulating and solving an optimization problem that jointly captures pairwise document relations and user feedback. These scores are then used to improve the selection of reliable documents and support trust-aware answer generation. Evaluation results show that TrustPropRAG improves both retrieval quality and exact match over baselines, and remains robust under sparse and noisy feedback.

Abstract:
Text-to-SQL has advanced rapidly with large language models, but complex database queries still require reasoning beyond one-shot generation, including multi-step decomposition, execution-based diagnosis, and targeted correction. We present EvoSQL, a co-evolution framework that formulates SQL synthesis as an iterative interaction between a generator and a critic. Moreover, EvoSQL maintains a contextualized candidate memory, verifies SQL candidates with both execution signals and LLM-based critique, and updates its memory through utility-guided aggregation. To strengthen the underlying generator–critic pair, we further introduce a Self-Distillation Policy Optimization (SDPO) fine-tuning stage that injects execution-aware supervision into current coding LLM backbones. Experiments on Spider and BIRD show that EvoSQL consistently improves open-source models over Maj@16 baselines, with particularly large gains on BIRD-Dev, ranging from +1.37% for Qwen3-4B to +9.19% for Qwen2.5-Coder-3B. SDPO initialization further improves selected backbones on Spider-Test and BIRD-Dev. These results suggest that memory-grounded co-evolution is an effective path toward more reliable and generalizable Text-to-SQL systems.

Abstract:
Dialogue histories often need to be compressed to fit within the context budgets of large language models (LLMs). However, when conversational compression occurs, turns containing critical information are sometimes removed. We study which turns should be retained to minimize information loss during task-oriented dialogue compression. On the MultiWOZ and SGD datasets, we compare four turn-level signals: Frame for task context, Evidence for question-conditioned relevance, Pivot for state changes, and Recency for later context. Evenly spaced selection (Stride) serves as a simple positional baseline. The effectiveness of each signal depends on the question type. Evidence achieves the highest accuracy among the four signals on current-slot questions in both datasets. At a 30% word budget, Pivot improves over Stride on MultiWOZ when a state change determines the final value (0.421 versus 0.255), but is slightly worse on SGD (0.234 versus 0.245). For questions that require information from both the task context and a final value located in separate turns, Stride performs best among the compressed methods. The best selector depends on the evaluation target, and no single method is best in all settings.

Abstract:
Large language models are increasingly trained on GPUs distributed across multiple regions, but geo-distributed training is challenging in practice. Real clusters often contain GPUs with different speeds and memory capacities, and they communicate over slow wide-area networks. Our analysis shows that this creates serious problems: existing synchronous methods preserve stable updates, but fast GPUs wait up to 20.9% of their runtime for slower ones, and all workers spend, on average, 65.8% of their runtime on synchronization. Recent asynchronous methods reduce waiting time but worsen the model accuracy due to stale updates. To address the problems, we present GeoMesh, a synchronous geo-distributed training framework for heterogeneous GPUs. GeoMesh balances per-worker workloads by assigning each GPU a suitable batch size and number of inner steps, so faster GPUs do more useful work instead of waiting. It also reduces communication volume by nearly 32 × by exchanging compressed sign-based pseudo-gradients with lightweight magnitude and token count. Across heterogeneous GPUs and Azure-derived WAN, GeoMesh reduces time-to-target perplexity by up to 70.2% over representative baselines and lowers straggler- and WAN-induced GPU idle by up to 8.0 × and 5.6 × , respectively, while preserving comparable zero-shot accuracy.

Abstract:
Despite the growing video understanding capabilities of recent Multimodal Large Language Models (MLLMs), existing video benchmarks primarily assess understanding based on models’ static, internal knowledge, rather than their ability to learn and adapt from dynamic, novel contexts from few examples. To bridge this gap, we present Demo-driven Video In-Context Learning, a novel task focused on learning from in-context demonstrations to answer questions about the target videos. Alongside this, we propose Demo-ICL-Bench, a challenging benchmark designed to evaluate demo-driven video in-context learning capabilities. Demo-ICL-Bench is constructed from 1200 instructional YouTube videos with associated questions, from which two types of demonstrations are derived: (i) summarizing video subtitles for text demonstration; and (ii) corresponding instructional videos as video demonstrations. To effectively tackle this new challenge, we develop Demo-ICL, an MLLM with a two-stage training strategy: video supervised fine-tuning and information-assisted direct preference optimization, jointly enhancing the model’s ability to learn from in-context examples. Extensive experiments with state-of-the-art MLLMs confirm the difficulty of Demo-ICL-Bench, demonstrate the effectiveness of Demo-ICL, and thus reveal future research directions for video in-context learning.

Abstract:
Conversational Recommender Systems (CRS) typically require domain-specific dialogue data, which is costly, scarce, and often unavailable in new domains. We conduct a systematic empirical study of zero-data CRS bootstrapping: generating synthetic conversational supervision from non-conversational signals—item reviews, metadata, and user-item interactions—without any in-domain dialogue corpus. We compare two information-theoretic selection strategies, Jensen-Shannon diversity and Fisher information, across domain signals, model architectures, datasets, and fine-tuning paradigms. Our results show that domain-grounded synthetic data consistently outperforms zero-shot prompting and naive synthetic baselines; active selection improves data efficiency over random sampling; metadata and collaborative filtering signals each improve selection quality; and, in low-resource settings, synthetic data can outperform scarce real dialogues while further complementing them. These findings establish non-conversational domain signals as a viable path toward building CRS without conversational training data. The code is available at https://anonymous.4open.science/r/zero_data_crs/ .

Abstract:
While Audio Language Models (ALMs) offer a promising shift towards explainable audio deepfake detection (ADD) via generated reasoning, whether these explanations reflect faithful perceptual grounding or fragile post-hoc rationalizations under adversarial attack remains under-explored. To address this gap, we introduce SARA (Shift Analysis of Reasoning in Audio), a diagnostic framework that evaluates the reasoning of five ALMs across three dimensions: acoustic perception, reasoning-verdict coherence and dissonance. Our experiments reveal three key findings. First, current ALMs exhibit weak acoustic perception ( < 50 % accuracy), and generating explicit rationales imposes an accuracy penalty (reducing real F1 score by up to 21%). Second, adversarial attacks show a modality asymmetry: acoustic attacks severely degrade reasoning-verdict coherence (average drop of 14.20%), whereas linguistic attacks achieve higher attack success (61.73%) while preserving fluent coherence (a drop of only 3.91%). Third, we demonstrate that decomposing reasoning shifts into prediction confidence and reasoning-verdict coherence enables a secondary defense layer that can detect adversarially perturbed audio with a 74.41% F1 score even with unseen attack strategies and speaker profiles. These findings suggest that reasoning traces also provide diagnostic utility that persists even when final classification outputs are compromised. Our code is available at [https://github.com/nqbinh17/sara_reasoning_shifts](https://github.com/nqbinh17/sara_reasoning_shifts).

Abstract:
Vision-language models (VLMs) often generate fluent but visually unsupported descriptions, especially by mentioning objects absent from the image. We propose QK Product Steering , a data-free, training-free, and zero-inference-cost weight edit for reducing object hallucination. The method directly edits the per-head query-key product, the operator that produces pre-softmax attention logits, by suppressing a small number of dominant singular modes in selected middle layers. The edited product is then mapped back to the query weights through a closed-form query-only update while keeping shared key weights fixed, making the edit compatible with grouped-query attention. We further decompose the QK product into symmetric and antisymmetric components to distinguish mutual content-similarity patterns from directional attention patterns. Across three GQA-based VLMs, QK Product Steering achieves an average relative CHAIR s reduction of 5.8%, while matched random-mode controls show negligible change. Interpretability ablations show that the hallucination signal is specific to dominant QK modes and is primarily localized to the symmetric mutual-attention channel. Overall, QK Product Steering offers a simple alternative to decoding-time mitigation, requiring no additional data, fine-tuning, or inference-time overhead while largely preserving general multimodal capability.

Abstract:
Retrieval-augmented generation (RAG) typically incorporates external knowledge by appending retrieved documents to the input prompt. While effective for knowledge-intensive tasks, this in-context formulation increases memory use and serving cost as the retrieved context grows. Motivated by this limitation, we propose Doc2Weight, an in-weights RAG framework that injects retrieved information into model parameters rather than the textual context. Doc2Weight converts retrieved documents into parameter-efficient low-rank updates and fuses them in weight space according to their relevance to the query. This enables retrieval augmentation with a query-only prompt, decoupling retrieval depth from prompt length while allowing multiple retrieved passages to influence generation within a single forward pass. We evaluate Doc2Weight on instruction-tuned language models of different scales, including LLaMA-3.2-1B, Qwen2.5-1.5B, and LLaMA-3-8B, across major QA benchmarks. Experiments show that Doc2Weight achieves the best average F1 among standard in-context RAG and prior in-weights RAG baselines across backbones and prompting settings, while matching the throughput of in-context RAG once document signatures are cached. Our code is publicly available.

Abstract:
Language models are increasingly used as agents to help humans decide what information is surfaced. This usage incentivizes content creators to optimize content in ways that appeal not only to humans, but also to agents that mediate access to them. In this paper, we study selection shifts induced by rewriting in agentic decision-making. Given a set of competing content snippets, we rewrite only one snippet while leaving the rest unchanged, and then measure how it impacts the agent’s choice. We operationalize this setup with AgentBait an advisor-rewriter framework in which the advisor learns to propose rewriting strategies and the rewriter revises the snippet. While a rewriter with a fixed prompt improves target snippet selection from 17.1% to 34.8%, our AgentBait raises its selection to 98.5%. We further show that the advisor trained with AgentBait effectively transfers to setups with different agents, languages, and snippets in other domains (e.g., scientific papers). However, higher target selection can reflect unsupported rewrites rather than better content. Adding a reward for support from the original snippet redirects the advisor toward more supported rewriting strategies, revealing a trade-off between factuality and target selection. Together, our results show that once agents mediate access to information, content can be rewritten to be chosen by the agent, even when selection and usefulness diverge.

Abstract:
Prompting is the primary interface through which users interact with large language models. Although individual prompting strategies have been extensively studied, it remains unclear whether combining multiple strategies can yield additive benefits. We present a systematic empirical study of prompt pattern composition across multiple LLMs and composition mechanisms, evaluating 18 prompt pattern categories over 7 benchmarks spanning 6 task types. Among the three models for which we exhaustively evaluate all 306 ordered pairs of functional prompt patterns, only 6.86%, 10.46%, and 4.25% of compositions outperform their stronger constituent prompt, respectively. We further find that prompt accumulation provides only limited early-stage gains, while local additive predictions increasingly overestimate actual performance as compositions grow longer. Moreover, selection-based mechanisms generally provide a more favorable performance profile than direct prompt accumulation. Overall, our results reveal a scoped less-is-more tendency in the prompt-composition settings studied here: effective use of multiple prompt strategies depends more on selecting suitable strategies than on simply accumulating additional instructions.

Abstract:
Reinforcement learning from verifiable rewards (RLVR) has become a promising paradigm for improving the reasoning capabilities of large language models (LLMs). However, existing RL-based training remains only a rough approximation to human learning, where humans leverage external experience to guide exploration and gradually internalize problem-solving strategies into stable knowledge. Motivated by this gap, we ask: how can LLMs better utilize and internalize experience during RLVR training? To answer this question, we propose D ual G uidance O ptimization ( DGO ), which treats experience as a bridge between external guidance and internal policy learning. Rather than using experience only as an inference-time aid or trajectories only as optimization signals, DGO couples them in a closed loop: experience guides exploration, improved trajectories renew the experience bank, and guided trajectories are rewritten as self-contained traces for policy distillation. This enables the model to better utilize external experience while yielding consistent gains under experience-free inference. Experiments across multiple benchmarks show that DGO consistently outperforms baselines under both experience-free and experience-guided inference, suggesting that coupling utilization and internalization leads to effective reasoning.

Abstract:
We study two modes of low-resource language safety: in-language alignment, where models refuse harmful instructions in the target language, and cross-lingual safety borrowing, where they rely on a better-aligned proximate high-resource language, including via response-language fallback. We evaluate 11 commercial APIs on English, Mandarin, Cantonese, Tibetan, and Russian, plus 3 Tibetan-specialized open-source models, using native-speaker-verified datasets. Results reveal two patterns: Tibetan, for which response-language fallback provides limited aggregate protection in our setting, shows a safety cliff with ASR of 12.69–60.96%, far above English; Cantonese, which shares script and vocabulary with Mandarin, keeps aggregate ASR in single digits, but Cantonese-reply ASR exceeds aggregate ASR for all 11 models (sign test, p < 0.001), exposing in-language safety gaps concealed by aggregate metrics. We release Cantonese-AdvBench and Tibetan-AdvBench.

Abstract:
Web agents need to navigate visually rich, long-horizon interfaces that change across sites, yet most previous agents still learn each task in isolation and discard the procedural knowledge they accumulate. Recent skill-augmented frameworks take an important first step, but they treat the skill library as a flat or two-tier prompt-side cache and offer no principled mechanism for compressing redundancy or composing skills recursively. We introduce Scaffold, a self-improving framework for visual web agents that (i) induces parametric, executable skills from successful trajectories under a multi-instance abstraction constraint, (ii) maintains a recursively composed hierarchy in which higher-level skills invoke lower-level ones, (iii) compacts the library via a minimum-description-length (MDL) criterion and behavioral equivalence checking, and (iv) periodically distills skill-augmented trajectories back into model weights to internalize the abstractions. Across WebArena, VisualWebArena, and a held-out split of Online-Mind2Web, Scaffold improves success rate by 11.1–17.2 absolute points over the strongest skill-augmented baseline and shows monotonic gains across five self-improvement iterations without library collapse. We release the code and documents in the Github [repository](https://github.com/BokwaiHo/SCAFFOLD).

Abstract:
Vision-Language Models (VLMs) are increasingly applied to embodied task settings, yet it remains unclear whether their task-progress judgments are grounded in physical state comparison or in input-order shortcuts. We study a diagnostic two-frame setting: given two robot manipulation states and a task goal, the model must decide which state is closer to completion, or whether the two states are effectively equal. We observe a strong Forward-Backward Performance Gap: models often answer correctly in the natural forward order, but fail when the same two physical states are shown with their image slots swapped. This reversal is not intended to mimic the dominant deployment order of robot execution; rather, it is a counterfactual diagnostic for testing whether the selected physical state remains consistent under input-slot inversion. To reduce this order dependence, we introduce Equal-State supervision as a zero-point calibration signal, training models to output equal when no task-relevant progress difference is present instead of defaulting to a directional answer. From real manipulation videos, we construct the Embodied Causal-Temporal Reasoning (ECTR) dataset and propose Tri-Stage Weighted Alignment (TSWA), a staged fine-tuning procedure that strengthens spatial description, task-conditioned comparison, and answer calibration in sequence. On Qwen2.5-VL-7B, TSWA reduces GAP from 60.50% to 5.14% and raises overall accuracy from 44.66% to 80.19%. This comes with a small decrease in forward accuracy, but large gains in backward accuracy, equal-state recognition, and pairwise consistency, suggesting that TSWA replaces order-dependent correctness with more calibrated state comparison.

Abstract:
Existing conversational retrievers commonly treat topical relevance as a proxy for answerability. However, a passage that closely matches the dialogue context is not necessarily the one that supports the correct answer. We identify this mismatch as a systematic answerability gap. To address this issue, we propose CLEAR, a framework that shifts conversational retrieval from topical relevance to answerability. The core of CLEAR is entailment distillation, which transfers answer-passage entailment supervision into a cross-encoder reranker so that the reranker discriminates answer-supporting passages from topical distractors at inference time, without requiring answers. CLEAR is complemented by a passage-centric abductive recall module that brings low-similarity yet answerable passages into the candidate pool by inferring answerable queries from passages with an LLM. Across TopiOCQA, QReCC, and out-of-domain TREC CAsT datasets, CLEAR consistently improves top-ranked precision over strong query-rewriting and dense-retrieval baselines, with the largest gains observed in conversations involving heavier topical noise. Moreover, applying our reranker on top of an LLM-driven query rewriter yields further gains.

Abstract:
Self-correction has emerged as a promising paradigm for enhancing LLM reasoning, yet recent studies reveal a troubling phenomenon: without external oracle feedback, iterative reflection often degrades rather than improves performance. Our analysis uncovers that toxic reflection, where correct answers deteriorate into incorrect ones, occurs at rates comparable to effective corrections. We identify the root cause as the shortsightedness of existing methods: models focus only on the previous output while neglecting the global reasoning state, rendering them unable to quantify uncertainty. To address this, we propose two complementary reflection signals — Reflecting Entropy for snapshot disagreement among reasoning trajectories and Belief Shift for inter-round belief revision — paired with a two-stage hierarchical controller that schedules reflection adaptively. We instantiate these ideas as REGAR (Reflecting Entropy-Guided Adaptive Reflection), a framework that maintains a pool of candidate reasoning trajectories and applies a single, uniform reflection primitive when correction is needed. Experiments on mathematical and commonsense reasoning demonstrate consistent improvements over strong baselines while reducing computational overhead, highlighting both effectiveness and efficiency.

Abstract:
While Large Language Models have achieved state-of-the-art performance in general machine translation, they frequently struggle to adhere to complex stylistic nuances and lexical constraints strictly and often revert to a neutral style. The scarcity of style-specific parallel data hinders existing solutions, whereas standard prompting lacks rigorous verification mechanisms. To bridge this gap, we propose Style-Ruler, an inference-only agentic framework that operationalizes style via unsupervised extraction and neuro-symbolic verification. Style-Ruler orchestrates three specialized agents: (1) a Style-Reverse Analyst (SRA) that mines quantifiable linguistic constraints from monolingual reference texts via unsupervised extraction; (2) a Constraint-Adaptive Translator (CAT) that employs a Style-Chain-of-Thought mechanism to plan stylistic adherence before generation explicitly; and (3) a Sandbox-Augmented Auditor (SAA) that provides heterogeneous feedback through deterministic code execution. These components form a closed-loop refinement mechanism where symbolic logic guides neural generation. Extensive experiments on the GuoFeng Webnovel dataset and WMT24++ benchmark demonstrate the superiority of our approach. Style-Ruler achieves a CSR of 82.4% on stylized translation, surpassing standard Chain-of-Thought prompting by over 50%, while improving BLEU scores by +1.0 on En → Zh.

Abstract:
Large language models (LLMs) show promise for scientific applications but struggle when domain knowledge is encoded in complex structures. Chemistry exemplifies this challenge: molecular structures encode rich relational data that dictates chemical meaning, yet standard formats like SMILES obscure this through arbitrary linearization. In this work, we systematically characterize which structural information lost during linearization most severely hampers LLM reasoning, decomposing the problem into IUPAC-grounded dimensions: substituent positions (locants), attachment points (anchoring), and modification constraints (valency). Our diagnostic reveals non-uniform failures across dimensions, with locant identification being especially challenging. We directly address the failures by grounding the linearized representations in structural context with each component targeting its corresponding diagnostic dimension. General-purpose LLMs with explicit grounding can outperform chemistry-specialized models on structure-aware molecule manipulation, suggesting structural context can compensate for domain-specific training. Moreover, we investigate a training-free inductive protocol that nearly doubles the accuracy of weaker models. These results point toward explicit structural grounding as a principled path to more reliable molecular reasoning across model scales.

Abstract:
Recent work has questioned whether large language models (LLMs) can be useful for temporal event prediction. Despite vast pretrained world knowledge, LLMs often underperform simpler specialized models. We ask whether this knowledge is truly inaccessible, or whether current methods fail to present event prediction in a form LLMs can use. We argue for the latter and show that LLMs become effective event forecasters when the interface is designed around two choices. The first grounds each event in a natural-language description, and the second separates event semantics from the prediction space to prevent semantic bias from label associations. To test this, we introduce DescribeEvent, a benchmark of ~150K events across seven datasets and six domains with event-level descriptions. We find that capable LLMs substantially benefit from descriptions and symbolic labels, outperforming description-free variants and non-LLM baselines, while weaker models gain little. These results suggest that accessing LLM world knowledge for event prediction is an interface problem, and that event descriptions offer a practical way to elicit the knowledge LLMs already encode.

Abstract:
Autonomous large language model (LLM) agents have shown strong potential for open-ended information-seeking tasks. However, existing agent frameworks primarily rely on sequential execution, making them vulnerable to error accumulation in long-horizon reasoning. Recent plan-and-execute approaches partially alleviate this issue, but their fixed planning strategies still struggle to recover from intermediate failures. We propose Logic Tree, a non-linear agentic reasoning framework formulated as a Directed Acyclic Graph (DAG). Instead of following a fixed plan with sequential execution, Logic Tree models complex task solving as a dynamically evolving and recoverable graph search process. The framework decomposes queries into parallel execution branches and incorporates a two-level self-correction mechanism. Local backtracking performs targeted rollback to repair failed reasoning states, while global replanning revises high-level execution strategies and preserves verified intermediate conclusions through a knowledge sanitation process. Experiments on BrowseComp, GAIA, and XBench show that Logic Tree consistently outperforms both sequential and plan-and-execute baselines. Further ablation studies demonstrate that combining DAG-based parallel execution with dynamic error recovery substantially improves robustness in long-horizon research tasks.

Abstract:
Latent chain-of-thought models move intermediate reasoning into continuous states, obscuring which internal object causally carries a computation. We introduce SCIT, the Suffix Cache Interchange Test, which patches exact source–recipient counterfactuals and crosses cache-segment sufficiency with K/V splits, hidden-state and semantic-source controls, decoded validation, and matched corruption. In GPT-2 arithmetic, counterfactual transfer concentrates in value-cache suffix trajectories rather than hidden states, keys, answer slots, or single-token triggers. Full sufficiency-and-necessity closure holds only for the main CODI-GPT2 checkpoint; the Sim-CoT-style reproduction supports sufficiency and decoded transfer but not necessity. Across broader competent cells, arithmetic-like GPT-2/1B tasks retain latent-tail value/KV transfer, whereas 8B and repaired non-arithmetic tasks shift to prompt-prefix or full-cache K/V; ungated cells receive no call. SCIT thus provides a cache-level diagnostic, a checkpoint-specific arithmetic mechanism, and a competence-gated carrier map—not a universal latent-tail claim. Code: https://github.com/YIDING4869/scit-emnlp-2026

Abstract:
Multimodal fake news proliferates on social media, yet its verification is hindered by context sparsity and analytical heterogeneity: deceptive posts often hinge on implicit social background, while distinct misinformation patterns demand different verification strategies. Existing methods rely on either representation-driven classifiers or LLM-based retrieval-augmented and tool-using pipelines, which act as opaque black boxes or follow rigid, predefined tool chains, failing to adapt reasoning to diverse news and yielding weakly grounded, uninterpretable decisions. To this end, we propose MCTL, a Meta-Cognitive Tool-Learning Framework that reformulates detection from static classification into in-context cognitive orchestration with dynamically planned tool-based reasoning. Specifically, MCTL retrieves cross-modal contextual evidence as external memory to compensate for context sparsity, performs meta-cognitive planning to construct a sample-specific Directed Acyclic Graph (DAG) reasoning blueprint, routes specialized tools along the DAG, and fuses conflicting judgments via consensus-aware arbitration, enabling context-grounded, sample-adaptive, and traceable verification. Extensive experiments demonstrate that MCTL consistently outperforms recent state-of-the-art agentic and RAG baselines, confirming its effectiveness and reliability.

Abstract:
While Large Language Models (LLMs) excel as static solvers, transforming them into autonomous agents remains challenging. This transition requires continuous environmental interaction, yet current agents lack the necessary persistent procedural memory. Existing approaches predominantly employ Retrieval-Augmented Generation (RAG) to inject explicit textual guidelines into model contexts. However, relying solely on symbolic instructions can introduce a text-action disconnect, frequently failing to activate the internal representations necessary for correct task execution. To address this, the paper introduces Neural Procedural Memory (NPM), a training-free framework that represents agent memory through implicit activation steering rather than explicit instructions. By distilling procedural skills from historical contrastive experiences into steering vectors in the activation space, NPM directly activates the task-relevant neural mechanisms to guide task execution. Evaluations across four agent benchmarks show that NPM performs comparably to baselines using explicit textual instructions. Furthermore, the results show that combining implicit steering with explicit workflows provides complementary advantages, leading to more robust task execution. Representational analyses indicate that these steering vectors encode consistent task logic, forming organized structures within the activation space. These findings suggest that implicit activation steering provides a promising approach for managing agent memory.

Abstract:
Evaluation of cultural value alignment in Large Language Models (LLMs) typically relies on direct prompting with survey-style questions, which often elicit neutral or safety-aligned responses and fail to capture the model’s underlying cultural orientation. To overcome these challenges, we propose a framework for probing and steering latent cultural representations in LLMs along the two Inglehart–Welzel axes of the World Values Survey (WVS). By translating social value questions into scenario-based dilemmas, we extract token-level probabilities to measure implicit values and apply activation steering, optionally combined with country-conditioned prompting, to shift model behavior without retraining. Across three 4B-open-weight LLMs, four countries and three social domains, we find substantial variation in steerability. Importantly, we identify latent entanglement, where interventions along one Inglehart–Welzel axis induce a shift along the other. This entanglement is consistent with correlations present in human WVS data and persists across our experimental settings. Our work identifies empirical constraints on cultural alignment in LLMs.

Abstract:
Reasoning prefixes shape the future trajectory of LLM problem solving, yet existing process reward models usually evaluate them through local step correctness. We argue that correctness is a useful but indirect proxy for the effect we ultimately care about: whether a prefix increases the probability of successful completion. We define this effect as prefix gain, the solve-rate improvement induced by conditioning lightweight student model group on a prefix, and use it to train a Prefix Utility Model (PUM) with a simple pairwise ranking objective. PUM learns outcome-grounded prefix utility and can score both complete trajectories and partial reasoning prefixes. Across Best-of-N selection, beam search, and reinforcement learning on mathematical reasoning, PUM provides a strong prefix-level supervision signal, especially when candidate pools are large, search budgets increase, or rule-based rewards are sparse.

Abstract:
Multi-agent debate is widely used to enhance reasoning in language models, yet the exchanged messages are rarely analyzed. We study these messages in a Debate+SFT self-distillation loop by tracking their round-level evolution, intervening on peer messages with T-CIC at inference time, and comparing against matched Solo controls. Across three 5-seed discovery tasks–MATH, GSM-Hard, and 2WikiMultiHopQA–we observe weak refinement on MATH (+0.9pp for Debate versus +9.0pp for Solo), harmful homogenization on GSM-Hard (-11.3pp, with messages 27.5% longer and more self-similar), and mild inflation on 2WikiMHQA (-0.5pp with moderate length growth). A 2-seed T-CIC check on round-4 adapters shows different forms of message dependence: MATH collapses when peer messages are removed, shuffled, or replaced (-16 to -23pp), GSM-Hard is selectively sensitive to mismatched context, and 2WikiMHQA is mostly stable except under cross-problem shuffling. Debate messages should therefore be measured and stress-tested before they are trusted as a training signal.

Abstract:
Multi-agent large language model systems offer a promising paradigm for multidisciplinary medical decision support by integrating heterogeneous expert perspectives. However, existing frameworks often rely on loosely structured interactions, where disagreement is insufficiently diagnosed and discussion may collapse into superficial consensus or parallel opinion aggregation. We propose Structured Adaptive Multi-disciplinary Collaboration (SAMAC), a structured and strategy-aware multi-agent framework for adaptive MDT-style medical reasoning. SAMAC introduces Structured Deliberation to detect opinion disagreement and majority not confident, along with a Consensus Purification mechanism that transforms disagreement into constructive and collaborative refinement. When disagreement arises, SAMAC adaptively selects between Self-refinement and Counter-questioning to enable evidence-grounded negotiation. Experiments on three medical datasets—RADCURE, HANCOCK, and OSCCPI—show that SAMAC consistently improves overall performance and collaborative gains over existing multi-agent baselines. In particular, SAMAC improves the Average Score by 4.29 points on the RADCURE dataset and achieves higher Relative Self-Correction Scores (RSS), indicating more effective resolution of disagreement and stable collective decisions. These results highlight the importance of controlled and diverse interaction dynamics in multi-agent medical reasoning. Code will be released upon publication.

Abstract:
Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and caveats, producing prompts that are 2–3× longer yet no more accurate. We trace this to three deficiencies - incomplete error observation, limited search diversity, and unreliable selection - and propose ESPO (Error-Structured Prompt Optimization), which decomposes prompt optimization into three phases: Diagnose clusters all training errors into structural patterns in one round; Propose generates candidates via four complementary strategies with independent biases; Select applies bootstrap stability selection. On seven public NLP benchmarks - Tweet, MMLU, GSM8K, HotpotQA, ScoNe, HoVer, and PUPA - ESPO wins 6/7 datasets with +3.76% average accuracy over the state-of-the-art (74.67% vs. 70.91% for GEPA), while producing prompts 47% shorter (1,004 vs. 1,878 chars) and faster at inference. Cross-model experiments across four additional student models (Gemma 3 12B, Mistral 14B, Qwen3 32B, Claude Haiku 4.5) show ESPO yields the best average accuracy on every model tested, with the largest gap on Qwen3 GSM8K (15.00% → 91.40%). A generalization bound (Appendix) grounds each phase in a corresponding term of the test-time gap, and the ablation confirms a key prediction: adding diversity without bootstrap selection actually hurts performance (-1.20%).

Abstract:
Graph anomaly detection (GAD) on text-attributed graphs (TAGs) is vital for applications such as fraud detection and academic integrity verification. Existing approaches generally fall into two paradigms. GNN-based methods capture structural patterns but struggle to capture fine-grained textual semantics. Methods integrating LLMs with graphs improve semantic understanding yet fail to fully comprehend topological relationships among neighbors. Moreover, both paradigms overlook the correspondence between textual semantics and graph topological relationships, limiting their ability to identify nodes whose semantics are inconsistent with their neighborhoods. In this paper, we formalize TAG anomaly detection as a node-to-neighborhood semantic consistency problem, where anomalies may arise from either textual semantic mismatch or topological deviation between a node and its neighbors. We propose N2NSC (Node-to-Neighborhood Semantic Consistency), a framework that captures the correspondence between graph topology and textual semantics through two complementary fusion paths. The two pathways work synergistically, enabling the LLM to jointly use textual and structural neighborhood information for anomaly detection. Extensive experiments across eight benchmark datasets, with additional validation on two real-world fraud datasets, demonstrate the effectiveness of N2NSC.

Abstract:
As LLM agents are deployed in long-horizon sessions, context accumulation drives up inference costs. Existing approaches utilize text pruning or dynamic memory eviction to minimize token footprints; however, their unconstrained sequence mutations alter layouts, introducing prefix mismatches and cache invalidation. This reveals a critical trade-off between text sparsity and prompt cache continuity. To address this, we present TokenPilot, a dual-granularity context management framework. Globally, Ingestion-Aware Compaction acts as a framework harness to stabilize prompt prefixes and eliminate open-world environmental noise at the ingestion gate. Locally, Lifecycle-Aware Eviction monitors the ongoing residual utility of context segments, enforcing a conservative batch-turn schedule to offload content segments only when task relevance expires. Experiments on ‘PinchBench‘ and ‘Claw-Eval‘ under both isolated and continuous modes demonstrate that TokenPilot reduces costs by 61% and 56% in isolated mode, and 61% and 87% in continuous mode, while maintaining competitive performance compared to prior systems.

Abstract:
Task-specific LoRA adapters offer a modular way to specialize large language and vision-language models. However, existing adapter composition methods are mostly static and cannot adapt to individual test inputs. To address these issues, we propose LiST, a label-free test-time LoRA fusion framework that converts an existing LoRA bank into a target-conditioned local simplex and searches sample-specific fusion weights at inference time. LiST builds joint task representations from LoRA parameter anchors and prompt-level behavior vectors, retrieves neighboring adapters as a local search space, and performs branch-preserving fusion without updating the backbone or adapters. Candidate weights are selected by a prompt-level energy with prior, geometric, and stochastic-consistency constraints, and are deployed only when they pass a safe acceptance rule. Otherwise, LiST falls back to a target-conditioned prior. Experiments on multimodal and language benchmarks show that LiST outperforms static LoRA merging and conventional test-time adaptation baselines, while preserving task-specific adapter utility and improving robustness on unseen tasks.

Abstract:
We study whether knowledge-gating benchmarks support transferable retrieval decisions or expose benchmark-specific structural regularities that can be exploited without full semantic interpretation. Knowledge gating is the binary decision of whether a dialogue system should retrieve external knowledge at a given turn. We audit this question with a lightweight logistic-regression probe over ten dialogue-metadata features: position, length, and question-form cues. Despite using only metadata and surface signals, the probe reveals non-trivial in-domain signal and markedly different structural patterns across benchmark families. Cross-benchmark transfer provides the clearest evidence: transfer within the related DSTC9/DSTC11 family is near-lossless (Macro F1 0.81–0.84; ROC-AUC ≈ 0.90), whereas transfer across the KETOD–DSTC benchmark-family boundary has weak ranking quality (ROC-AUC 0.41–0.48) and poor detection of knowledge-needed turns (positive-class F1 ≤ 0.22 in one direction). Sentence embeddings and fine-tuned BERT show the same qualitative weakness in the DSTC→KETOD setting: MiniLM ranking is sub-chance (ROC-AUC 0.40–0.44), while BERT remains near chance (0.50–0.53) despite having the strongest KETOD in-domain positive F1. These results suggest that strong in-domain gating scores can partly reflect benchmark-specific regularities and should not, by themselves, be interpreted as evidence of transferable retrieval reasoning. Code and reproducibility materials are available at https://github.com/zilongcheng582-cyber/RAGate-analysis.

Abstract:
Embodied AI and wearable assistants require agents to retrieve sparse evidence from continuous days-level videos. Existing memory-based methods typically follow a passive recording paradigm based on fixed-window captioning and periodic memory construction, which produces redundant memories and inefficient long-horizon reasoning. Inspired by human memory formation, we propose Active Explore Memory Agent (AEMAgent), which transforms passive recording into active memory formation by deciding when to remember, how to explore events, and when to reflect. These decisions are instantiated as a coherent pipeline: Surprise-guided Event Selection detects salient event changes for selective memory writing, Exploratory Episodic Memory Encoding enriches accepted events through self-questioning, Predictive-Coding-Guided Reflection updates knowledge from expectation-observation mismatches, and Semantic Memory Consolidation distills noisy episodic updates into compact long-term memory. Experiments on multiple benchmarks show that AEMAgent achieves the best overall performance among all compared methods while maintaining lower storage overhead than strong full-memory baselines, demonstrating the effectiveness of selective and efficient memory formation for scalable very-long video understanding.

Abstract:
Existing LLM-based molecular optimization methods frame the task as molecule-to-molecule mapping, forcing models to implicitly learn structure-property relationships from paired SMILES alone. We argue this misses a key ingredient: task-level chemical knowledge that tells the model which structural motifs to prefer or avoid for a given property direction, before it ever looks at a specific molecule. We propose SkillMol, which constructs compact task skills – sets of recommended and discouraged structural motifs for each optimization objective – from domain knowledge, and pairs them with an atom-anchored action interface that decomposes editing into discrete, executable operations. Skills are used twice: as explicit training context so the model learns skill-conditioned edit actions, and as functional-group constraints during inference search. A deterministic RDKit executor applies the proposed graph edits. On the MoleculeSTM benchmark across 14 molecular editing tasks, SkillMol achieves 92.5% loose and 80.0% strict hit ratio on single-property tasks and 65.3% average across loose and strict multi-property columns, outperforming strong LLM baselines. Our code can be found at https://github.com/jiajunyu1999/SkillMol.

Abstract:
Large Language Models (LLMs) use KV cache for efficient auto-regressive generation, but its linearly growing memory footprint bottlenecks long-context inference. Existing compression methods typically merge evicted information into retained tokens by additively combining both key and value vectors, despite their distinct roles in the attention mechanism. However, this uniform treatment overlooks the distinct roles of keys and values in attention: perturbations to keys are amplified through the Softmax exponential, whereas perturbations to values remain bounded and linear after Softmax. Based on this observation, we propose a differentiated strategy that evicts keys while merging values. Specifically, we formulate value merging as an MSE minimization problem and derive Expected Contribution Matching (ECM), a closed-form scheme that redistributes evicted values by their historical attention contributions. We further preserve semantic coherence by routing evicted information to semantically similar retained tokens via global semantic matching. Experiments on LongBench and LongBenchV2 demonstrate the effectiveness of our method, which achieves consistent improvements over existing training-free merging-based and eviction-based approaches under the same cache budget. Our code is available at https://github.com/Lpzhan931/ECM.

Abstract:
Long-horizon conversational agents need persistent memory beyond a fixed context window, yet memory effectiveness depends on the units stored and retrieved. Existing systems either discard fine-grained details during compression, retrieve units that are too coarse or context-poor, or fragment discourse into entity-relation triples. We propose Neo-Davidsonian Discourse Units: enriched Elementary Discourse Units (EDUs) that each express one event or fact with normalized entities, temporal grounding, and enough context to stand alone. The resulting units are non-lossy at the session level, atomic for retrieval, and naturally expose event arguments for graph-based associative recall. We instantiate this representation in EMem, which retrieves EDUs with dense representations and relevance filtering based on large language models, and EMem-G, which further organizes sessions, EDUs, and arguments in an event graph and applies Personalized PageRank. On LoCoMo and LongMemEval S benchmarks, our methods improve over prior state-of-the-art by up to 21.3% while using up to 7 × shorter effective context.

Abstract:
This paper studies retrieval-augmented generation (RAG), which leverages external knowledge to improve the performance of language models. Despite the remarkable progress, current RAG approaches are still far from satisfactory due to inadequate retrieval and potential hallucinations. Towards this end, we propose a novel approach named Task Refinement with Individual Optimization (TRIO) for RAG. The core idea of our TRIO is to equip three well-designed agents with an iterative refinement paradigm by aligning their individual optimization objectives. Specifically, we first expand ambiguous queries with a query extension agent and quantify the reward based on the improved answer quality, which can be utilized to refine our agent for higher expansion efficiency. To enhance the retrieval quality, we include a unanimity voting to annotate the expanded query as insufficient or sufficient, and their generative outcomes are utilized as ground truth to optimize the judge agent. To further improve evidence utilization and answer accuracy, an answer agent is optimized with dynamic matching-based rewards with curriculum learning for final outputs. Extensive experiments across multiple benchmark datasets validate the effectiveness of the proposed TRIO in comparison with state-of-the-art approaches. Our code is available at https://github.com/yingtaoren/TRIO.

Abstract:
Multimodal behavioral analysis is typically studied using separate task-specific corpora. Practical applications, however, would benefit from unified models that can support several behavioral predictions despite fragmented supervision. We study this problem as disjoint-corpus multimodal multi-task learning across emotion recognition on CMU-MOSEI, apparent Big Five personality assessment on ChaLearn FIv2, and ambivalence recognition on BAH. We propose Task-Aware Cross-Modal Experts (TACME), a multimodal fusion model with a shared bank of directed cross-modal Transformer experts and task-specific expert gates. Each expert models one ordered modality interaction, while task-specific gates select different expert mixtures for different behavioral tasks. We further adapt feature- and prediction-level flow-matching regularization to this disjoint-corpus behavioral setting and propose a Progress-Aware Gradient Balancing (PAGB) multi-task learning strategy for heterogeneous task optimization. Across three independent corpora and six task-specific metrics, TACME with PAGB and feature- and prediction-level flow-matching regularization achieves the best average rank of 1.00 among the compared multimodal fusion models. Although single-task TACME is stronger on the source-corpus metrics, the multi-task TACME model improves zero-shot unweighted average recall (UAR) on the unseen MELD test set from 28.9% to 35.9%, demonstrating improved generalization to unseen data.

Abstract:
Controllable creative writing is a text generation task that requires both high generation quality and diverse outputs. However, existing methods often improve one aspect at the expense of the other. To address this challenge, we propose SERGE, a reinforcement learning (RL) framework that introduces a diversity-guided self-refinement mechanism, where the policy itself refines low-diversity candidate texts, and employs policy shaping to amplify the gradient contributions of samples with both high generation quality and diversity. We also propose a fine-grained reward modeling framework to provide reliable quality reward signals to support RL optimization. Experiments on the short-form video copywriting generation task show that SERGE outperforms the base model and conventional diversity-enhancement strategies on both diversity metrics (DisNGram, PairBLEU, CosSim) and generation quality, effectively alleviating the "diversity collapse” issue in RL. Our code is available at: https://github.com/peng-mn/SERGE.

Abstract:
In pointwise document reranking, Chain-of-Thought models typically underperform direct scoring models. While existing diagnostics attribute this to inferior classification, score polarization, or calibration breakdown, whether targeted training can bridge this gap remains unclear. Our empirical study first confirms that this gap is stable across scales up to 32B parameters, ruling out model and data capacity confounders. We then apply stress tests utilizing reinforcement learning, fine-grained supervision, and architectural decoupling to explicitly repair these deviations. Although these interventions improve classification accuracy and absolute scores, the relative ranking gap persists. These findings suggest that, within the pointwise scoring paradigm, routing continuous relevance semantics through discrete text constrains ranking signal resolution, revealing a bottleneck that is stable and difficult to overcome under current standard methods, rather than an easily resolvable training bias.

Abstract:
Defenses against jailbreak attacks on Large Language Models (LLMs) operate at different pipeline stages, such as input modification or output guard, but it remains unclear which defenses to deploy at each stage and how to combine them. Prior empirical studies, fragmented by inconsistent attack-success-rate definitions and experimental settings, have evaluated defenses largely in isolation. Here we present the first systematic study, to our knowledge, of defense combinations both within and across pipeline stages, under a consistent threat model of direct, black-box, single-turn attacks. Our decision framework standardizes evaluation through a principled attack-success-rate formulation with controlled query budgets, together with explicit fairness rules. Across 19 attacks and 15 defenses, we find that no single defense is universally best, but well-chosen combinations achieve substantial safety with minimal utility degradation, yielding practical recommendations for layered defense pipelines.

Abstract:
Multilingual language models develop shared cross-lingual representations, and various interpretability methods claim to quantify this sharing. These methods have been developed largely in isolation, and when they disagree, it is unclear whether the disagreement reflects a property of the model or an artifact of the measurement. We compare four sharing metrics—CKA, ANC, GMM dominance per token, and ILO—across 21 base models from five families (125M–14B parameters) and correlate each with cross-lingual transfer on five downstream tasks. We find that the metrics differ in their quantification of cross-lingual sharing in these models and suggest that the disagreement traces to anisotropy, the tendency of representations to cluster in a narrow cone of the embedding space. Only ILO’s correlation with cross-lingual transfer (Spearman’s ρ = 0.90) survives controls for model size, family, and per-task variation. We therefore recommend ILO as the primary sharing metric, to be reported alongside anisotropy diagnostics.

Abstract:
Despite rapid advances in Code LLMs, their training datasets are often opaque due to proprietary and competitive concerns. Since many evaluation benchmarks are open-source and widely available online, models may unintentionally train on them, leading to data contamination that compromises evaluation validity. Reliable techniques for detecting whether a code snippet appears in a model’s training data are therefore essential. Compared with natural language, code contamination detection presents two unique challenges. First, code tokens from different syntactic categories contribute unequally to contamination signals, making uniform token treatment ineffective. Second, models trained on semantically equivalent code variants can evade detectors based solely on text matching. To address these challenges, we propose a syntax-aware framework that explicitly models token-type contributions. With a lightweight supervised detector trained on few labels, it captures contamination signals efficiently. Experiments across 8 Code LLMs, 2 languages, and 4 code transformations show absolute AUC gains of 8–10% for pretraining data detection and 15–20% for fine-tuning data detection, with AUC exceeding 90% on most fine-tuned tasks. Our work demonstrates that syntax-aware modeling is essential for trustworthy Code LLM evaluation.

Abstract:
Modern translation workflows demand more than semantic equivalence. Users routinely require models to preserve JSON or HTML schemas, honor curated glossaries, disambiguate with provided context, and match prescribed registers, often several at once. Conventional metrics such as BLEU and xCOMET capture semantic fidelity but provide little signal on constraint adherence, while general instruction following benchmarks ignore the cross-lingual nature of translation. We introduce IFMTBench, a benchmark for multilingual translation instruction following covering seven high-resource languages, with 4,506 single-constraint and 2,838 multi-constraint items spanning seven constraint dimensions and five compositional patterns with instructions issued in all seven languages. Constraints are split into a gating subset verified by deterministic checkers and a continuous subset scored by a rubric-based LLM judge, combined under a multiplicative rule that resists reward hacking. Evaluating 15 models reveals systematic gaps that prior protocols miss: Instruction following scales with size more sharply than translation quality, glossary and structured-format constraints dominate the difficulty gradient, and general instruction following rankings correlate only weakly with translation behavior. Our benchmark is available at https://github.com/Tencent-Hunyuan/Hy-MT2/tree/main/IFMTBench.

Abstract:
Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus mixtures or traced specific token groups from released tokenizer vocabularies; in contrast, we estimate corpus ratios for arbitrary target tokens. We first show that BPE tokenizers trained on different corpora share stable token ID–ratio distributions, motivating distribution transfer from known corpora to a target tokenizer trained on hidden corpora. We then propose Quantile-Guided Density Estimation (QGDE), which approximates this distribution with multiple quantile trends and uses local density weighting to produce token-level estimates. In controlled settings and a realistic setting using the released SmolLM tokenizer, QGDE achieves mean relative errors as low as 3.00% for token-level estimation and 3.08% after aggregation into category-level mixtures. These results suggest that released tokenizer vocabularies provide a useful signal for fine-grained corpus estimation beyond coarse composition inference.

Abstract:
Large language models (LLMs) are increasingly used to generate or assist reviews. This raises the risk of generative monoculture, where models converge on homogeneous judgments and language patterns, limiting the novelty of scientific progress. We study this phenomenon over 1,943 human reviews from 509 ICLR papers with 10,172 reviews generated by five state-of-the-art LLMs under multiple prompting strategies. We evaluate human and LLM review diversity across several dimensions, including praise-to-critique balance and semantic similarity. Across metrics, LLM-generated reviews exhibit systematic convergence: for example, they are substantially more praise-heavy, and more semantically similar across unrelated papers, suggesting a lack of paper-specific feedback. Specialized review agents reduce—but do not eliminate—these effects. Our work demonstrates the risk of introducing epistemic monoculture into peer review and develops methods to measure it. If left unchecked, this monoculture could narrow the range of perspectives that shape scientific evaluation.

Abstract:
Recent controllable text generation (CTG) for sentiment control has largely focused on decoder-based large language models, making causal attention the dominant paradigm. While effective for fluent generation, these models still struggle to satisfy complex constraints and follow fine-grained sentiment signals specified by users. Existing sentiment-aware CTG methods typically simplify the problem by treating sentiment either as a coarse categorical label (e.g., positive or negative) or as a single fine-grained control signal applied to an entire document. Consequently, more challenging settings such as sentence-level sentiment control within long-form text remain underexplored. To address these limitations, we introduce SenseShift, an encoder-based framework for fine-grained sentence-level CTG. Unlike standard decoder architectures, SenseShift leverages bidirectional attention, quantized sentiment signals, and iterative mask infilling to generate local sentences conditioned on target sentiment intensity. Empirical evaluations on story and review generation demonstrate that SenseShift achieves stronger sentiment controllability while maintaining text quality and robustness to out-of-domain generation compared to larger decoder-based baselines.

Abstract:
Recent Hypergraph-based retrieval-augmented generation (HyperRAG) methods use hyperedges to connect multiple entities simultaneously, enabling more efficient multi-entity evidence organization than pairwise graph structures. However, existing HyperRAG methods often rely on LLM-generated summaries to construct hyperedges, which can introduce hallucinations while also incurring high indexing costs. In addition, during retrieval, existing methods typically rely on either one-hop neighbor expansion or PageRank diffusion. The former may miss useful multi-hop evidence, while the latter can suffer from uncontrolled propagation over excessive hub nodes, leading to semantic drift and noisy reasoning chains. To address these challenges, we propose HyperSU, a novel hypergraph-based RAG framework featuring semantic-unit hyperedges and clue-guided bidirectional retrieval. During construction, HyperSU formulates hyperedge construction as an entity-aware minimum-description-length (MDL) optimization problem, inducing source-grounded semantic-unit hyperedges that balance sentence-level semantic coherence and entity compactness. It then constructs a hypergraph by modeling each semantic unit as a hyperedge over its co-mentioned entities. During retrieval, HyperSU performs clue-guided bidirectional expansion over the semantic-unit hypergraph, enabling both multi-hop evidence discovery and evidence-anchored noise reduction. Experiments show that HyperSU consistently improves answer accuracy over standard, graph-based, and hypergraph-based RAG baselines, raising average accuracy on GraphRAG-Bench by 5.5 points over the strongest baseline (68.7 vs. 63.2), with larger gains on reasoning-intensive tasks, where the relative improvement reaches 14.1%. Our code is available at https://github.com/RAIDSLab/HyperSU.

Abstract:
Safety in large language models is often framed as requiring a single correct response, even in safety-sensitive contexts where multiple perspectives may exist. Existing approaches largely ignore Overton pluralism (inclusion of multiple reasonable viewpoints for a single query) and reduce complex context to monolithic judgments that fail to capture the range of reasonable viewpoints. Moreover, existing frameworks are not grounded in safety settings, which limits their ability to capture the diversity of perspectives required in safety-critical situations. Existing safety datasets do not provide Overton-style annotations, limiting their ability to represent multiple harm perspectives within the same safety-critical situation. To address this, we construct SOS, the safety-focused Overton dataset of 1,770 real-world safety situations annotated across 14 harm taxonomy. We further propose PDOS, a lightweight, training-free framework that retrieves personas from a synthetic pool and integrates their perspectives to generate unified Overton safety responses. PDOS achieves the highest average NLI coverage of 0.4226 among several baselines, and outperforms the strongest baseline by 13.3%. Our findings show that Overton-based safety pluralism requires adaptable, context-aware approaches, providing insights into how these methods can better capture diverse perspectives.

Abstract:
Large language models (LLMs) have shown strong knowledge reserves and task-solving capabilities, but still face the challenge of severe hallucination, hindering their practical application. Though scientific theories and rules can efficiently direct the behaviors of human manipulators, LLMs still do not utilize these highly-condensed knowledge sufficiently through training or prompting. To address this issue, we propose SciDC, an LLM generation method that integrate subject-specific knowledge with strong constraints. By adopting strong LLMs to automatically convert flexible knowledge into multi-layered, standardized rules, we build an extensible framework to effectively constrain the model generation on domain tasks. Experiments on scientific tasks including industrial formulation design, clinical tumor diagnosis and retrosynthesis planning, consistently demonstrate the effectiveness of our method, achieving a 12% accuracy improvement on average compared with vanilla generation. We further discuss the potential of LLMs in automatically inductively summarizing highly-condensed knowledge, looking ahead to practical solutions for accelerating the overall scientific research process. All the code of this paper can be obtained.

Abstract:
AI agents combine LLMs with external tools to solve complex tasks. While tools extend agents’ capabilities, improper permissions introduce risks such as indirect prompt injection and tool misuse. We characterize these failures as unbalanced tool-driven agency, a unified view in which agents either retain unnecessary permissions (excessive agency) or fail to invoke required tools (insufficient agency), increasing risk and reducing performance. We introduce AgenTRIM, a framework that addresses this imbalance without altering agent reasoning, through complementary offline and online stages. Offline, it reconstructs the agent’s tool interface from code and validates tool behavior via controlled executions and trace analysis. At runtime, it enforces per-step least-privilege access through adaptive capability filtering and isolated validation of tool calls, implicitly decomposing tasks without explicit planning. On the AgentDojo benchmark, AgenTRIM substantially reduces attack success while improving task performance over the baseline, unlike other leading defenses. To assess robustness beyond prompt injection, we conduct additional experiments covering description-based attacks and policy enforcement. Together, these results establish AgenTRIM as a general, capability-preserving framework for mitigating tool-driven risks in LLM-based agents.

Abstract:
Prompt-injection detectors often fail on cross-prompt-injection attacks (XPIA), where the adversarial instruction is hidden inside a document, email, webpage, or tool output. This paper argues that the failure is not only a distribution-shift problem, but also a representation-level problem. Inside the hidden states of frozen detector backbones, user prompt-injection attacks (UPIA) and XPIA are encoded along distinct, weakly aligned directions. A detector whose readout is primarily aligned with the UPIA direction can therefore miss the XPIA signal, even though both signals are present in the same model’s representation. A two-axis linear-discriminant probe recovers this missing signal: UPIA attacks fire the UPIA axis in 99.9% of caught cases, while XPIA attacks fire the XPIA axis in 97.3% of caught cases. Axis ablations and activation-level steering suggest that these directions are not only predictive but may also be behaviorally relevant under controlled perturbations. The geometry persists across model families, training stages, scales, and external prompt-injection benchmarks, suggesting that XPIA should be treated as a distinct threat axis rather than as a harder version of UPIA.

Abstract:
Large Language Models with reasoning capabilities have recently shown strong promise for medical Question Answering. Current approaches report competitive results on English benchmarks through knowledge distillation from large models, yet multilingual coverage remains limited. We extend this line of research by presenting a method to generate multilingual reasoning traces that enrich the distillation paradigm with factual information from Wikipedia. Using a retrieval-augmented generation pipeline over WikiProject Medicine, we produce 500k reasoning traces in English, Italian, and Spanish. The traces are generated to solve medical questions drawn from MedQA and MedMCQA, both of which we extend to Italian and Spanish. We evaluate our approach across in-domain and out-of-domain Medical QA benchmarks, demonstrating consistent performance gains in both few-shot in-context learning and supervised fine-tuning settings, achieving state-of-the-art results among 8B-parameter models across multiple model families. With the aim of fostering the development of multilingual clinical decision-support tools, we release the complete suite of resources: reasoning traces, translated datasets, Medical Wikipedia, and fine-tuned models.

Abstract:
Tool-integrated reasoning has emerged as a promising paradigm for Text-to-SQL, framing SQL generation as sequential decision-making interleaved with tool execution. However, existing reinforcement learning approaches rely on coarse-grained outcome supervision, creating a credit assignment problem: models receive identical rewards regardless of whether intermediate steps are redundant or erroneous. We observe that tool-integrated reasoning offers a unique opportunity: each tool invocation produces structured, verifiable outputs that can serve as training-free step-level signals, eliminating the need for expensive reward model training. Building on this insight, we propose FineStep, a step-level credit assignment framework that leverages training-free process rewards derived from tool execution feedback. FineStep constructs a dual-channel value estimate for each step, combining backward-propagated outcome value with forward-smoothed process value, and synthesizes them via group-based normalization into a unified step-level advantage for policy optimization. Experiments on Text-to-SQL benchmarks show that FineStep achieves SOTA performance with a 3.25% EX gain over GRPO at the 4B scale, while results on AIME24 demonstrate transferability beyond SQL.

Abstract:
Recent advances in large language models have accelerated open-vocabulary EEG-to-imagined-text decoding, where non-invasive neural activity recorded during visual perception is translated into coherent natural language descriptions of viewed stimuli. However, existing systems remain highly vulnerable to biological noise, where corrupted neural projections induce hallucinated or semantically unstable generation in frozen language models. We introduce SYNAPSE (Symbolic Neural Alignment for Precise Semantic Extraction), a lightweight neuro-symbolic framework that stabilizes neural text generation through inference-time symbolic regularization. By purifying EEG-derived semantic candidates using commonsense graph structure and latent exemplars, SYNAPSE improves semantic stability without end-to-end LLM fine-tuning. Experiments across popular EEG decoding benchmarks and multiple frozen LLM backends demonstrate consistent gains over unconstrained prompting baselines, robustness under object-label ablation, and performance commensurate with substantially more resource-intensive fine-tuned systems, while preserving biometric privacy by localizing raw EEG processing entirely within the encoder stack.

Abstract:
Browser-use agents require seamless alignment between structured web metadata and visual information, while preserving relevant context across long interactions. Existing interfaces often rely on either screenshot-level action prediction or static Set-of-Marks overlays, leaving the model to resolve dense DOM-pixel alignment before every operation. We introduce Probe to Act (P2A), an active probing framework for the browser-agent loop that moves this alignment into decision time. P2A addresses an asymmetric bridge between symbolic DOM hypotheses and screenshot layout by rendering on-demand symbolic DOM structure back into pixels. Before committing a state-changing browser operation, the agent can issue lightweight probes to translate DOM handles into pixel evidence, map screen regions back to DOM candidates, register visual-only targets, and commit verified notes. These interleaved processes naturally produce evidence-based memory: only probed, acted-on, or explicitly committed observations are kept across steps, preserving only decision-critical evidence in long-horizon contexts. P2A can be used as a prompting strategy for proprietary models under the standard DOM+SoM interface, and can be distilled into open-weight models through cold-start synthesis and self-bootstrapped SFT. Across three browser-use benchmarks, P2A shows clear gains on task success rate for both proprietary and fine-tuned models; on VisualWebArena, for example, it improves Gemini-3-Pro from 54.1% to 61.2% and Qwen3-VL-8B from 24.6% to 32.9%, while matching the costly full-observation history (3̃×) at near context cost of action-only history (1̃.2×).

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has been central to the recent success of Large Reasoning Models. However, while RLVR significantly improves single-sample accuracy, it often fails to expand the model’s intrinsic reasoning coverage (pass@ k ) due to limited exploration during training. To address this, we optimize the structural design of train-time rollouts to enhance pass@ k . Our analysis identifies three key design principles: (1) difficulty-adaptive rollout can play an important role in expanding pass@ k , beyond serving as an efficiency heuristic; (2) tree-based rollout outperforms parallel sampling in discovering correct answers; and (3) sentence-entropy-guided forking overcomes the localization phenomenon of token-level branching to maximize semantic diversity. Building on these insights, we propose DATPO (Difficulty-Adaptive Sentence-entropy-guided Tree-structured Policy Optimization). DATPO integrates difficulty-adaptive tree search with a sibling-diversity advantage term, explicitly promoting semantic diversity to expand reasoning coverage during training. Experiments on mathematical reasoning benchmarks demonstrate that DATPO outperforms baselines especially in pass@ k , which directly translates to superior test-time scaling performance.

Abstract:
ision-Language Models (VLMs) demonstrate exceptional visual reasoning capabilities, yet their inference costs escalate rapidly with the proliferation of visual tokens. Existing visual token pruning methods exhibit two fundamental limitations. First, most approaches operate exclusively post-vision encoder, leaving the substantial latency of the visual encoding phase unoptimized. Second, under strict token budgets, these methods often fail to jointly preserve holistic visual contexts and fine-grained details, leading to performance degradation. To address these bottlenecks, we propose PACE (Pixel-Adaptive Condense and Extract), a training-free inference framework that accelerates both the vision encoder and the Large Language Model (LLM) via a unified Condense-and-Extract paradigm. During the Condense stage, an Adaptive Pixel Compressor (APC) evaluates visual information density prior to encoding, adaptively downsampling redundant inputs, curtailing encoder computation while preserving global context and essential visual cues. In the Extract stage, a Dynamic Dual-Attention Extractor (DDAE) selectively retains visual tokens via a fusion of internal visual signals from the encoder and semantic signals from the LLM, safeguarding task-critical details. By integrating PACE into Qwen2.5-VL-7B, the model retains 93.8% of its original performance while utilizing only 10% of the visual tokens, yielding a 3.1× speedup in time to first token (TTFT). Our code is available at https://github.com/jjL357/PACE

Abstract:
While dense biomedical embeddings achieve strong performance, their opaque dimensions limit transparency in biomedical NLP. Recent question-based interpretable embeddings represent text through binary answers to natural-language questions, but existing approaches rely primarily on corpus-driven signals, often capturing topical or stylistic differences rather than fine-grained biomedical distinctions. We propose QIME, an ontology-grounded framework for interpretable biomedical text embeddings in which each dimension corresponds to an explicit biomedical-domain yes/no question. QIME leverages a biomedical ontology to guide contrastive question generation from semantic clusters, producing atomic, domain-grounded questions. It constructs embeddings via similarity-based semantic activation with MMR-based diversity-aware dimension selection, yielding sparse representations efficiently. Experiments on biomedical clustering, STS and retrieval benchmarks show that QIME consistently outperforms prior interpretable embedding methods and substantially narrows the gap to strong black-box biomedical encoders. It also imposes substantially lower cognitive burden than existing methods, providing concise and domain-specific interpretations. The code is available at https://github.com/L1nzh/QIME.

Abstract:
Long-document understanding often requires reasoning over many visually rich pages, making inference costly and prone to context rot. In this work, we propose InSight-doc, an agentic visual perception framework that treats visual resolution as an adaptive reasoning-time resource. InSight-doc starts from low resolution and selectively zooms into high-resolution regions for finer evidence, without relying on any external retriever. To train such an agent, we construct an active-perception corpus of 17.9K high-quality SFT examples with region-level zoom-in trajectories, accompanied by 19.2K hard RL examples. Through SFT+RL, InSight-doc-8B improves the baseline by 4.3–16.4 accuracy points over document VQA benchmarks. On long documents, it reduces hallucination by more than 40% and inference latency by 41%–68% while maintaining an accuracy lead. Our code, datasets, and model are released at https://github.com/m-Just/InSight-doc.

Abstract:
We present a controlled empirical study of auxiliary supervision for low-resource emotion classification. Using a shared-encoder multi-task learning framework, we compare three auxiliary objectives - coarse affective valence, valence-activation structure, and positive-affect detection - across three supervision sources: aligned labels deterministically derived from gold emotion annotations, rule-based weak labels generated with VADER, and synthetic labels generated by Qwen2.5-14B. We evaluate on MELD, GoEmotions, and DAIR-AI Emotion under matched training procedures and multiple data regimes. Auxiliary supervision yields its clearest, though modest, gains in low-resource settings, especially on MELD and GoEmotions, while improvements on DAIR-AI Emotion are smaller and less consistent. Error and per-class analyses suggest that gains concentrate in semantically overlapping or low-frequency categories such as anger/disgust, fear/sadness, remorse, and embarrassment. We further find that agreement with aligned auxiliary labels is only weakly associated with downstream transfer, suggesting that useful auxiliary supervision need not exactly reproduce gold-derived affective mappings. Overall, our results support auxiliary supervision as a lightweight inductive bias for low-resource emotion classification, while also showing that its benefits are dataset-dependent and sensitive to auxiliary-task design.

Abstract:
Object hallucination remains a primary obstacle to the reliable deployment of Multimodal Large Language Models (MLLMs). Current inference-time mitigation methods mainly assume that hallucinations stem from visual neglect and therefore steer models toward greater visual reliance. In contrast, our systematic interventions on multiple MLLMs show that increasing visual reliance may exacerbate hallucinations on some models, whereas reducing it may mitigate them. This result suggests that visual neglect alone provides an incomplete explanation of hallucination. We characterize the image as a context that competes with both the model’s parametric knowledge and its textual context. Based on this characterization, we propose a training-free framework, Context-Preference Activation Steering (CAS). CAS extracts two Context Preference Vectors (CPVs), designed to capture different context-preference relationships, from two small sets of conflict samples and injects signed residuals into the MLP-sublayer outputs before residual addition at four mid-early layers in a single forward pass. Experiments show that CAS substantially mitigates object hallucinations with negligible decoding overhead while maintaining F1, lexical diversity, and repetition rates close to those of the base models. Code and data are available at https://github.com/wss7/CAS.

Abstract:
The mid-training phase has emerged as a critical bridge for enhancing the reasoning capabilities of Vision-Language Models, yet its effectiveness is constrained by the scarcity of high-quality visual reasoning data. While data synthesis offers a promising alternative, conventional stateless synthesis paradigms produce shallow trajectories that quickly succumb to utility plateaus. To overcome this, we propose VisMind, a novel multi-agent collaborative framework that generates deep-reasoning Stateful Synthetic Dialogue (SSD) from visual inputs. By orchestrating iterative problem-solving, where peer student agents engage in bidirectional critique under the ground-truth-anchored guidance of a teacher, VisMind transforms static, fragmented responses into rich, multi-turn logical derivations. Utilizing this interactive paradigm, we construct a dense mid-training corpus of approximately 1.14B tokens. Extensive experiments across the Qwen2-VL and InternVL3 model families demonstrate that applying this corpus via Continued Pretraining consistently outperforms a representative stateless synthesis approach and elevates standard baselines by up to a 2.24% absolute average gain across comprehensive mathematical and logical benchmarks. Furthermore, in-depth analysis reveals that the SSD paradigm exhibits continuous linear scaling without early saturation, fosters intrinsic reasoning generalization, and establishes a robust cognitive foundation that enhances the efficacy of subsequent Reinforcement Learning with Verifiable Rewards.

Abstract:
Backdoor attacks are an important safety risk for large language models (LLMs), yet existing studies often rely on manually crafted triggers and fixed poisoning pipelines. This limits the diversity of attack behaviors and makes it difficult to systematically evaluate defense robustness. We introduce AutoBackdoor , an agent-driven framework for scalable LLM backdoor red-teaming. Given a target scenario, AutoBackdoor automatically generates triggers, constructs poisoned samples, refines them through self-feedback, and produces backdoored models through fine-tuning. We evaluate the framework across four realistic scenarios: Bias Recommendation , Hallucination Injection , Error Code Generation , and Peer Review Manipulation . Experiments on LLaMA-3, Mistral, Qwen, and GPT-4o show that AutoBackdoor can achieve over 90% attack success with only a small number of poisoned samples. We further find that existing defenses provide limited protection against these agent-generated attacks. These results suggest that automated red-teaming can serve as a useful tool for stress-testing LLM backdoor defenses and understanding emerging risks from agent-driven model customization pipelines. Our code is available at https://github.com/bboylyg/AutoBackdoor .

Abstract:
Knowledge editing aims to update specific facts in language models to ensure reliable and up-to-date behavior. A key challenge is whether edited knowledge generalizes beyond the format in which the edit is applied, yet existing evaluations primarily assess generalization using paraphrases. In this work, we identify a critical blind spot: generalization across heterogeneous task formats, including multiple-choice and verification-based question answering. We show that paraphrase-based evaluations overestimate robustness, as models that perform well on paraphrases exhibit substantial performance drops across task formats. While model predictions are consistent across formats prior to editing, editing introduces format-dependent inconsistencies. We further observe that heterogeneous task formats exhibit distinct decision landscapes, where single-format objectives lead to format-specific local optima with divergent edit directions, resulting in misaligned outputs across formats. To address this issue, we demonstrate that aligning edit directions across task formats improves cross-format generalization and consistency. These findings highlight the need to move beyond paraphrase-based evaluation and to develop editing methods that maintain consistent factual behavior across diverse task formulations.

Abstract:
LLMs are transforming customer service, yet evaluating service agents across diverse business scenarios remains costly and fragmented, as each new domain requires building a dedicated benchmark from scratch. Existing benchmarks rely on static, scenario-specific scripts with two fundamental limitations: Scenario Silos, where each domain demands costly independent annotation with no shared evaluation structure, and Dialogue Dynamics Gap, where fixed scripts fail to capture the full range of real user behaviors. We propose SAGE (Service Agent Graph-guided Evaluation), to our knowledge the first scenario-extensible benchmark to jointly assess logical SOP compliance and conversational quality, abstracting any SOP as a directed graph and bridging Scenario Silos through a lightweight two-step extension mechanism. SAGE further enables dynamic multi-turn simulation via adversarial user agents to overcome the Dynamics Gap. Across six industrial scenarios representative of real-world customer service deployment, we evaluate 27 mainstream LLMs spanning both closed-source systems and open-source families, offering a fine-grained capability map to guide real-world model selection. Our experiments further reveal two systematic failure patterns shared across all tested models: while models generally succeed at recognizing user intent, they frequently struggle to follow complex business logic and execute the correct subsequent action; and they sustain a warm and empathetic conversational tone even when their underlying procedural reasoning has already failed. Code and resources are available at: https://github.com/lingshi6565/SAGE-Bench.

Abstract:
This paper studies the problem of uncertainty-aware reasoning over knowledge graphs (KGs), which aims to incorporate uncertainty quantification into unseen knowledge predictions over KGs. Previous works typically incorporate uncertainty measurement tools such as conformal predictions into graph representation learning. Despite the progress, the performance of these approaches remains limited since they usually neglect both structural information in KGs and intrinsic asymmetry between over-estimation and under-estimation. Towards this end, we propose a novel approach named Context-anchored Asymmetric Conformal Prediction for uncertainty-aware REasoning over KGs (CARE). The core of our CARE is to identify the context of each query using anchors, which guide the construction of triple-specific asymmetry prediction intervals. In particular, we first take the subset of the calibration set as a reference pool. For each triple query, we summarize the KG structural statistics into a vector, which guides the identification of its anchor samples from the pool. The confidence residuals between query and anchors are utilized to quantify over-estimation and under-estimation, producing contextual nonconformity scores. Finally, we utilize the unselected calibration dataset to determine cutoff levels for constructing asymmetric prediction intervals. Extensive experiments on benchmark datasets validate the effectiveness of CARE. The code can be found at https://github.com/fanboyang/CARE.

Abstract:
Multi-Agent Systems (MAS) have recently moved from static workflows toward dynamically generated collaboration topologies. However, existing topology generation methods rely primarily on the parametric knowledge of large language models, with external search or retrieval used only as a reactive tool rather than an explicit determinant of collaboration structure. This leads to structure–knowledge misalignment, where systems exhibit redundant interactions or insufficient verification in knowledge-intensive tasks. We propose K-GAT (Knowledge-Guided Agent Topology Generator), a neuro-symbolic framework that formulates collaboration topology design as a knowledge-conditioned structure learning problem, integrating external evidence directly into autoregressive graph generation. Extensive experiments on knowledge-intensive benchmarks demonstrate K-GAT’s efficiency and effectiveness. Notably, on the expert-level GPQA dataset, K-GAT outperforms the LLM-Debate baseline by a substantial margin of +15.7% in accuracy, while consuming less than half the computational tokens.

Abstract:
Large Language Models (LLMs) are increasingly evaluated with input attribution methods, yet comparing such explanations remains challenging. Existing soft-perturbation faithfulness metrics, such as Soft-NC and Soft-NS, can conflate attribution quality with the number of words retained during perturbation: attribution methods with larger average scores may keep more words and therefore obtain inflated scores. To address this issue, we propose 𝜋 -Soft-NC and 𝜋 -Soft-NS, an evaluation framework that compares attribution methods under the same expected retaining probability, thus controlling the number of retained words. We further introduce Grad-ELLM, a gradient-based attribution method tailored to autoregressive decoder-only LLMs, which combines gradient-derived channel importance with attention-derived token importance at each decoding step. Experiments on classification and open-generation tasks with Llama and Mistral show that Grad-ELLM achieves strong comprehensiveness-oriented faithfulness under 𝜋 -Soft-NC, while there is no dominant method under 𝜋 -Soft-NS. Our evaluation metric serves as a controlled evaluation framework to compare XAI methods for LLMs, which will support progress in the field.

Abstract:
Text-guided medical image editing must satisfy the requested pathology while preserving anatomy, modality-specific appearance, and clinical plausibility. However, existing datasets largely supervise editors with final accepted edits and discard the failed attempts produced during generation. We argue that these failures provide essential supervision for quality control: they specify what should be rejected, why an edit is medically or visually invalid, and how the instruction should be revised. We present Med-Banana, a trajectory-supervised framework for quality-controlled medical image editing. We introduce Med-Banana-80K, a large-scale resource of success-and-failure editing trajectories with candidate images, verification outcomes, rejection reasons, and prompt refinements. Building on it, Med-Banana jointly trains an editor, verifier, and refiner, enabling edit–verify–refine inference from accepted and rejected attempts. Experiments across MLLM judges, blind expert assessment, source-preservation and real–synthetic separability probes demonstrate consistent improvements over open medical image editors. Code and data are publicly available.

Abstract:
Large Language Models often generate incorrect factual answers, yet it remains unclear how they internally distinguish facts they know from those they do not. Existing hallucination-detection methods largely rely on behavioral signals or opaque scalar scores, leaving the structure of factual knowledge in hidden state space poorly understood. We address this gap by introducing a coarse-to-fine framework to discover a knowledge boundary: a decision surface in transformer hidden states that separates representations leading to correct factual answers from those leading to errors. Across multiple datasets and model families, we show that factual correctness is predictable from hidden states, and that this signal is concentrated in specific layers and a small number of influential dimensions. Using Qwen3-8B as a case study, we further identify a compact "knob" subspace that retains near-full boundary-prediction performance and supports targeted interventions that bidirectionally modulate the likelihood of the full reference answer under teacher forcing.

Abstract:
Scientific domain entity linking (EL) differs from general domain EL because mentions and entity names often lack lexical overlap. Another challenge is that specialized terminology is used in the scientific domain, which is rarely encountered in models pretrained on general domains. Therefore, models trained on general domains transfer poorly to scientific domains. To address this, in-domain fine-tuning is the natural remedy. However, many scientific domains lack expert-annotated data, motivating the need for a zero-human-annotation approach. Existing zero-shot methods heavily rely on LLMs to generate aliases across entire mention corpora, which incurs substantial computational cost, and those methods provide no mechanism to filter out noise from LLMs. To address these challenges, we propose Sci-ZSEL, a framework that selectively generates entity aliases with an LLM to control computational cost, and applies an ontology-aware filter to remove aliases that semantically drift toward ontology neighbors. Then, filtered aliases are used to construct pseudo-labeled mention-entity pairs for fine-tuning. To enable evaluation of EL under low lexical overlap, we also release a new animal science EL benchmark linked to three livestock trait ontologies, where mentions and entities exhibit substantially lower lexical overlap than in existing benchmarks. Across five benchmarks, Sci-ZSEL outperforms the non-fine-tuned baseline, is most useful on nonoverlapping mentions, and combining it with curated synonyms gives the best performance in most settings.

Abstract:
Generating the first postoperative outpatient note requires selecting which information from heterogeneous clinical records should be reflected in the final note, rather than merely summarizing all inputs. We evaluate this task on an original 170-case set of de-identified clinical cases collected at Seoul National University Hospital (SNUH), involving two surgical divisions and five physicians. We then extend the evaluation to 1,329 cases across 10 surgery groups within the same institution to examine whether the main trends persist at a larger scale. We introduce physician- and surgery-specific same-surgery references: structured reflection maps that capture how information is selected and rewritten in prior same-surgery cases. Experiments show that these references become most effective when carried into the merge stage, suggesting that the main bottleneck is deciding what to reflect across records. We further show that cross-patient references can induce unsupported transfer from prior cases. To analyze this trade-off, we use a human-validated two-source FactScore that checks each generated fact against both the source records and the physician-written note. A lightweight grounding constraint reduces hallucination while maintaining or improving alignment with physician-written notes across divisions and LLMs.

Abstract:
Recent advances in reinforcement learning–based fine-tuning of multimodal large language models (MLLMs) have highlighted the importance of entropy control for stable optimization. While entropy has been extensively studied in reasoning-centric settings, its behavior and optimization role in perception-oriented tasks remain insufficiently understood. In this work, we reveal that entropy exhibits fundamentally different roles in perception tasks compared to reasoning, particularly under continuous prediction and reward. Using visual grounding as a case study, we show that entropy can remain misaligned with optimization objectives due to persistent low-probability yet high-advantage samples. Building on these findings, we introduce ECVGPO (Entropy Control Visual Grounding Policy Optimization), an interpretable algorithm designed for effective entropy regulation. Through entropy control, the trade-off between exploration and exploitation is better balanced. Experiments show that ECVGPO achieves broad improvements across various benchmarks and models.

Abstract:
Continual pre-training (CPT) adapts a pretrained language model to a target domain, but it often degrades general capabilities even when general-data replay is used to mitigate forgetting. We revisit this failure through replay supervision: prior CPT replay mainly tunes data mixtures and token budgets while applying one-hot next-token prediction (NTP) uniformly to domain and replay tokens. This target is natural for learning domain text, but on replay tokens it introduces target variance: it is a noisy, corpus-dependent proxy for preserving the frozen base model’s conditional behavior. We study self-distilled replay: domain tokens keep NTP, while replay tokens are trained to match the frozen base model distribution. Diagnostics show that this target substitution lowers target error, produces more directionally consistent replay gradients, and reduces base-behavior drift. Empirically, self-distilled replay improves the SmolLM3 domain–general trade-off over matched NTP replay across diverse token budgets and replay ratios. It also remains effective for Qwen3 models whose pretraining mixtures are unavailable: under the same mismatched, non-source replay setting, it improves both general score and PubMedQA in all eight matched comparisons. Overall, these results suggest supervising CPT replay to preserve base behavior, rather than fit empirical next tokens, as a way to reduce target variance.

Abstract:
Users often ask large language models (LLMs) to report how confident they are, but it is unclear whether such linguistic confidence tracks the model’s internal confidence. We study this question across 8 classification tasks, 2 generation tasks and 30 models from three families. For classification, we compare linguistic confidence with logits-based confidence along three axes: association, magnitude agreement and calibration. For generation, we test whether linguistic confidence tracks semantic-entropy-based uncertainty. The axes frequently diverge. Instance-level association is weak on average, although it improves on easier items and for stronger base models. Instruction-tuned models often report higher confidence and sometimes show higher association, but they also have larger confidence gaps and worse calibration. Prompt design mostly changes the distribution of reported confidence. Attitude cues inflate confidence without improving alignment, while score exemplars can preserve rank-order signal when they avoid collapsed confidence values. Regression analyses show that distributional properties of confidence scores explain much of the observed alignment pattern, with model metadata playing a smaller role after controls. These results support a lossy-channel view of linguistic confidence. A more dispersed verbal confidence distribution can carry useful rank information, but it does not make the scores calibrated. Linguistic confidence should therefore be evaluated with multi-axis diagnostics before being used in downstream reliability pipelines.

Abstract:
Structured knowledge fact checking aims to determine the truthfulness of natural language claims by reasoning over structured evidence. Recent program-generation approaches leverage large language models (LLMs) to generate executable graph reasoning programs, achieving strong performance on structured knowledge fact checking benchmarks. However, these methods remain limited by invalid relation generation, single-path reasoning that lacks self-correction, and biased evidence assessment that tends to overestimate supporting signals. We propose Dialectical Agentic REasoning (DARE), a multi-agent framework that formulates structured knowledge fact checking as an iterative retrieve-reason-reflect process. DARE integrates relation-grounded evidence retrieval to constrain reasoning to valid structures, dialectical bidirectional verification to evaluate evidence from both supporting and refuting perspectives, and confidence-driven meta-reflection to dynamically determine whether additional evidence exploration is necessary. Extensive experiments demonstrate the effectiveness of DARE in structured knowledge fact checking, with an 8B backbone achieving 88.12% accuracy and matching or surpassing GPT-4o-based program-generation baselines, which attests to the efficacy of dialectical agentic reasoning in eliciting the latent reasoning capabilities of LLMs. Our code is available at https://github.com/inflee/DARE.

Abstract:
As large language models increasingly act through external tools, deciding when to call a tool has become a central problem alongside deciding how to use it. Unnecessary tool calls introduce latency, cost, retrieval noise, and error propagation, while missed calls hurt knowledge-intensive queries or questions requiring up-to-date evidence. Existing methods typically trigger tools from absolute query or generation signals, such as difficulty, confidence, or final task reward, and therefore lack an explicit estimate of the instance-level marginal benefit of tool use. We propose CoBRA, a counterfactual boundary-learning framework for tool-augmented language models. CoBRA first constructs internal and external experts from the same base model, collects paired trajectories, and estimates the reward margin between answering with and without tools. This margin partitions data into internal-favored, external-favored, and ambiguous cases. CoBRA then uses clear-margin samples for Boundary-Aware Cold-Start SFT, followed by MARS-RL with reference-split rollouts and counterfactual marginal advantages to optimize boundary decisions. Experiments with retrieval as the main tool on Qwen3-4B show that CoBRA improves tool-use efficiency and boundary-sensitive answer accuracy while maintaining strong performance on tool-dependent out-of-distribution questions.

Abstract:
Scientific contributions rarely develop in isolation, but instead build upon prior discoveries. We formulate the task of automated technological roadmapping as extracting scientific contributions from scholarly articles and linking them to their prerequisites. We present the Scientific Contribution Graph, a large-scale resource containing 6 million detailed scientific contributions extracted from 655k open-access papers spanning computer science, medicine, biology, physics, chemistry, and other sciences, and connected by 36 million prerequisite edges. We further introduce scientific prerequisite prediction, a scientific discovery task in which models predict which existing technologies can enable future discoveries, and show that contemporary models are rapidly improving on this task, reaching 0.48 MAP when evaluated using temporally-filtered backtesting. We anticipate technological roadmapping resources such as this will support scientific impact assessment and automated scientific discovery.

Abstract:
Deep research (DR) systems are increasingly used for complex information-seeking tasks, but existing works mainly focus on generating reports and summaries. In contrast, many enterprise tasks instead require an agent to identify concrete workflows which is a sequence of action-steps. For example, rather than summarizing budgeting policies, an agent should be able to determine the steps needed to answer a question such as: "How do I request new headcount given a fixed budget?". Therefore, we introduce DRFLOW, a benchmark for evaluating personalized workflows predicted by agents from heterogeneous sources. Each task requires the agent to identify relevant evidence from scattered sources, then use that evidence to predict the correct action-step sequence for the user’s task. DRFLOW contains 100 tasks across five domains, with 1, 246 reference workflow steps grounded in more than 3, 900 sources. We define seven diagnostic metrics covering factual grounding, step recovery, structural ordering, condition resolution, and personalization. We further present DRFLOW-Agent (DRFA), a workflow-oriented reference agent to predict personalized workflow. We show that although DRFA improves over strong baseline agents (upto 10.02% average F1 score), there is substantial room for improvement remains across these workflow metrics, indicating that predicting complete and correct personalized workflows remains a challenging frontier for deep research.

Abstract:
Recent multilingual large language models achieve strong translation performance, yet the internal mechanisms responsible for low-resource multilingual translation remain insufficiently understood. In this paper, we study translation circuits in multilingual decoder-only LLMs for Hindi→Urdu and Hindi→Sindhi (Devanagari). We propose TITAN, a mechanistic framework that combines causal intervention, sparse feature decomposition, and targeted preference optimization to identify and optimize translation crucial transformer components. Unlike conventional full-parameter fine-tuning, TITAN restricts optimization to sparse causally important circuits, enabling scalable and parameter-efficient adaptation while updating fewer than 5% of model parameters. Experimental results show that TITAN achieves BLEU improvements of +33.75% on Hindi→Urdu and +45.86% on Hindi→Sindhi over the strongest multilingual baselines. Human expert evaluation further confirms substantial gains in adequacy and fluency across both language pairs. Additional analyses reveal that multilingual translation is implemented through sparse structured circuits operating over a shared semantic subspace rather than through an English centric pivot representation. Overall, our findings demonstrate that mechanistic circuit discovery can directly improve low-resource multilingual translation while providing interpretable insights into multilingual transformer behavior.

Abstract:
We study whether LLMs can produce executable workflow artifacts that remain consistent across graph structure, tool implementation, schema bindings, and runtime wiring. In this setting, correctness depends on cross-layer consistency: a workflow may be structurally plausible, yet still fail because tool implementations, schema bindings, or runtime execution do not align. Existing benchmarks largely evaluate these capabilities in isolation or rely on trajectory-level proxies, leaving open whether generated workflow artifacts execute end-to-end. We introduce MCPGen, an executable benchmark for Model Context Protocol (MCP) workflow development. MCPGen contains 100 self-contained MCP projects across 16 application domains and evaluates three diagnostic tasks: workflow reconstruction, tool creation, and backward-compatible workflow extension. We evaluate 11 representative LLMs in a single-turn foundation-model setting, assessing generated artifacts through static analysis, unit and integration tests, and process-isolated end-to-end execution. Models reach 88.5% on workflow reconstruction, but no model exceeds 57% end-to-end execution success. Per-tool unit-test pass rates reach 63.8%, while project-level integration success does not exceed 45%, suggesting that integration remains a major bottleneck even when isolated tool tests pass. Code and data are available at https://github.com/zoe-yyx/MCPGen_Official .

Abstract:
We study whether media takeovers are associated with changes in the political positioning of news outlets. Across nine ownership changes spanning four languages and varied acquirer profiles, we find that aggregate outlet-level shifts are generally modest, but mask selective, actor-specific repositioning toward particular political figures and ideological blocs whose magnitude and direction vary by acquirer. A three-level decomposition — outlet, ideological bloc, individual actor — is combined with in-time placebo tests, within-outlet triple-interaction contrasts and entity-month re-estimation to separate the effects that withstand pre-trend and multiplicity checks from those that do not. We obtain these findings using a multilingual, entity-centered pipeline over CC-News that masks target politicians before zero-shot target-oriented sentiment classification and estimates changes via difference-in-differences against country-matched control media. These results suggest that aggregate stability masks a redistribution of tone across politically salient actors rather than uniform outlet-wide slant shifts.

Abstract:
Prior work establishes that controlled contrastiveness between self-generated responses from large language models, set via reward scores, improves downstream preference tuning in English. We extend this method to multiple languages and evaluate two models across a total of 14 high and low-resource languages on a diverse set of tasks. Our central finding is that cross-lingual contrastive preference tuning on self-generations (CroCo) transfers without language-specific preference annotation. A reward model trained on English preferences (atop a multilingual base) produces useful within-language rankings across most languages, and pairing in either a monolingual or multilingual setting improves over each model on the majority of setups while preventing the catastrophic forgetting of supervised fine-tuning. We observe that the gains require on-policy data. Off-policy responses reduce the benefit and online preference optimization fails to improve over the offline variant. Specifically, on structured tasks, our method matches or exceeds the base in 6/7 languages for EuroLLM-9B and 4/7 settings for Aya-3B. On open-ended generation, evaluated by two judges, both tuned models win 28/30 times against their respective base across 15 evaluated languages (high and low-resource). Overall, we show promising directions for multilingual preference tuning using self-generations.

Abstract:
We assess the robustness of large language models (LLMs) on a simple task—fast-food order specification—in the face of user-initiated amendments. In amending an order, users must communicate, explicitly or implicitly, that a change is being made, the type of change, and what is being changed. We generate synthetic food orders with single amendments, controlling the type of amendment, which components are implicitly or incompletely specified, and the pragmatic inferences required to resolve them. On a dataset broadly covering the space of possible combinations, 7 flagship models average 97.7% accuracy. On a more targeted dataset with denser coverage of empirically difficult patterns, accuracy drops to 93.4%, with the hardest combinations dropping below 80% accuracy. We also test two further prompting methods: generating an order before interpreting amendments in a turn-taking interaction; and function-calling to parameterize explicit edit operations on the order itself. These somewhat improve performance (+0.2 pp and +3.2 pp respectively), but neither entirely eliminates the failure modes. We also find that asking clarification questions about implicit components improves accuracy on a subset of difficult examples (+5.4 pp), but models ask on only 11% of examples. Given the task’s deliberate simplicity—small fixed menu, a single standard order, with a single unambiguous amendment—these results illustrate the challenge of robustly performing the pragmatic inferences that real-world conversational interaction demands.

Abstract:
As text-to-image (T2I) diffusion models become increasingly deployed in real-world applications, concerns about backdoor attacks have gained significant attention. Prior works largely focus on models with a single text encoder, leaving recent multi-encoder architectures, e.g., Stable Diffusion 3 (SD 3), underexplored. In this paper, we study SD 3, which uses three distinct text encoders, and systematically analyze its text-encoder-based backdoor vulnerabilities. To understand each text encoder’s role in backdoor attacks, we define four target categories spanning from global-level to entity-level semantics and identify the minimal encoder subset required for each attack type. Furthermore, we propose Multi-Encoder Lightweight aTtacks (MELT) and validate MELT’s effectiveness and parameter efficiency. MELT matches full encoder fine-tuning attack performance while using fewer than (0.2%) of its parameters, demonstrating that effective backdoors can be implanted under limited parameter budgets.

Abstract:
In this work, we analyze shortcomings in cross-lingual knowledge transfer in large, modern reasoning LLMs. We demonstrate that the perceived gap in knowledge transfer is primarily a script barrier. First, we conduct an observational data analysis on the performance of thinking models on two datasets with local knowledge from around the world, ECLeKTic and MultiLoKo. Our regression analysis shows that script match—not language or family—is the primary predictor of knowledge transfer failure once model capability and question difficulty are accounted for. We further this finding by providing the LLMs with the key entities of the questions in their source language and find that this disproportionately improves cross-script questions. We then posit that these LLMs could be reasoning better at test-time. To evaluate this, we develop a synthetic generation pipeline to design SFT samples to encourage the model to better reason about transliteration ambiguities when trying to fetch parametric knowledge at inference-time. We show that teaching two models to reason better reduces the cross-script transfer gap. As a result, we conclude that there is potential to improve cross-lingual parametric knowledge transfer during post-training.

Abstract:
The goal of this work is to enhance balanced multimodal understanding in audio-visual large language models (AV-LLMs) by addressing modality bias without additional training. In current AV-LLMs, audio and video features are typically processed jointly in the decoder. While this strategy facilitates unified multimodal understanding, it may introduce modality bias, where the model disproportionately relies on a single dominant modality. In this work, we provide an in-depth analysis of this phenomenon and propose Fork-Merge Decoding (FMD) as a targeted solution. FMD is a simple, training-free inference strategy that rectifies this imbalance without architectural modifications. Specifically, FMD isolates audio and visual inputs in the early decoding layers (fork) to enforce modality-specific reasoning, before integrating them for joint interpretation (merge). This two-stage mechanism effectively mitigates the dominance by ensuring balanced contributions from both modalities. Experiments on VideoLLaMA2, video-SALMONN, and Qwen2.5-Omni demonstrate that FMD consistently enhances multimodal understanding across diverse benchmarks.

Abstract:
Large language models (LLMs) are typically pretrained on short sequences and then extended to work on longer sequences with additional training. However, LLMs still struggle to generalize to very long sequences. We propose Randomized YaRN, a training method that improves length generalization by combining YaRN-based positional extrapolation with randomized positional encoding and a length curriculum. During training on short context data, tokens are assigned YaRN positional encodings sampled from a larger position range, exposing the model to out-of-distribution positional representations without requiring long-context inputs. We evaluate Randomized YaRN on two challenging long-context reasoning benchmarks, BABILong and Multi-Round Coreference Resolution (MRCR). When training on data with <8K context, Randomized YaRN consistently improves reasoning performance on context lengths from 16K to 128K and outperforms standard fine-tuning baselines, with the largest gains appearing at far out-of-distribution lengths. Our results suggest that progressively exposing models to OOD positional distributions provides an effective recipe for generalizable long-context reasoning.

Abstract:
A model can know when two events occurred yet still answer which came first incorrectly. Multi-hop QA accuracy conflates such composition errors with missing facts. We introduce a double-gate protocol for temporal factual chains and evaluate a case for composition only if the model (i) answers every constituent fact correctly across paraphrased probes and (ii) answers every aligned sub-question correctly in isolation. If the original multi-hop answer remains incorrect, the case is a residual composition failure; we call the systematic occurrence of such failures composition collapse. On D4v2, a benchmark of 390 temporal chains, two post-trained models differ by only 0.3 percentage points in atomic stability yet by 47.4 points in depth-2 (two-fact) residual failure. In a controlled same-base comparison, outcome-verified reinforcement learning reduces depth-4 residual failure by 54 points relative to reasoning-trace SFT under matched data and LoRA budget. On 80 examples that pass both gates for every GRPO variant, failure decreases monotonically, showing that model-specific gate subsets are unlikely to drive the trend. Finally, chain-of-thought decoding recovers 70–75% of gate-passing failures, consistent with an important role for inference-time computation. These results motivate reporting atomic stability and gate-controlled composition separately when evaluating multi-hop reasoning.

Abstract:
Recent studies have explored adapting LLMs to time series tasks by leveraging language reasoning paradigms. However, it remains unclear whether reasoning strategies are effective across time series tasks with diverse temporal structures. In this work, we systematically investigate this practice using five general reasoning strategies across 12 LLMs and 20 public datasets, together with a self-constructed benchmark of approximately 18k QA pairs. Our results show that general reasoning strategies are not universally beneficial: their effectiveness depends on alignment with task-specific temporal decision structures and output spaces, rather than on model scale or complexity. We further observe that prior information and common time series pre-processing techniques exhibit strong task-dependent effects. Based on these findings, we propose three time-aware reasoning strategies that explicitly model temporal regimes, latent dynamics, and output uncertainty. Across four tasks, they consistently outperform generic reasoning strategies, and enable reasoning to better benefit from larger model scales and multi-sampling. Our study highlights the importance of temporal structure awareness in designing reliable reasoning strategies for LLM-based time series analysis.

Abstract:
Automatic identification of code-switched (CS) utterances remains a challenge for language identification (LID) systems, causing such texts to be underrepresented in the training data of Large Language Models. In this paper, we revisit MaskLID, a state-of-the art approach for CS identification, which requires no training and detects arbitrary language combinations. We make three main contributions: (a) we reveal, and address, a major issue of MaskLID: its overreliance on word-level language association scores; (b) we reformulate the underlying optimization algorithm as an Integer Linear Program, enabling us to experiment with a large set of clear and interpretable constraints; (c) each of these improvements vastly improves the baseline system, as we illustrate in experiments involving 10 diverse languages, where we observe a strong boost in performance on CS benchmarks.

Abstract:
Conversational voice agents have advanced significantly, offering increasingly natural human-machine interactions through both cascaded and end-to-end architectures. However, while recent benchmarks extensively evaluate dyadic interactions and passive audio comprehension, they largely overlook a prevalent real-world scenario: multi-party conversations. Evaluating agents in these settings is fundamentally more challenging than in dyadic interactions due to the exponentially greater conversational complexity. For voice agents to integrate seamlessly into human group dynamics, they must not only generate contextually appropriate responses but also demonstrate a nuanced understanding of open turn-taking. To address this gap, we introduce Multiparty Bench (MP-Bench), the first benchmark specifically designed to objectively evaluate conversational speech systems as active participants within multi-party contexts. MP-Bench assesses agent behavior along two primary dimensions: turn-taking awareness and response appropriateness. Additionally, we incorporate comprehension-based question-answering tasks as a complementary evaluation. By benchmarking 12 voice agents, we find that real-time voice agents stay at or below 22% on multiparty comprehension and remain near chance on multiparty turn-taking, exposing an open challenge for real-time voice agents under multiparty scenario.

Abstract:
Although large language models (LLMs) perform much worse than traditional supervised models on the semantic role labeling (SRL) task, our closer investigation reveals two important findings. First, the existing labeled SRL data is substantially flawed, i.e., of unsatisfactory quality. Second, LLMs can serve as an effective tool for detecting annotation errors and deciding which labels require human review, from the perspective of minimizing human effort. This paper proposes an LLM-assisted data quality improvement approach, and applies it to two SRL evaluation datasets from two different domains. With our approach, human annotators need to review only 11% of the originally annotated arguments flagged for manual inspection, among which 40% are truly erroneous. Meanwhile, our approach suggests 20% new arguments potentially missed in the original labels for human review, among which 60% are indeed missing. In total, our approach locates and corrects 21% errors in the two evaluation datasets. Experiments show that both traditional supervised models and LLMs perform much better on the revised evaluation datasets.

Abstract:
Large Language Models (LLMs) suffer from temporal misalignment, often due to the contradictory nature of their training corpora. While current mitigation strategies rely on computationally expensive fine-tuning or context-heavy retrieval augmented generation (RAG), the internal mechanisms governing time sensitive recall remain under-explored. Unlike prior studies that identify temporal components such as attention heads and MLP layers, we provide the first feature level map of temporal recall by isolating individual MLP features via transcoder circuit tracing. We identify three node categories (common temporal, common to the year and chrono-semantic) which interact to generate a temporal filter during factual recall. By analysing Gemma 2 2B, LLaMA 3.2 1B, and Qwen3-4b, we show that these features do not follow a simple linear pipeline but represent time through a parallel and mixed syntactic-semantic interplay across layers. We additionally discover a class of higher-layer temporal components invisible to existing EAP-IG methods, establishing transcoders as a more complete lens for temporal interpretability for time sensitive factual recall. These findings present MLP components for potential targeted interventions in time-sensitive factual recall. Our code can be found at https://github.com/sg-sy/timewarp-temporaltranscoder.

Abstract:
Instruction hierarchy (IH) alignment teaches language models to prioritize higher-level instructions when inputs conflict. While studied primarily in text-only settings, vision-language models (VLMs) introduce new challenges for IH: instructions may be embedded in images, split across modalities, visually transformed, or encountered during agentic tasks. Positing multimodal IH alignment as a reasoning problem, we train VLMs using reinforcement learning with rule-based rewards, comparing text-only, image-only, and mixed-modality supervision. We find that text-only IH training partially transfers to multimodal attacks, failing when models must decode, reconstruct, or reason over instructions across modalities. Image-based training improves robustness beyond text-only supervision, while mixed-modality training performs best overall. Importantly, the benefits generalize beyond the synthetic typographic training setting to real-image and web-agent safety tasks, while largely preserving general multimodal capability, showing that lightweight, verifiable supervision can meaningfully improve VLM robustness under adversarial, cross-modal, and interactive instruction conflicts

Abstract:
Multi-hop question answering (QA) requires integrating evidence across multiple documents and reasoning over interdependent intermediate steps, posing significant challenges for retrieval-augmented generation (RAG) systems. While recent RAG-based approaches interleave retrieval and reasoning to improve factual grounding, they often struggle with complex multi-hop questions due to retrieval–reason mismatch, ignored dependencies among sub-questions, and error propagation during decomposition. In this work, we propose SDRAG, a sub-question dependency-aware RAG framework that explicitly models the intrinsic logic of multi-hop questions as a dependency graph. SDRAG first decomposes the question into fine-grained sub-questions and explicitly models their dependency relations, then refines the graph by pruning unreliable or redundant nodes, and finally traverses the pruned graph to coordinate retrieval and reasoning. To improve graph construction quality, we further introduce a supervised fine-tuning strategy that independently trains task-specific LoRA adapters for decomposition, dependency prediction, and pruning. Experiments on three common multi-hop QA benchmarks show that SDRAG consistently outperforms strong RAG-based baselines, while additional analyses demonstrate the effectiveness of dependency modeling, graph pruning, and fine-tuning.

Abstract:
Proactive agents are expected to anticipate user needs and provide autonomous assistance by perceiving environmental context without explicit instructions. A fundamental capability of such agents is to identify and track users’ upcoming events, enabling continuous and event-specific assistance. For example, by recording the time and location of a planned hike, an agent can deliver weather reminders in advance or provide navigation support before departure. However, existing works on proactive agents largely overlook event-centric assistance, and the open-ended nature of proactive assistance poses challenges for reliable evaluation. To bridge these gaps, we introduce ProEvent, the first event-centric benchmark designed to assess an agent’s ability to proactively maintain a user’s timetable based on ongoing instant messaging chats. ProEvent provides synthesized yet realistic chats that consider the dynamic interaction among users, concurrent chat threads, and noise in the real world, and evaluates proactive agents on response timing, single-step response correctness, and multi-step response correctness. Experiments on eight LLMs and pipelines reveal that current agents frequently overact and struggle with event cancellation. Notably, even the state-of-the-art GPT-5.1 provides redundant assistance in 30% of cases and achieves only 26.7% recall in event cancellation scenarios. Further qualitative analysis reveals fundamental limitations of current LLMs as proactive agents, particularly in detecting implicit events and reasoning from the user’s first-person perspective.

Abstract:
Hallucinations remain challenging for large language models (LLMs), especially in factual generation. Existing mitigation methods often rely on retrieval, supervision, representation editing, or output-level decoding, while paying limited attention to how factual support is internally coordinated across attention heads. In this work, we study factual decoding from a head-level coordination perspective and find that unreliable generations are often associated with fragmented and unstable use of factual anchors across heads. To improve such coordination, we propose Head Coordination Graphs (HCG), a plug-and-play inference-time framework that models attention heads as interacting nodes. HCG uses explicit factual anchors as observable support signals to construct sparse head graphs and applies risk-gated propagation to adaptively regulate cross-head influence without parameter updates, additional training, or external knowledge. Experiments across multiple factuality and hallucination benchmarks demonstrate broad improvements in factual consistency across open-source LLMs, while analyses and ablations demonstrate the importance of head-level coordination, sparse graph construction, and risk-gated propagation.

Abstract:
Transformer-based models are highly susceptible to backdoor attacks via supervised fine-tuning (SFT). To red-team existing data-poisoning defenses, prior work has increasingly focused on stylized triggers, synthetic artifacts, and token-level perturbations designed to evade detection. However, this trend has shifted threat models away from naturally occurring semantic triggers and realistic low-budget poisoning settings. Addressing this gap, we introduce SteganoBackdoor, an optimization-based framework that transforms semantic-trigger seeds through autoregressive token replacement, sequentially minimizing embedding overlap with the inference-time trigger while preserving a strong per-sample training-time payload. The resulting SteganoPoisons maintain linguistic fluency and encode the payload across ordinary tokens, such that no individual token carries a concentrated signal and the full payload instead emerges from their exact combination and ordering. Across 18 encoder-based and decoder-only models spanning 120M to 14B parameters, SteganoBackdoor achieves high attack success under sub-percent poisoning budgets and exposes limitations in existing data-poisoning defenses. Project page: github.com/exruiec/SteganoBackdoor

Abstract:
This paper argues that evaluation of educational LLMs should move beyond decontextualized task-based benchmarks toward situated benchmarks centered on pedagogical action. Prior work in human-computer interaction and the learning sciences has long emphasized situated use, user context, and interaction processes; our focus is narrower and complementary: how these concerns should reshape benchmark design for educational LLMs. Existing evaluations already cover question generation, knowledge answering, error diagnosis, feedback generation, and teaching analysis, but their dominant unit remains the single input-output episode, making it difficult to compare whether a model genuinely supports learning progress, instructional judgment, or subsequent revision in context. Educational activity is processual, context-dependent, and development-oriented, so explanations, feedback, and hints can only be evaluated properly in relation to learner state, interaction history, and pedagogical goals. We review a landscape still dominated by task-based paradigms and argue, from educational conceptions, educational values, and research methodology, that next-generation educational benchmarks should make pedagogical function in context a core target. We further propose layered, scalable, and benchmarkable situated protocols built through simulation-first development and follow-up empirical calibration, extending evaluation beyond task quality to contextual fit and downstream consequences for learning and teaching decisions.

Abstract:
Prompt injection attacks pose a serious security risk to large language models in real-world applications. However, existing prompt injection benchmarks primarily focus on short-context inputs, leaving the attacks and defenses in long-context settings largely unexplored. This gap leads to a substantial overestimation of the effectiveness of current defenses. In this paper, we bridge the gap by introducing LongPIBench, a long-context benchmark for prompt injection covering 4 realistic application scenarios: paper peer review, resume screening, code review, and email summary. For each scenario, we construct a synthetic dataset and a real-world dataset, with context lengths ranging from thousands to tens of thousands of tokens. The evaluation results on LongPIBench reveal significant vulnerabilities of prompt injection defenses under long-context settings: even simple heuristic prompt injection attacks achieve high success rates and frequently bypass state-of-the-art defenses. We hope LongPIBench can serve as a practical benchmark for systematically evaluating prompt injection defenses in realistic long-context scenarios.

Abstract:
LLMs deployed as judges in high-stakes classification are commonly assumed to let their generated rationale shape the verdict; we show this assumption fails in disinformation detection. We introduce the Multi-Bias Injection Protocol (MBIP), which perturbs the reasoning stage along four cognitive axes (confirmation, anchoring, framing, authority) while holding the input fixed, and quantifies induced rationale variation with technique-count metrics anchored on the number of persuasion techniques the model reports. Across five disinformation datasets, MBIP produces substantial rationale variation but near-zero verdict flipping; an explicit rationale-deference prompt raises the flip rate only modestly, to 0.118–0.300 across three open-weight models, with API-only gpt-4o-mini in the same range. We call this verdict-level invariance Decision Rigidity. At the representational level, our residual-stream probe localizes a candidate commitment band late in the network, where biased and neutral final-class trajectories converge. The band’s location is architectural, replicated by a neutral-vs-neutral null; a within-band alignment-rate elevation and a random-lexicon control gap co-occur with the behavioral invariance and attenuate in Gemma-2-9B. All representational evidence is correlational. Generalization to other high-prior, evidence-dependent classifications and attenuation on reasoning-intensive tasks remain falsifiable hypotheses.

Abstract:
Hypergraph-based RAG systems surpass traditional graph-based approaches by organizing complex n-ary atomic facts among entities, rather than relying solely on binary relationships. Despite the advancements in multimodal large language models (MLLMs) with enhanced visual capabilities, current hypergraph-based RAG frameworks predominantly restrict knowledge retrieval and reconstruction to a unimodal, text-centric paradigm. This limitation prevents them from fully leveraging the powerful visual perception capabilities of modern MLLMs. To address this gap, we systematically explore the integration of hypergraph awareness in RAG systems through visual cues. By incorporating visual representations of hypergraphs into the RAG pipeline, we introduce VizRAG, the first RAG system to support visual hypergraph structure awareness. Experimental results demonstrate that VizRAG significantly outperforms strong baselines, validating the promising potential of hypergraph visualization as a novel approach for RAG systems.

Abstract:
Large language models (LLMs) have shown strong multimodal reasoning abilities, yet their understanding of purely numerical time-series remains limited. Existing approaches mainly focus on forecasting or trend description, providing little insight into the underlying events that have occurred or are unfolding within observed numerical dynamics. We study number-to-event reasoning and decoding, a task that infers interpretable and structured event hypotheses directly from numerical time-series when explicit event information is unavailable at inference time. The task focuses on aligned and plausible event semantics that help explain observed numerical dynamics. To address data scarcity and semantic alignment challenges, we propose a reasoning-aware framework that combines an agent-guided event extractor, a Hawkes-process-based synthetic data generator, and a multi-stage fine-tuning pipeline integrating a time-series encoder with a structured event decoder. The model generates intermediate reasoning steps and outputs constrained event representations. Experiments across multiple real-world domains show that our approach consistently outperforms strong LLM baselines in event-level precision and recall, demonstrating its effectiveness in bridging numerical reasoning and semantic event understanding. The code is available at https://github.com/feng1201/NUM2EVENT_IDEA

Abstract:
The rise of agentic AI enables LLMs to access diverse user data, raising critical privacy concerns. Prior work on contextual privacy studies whether LLMs regulate information disclosure according to context-dependent norms. However, acceptable disclosure boundaries may vary across users even within the same context. To address this limitation, we introduce personalized privacy, which incorporates user-specific disclosure preferences into privacy control. We further present P3Bench (Personalized Privacy Preservation Benchmark), a novel benchmark extending contextual privacy policies with personalized disclosure policies. Experiments show that prompt-based policies fail to reliably enforce personalized privacy policies, with Qwen2.5-7B and Gemma3-4B showing average policy ignorance ratios of 51.25% and 74.28%, respectively. Finally, to address this problem, we propose Repair, a robust inference-time attention head intervention method that adjusts disclosure behavior toward policy-consistent responses. Our method significantly improves adherence to user-specific privacy preferences by reducing cases where the model fails to follow the given policy.

Abstract:
In this work, we introduce DMDIntel which uses dynamic mode decomposition (DMD) to make the predictions made by LLMs in a classification task interpretable. It develops an input attribution pipeline, that first decomposes the hidden states of an LLM into prominent patterns, also known as modes, and then associates ranks to the input tokens based on the projection values on those modes. Rigorous experiments across three datasets and three model families consistently show that the ranked attribution of input tokens obtained using DMDIntel by far outperforms state-of-the-art techniques such as principal component analysis, integrated gradients and SHAP.

Abstract:
Existing Supervised Fine-Tuning paradigms, particularly Full Parameter Fine-Tuning are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting, which current methods typically address in isolation and lack a unified optimization signal to bridge data selection, parameter updates, and knowledge preservation. To address this, we propose Neuron-Guided Fine-Tuning (NGFT), a holistic framework that leverages neuron activation patterns as a universal proxy to unify the fine-tuning lifecycle. NGFT operates via three synergistic mechanisms: (1) Adaptive Task-Specific Neuron Selection, which identifies essential neurons in a single forward pass to concentrate updates and reduce redundancy; (2) Activation-Based Data Selection, which prioritizes information-dense samples that maximize contribution to key neurons; and (3) Neuron Activation Alignment, a novel loss function that anchors activations to pre-trained states, deepening representation learning and preserving general knowledge. Experimental results across three models across both domain-specific and general benchmarks demonstrate that NGFT significantly outperforms existing mainstream fine-tuning methods in both efficiency and performance, while effectively mitigating catastrophic forgetting.

Abstract:
Reinforcement learning fine-tuning of large language models increasingly adopts multiple reward dimensions, including verifiable rules, task-specific evaluators, and learned reward models, to provide richer supervision across diverse capabilities. These dimensions are commonly scalarized with fixed aggregation weights. We identify a failure mode in which aggregation itself induces reward hacking: static projection aliases qualitatively different reward profiles into a single scalar, steering optimization toward whichever dimensions are easiest, densest, or systematically favored by the reward signal. Over training, this traps the policy in suboptimal profiles and prevents convergence to better-balanced ones that would yield higher task performance. To address this, we propose Adaptive Multi-Reward Projection (AMRP), a lightweight online method that reallocates aggregation weights using three signals—relative shortfall, reward volatility, and recent progress—increasing pressure on lagging, unstable, or stagnant dimensions while relieving saturated ones. Across structured reasoning, citation-grounded generation, and open-ended alignment under GRPO, AMRP consistently improves reward-profile balance and downstream performance over fixed and dynamic weighting baselines; it also remains effective with GDPO and PPO, supporting compatibility across RL algorithms. Our code is available at https://github.com/yyhappier/AMRP.git .

Abstract:
Large language model (LLM) inference is constrained by the quadratic scaling of self-attention and the linear scaling of the KV cache, increasing latency, energy consumption, and GPU memory demand as context length scales. Existing soft-compression methods either lack query-guided memory selection at inference time, train without answer-targeted supervision, or couple compression tightly to a specific decoder architecture. We propose a Context-to-Answer-Aligned Memory Compression (CMC) framework, which compresses long input contexts into compact Context Memory Embeddings (CMEs) aligned to any frozen decoder’s embedding space, reducing inference costs without modifying decoder weights. CMC introduces a two-tier KV cache that combines question-guided CME selection with a local context window, and trains the compressor with answer-targeted distillation from a frozen LLM. Experiments across nine encoder-decoder combinations and four QA benchmarks show that CMC consistently outperforms the baseline, achieving up to 7.3 EM and 4.0 F1 point gains on SQuAD, while reducing inference time and energy consumption by up to 20% and peak reserved GPU memory by up to 50% at 3,000 generation tokens. Ablation studies confirm that each architectural component and training objective contributes to the performance.

Abstract:
Legal judgment prediction (LJP) is important for supporting judicial decision-making and improving judicial efficiency. Existing LLM-based approaches often rely on manually specified reasoning pipelines or external legal structures, while feedback-based methods typically provide generic signals of correctness or coherence rather than case-specific legal supervision. We propose IALR: Instance-Adaptive Legal Rubric Supervision for Legal Judgment Prediction. For each training instance, IALR contrasts model responses of varying quality and distills their legally consequential failures into a weighted Instance-Adaptive Legal Rubric. These rubrics convert judgment-relevant conditions into discriminative preference signals for Direct Preference Optimization, enabling the model to learn charge prediction without following a predefined reasoning paradigm. At inference time, IALR requires no additional evaluator or correction module. Experiments on four Chinese criminal charge-prediction benchmarks show that IALR consistently improves Qwen2.5-7B, yielding average absolute accuracy gains of 5.68% in-domain and 8.73% out-of-domain. These results demonstrate the effectiveness and cross-domain transferability of Instance-Adaptive Legal Rubric supervision. The code and data are available at https://github.com/Baii668/IALR.git.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) improves language-model reasoning, but how these gains relate to inference-time decoding and search remains unclear. Does RL produce reasoning behavior the base model does not have, or does it shift the rollout distribution toward trajectories the base model can already reach but rarely samples? We study this behaviorally with a Unified Decoding Framework (UDF), which expresses token-level sampling, beam-like search, tree search and sequence-level resampling as executable policies over a shared budgeted operating space. Operating points are scored after generation with pass@ k , self-consistency, best-of- N and first-finish success, so the generation policy stays separate from the evaluation metric. Using paired Base/RL checkpoints from SimpleRL-Zoo, we ask whether an RL default-policy curve can be approximated by a structured path of Base operating points (𝜋, N Base ) rather than by post-hoc per-budget matching. On Math500, AIME, GPQA and IFEval, the pass@ k recovery path follows a Budgeted Operating-Point Transition Rule (BOPTR), N Base ≈ 𝛼 N RL 𝛽 , with benchmark-conditioned exponents. On Qwen2.5-7B, BOPTR gives the lowest transfer error among the non-oracle rules we test, 3.41 pp. Across models the rule family remains informative, but its parameters and selected policies are model-specific: Qwen models trained on the same difficulty split share operating-point structure, while cross-family models and different training distributions mark the limits of direct transfer. These results support a qualified internalized-search reading: under the recipe we test, much of the measured RL gain corresponds to a change in sampling efficiency, toward operating points the base model can already reach under search. We treat the benchmark-wise scaling patterns as descriptive of this recipe and cohort, report the cases in which they break down, and use UDF and BOPTR as behavioral diagnostics rather than as evidence of parameter-level equivalence or as general predictors of post-RL performance.

Abstract:
Entropy-based deep reasoning has emerged as a promising direction for improving the reasoning capabilities of Large Language Models (LLMs), but existing methods often either increase response length indiscriminately or shorten responses at the cost of accuracy. To better balance this trade-off, we introduce Conditional Entropy Shaping (CES), a framework that dynamically controls token-level response entropy, enabling LLMs to produce concise solutions on simple problems while encouraging deeper exploration on hard ones. Built on DAPO, CES uses token-level entropy as an uncertainty signal and applies a conditional bidirectional policy: it penalizes high-entropy "forking point" tokens on correct reasoning paths to improve conciseness, and rewards them on incorrect paths to encourage exploration and error correction. We implement CES on DeepSeek-R1-Distill-Qwen-7B and evaluate it on 12 mathematical benchmarks. CES consistently improves average accuracy while reducing response length relative to DAPO, and supplementary experiments show similar trends on a smaller 1.5B backbone and on out-of-domain benchmarks.

Abstract:
Reasoning language models generate lengthy chain-of-thought (CoT) sequences whose key-value (KV) cache grows linearly and becomes a memory bottleneck during decoding. Existing compaction methods treat reasoning trajectories as flat token sequences and apply uniform compression, ignoring the hierarchical structure of CoT reasoning where different steps vary drastically in importance. We propose Thought-Aware Attention Matching (TAM) , which exploits this structure through three mechanisms: (i)thought segmentation that decomposes the trajectory into reasoning blocks, (ii)adaptive budget allocation that assigns compression budget based on each segment’s importance and size, and (iii)pivotal token protection that preserves high-attention reasoning anchors. We prove that the allocation rule is optimal under a convex error model and that cumulative error under sequential compaction remains bounded. Experiments on AIME 2024 and MATH-500 with Qwen3-4B show that TAM improves accuracy over uniform compaction at the same memory footprint, with periodic compaction bounding peak memory to 3.1–3.2,GB (a 65% reduction) while maintaining competitive accuracy.

Abstract:
Supervised fine-tuning of large language models relies on human-annotated data, yet annotation pipelines routinely involve multiple crowdworkers of heterogeneous expertise. Standard practice aggregates labels via majority vote or simple averaging, discarding annotator identity and causing the model to absorb the errors of unreliable annotators into its parameters. We propose REALM, which jointly learns the model parameters and a scalar expertise value for each annotator, entirely unsupervised and requiring nothing beyond annotator identity. The key idea is to model each observed label as a mixture between the model’s prediction and a uniform random guess, weighted by the annotator’s learned expertise. REALM applies to any task with a fixed label set, and extends to multiple tasks via a learned expertise matrix. On four text-classification datasets with real crowdsourced annotations, REALM is the best method in all 12 configurations of the three heterogeneous-annotator datasets, improving on the strongest applicable baseline, including majority vote and Dawid–Skene aggregation, by +2.9 points on average. On five question answering benchmarks with simulated noisy labels, it outperforms naive noisy fine-tuning in 152 of 162 configurations, by +5.0 points on average, with gains that grow with model capacity. The learned expertise additionally recovers annotator reliability without ever observing it. Our code is available at https://github.com/sajjad-ucsb/REALM .

Abstract:
Large language models (LLMs) are increasingly used as judges to evaluate agent performance, particularly in non-verifiable settings where judgments rely on agent trajectories including chain-of-thought (CoT) reasoning. This paradigm implicitly assumes that the agent’s CoT faithfully reflects both its internal reasoning and the underlying environment state. We show this assumption is brittle: LLM judges are highly susceptible to manipulation of agent reasoning traces. By systematically rewriting agent CoTs while holding actions and observations fixed, we demonstrate that manipulated reasoning alone can increase false positive rates of state-of-the-art vision-language model (VLM) judges by up to 117% relative to the unmanipulated baseline across 800 trajectories spanning diverse web tasks. We study manipulation strategies spanning style-based approaches that alter only the presentation of reasoning and content-based approaches that fabricate signals of task progress, and find that content-based manipulations are consistently more effective. We evaluate mitigation strategies including manipulation-aware prompting and judge-time scaling, which can reduce but do not eliminate the vulnerability. Our findings reveal a fundamental vulnerability in LLM-based evaluation and highlight the need for judging mechanisms that verify reasoning claims against observable evidence.

Abstract:
Existing memory systems for long-horizon LLM agents often retrieve prior traces as passive context rather than converting them into executable capabilities. In this paper, we propose MSCE, a training-free Memory–Skill Co-Evolution framework that organizes agent experience into grounded step traces, reusable procedural policies, and declarative environmental cognition. MSCE crystallizes evidence-backed L2 policies with positive estimated gain into callable skills that retain evidence links, applicability boundaries, decision guidance, verification rules, and reliability estimates. It further introduces reflection-weighted value backfilling, which propagates sparse terminal feedback through dense local self-reflections to produce evidence-calibrated trace values for governing memory and skill evolution. Experiments on EvoAgentBench and LoCoMo show that MSCE achieves the best aggregate performance among the evaluated skill-augmented and memory-driven baselines, with statistically supported aggregate gains, positive cross-domain transfer, and continual improvement as online experience accumulates. Our code is available at https://github.com/MemTensor/MemOS/tree/main/apps/memos-local-plugin.

Abstract:
In-context learning (ICL) has emerged as a powerful paradigm for adapting large language models (LLMs) to new, data-scarce tasks using only a few carefully selected task-specific examples embedded in the prompt. However, given the limited context size of LLMs, a fundamental question arises: Which examples should be selected to maximize performance on a given user query? While nearest-neighbor-based methods have been widely adopted for this purpose, they suffer from well-known drawbacks in high-dimensional embedding spaces, including poor generalization and limited diversity. In this work, we study example selection in ICL from a principled, information-theoretic perspective. We frame the example selection task as a query-specific high-dimensional optimization problem: selecting a subset of exemplars from a larger example bank to minimize prediction error for a given query. To solve this, we first derive a principled surrogate objective that is approximately submodular, enabling the use of a greedy algorithm with an approximation guarantee. We further enhance our method by introducing an optimal design-based regularizer to encourage diversity in the selected examples. Empirically, we demonstrate significant improvements over standard retrieval methods across a suite of classification tasks, highlighting the benefits of structure-aware, diverse example selection for ICL in real-world, label-scarce scenarios.

Abstract:
Cross-cultural mediation by large language models (LLMs) requires deciding both when to intervene and how to respond in culturally grounded conflicts. Progress on this problem has been limited by the lack of (1) mediation datasets with measurable downstream effects and (2) principled metrics for evaluating intercultural stance change. To address these gaps, we introduce CC-Mediation, a cross-cultural mediation benchmark of 1,661 ten-turn dialogues grounded in the Developmental Model of Intercultural Sensitivity (DMIS), containing culturally grounded conflicts, mediation interventions, and post-intervention trajectories. We further propose two DMIS-based evaluation metrics: Trajectory AUC, which measures the persistence of intercultural improvement over time, and a signed Wasserstein-1 distance, which measures the magnitude and direction of shifts in intercultural stance. Both metrics show strong agreement with human judgment. Using CC-Mediation, we find that current LLMs have limitations on both axes: intervention timing (when) failure stems from a positional prior that ignores dialogue content, while mediation strategy (how) failure arises from a late-layer elicitation collapse rather than a knowledge deficit.

Abstract:
Personalized Question Answering (PQA) requires models to shift from a fact-centric to a user-centric paradigm, inferring implicit intents from historical interactions. However, standard Retrieval-Augmented Generation (RAG) struggles with narrative noise and fragmented semantics. Similarly, while Agent Memory was proposed to maintain long-term consistency, existing implementations predominantly focus on accumulating objective facts, neglecting the subjective attitudes essential for personalization. We argue that for PQA, while objective facts provide the necessary foundational context, relying solely on them is insufficient; they must be augmented by explicit modeling of subjective attitudes to capture a comprehensive user profile. To address this, we introduce Mem-ABC, an Attitude-Aware Agent Memory framework grounded in the Tripartite Model of Attitudes (Affect, Behavior, and Cognition). Mem-ABC employs a two-stage offline process: first autonomously refining raw history into general memories to eliminate noise, and subsequently reorganizing them into a structured attitude-aware memory. This structure enables the agent to explicitly retrieve context aligned with user psychology. Experiments on the LaMP-QA benchmark demonstrate that Mem-ABC consistently outperforms strong baselines across multiple domains.

Abstract:
In existing memory benchmarks for Large Language Models (LLMs), the evaluated dialogue sessions often lack long-term semantic consistency, and the underlying personas tend to be flat and static. Furthermore, in real-world scenarios, interactions between users and assistants involve more diverse, heterogeneous data streams, such as documents and emails. These shortcomings significantly limit the realism and effectiveness of current evaluations. To address these limitations, we introduce RHELM ( ̲R ealistic, ̲H eterogeneous, and ̲E volving ̲L ong-term ̲M emory). Driven by meticulously crafted user profiles and a novel LOOP (p ̲L an-r ̲O llout-ev ̲O lve- ̲P rune) module, we construct realistic dialogues across diverse interaction scenarios that exhibit dynamic temporal evolution and long-term coherence. Crucially, these dialogues are deeply integrated with heterogeneous external sources synchronized with the user’s temporal event trajectory. The resulting benchmark encompasses challenging question-answer pairs spanning seven inquiry types, with each question mapping to at least one of 27 critical memory characteristics that we identify as essential yet underexplored in current research. Comprehensive experiments across full-context models, retrieval-augmented generation (RAG) methods, and representative memory frameworks reveal that contemporary approaches still expose critical weaknesses in complex, real-world settings, particularly in resolving multi-source aggregation and real-world contextual reasoning.

Abstract:
Affordance reasoning, the inference of an object’s action possibilities from its physical properties (e.g., shape and material), is fundamental to human physical understanding and increasingly critical for Large Language Models (LLMs). However, existing affordance benchmarks largely expose explicit object identities in the evaluation setup, allowing models to rely on memorized object-affordance mappings rather than reasoning over physical properties. To address this gap, we introduce Affordance20Q, a novel affordance reasoning benchmark formulated as a 20-Questions game without exposing the object’s identity. In each game, the model identifies a hidden object’s affordance from a candidate set by asking yes/no questions about its physical properties. Affordance20Q comprises 1,009 games over 454 objects and 59 affordances, all manually filtered, refined, and annotated. We conduct comprehensive experiments with 15 state-of-the-art LLMs and find a substantial gap (~20 points) compared to human performance. A KL-based information-gain (IG) analysis further shows that models fail to ask discriminating questions as the game progresses. To close the gap, we develop KB-Anchored Rule Induction (KARI), a pipeline based on LLMs that generates affordance rules grounded in evidence from knowledge bases (KBs). KARI improves open-source LLMs by up to 15.2 points, while the limited coverage of KBs hinders further gains. We release all our code and data at https://github.com/1171-jpg/Affordance20Q.git.

Abstract:
Given a strategically complex board game, human players can quickly learn to devise strategies after playing a few rounds. Autonomous agents require similar capabilities in realistic interactive environments, yet existing agent benchmarks often fail to fully capture such strategic and evolving decision-making scenarios. We present PTCG-Bench, a benchmark built on the Pok’ e mon Trading Card Game (PTCG) that evaluates LLM agents at two complementary levels: (1) their decision-making performance within a single complex environment, and (2) their ability to self-evolving through accumulated experience. We further include a modular harness ablation to better interpret agent performance without conflating it with model capability. Our experiments show that, although LLM agents can achieve non-trivial gameplay performance, sustained and stable self-evolution remains challenging, and performance is sensitive to harness design. We hope that PTCG-Bench will facilitate future research on harness-aware and self-evolving agents in realistic interactive environments.

Abstract:
Retrieval-based fact-checking systems ground verification in large document collections, creating a critical vulnerability: adversarial evidence contamination. We demonstrate that factually inconsistent but contextually coherent evidence generated to satisfy both retrieval-similarity and linguistic-fluency constraints can systematically mislead verification models. Contaminating just 3.1-7.4% of training corpora with five label-specific strategies causes adversarial content to penetrate up to 35.2% of top-k retrievals, degrading verification accuracy by up to 46.9% relative F1. Existing retrieval-stage defenses, validated against gradient-optimized or syntactically anomalous attacks, transfer poorly to this threat model: detection rates collapse and F1 recovery exhibits high variance across seeds. We propose GLAD (Geometric-Lexical Adversarial Defense), which fuses dense embedding peer-coherence with lexical distributional isolation. Across two benchmarks, two embedding models, and five random seeds, GLAD achieves the highest detection rates (70-86%) and the most consistent F1 recovery, reclaiming 73-80% of contamination-induced degradation on LIAR-RAW while approaching clean baselines on RAWFC. Crucially, unlike defenses that trade clean performance for robustness, GLAD imposes negligible cost on uncontaminated corpora.

Abstract:
LLMs expand accessibility and provide wide-reaching access to information. Yet these interactions also create opportunities to embed subtle, goal-oriented behaviours that shape what users think and how they behave, a concern reflected in governance frameworks that prohibit manipulative AI. We refer to these behaviours as hidden intentions: covert agendas embedded in a model’s outputs that can manipulate users’ beliefs and actions. In this work, we examine whether hidden intentions can be identified and characterised, and assess whether detection can serve as a mitigation strategy. To operationalise this, we introduce a social-science-grounded set of ten hidden intention categories and show that they are trivially inducible. A case study further confirms that all ten categories manifest in deployed LLMs. We then evaluate static classifiers and LLM judges on these categories, providing the first systematic analysis of why hidden intentions are difficult to detect. Our stress tests show that, unless false-positive rates are vanishingly small, auditing is dominated by precision-prevalence trade-offs. Capability scaling and reasoning models do not close this gap, suggesting a fundamental challenge for open-world detection. These findings expose a core gap of current AI governance: without new auditing paradigms for open-world, low-prevalence risks, bans on manipulative AI remain difficult to enforce.

Abstract:
Large Reasoning Models (LRMs) improve performance by allocating additional inference-time compute to generate extended chain-of-thought reasoning. However, recent studies reveal that sequential test-time scaling often yields diminishing or even negative returns, as longer traces exhibit increased uncertainty, error compounding, and drift from the original problem. We propose ThinkRetrieve, a test-time scaling framework that augments the reasoning traces of LRMs with dynamically retrieved solved examples at each reasoning step. Given an external corpus of problems paired with step-by-step solutions, ThinkRetrieve retrieves relevant exemplars at each intermediate step and injects them directly into the thinking trace, providing the model with guidance on how to reason rather than merely what facts are relevant. Experiments across five reasoning models (1.5B-8B parameters) on GSM-8K, MATH-500, AIME 2025, and SciQ demonstrate that ThinkRetrieve consistently improves accuracy over standard test-time scaling, with relative gains of up to 60% on AIME 2025.

Abstract:
Zeroth-order (ZO) optimization is a memory-efficient alternative to backpropagation for fine-tuning large language models, but its deployment is limited by the high variance of gradient estimation. We propose GRZO, a Group-Relative Zeroth-Order optimizer that draws one pseudo-independent perturbation per mini-batch example and aggregates the per-example losses through group-relative normalization, raising the effective gradient-direction count from one to the batch size at no additional forward cost while preserving inference-level memory. We prove that GRZO is directionally unbiased with variance shrinking proportionally to the batch size, yielding a tighter nonconvex convergence bound than MeZO. Across RoBERTa-large, Llama3-8B, and OPT-13B over multiple tasks, GRZO improves average accuracy on Llama3-8B by +3.0 over MeZO while staying within 0.5% of the forward-only inference memory floor; as a drop-in replacement for the MeZO core, it lifts sparse, low-rank, and quantized ZO variants by +4.9 on average.

Abstract:
Vision-language-model-based embodied agents can complete instructed tasks but often violate safety constraints in the process, a problem recently framed as interactive safety. Training such agents to act safely is difficult, since safety and task success are distinct objectives, and safety arises only at a small number of safety-critical steps within a trajectory. Standard supervision is insufficient: imitating safe trajectories teaches behavior without explaining why it is safe, and contrasting arbitrary safe and unsafe trajectories mixes the safety signal with unrelated differences. We propose SafeBranch, a framework that aligns an embodied actor on safety through branch pairs constructed from the actor’s own unsafe rollouts via environment rollback. SafeBranch rolls each unsafe rollout back to the safety-critical step that caused the violation, queries the actor for a safe alternative, and pairs the original action with the alternative so that the two branches differ only at that step. The trained actor acts safely at deployment with no critic in the loop. On IS-Bench, SafetyALFRED, and out-of-distribution variants with unseen tasks and objects, it improves safe task completion, achieving roughly ten times more safe successes than the untrained baseline on the unseen-object variant. Project page: https://github.com/98hyunse/SafeBranch.

Abstract:
Textual anomaly detection (AD) aims to identify samples that deviate from the underlying distribution of normal data, without requiring explicit supervision on anomalies. Existing literature has focused almost exclusively on the semi-supervised setting, assuming access to clean training datasets composed solely of normal samples. However, this assumption rarely holds in real-world scenarios, as datasets used to infer the normal distribution are frequently contaminated by anomalies. In this work, we first highlight the sensitivity of conventional anomaly detection methods to contamination, before demonstrating the relevance of data depth functions for robust textual anomaly detection. Building on recent findings regarding the geometry of language model embeddings, further supported by our empirical observations, we introduce the geodesic depth to better capture the structure of contextual embedding spaces. Our results show that the geodesic depth consistently outperforms commonly used anomaly detection methods once contamination exceeds 10% in the training data, with substantially larger improvements observed in the most challenging scenarios.

Abstract:
Group-relative policy optimization (GRPO) provides a stable recipe for outcome-based RL on language model reasoning, but its credit assignment is coarse: it broadcasts a single trajectory advantage to every token, regardless of how individual steps contribute to the outcome. We call this the uniform-broadcast failure mode and address it with RLPgR, a dependency-aware rescaling of the existing GRPO advantage. RLPgR parses each rollout into a Progress Dependency Graph (PDG), where each node groups a contiguous span of steps into a milestone-level unit with a locally validatable artifact, and directed edges connect a node to the artifacts it depends on. Using the PDG, RLPgR rescales the group-relative advantage by a small, bounded, sign-preserving factor at the node level. This preserves the outcome-governed rollout direction while each step’s update better reflects its dependency role, avoiding the reward-hacking drift of densely shaped process rewards. To reduce PDG construction cost, we train an open Qwen3-8B progress annotator amortizing a one-time proprietary bootstrap. Across model families and sizes on five math reasoning benchmarks, RLPgR is competitive with strong outcome-RL and process-RL baselines under matched training budget. Three controlled studies (scaling-strength sensitivity, segment-level baselines, and PDG label perturbation) support the view that the gain comes from structured credit assignment rather than incidental tuning.

Abstract:
Large Language Models (LLMs) often fail to maintain instruction hierarchies (IH) when processing multi-source inputs with varying role-level priorities, paradoxically adhering to lower-priority directives during conflicts. While existing defenses mitigate this issue, they are largely restricted to single-turn scenarios and require expensive fine-tuning. In this paper, we formalize this failure mode in multi-turn contexts via a Jensen-Shannon Divergence (JSD) framework, uncovering a pervasive role-influence inversion phenomenon where subordinate inputs override superior roles. To rectify this without training, we propose IHDec (Instruction Hierarchy-steered Decoding). IHDec leverages JSD to automatically detect token-level hierarchy violations and dynamically executes contrastive decoding to suppress misaligned subordinate roles. Extensive evaluations demonstrate that IHDec outperforms training-based baselines in multi-turn conflicts while fully preserving general response quality. Furthermore, IHDec strengthens safety against adversarial prompt injections and exhibits a robust scaling synergy with larger models.

Abstract:
Chain-of-thought (CoT) reasoning improves the reasoning ability of large language models by introducing intermediate computation, but explicit rationales increase decoding length, latency, and context cost. Implicit CoT offers a more efficient alternative by moving intermediate reasoning into continuous latent states. However, latent reasoning can be unstable: successive latent states may become overly similar and collapse toward a shared dominant direction, reducing the diversity of the reasoning trajectory. In this work, we identify latent rank collapse and propose WIDER , a lightweight spectral regularizer for implicit CoT. During training, WIDER estimates the shared direction of each latent trajectory and penalizes projections onto this direction, encouraging latent states to span a broader representational subspace. The method is plug-and-play and leaves the backbone model, latent schedule, and inference-time decoding procedure unchanged. We further formulate this collapse as a geometric bottleneck in implicit reasoning, casting its mitigation as a training-time regularization problem rather than an inference-time decoding change. Extensive experiments show that WIDER improves matched implicit CoT baselines, while mechanistic analyses reveal higher effective rank, lower dominant-direction energy, and reduced redundancy among latent steps. These results highlight latent subspace utilization as an important factor for efficient continuous reasoning, providing a geometric perspective for analyzing and improving implicit CoT. Code is available at https://github.com/whitesweater/WIDER.

Abstract:
Optimizer design plays a central role in efficient language model pretraining, directly affecting optimization dynamics, convergence speed, and compute cost under fixed training budgets. Muon has emerged as a strong optimizer by orthogonalizing momentum updates, yielding a matrix-valued analogue of sign-based normalization. However, unlike Adam-style methods, Muon does not explicitly incorporate gradient-variance information into its updates. Motivated by Adam’s variance-adaptive interpretation, we propose Muon-NSR and Muon-VS, two variance-adaptive Muon variants for language model pretraining. Muon-NSR applies noise-to-signal ratio (NSR) modulation before Newton–Schulz orthogonalization, whereas Muon-VS uses variance scaling (VS) without introducing any additional hyperparameters beyond those of Muon. Both methods preserve Muon’s spectral normalization structure while requiring only one additional variance buffer. Experiments on Llama-style and GPT-2 pretraining across model scales from 125M to 1.2B parameters show that our methods improve over well-tuned Muon baselines and remain competitive with representative adaptive Muon-family baselines. On Llama-1.2B, Muon-VS achieves a 1.33× step-to-target speedup over a well-tuned Muon baseline, with Muon’s final validation loss as the target. These results indicate that variance-adaptive modulation is a simple and effective mechanism for improving Muon-style optimizers in language model pretraining.

Abstract:
Large reasoning models have achieved remarkable performance through extended chain-of-thought sequences, yet this computational freedom leads to excessive token generation even for simple problems. We present Length-Adaptive Policy Optimization (LAPO), a novel framework that transforms reasoning length control from an external constraint into an intrinsic model capability. Unlike existing approaches that impose rigid limits or rely on post-hoc interventions, LAPO enables models to internalize an understanding of appropriate reasoning depth through a two-stage reinforcement learning process. In the first stage, models learn natural reasoning patterns by discovering the statistical distribution of successful solution lengths. In the second stage, these learned patterns are embedded as in-context, self-declarative guidance, teaching the model to proactively plan its reasoning budget. Experiments on mathematical reasoning benchmarks demonstrate that LAPO reduces token usage by up to 38.5% while improving accuracy by 2.3%. Our analysis reveals that models trained with LAPO develop emergent abilities to allocate computational resources based on problem complexity, achieving efficient reasoning without sacrificing quality.

Abstract:
Long videos contain far more visual content than Large Vision-Language Models (LVLMs) can process under a fixed visual-token budget, making frame selection essential. Existing query-aware selectors usually estimate frame-query relevance and build a compact subset from high-scoring frames. Although their mechanisms differ, the similarity sequence is still often treated primarily as values to rank or sample from, rather than as an ordered signal whose shape reflects how query-relevant evidence emerges, peaks, and fades over time. This can obscure frames that explain, contextualize, or follow an event, because such evidence may lie on the rising or falling sides of a nearby relevance peak and receive lower absolute scores. We propose RIDGE, a frame selection framework that reads the frame-query similarity curve as a temporal signal. By using local changes and curvature, RIDGE partitions the timeline into structural regions and applies region-specific selection to preserve event cores, transitions, buildup, aftermath, and contextual frames under a fixed budget. It is a lightweight post-processing step on precomputed frame-query scores and requires neither training nor iterative LVLM calls. Across four long-video benchmarks and three backbones, RIDGE achieves the best performance in most settings and remains competitive in the others.

Abstract:
Large language models have achieved strong single-turn language understanding and generation abilities, but long-term agent applications require models to handle dynamic interaction history beyond an isolated inference call. Existing agent memory systems address this need through external storage and retrieval and have achieved promising results. However, many existing frameworks rely on full storage or highly structured memory representations, which can increase cost and noise while disrupting event coherence. Inspired by cognitive views of event organization and adaptive forgetting, we introduce ECO-Mem, an event-centric organization framework for visibility-controlled long-term memory. ECO-Mem segments dialogue into source-linked event episodes and connects them through typed event relations, representing memory as linked events rather than isolated facts. It implements forgetting as non-destructive visibility control instead of destructive deletion. At query time, it traverses the event-relation graph to identify relevant hidden support and reactivates it only when the source evidence confirms the query context. Experiments on LoCoMo and RealTalk validate ECO-Mem’s effectiveness and efficiency for long-term agent memory.

Abstract:
Large language models (LLMs) are increasingly used as sources of health advice, raising urgent concerns about reliability, safety, and personalization. While prior work has studied health advice conflicts, these efforts typically ignore patient context and treat contradictions as universal. In this paper, we propose the task of contextualized divergence detection, which frames divergence not only as strict contradictions but also as context-dependent mismatches. To support this task, we introduce a novel dataset of 10,545 samples that combines user profiles, consumer medication guideline documents, and advice statements, with divergences synthetically generated across four categories: direct, conditional, temporal, and sub-typical. We benchmark a set of LLM-based inference approaches, including prompt-based inference, retrieval-augmented generation (RAG), supervised fine-tuning, and agent-based reasoning. We observe that relevance-filtered RAG achieves the highest accuracy and robustness for large models, while agent-based reasoning improves interpretability at the cost of accuracy, and supervised fine-tuning is essential for smaller models. A qualitative error analysis highlights recurring challenges such as hedging language, temporal mismatches, and profile-advisory inconsistencies. This work provides the first dataset and systematic evaluation of contextualized divergence detection in health advice, paving the way toward safer, more personalized interactions with LLMs.

Abstract:
Powerful code agents can execute scripts, call tools, and manage files, yet many important applications remain accessible primarily through graphical user interfaces. We argue that screenshot-and-click is an inefficient interface for software-operating agents: screenshots are state-incomplete, and GUI actions are brittle, semantically weak, and poorly matched to long-horizon planning. We introduce ASIL (Agent-Software Interaction Layer), an agent-native interface that exposes software through structured JSON observations and code-executable semantic actions, realized through the deepest feasible access path for each application. We instantiate ASIL across 15 applications and a benchmark of 300 single-application and 80 multi-application tasks. ASIL reaches above 80 with closed models while executing fewer than five actions per task. Under a repaired runtime and a 50-step screenshot budget, the same tasks yield 6.6 and 26.6 strict success under screenshot-and-click control, rising to 15.0 and 53.3 on an easier OSWorld-comparable band. Against application-native interfaces on matched tasks, ASIL exceeds LibreOffice’s UNO API by 28-38 strict points but only matches draw.io’s MCP content contract. The structured modality also suits training: small-scale SFT raises Qwen3.5-2B from 58.0 to 72.1 and Qwen3.5-9B from 66.6 to 80.4, and resource-limited on-policy RL further raises them to 74.4 and 82.2.

Abstract:
While reinforcement learning has enabled LLM-based search agents to invoke external tools, existing methods train under fixed budgets and cannot adapt when constraints vary at deployment. We propose AnySearch, a framework that enables a single policy to perform budget-aware search under any budget constraint through a training scaffold and curriculum reinforcement learning. In the first phase, we train the agent with explicit budget state injection and structured reasoning prompts that guide efficient allocation under linearly decaying budgets. In the second phase, the scaffold is removed and the agent learns to operate autonomously under adaptively sampled budget constraints, matching inference conditions. Both phases are optimized with a composite reward that couples answer accuracy with budget efficiency through absolute and relative signals, where an adaptive weight amplifies the efficiency signal for high-accuracy queries and attenuates it for low-accuracy ones. Extensive experiments on seven general and multi-hop QA benchmarks show that our method outperforms baselines across all budget scales, generalizes to unseen constraints beyond the training range, and achieves superior tool productivity without excessive token overhead.

Abstract:
The proliferation of large-scale language models has significantly advanced natural language processing capabilities; yet their substantial computational requirements create barriers for deployment in resource-constrained environments and limit accessibility for under-represented language communities. To address this fundamental issue, we present a novel multi-objective pre-training approach for developing parameter-efficient language models. Our method, implemented in the ARJUN model family, strategically combines four complementary pre-training objectives within a unified transformer encoder-decoder framework: (1) Masked Token Prediction (MT) for bidirectional understanding, (2) Next Token Prediction (NT) for autoregressive generation, (3) Masked Span Prediction (MS) for multi-token dependencies, and (4) Complete Sentence Prediction (CS) to bridge discriminative and generative tasks. Using this approach, our ARJUN-Base model (69M parameters) consistently outperforms standard baselines 2.5-4x larger. Extending this architecture, our scaled-up ARJUN-Large variant (up to 256M parameters) demonstrates superior architectural efficiency, establishing strong performance baselines within matched parameter budgets. Evaluations across 20 discriminative and generative tasks in Bangla, Tamil, and Hindi show prominent gains, including up to +6.5 percentage points (pp) over comparable baselines on macro-F1 scores and +17.5 pp sacreBLEU on generation. Finally, ablation studies confirm that joint training on these four objectives yields synergistic representations that consistently outperform single-task configurations across all scales.

Abstract:
Modern Vision-Language Models (VLMs) perform well above the human baseline in image geolocalization, a task critically important in disaster response, OSINT verification, and location privacy. However, most efforts to study AI behavior on the task remain limited to static image-based retrieval, classification, and predictions. We argue that faithful recreation of the task should involve embodied navigation, where a multimodal agent autonomously explores its surroundings to gather observations before submitting a prediction. To this end, we introduce GeoAgent, an agentic environment-based benchmark that requires agents to navigate Street View environments to refine their geolocalization through sequential reasoning. Our analysis shows that modern VLMs struggle to discern regional patterns while succeeding at country- and continent-level predictions. When compared to static image-based baselines, agentic navigation significantly improves accuracy across established metrics. We also note severe bias in a developed/developing region context across frontier model architectures and poor self-improvement capabilities given incorrect priors. Overall, our work establishes the challenges of embodied navigation and geospatial reasoning. We publicly release our code and the GeoAgent environment: https://geoagent-benchmark.github.io

Abstract:
Aligning large language models (LLMs) with human preferences is challenging due to the inherent heterogeneity and multi-dimensionality of these preferences. Existing inference-time multi-objective alignment methods often rely on fixed weights to combine preference models. However, different tokens contribute unequally to preference objectives during decoding. Ignoring such token-level variation may cause the loss of preference information and lead to objective interference. To alleviate this, we utilize the implicit rewards of preference models to estimate importance differences between tokens, and propose a new training-free and reward-free framework TAW to dynamically balance multiple objectives for inference-time alignment. We design the relative entropy and marginal entropy estimators to quantify the informativeness and uncertainty of preference objectives. TAW with the estimator ensures that more informative or reliable preferences are emphasized, while weak or uncertain signals are suppressed. Experiments on two LLM alignment benchmarks show that TAW outperforms strong baselines without predefined weightings, while also achieves an overall optimal Pareto front when explicit preference weighting is applied.

Abstract:
Multimodal Named Entity Recognition (MNER) uses supplementary images to resolve ambiguity in entity mentions in noisy, short-form social-media text. Prevailing approaches typically fuse text with visually aligned features, but such cues often lack sufficient discriminative power and may support conflicting entity-type hypotheses. To address this issue, we propose TCHV, a lightweight framework grounded in a type-conditioned hypothesis-verification perspective. By formulating each span-type pair as a distinct hypothesis during inference, TCHV predicts the latent feature evidence expected under each hypothesis and validates it against observed multimodal representations through bidirectional text-image matching, using a visual relevance gate to suppress unin- formative images. Verification scores serve as surrogate evidence statistics; TCHV improves over established non-VLLM multimodal baselines on Twitter-2015/2017, and TCHV† leads the compared VLLM-based methods on Twitter-2017. Extensions demonstrate applicability to grounded and fine-grained settings. Code: https://github.com/ hanfeiyu703/ImaginE.git.

Abstract:
Agentic reinforcement learning (RL) has proven effective for training LLM-based agents with external tool-use capabilities. However, we identify that agentic RL training induces increasing redundant tool calls and blurs the model’s intrinsic knowledge boundary, where the model fails to distinguish when tools are needed versus when parametric knowledge suffices. Existing solutions based on reward shaping create coarse-grained optimization targets that tend to incentivize indiscriminate tool-call suppression, leading to reward hacking. In this paper, we propose AKBE (Agentic Knowledge Boundary Enhancement), an on-policy method that dynamically probes the model’s intrinsic knowledge boundary through dual-path (with-tool and no-tool) rollouts during training. We define the knowledge boundary as the per-instance determination of whether tools are required and the minimum tool calls necessary. By comparing correctness across paths, AKBE categorizes trajectories and constructs targeted supervisory signals that guide efficient tool-use patterns for each question. These signals are integrated seamlessly into the agentic RL training loop. Experiments on seven QA benchmarks demonstrate that AKBE improves task accuracy by on average +1.85 and reduces tool calls by 18% over standard agentic RL, yielding 25% higher tool productivity without any accuracy-efficiency trade-off. Further analysis suggests its plug-and-play compatibility across different RL algorithms and the mechanism of each signal category.

Abstract:
Recent VLMs improve multimodal reasoning by generating explicit chains of thought, but often overthink with redundant reasoning. Although conventional length penalties mitigate this redundancy, they rely on coarse global rewards that fail to discern intermediate process quality or adapt to task complexity. To address this dilemma, we propose Binned Relative Policy Optimization (BRPO), a training-efficient framework that enables non-trivial process-level evaluation without introducing costly process reward models. Specifically, BRPO leverages high entropy lexical markers to segment reasoning trajectories into discrete intermediate prefixes, evaluating their problem-solving potential through forced answer completions. Building on this, we introduce Bin Normalization along with a length-aware adjustment to map prefixes into process bins, dynamically calibrating rewards and regulating length constraints based on reasoning stages and visual complexity. Experiments on six multimodal math benchmarks show that our method improves both accuracy and efficiency: on Qwen2.5-VL-7B, it raises average accuracy from 47.8% to 53.6% while reducing the average response length by 39.0%.

Abstract:
We show that attention sinks and compression valleys create a vulnerable region in decoder-only Transformers, where small activation perturbations can be amplified through the autoregressive trajectory. Based on this, we propose Sensitivity-Scaled Steering (SSS), a progressive activation-space attack that anchors perturbations at the beginning-of-sequence token and adaptively reinforces them at sensitive layers and tokens. Instead of forcing an abrupt behavioral change, SSS induces a staged drift, making outputs gradually shift toward the target behavior while remaining fluent and benign-looking in early generations. Across multiple open-weight models and four behavioral axes, SSS achieves high attack success, preserves coherence, and causes negligible degradation to general capabilities. These results show that attention sinks and compression valleys are not merely mechanistic features; rather, they expose exploitable amplification mechanisms that can be treated as hidden-state weaknesses for activation-space attacks in white-box and supply-chain LLM deployments.

Abstract:
Personalizing large language models (LLMs) requires reflecting stable user preferences while adapting their expression to each target context. Soft-prompt personalization has become a practical route to this goal, but existing methods produce soft prompts in which stable preferences, target-induced content, and context-dependent expression are bundled into an entangled conditioning signal without separately addressable components, leaving inference with no principled space to intervene on individual aspects of personalization. We propose Factorized soft prompts with Latent Control (FaLCon), an LLM personalization framework built on two coupled stages. First, factorized soft prompts replace this entangled conditioning with a set of separately addressable preference factors, forming a structured control interface. Second, inference-time latent control treats the resulting factor weights as a continuous action space and adapts them per instance from teacher-forced likelihood feedback, keeping the LLM frozen and updating only a low-dimensional factor-weight posterior without back-propagating through the LLM. Experiments on multiple personalized generation tasks reveal that FaLCon consistently outperforms strong baselines, validating the individual necessity and joint effectiveness of the structured interface and the inference-time controller.

Abstract:
Long-form video understanding remains challenging for video agents due to the mismatch between query demands and evidence acquisition strategies. Although recent planning-before-perception methods outperform query-agnostic pipelines, they often rely on a single dominant strategy, either generation-based strategy or retrieval-based strategy, limiting their ability to handle diverse query demands. We propose Route2Look, a lightweight and model-agnostic framework for query-adaptive evidence acquisition in long-form video understanding. Route2Look operates in a Route-Look-Memorize loop with three tools: Global Browse for holistic context, Temporal Ground for explicit temporal cues, and Semantic Retrieve for semantic search. The core component is a routing policy that dynamically selects evidence acquisition tools based on the query. To build this policy, Route2Look adopts a two-stage design: first distilling the routing skill from differential contrastive analysis between generation-based and retrieval-based trajectories, and then applying the distilled skill with hard routing rules and continue-or-stop criteria during inference. Experiments on challenging long-video benchmarks show that Route2Look achieves state-of-the-art performance while maintaining strong frame efficiency across datasets and query types. Oracle routing analysis further reveals the potential of query-adaptive evidence acquisition for future long-form video understanding.

Abstract:
When a vision-language model (VLM) is shown an irrelevant image containing a digit (the anchor image) alongside a numeric question, the model’s numeric answer shifts toward that digit. This matters because multi-image inputs (retrieval-augmented QA, multi-screenshot analysis, VLM-as-judge) routinely contain irrelevant numeric content. We call this phenomenon cross-modal numerical anchoring: across six VLMs and five numeric VQA datasets, the per-model mean SDF (signed direction-follow rate) ranges from 3% to 24%, with the effect growing stronger as the model’s confidence in the base answer decreases. Masking only the digit pixels of the anchor image, leaving the rest of the image intact, recovers the non-anchored answer, isolating the digit pixels as the causal channel, with the effect concentrated at low confidence. We propose a mitigation that uses this anchored-masked contrast as a calibration signal: at a single per-model integration site, we estimate a low-rank subspace spanning the anchoring directions and remove it via a projection at inference time. On LLaVA-OneVision-7B, the projection reduces anchoring ( 𝛥 SDF a =-1.64 pp, rel% -30.0% ) without degrading accuracy on non-anchored inputs or average performance on six capability benchmarks (within ±1 pp).

Abstract:
Multi-agent LLM systems coordinate through natural-language messages that consume 40–60% of their token budget. Replacing these with structured graphs reduces cost but fails on tasks requiring adaptive reasoning. We propose Routed Graph Handoff , where a lightweight LLM router (155 tokens, 0.15% overhead) selects between a typed dependency graph and natural language for each delegation. On four benchmarks (1,050+ trajectories), the routed system matches or exceeds NL-only on every task: +12.7,pp on Tau-retail at 3.2 X compression (p<0.01), +8.7,pp on BrowseComp at 2.2 compression (p<0.05), and parity on BFCL and AppWorld. Without the router, graph-only delegation regresses 14.6,pp on AppWorld—the router eliminates this at near-zero cost. A graph-aware executor prompt is load-bearing: the same schema without interpretation guidance yields no gain. An oracle analysis reveals 8.6,pp of additional headroom, motivating execution-time adaptive routing as future work.

Abstract:
Integrating supervised fine-tuning (SFT) and reinforcement learning (RL) has become a common post-training paradigm for improving the reasoning capabilities of large language models. Existing methods typically rely on sequential SFT–RL pipelines or static loss weights, overlooking sample-level heterogeneity and evolving learning states. We propose Sample Learning State (SLS), a rollout-based characterization of each training sample using two complementary signals: sample mastery and exploration-trajectory dispersion. These signals estimate whether reward-driven updates are reliable for the core task and whether current rollouts provide informative contrast for RL optimization, respectively. Based on SLS, we design a state-aware coefficient that dynamically balances SFT and RL losses, increasing the RL weight only when the model demonstrates sufficient core-task competence and the sampled trajectories provide useful reward contrast. Experiments on Qwen2.5-Math models show that SLS achieves state-of-the-art results on four in-distribution and two out-of-distribution mathematical reasoning benchmarks. In multi-reward settings, SLS preserves core reasoning performance while mitigating reward hacking induced by auxiliary rewards.

Abstract:
Whether intermediate reasoning is computationally useful or merely explanatory depends on whether chain-of-thought (CoT) tokens contain task-solving information. We present a mechanistic causal analysis of CoT on GSM8K using _activation patching_: transferring token-level hidden states from an arithmetic CoT generation to a direct-answer run for the same question, then measuring the effect on final-answer accuracy. Across models, patching can elicit generations that lead to correct answers, revealing that individual CoT hidden states can encode sufficient task-solving information even when the original CoT trace is incorrect. This information is more prevalent in correct than incorrect CoT runs and is unevenly distributed across tokens, concentrating in mid-to-late layers and appearing earlier in the reasoning trace. Moreover, patching from language tokens such as verbs and entities more frequently elicits correct reasoning traces, whereas patches from mathematical tokens more often produce answer-proximal generations that rarely succeed. Further analysis shows that earlier CoT tokens concentrate recoverable information, while later ones require broader search to find the correct answer. Together, these findings show that CoT encodes recoverable, token-level task-solving information, revealing how arithmetic reasoning is represented, accessed, and disrupted.

Abstract:
Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across languages remain underexplored. We investigate the cross-lingual transferability of FVs using multilingual multi-label emotion recognition as a challenging semantic classification benchmark. Specifically, we examine whether FVs extracted from a source language can steer task behavior in another language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture language-agnostic, task-relevant signals rather than purely language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of attention heads for constructing effective FVs, and the pattern remains consistent across languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.

Abstract:
Vision-language models must reconcile visual evidence with memorized world knowledge when the two conflict. How they resolve this conflict shapes the reliability of multimodal systems, yet prior work characterizes it behaviorally without a component-level causal account. We combine activation patching across three granularities (residual stream, attention heads, and MLP sublayers) with model-component ablation studies and mechanistic analysis. Across three VLM families, we find that visual grounding emerges by default, whereas prior grounding depends on a small set of causally necessary attention heads (2.5-4.8%) concentrated in the second half of the network. These heads enable answers from stored world knowledge (e.g., “red” for a strawberry) despite conflicting visual input. Ablating them flips predictions from knowledge-grounded to visually grounded answers in 68-96% of cases under prior-knowledge prompts, but changes only 0.8-7.5% of visually grounded predictions, establishing an asymmetric causal structure. The identified heads decompose into routing heads, which modulate information flow, and writing heads, which directly project answer tokens into the residual stream. This structure is consistent across model families and scales, revealing a sparse causal circuit underlying perception-knowledge conflict in VLMs.

Abstract:
While in-context learning is generally shown to be effective for Large Language Models (LLMs), misleading or flawed contexts can cause performance degradation and mode collapse, an empirical phenomenon we term "pigeonholing". Unintentionally bad contexts can happen without jailbreak intent. For example, a user prompts the model to justify an incorrect math theorem or fails to correct buggy agent code. Specifically, we investigate pigeonholing in two settings: (1) when a user suggests an incorrect solution, and (2) when the multi-turn dialogue history contains the assistant’s incorrect answers from prior turns. Our evaluations across 10 verifiable and open-ended tasks with 10 different models show that pigeonholing manifests in three distinct ways: (1) repeating the incorrect answers from the context (causing a 38-40% performance drop), (2) reducing answer diversity in open-ended text generation, and (3) reversing stances on controversial topics to align with the previous user or assistant’s perspective. Further, we find that pigeonholing worsens almost monotonically with context length (i.e., conversation turns). Finally, while standard RLVR-trained models show improvements on clean prompts, they fail to maintain performance on corrupted contexts. We take a step towards mitigating model vulnerability by training with RLVR on synthetic errors, improving model performance on bad contexts by 43–60% over vanilla baselines.

Abstract:
Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more effectively. We study whether Large Language Models (LLMs) can similarly benefit from such experiential abstractions. From LLMs’ solution traces on a training set, a stronger teacher or the LLMs themselves extract natural-language abstractions into a retrievable library. We explore two modes of abstraction usage: (1) inference-time retrieval and (2) reinforcement learning (RL) with abstraction-augmented training prompts. Experiential abstractions improve LLM performance on mathematical and logical reasoning benchmarks. Self-extracted abstractions match teacher-extracted ones, and our abstraction usage framework can transfer to other datasets and models. These findings suggest LLMs can extract and apply experiential abstractions much as humans leverage distilled experience.

Abstract:
Open scientific document embedders are typically trained on binary citation edges between paper title-abstract pairs. This signal discards the citation context sentence: a domain-expert description of the cited paper, written by the citing author. We use these sentences as the primary contrastive supervision signal to train SciEmbed, an open 149M ModernBERT-base encoder. On the official SciRepEval benchmark it reaches 66.85 over three seeds, surpassing prior open scientific embedders such as SPECTER2 and SciNCL. To isolate what the supervision source contributes, we compare against IBM Granite R2, a same-size ModernBERT peer trained on general retrieval data. Using only open citation-context data, SciEmbed matches Granite R2 to within 0.6 on the SciRepEval macro overall while specialising in the opposite direction: it leads regression tasks tied to paper impact by 2.3 points, where Granite R2 instead leads classification, proximity, and search. A 28-configuration sweep over recipes and alternative supervision signals does not improve on this result, consistent with the specialisation tracking the supervision source rather than the training recipe. Training at longer contexts yields monotonic gains on a body-fact retrieval probe within our recipe. We release the model, training pipeline, and evaluation harness as a fully open, citation-context-supervised baseline with a distinct, reproducible specialisation pattern.

Abstract:
Hybrid language models combine attention with a fixed-size recurrent state, but the role of each channel remains unclear. We introduce two cache-level interventions. Split-prefill keeps only the KV cache or only the recurrent state from a prefilled context, then generates an answer. State-swap pairs the KV cache from one context with the recurrent state from another in a single forward pass. On Qwen3.5 and Falcon-H1, the two channels split cleanly. Exact retrieval survives only through attention (64–98% of full accuracy) and collapses to zero through recurrence. Output language and persona reverse the pattern: both survive recurrence (70–80%) while KV-only drops to 1% language accuracy. State-swap confirms this causally: the answer takes its value from the KV side and its language from the recurrent side. Recurrent-only generation also accepts words that were never in the context but share meaning or parts with seen items. Attention provides a lookup over what was said; the recurrent state shapes how the model says it next.

Abstract:
Decoding language from intracortical activity is a sequence transduction problem that stress-tests modern NLP methods: inputs are high-dimensional and nonstationary, alignment to linguistic units is unknown, and training data are orders of magnitude smaller than in standard translation or speech recognition. We study attempted-speech decoding from intracortical recordings as a neural-to-language translation task and ask what contextual sequence-to-sequence modeling contributes beyond the framewise connectionist temporal classification (CTC) pipelines that dominate this domain. We introduce a multitask Transformer encoder–decoder that jointly predicts phonemes, words (via a BART-initialized decoder), and auxiliary acoustic features, together with a day-conditioned calibration module (Neural Hammer Scalpel, NHS) that combines a global affine transform with feature-wise linear modulation to handle session-level distribution shift. On the Willet et al. dataset, our model achieves state-of-the-art phoneme error rate (14.3%); direct word decoding reaches 25.6% WER, falling to 19.4% with rescoring, however still trailing multi-stage systems with heavy language-model refinement. Ablations and held-out-day experiments quantify the contributions of multitask supervision and NHS and the cost of session drift, while attention analyses reveal recurring temporal chunking that phoneme and word decoders consume differently. Together, these results position seq2seq modeling as both a competitive decoder and an interpretable probe for how language evidence is organized in neural inputs.

Abstract:
Aligned large language models are trained to comply with benign requests while refusing harmful ones. However, recent work shows that this refusal behavior is fragile and can often be disrupted by simple activation- or weight-space interventions. In particular, refusal has been shown to be largely mediated by a single residual-stream direction, typically computed as a difference-of-means vector between harmful and harmless prompts. However, this standard construction aggregates heterogeneous harmful prompts into one uniform average, potentially conflating distinct refusal-related phenomena. In this work, we study whether refusal interventions can be strengthened by learning how to combine prompt-specific refusal directions. We propose Weighted Refusal Direction Optimization (WRDO), a procedure that constructs a composite refusal direction from a bank of per-prompt directions using learned aggregation weights. This approach allows different harmful prompts to contribute unequally to the final intervention direction. We evaluate WRDO against prior single-direction methods on three open-weight language models using three guard judges and HarmBench as an external evaluator. We further measure the effect of the intervention on general model capability through standard utility benchmarks. Finally, we investigate a defense-oriented setting in which the same optimization machinery is used in reverse to restore refusal behavior in an abliterated model checkpoint. Our results show that weighted aggregation provides a more effective and flexible mechanism for manipulating refusal representations. Code is available at https://github.com/brain-lab-research/WRDO .

Abstract:
Reasoning over language instructions in embodied tasks such as robotics often requires understanding spatial relations from a speaker’s situated perspective. Humans infer such perspectives from shared environmental knowledge, activity context, and commonsense. Recent vision-language models (VLMs) appear capable of spatial reasoning, but their ability to infer a speaker’s viewpoint from contextual cues and interpret situated spatial relations from that viewpoint remains unclear. We call this capability contextual observer grounding. To study this capability, we construct the Point-of-View Benchmark (POVBench), a dataset of 3D scenes and queries that disentangles Inferred, Stated, and Given forms of observer grounding in natural embodied communication. Given multi-view observations and a natural-language sentence, models must localize unseen or underspecified targets from situated spatial and contextual cues. Across state-of-the-art VLMs, localizing targets from directional language remains challenging, even when observer grounding is made explicit. We find that explicit breakdowns of observer-relative spatial reasoning improve target localization. Our project page is available at https://mimo-owl.github.io/POVBench/.

Abstract:
Multimodal intent recognition predicts a speaker’s intent from text, audio, and video, enabling conversational AI and human–computer interaction. However, existing methods share two limitations: (I) the three modalities are assigned equivalent roles even though text alone carries most of the predictive signal; (II) nonverbal cues that complement text are mixed with cues that merely repeat it. We propose Asymmetric Multimodal Learning with Residual Decomposition (AML), in which text produces the primary prediction and nonverbal signals enter only as a correction. The correction splits into Hierarchical Modality Calibration (HMC), which removes cues already present in text, and Asymmetric Multimodal Disentanglement (AMD), which carries cues that add new information. Extensive experiments on MIntRec2.0 and IEMOCAP-DA show that AML consistently surpasses prior state-of-the-art methods across all reported metrics, and additional analyses confirm the structural advantages of the asymmetric design.

Abstract:
Personalizing large language models requires adapting model behavior to individual users while preserving robustness and deployment-scale efficiency. Existing approaches typically personalize LLMs either at the input level, by retrieving user histories or constructing profile prompts, or at the parameter level, by maintaining user-specific parameter-efficient modules. The former makes personalization sensitive to retrieval quality and prompt design, whereas the latter incurs storage and maintenance costs that grow with the user population. To address these limitations, we propose TAP-PER (Temporal Attentive Prefix for PERsonalization), a prefix-based framework that encodes user preferences as learnable representations, avoiding the serialization of user histories into prompts and replacing heavy per-user adapters with lightweight user-state prefix embeddings. Inspired by personalized recommendation systems, TAP-PER decomposes user modeling into user-state and query-conditioned components, and incorporates temporal signals to capture the evolving nature of user interests. Experiments on six LaMP tasks show that TAP-PER consistently outperforms prompt-based and model-based baselines across classification, rating, and generation settings. Moreover, TAP-PER uses 130× fewer per-user parameters than OPPU and roughly half the total parameter footprint of PER-PCS at the 1,000-user scale, demonstrating scalable personalization without prompt-serialized histories or heavy per-user adapters.

Abstract:
Both models and minds not only encode features that describe an entity, but can specifically represent which features of that entity are relevant for a given context. This core capability underlies in-context learning in language models and a variety of cognitive phenomena in minds, yet it is unclear how intelligent systems identify and promote relevant features. In this work, we propose Amortized Direction Accentuation (ADA), an algorithmic hypothesis that provides an explanation for how relevance is determined in the context of few-shot concept induction—the ability to infer a latent concept from a small number of positive examples. Under ADA, concept induction proceeds by forming a summary representation of the examples in a feature space that is shaped by amortized knowledge about which features tend to be useful for finding commonalities between items. We investigate and validate ADA in language models using a few-shot concept induction task. This work provides a novel perspective on the mechanisms supporting contextualization in language models and a promising hypothesis for a core aspect of cognition.

Abstract:
Prompt ambiguity is a common source of failure in large language models, but is difficult to localize because it is a latent property of the prompt, while existing attribution methods are designed to explain observable outputs such as logits or generated tokens. We introduce PRIG, a gradient attribution method that uses a probe logit to attribute latent properties to token positions. Specifically, PRIG trains a linear probe to distinguish clear prompts from ambiguous prompts and attributes the probe score to earlier token representations in the residual stream. To enable token-level evaluation, we construct synthetic ambiguity datasets across coding, math, and writing by rewriting one task-critical sentence per prompt, and complement them with a human-written gold benchmark. In this setting, PRIG localizes ambiguous spans substantially better than gradient attribution baselines, achieving 0.840 AUROC on the combined synthetic benchmark. It also outperforms GPT-5.4 on sentence-level ambiguity identification and retains useful signal out-of-domain. More broadly, these results show that latent prompt properties can be localized through intermediate representations, rather than through output-level attribution.

Abstract:
Post-training quantization (PTQ) methods are typically architecture-specific. Yet two distributional pathologies they target, dense cores with sparse outliers and input-conditioned activation clusters, recur across model families. We argue that for these pathologies the unit of selection should be the tensor distribution, not the architecture. We propose two composable primitives, dispatched from one calibration pass by a two-statistic diagnostic: Dual-Band Affine Folding (DBAF) where a per-tensor outlier gate fires, and Prompt-Conditioned Scale Anchoring (PCSA) where a per-prompt compactness ratio does. The diagnostic is a selection heuristic, not a calibrated predictor. Four prediction-then-outcome checks are consistent with the dispatch: a rotation cliff on a six-host LLaMA-3-8B W4A4 matrix, cross-family transfer to Qwen-2.5-7B, per-site gating on the KV-cache and SAM-B (+5pp NIAH at 8k, +4.8 mAP), and a negative SwinIR-×3 prediction at training-free W4A16. Both primitives are training-free and calibrate in seconds. The gate’s boundary is learnable, not merely hand-set: a logistic dispatcher trained with each model family held out reproduces it at mean AUC 0.962. Calibrated once on LLaMA-3-8B, the same thresholds dispatch across seven model families in six domains; off distribution the gate errs toward skipping.

Abstract:
We present a novel approach for claim verification from tabular data documents. Recent LLM-based approaches either employ complex pretraining/fine-tuning or decompose verification into subtasks, often lacking comprehensive explanations and generalizability. To address these limitations, we propose a Multi-Agentic framework for Claim vErification (MACE) consisting of three specialized agents: Planner, Executor, and Verifier. Instead of elaborate fine-tuning, each agent employs a zero-shot Chain-of-Thought setup to perform its tasks. MACE produces interpretable verification traces, with the Planner generating explicit reasoning strategies, the Executor providing detailed computation steps, and the Verifier validating the logic. Experiments demonstrate that MACE achieves state-of-the-art (SOTA) performance on two datasets and performs on par with the best models on two others, while achieving 80–100% of best performance with substantially smaller models: 27–92B parameters versus 235B. This combination of competitive performance, memory efficiency, and transparent reasoning highlights our framework’s effectiveness.

Abstract:
Shopping assistants are shifting from ranked product lists toward structured decision support, where systems must synthesize shopper context, product evidence, and next-step guidance into a coherent recommendation experience. This changes the unit of evaluation: a fluent response can still fail by ignoring shopper context, contradicting itself across components, or leaving defects too vague to localize. Existing personalization, grounding, and LLM-as-a-judge benchmarks cover pieces of this problem, but they do not define a joint evaluation target for structured shopping-assistant responses. We formulate this missing evaluation target as PACE: Personalized, Actionable, Compositional, and Evidence-grounded evaluation. We instantiate PACE with two artifacts: PACEShop, a benchmark dataset that makes the target measurable through 22,625 controlled records with structured personas, auditable evidence pools, GOOD/BAD labels, and gold defect family and location annotations; and PACEJudge, a training-free judging protocol that makes the target reportable through a structured output contract. Our experiments show that generic judges can recognize broad quality but fail to recover the diagnostic fields required for PACE; PACEShop makes these failures verifiable, and PACEJudge improves persona-source, cross-component, grounding, and family/location closure without retraining, showing that realistic shopping-assistant evaluation requires a task-matched output contract rather than only a stronger backbone or scalar prompt.

Abstract:
Retrieval-Augmented Generation (RAG) grounds large language models in external corpora, but implicit trust in retrieved documents creates a critical attack surface: PoisonedRAG shows that a handful of crafted passages can dominate dense retrieval and steer generation toward attacker-chosen answers. We present the Tri-Layer Sieve, a middleware defense that sanitizes retrieved evidence through cross-embedding-space clustering with an independent judge model, structural filtering of trigger–payload artifacts, and LLM consistency verification. The design exploits a key weakness of retrieval-stage poisoning: a single document must satisfy one embedding geometry, one internal Trigger–Payload structure, and one generation objective — rarely all three simultaneously, a fragility that persists even against an adaptive attacker who paraphrases around it. On Natural Questions, HotpotQA, and MS-MARCO with Contriever retrieval ( k=50 ), the Sieve reduces black-box Attack Success Rate from 67.0/87.0/64.0% to 3.0/14.0/4.0%, mitigates white-box HotFlip attacks from ∼ 74% to 27.8% on NQ with Layer 3 enabled, and drives poisoned-document MRR to 0.000, while restoring clean accuracy from 13–33% under attack to 58–76%. Under an architecture-aware adversary who paraphrases triggers to evade the structural filter, enabling the consistency layer halves adaptive ASR (32.0% → 15.0% on NQ) while raising clean accuracy by 18 points, at an added latency of ∼\!16 –19 s/query under live retrieval.

Abstract:
Speech inpainting reconstructs missing or corrupted speech segments, enabling restoration of damaged virtual lectures and online presentations. While existing audio-only approaches perform well for short gaps, their performance degrades significantly for long-duration missing regions. In this work, we introduce a novel multimodal speech inpainting framework that leverages accompanying presentation slides as semantic grounding for reconstructing both short and long missing speech segments (10 ms–1500 ms). Our approach reformulates speech inpainting from a local signal-reconstruction problem into a contextual speech generation task using a LALM . To improve acoustic flexibility during generation, we propose ASG , a token-grouping strategy that relaxes conventional hard-label training. In addition, we incorporate an auxiliary CTC objective to improve alignment between generated speech and target lexical content. Experiments on the M3AV-CHI benchmark demonstrate that the proposed multimodal framework substantially outperforms an audio-only baseline and consistently improves intelligibility and reconstruction quality across challenging long-duration gaps. Code and demo samples are available online. https://github.com/yanir1/Speech-Inpainting

Abstract:
LLM agents have recently shown promise in automating machine learning engineering by editing model and training code under execution feedback. Data, however, remains largely outside this agentic optimisation loop. We frame pre-training data selection as heuristic engineering over per-document features, i.e., lexical statistics, categorical labels, and perplexity. We introduce AutoData, an agent that searches directly over executable selection algorithms. Unlike prior data mixture methods that optimise weights over a fixed set of domains, AutoData searches a richer program space of scoring, stratification, and stochastic selection rules, discovering feature interactions automatically by iteratively refining algorithms with validation feedback from a proxy model. Within an overnight search, AutoData discovers a selection algorithm that outperforms existing human-designed curation pipelines. Despite being searched only on this small proxy, the discovered recipe transfers to larger scales and improves the downstream metric CORE. These results suggest that data engineering can be treated as an agentic machine learning problem, extending autonomous research from model and training-code optimization to the data.

Abstract:
Chain-of-thought (CoT) reasoning sometimes fails to faithfully reflect the true computation of a large language model (LLM), hampering its utility in explaining how LLMs arrive at their answers. Moreover, optimizing for faithfulness and interpretability in reasoning often degrades task performance. To address this tradeoff and improve CoT faithfulness, we propose Reasoning Execution by Multiple Listeners (REMuL), a multi-party reinforcement learning approach. REMuL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful. A speaker model generates a reasoning trace, which is truncated and passed to a pool of listener models who “execute” the trace, continuing the trace to an answer. Speakers are rewarded for producing reasoning that is clear to listeners, with additional correctness regularization via masked supervised finetuning to counter the tradeoff between faithfulness and performance. On multiple reasoning benchmarks (BIG-Bench Extra Hard, MuSR, ZebraLogicBench, and FOLIO), REMuL consistently and substantially improves three measures of faithfulness – hint attribution, early answering area over the curve (AOC), and mistake injection AOC – while also improving accuracy. Our analysis finds that these gains are robust across training domains and are associated with shorter and more direct CoTs.

Abstract:
Interviews collect information through conversation, requiring interviewers to ask appropriate questions, adapt to respondent behavior, and recover structured information from dialogue. Yet, AI systems’ abilities as interviewers remain under-explored. We present InterviewBench, a benchmark for evaluating LLM-based AI interviewers through both local interviewer decisions and dynamic multi-turn interviews. The static setting abstracts real interviews into controlled local-decision instances, while the dynamic setting evaluates end-to-end interviewing through long-horizon conversational simulations. Built from real interview transcripts, InterviewBench contains 5,500 classification questions, 3,000 generative questions, and 184 dynamic scenarios. We further model interviewer evaluation as contextual decision selection, where the model must choose or generate the next interviewer action conditioned on dialogue context. Evaluating 23 open-source and proprietary LLMs, we find that current models can produce fluent interviewer utterances, but still struggle with strategy selection, explicit evidence elicitation, long-horizon interaction management, and recovery from non-ideal interviewee behavior.

Abstract:
Watermarking has become a key mechanism for distinguishing AI-generated from human-written texts. In practical settings, however, content produced by large language models (LLMs) is often subject to critical post-generation edits, posing challenges for detecting and localizing such modifications. In this work, we propose incorporating error-correcting codes, specifically Varshamov-Tenengolts (VT) codes, by embedding VT code strings into watermarked LLM outputs, thus enabling post-generation edit detection of token insertions and deletions. We accompany such a watermarking procedure with a watermark detector and a post-edit detector. We evaluate our method on open-source LLM using synthetic edits and demonstrate excellent empirical performance of post-generation edit detection.

Abstract:
We investigate the effect of script variants on language modeling in Japanese, which uniquely combines logographic (kanji) and phonetic (kana) writing systems. Motivated by educational curricula in which learners are gradually exposed to increasing numbers of logographic characters, we simulate this staged exposure during training to study its effect on model behavior. We construct multiple versions of a corpus with controlled script transformations, including fully transliterated kana text and intermediate variants corresponding to curriculum-based kanji levels, and train language models under different schedules that vary the timing and progression of script exposure during pre-training. In addition, we propose a cross-script evaluation framework using multiple versions of a minimal-pair benchmark with systematically varied kanji usage to assess the robustness of language models to script variation. Our results show that training on a mixture of two extreme script variants improves robustness across script conditions, whereas curriculum-based progression does not consistently improve performance.

Abstract:
Subword tokenization introduces a computational layer in language models, where many distinct token sequences decode to the same surface form while preserving meaning, yet induce different internal computations. Despite this non-uniqueness, language models are typically trained using a single canonical deterministic tokenization. In this work, we use non-canonical subword variants as a strictly meaning-preserving form of data augmentation. We introduce a lightweight training architecture that conditions canonical next-token prediction on sampled subword variants via an auxiliary causal encoder and block-causal cross-attention, without modifying the training objective or token interface. In data-constrained pretraining, augmentations with subword variants consistently delay overfitting under repeated data exposure and reduce held-out next-token loss across evaluation datasets. In multilingual fine-tuning, we find that the effectiveness of training with auxiliary subwords depends on tokenizer quality: gains are strongest when canonical tokens are highly compressed and diminish when the tokenizer already over-fragments the input.

Abstract:
Human reasoning is inherently multimodal: when problems become difficult, we rarely think in words alone. We often externalize our reasoning by sketching diagrams or drawing grids to understand the underlying conceptual structure and avoid mistakes. Building on this premise, our research investigates: (a) whether grounding multi-hop textual-spatial stories into geometry-aware modalities, such as layouts or grids, improves reasoning compared to natural language-based inference; and (b) whether a model can decide when to rely on natural language reasoning and when to switch to a structured modality. We address these questions by introducing a switching metric based on trustworthiness and complexity signals, which estimates when grounding a spatial story into a structure is likely to improve performance. This takes a first step toward principled modality selection in large language model (LLM) reasoning. Across our settings, switching from natural language-based reasoning to a grid-based representation improves LLM performance by up to 42%, highlighting the importance of modality choice in shaping reasoning outcomes.

Abstract:
Cloud-hosted vision-language models (VLMs) may be silently fine-tuned, poisoned, compressed, or replaced after deployment, motivating black-box fingerprint verification. RESF tests response distributions but requires estimating the service temperature, making its reliability sensitive to estimation error. We propose Latent Visual Sensitive Fingerprinting ( LVSF ) , based on a local Pearson-sensitivity decomposition of the conditional top- k distribution of the first generated token. This decomposition separates observable output shifts from input-dependent exposure to parameter changes. Modeling tampering-induced relative top- k logit shifts as isotropic, we show that a uniform top- k anchor maximizes average sensitivity. Accordingly, LVSF equalizes recorded top- k logits, maintains a margin from out-of-support tokens, and maximizes full-parameter Fisher sensitivity, yielding a reference distribution invariant to positive temperature scaling. These principles apply to both LLMs and VLMs. For VLMs, LVSF fixes a natural-language query and optimizes an image in a pretrained VAE latent space to produce natural-looking fingerprints. Repeated API queries detect excess out-of-support first tokens or shifts in conditional in-support frequencies. Across four VLMs and multiple tampering types, LVSF achieves consistently high detection rates with no observed false positives and substantially higher detection rates than evaluated baselines on Qwen2.5-VL-7B-Instruct.

Abstract:
Voice agents are increasingly deployed across enterprise applications. However, no existing benchmark jointly addresses realistic conversation simulation and comprehensive voice-specific evaluation. We present EVA-Bench, an end-to-end evaluation framework that addresses both. On the simulation side, EVA-Bench orchestrates dynamic bot-to-bot audio conversations with automatic simulation validation that detects user simulator error and appropriately regenerates conversations before scoring. On the measurement side, EVA-Bench introduces two composite metrics: EVA-A (Accuracy) and EVA-X (Experience). EVA-Bench includes 213 scenarios across three enterprise domains, a controlled perturbation suite for accent and noise robustness, and multi-trial measurements that distinguish peak from reliable capability. Across 12 systems spanning three architectures, we find: (1) no system simultaneously exceeds 0.5 on both EVA-A pass@1 and EVA-X pass@1; (2) peak and reliable performance diverge substantially (median pass@k - pass^k gap of 0.44 on EVA-A; and (3) accent and noise perturbations expose substantial robustness gaps, with effects varying across architectures, systems, and metrics (mean Δ up to 0.314). We release EVA-Bench under an open-source license.

Abstract:
Retrieval-augmented generation (RAG) is a common strategy to reduce hallucinations in Large Language Models (LLMs). While reinforcement learning (RL) can enable LLMs to act as search agents by activating retrieval capabilities, existing ones often underutilize their internal knowledge. This might lead to redundant retrievals, potential harmful knowledge conflicts, and increased inference latency. To address these limitations, an efficient adaptive search agent capable of discerning optimal retrieval timing and synergistically integrating parametric (internal) and retrieved (external) knowledge is in urgent need. This paper introduces the Reinforced I nternal-External K nowledge Synergistic R E asoning A gent ( IKEA ), which could indentify its own knowledge boundary and prioritize the utilization of internal knowledge, resorting to external search only when internal knowledge is deemed insufficient. This is achieved using a novel knowledge-boundary aware reward function and a knowledge-boundary aware training dataset. These are designed for internal-external knowledge synergy oriented RL, incentivizing the model to deliver accurate answers, minimize unnecessary retrievals, and encourage appropriate retrievals only when its own knowledge is lacking. Evaluations across multiple knowledge-intensive reasoning tasks demonstrate that IKEA significantly outperforms baseline methods, reduces retrieval frequency significantly, and exhibits robust generalization capabilities.

Abstract:
Prompt optimization can improve multi-agent LLM systems, but the prompts being optimized often serve two entangled roles: generating task-relevant content and specifying execution-critical protocols, such as message routing, output formatting, and termination signals, on which the underlying code relies. As a result, a prompt edit intended to improve content generation can inadvertently corrupt the protocol and cause the entire agent pipeline to fail. Our key observation is that these two roles have different representations: execution protocols are typically structured, while task-relevant content is usually expressed in unstructured language. Based on this, we propose control-data flow separation, where execution-critical control is represented as typed, validated program objects, while task-relevant language remains the optimizable data flow for agent communication. This design allows optimizers to improve multi-agent behavior without exposing the routing or formatting interface to prompt drift. Across synthetic reasoning, collaborative review generation, and insurance rating workflows, our framework empirically achieves 100% eventual protocol validity while consistently improving task performance.

Abstract:
Tool-integrated LLM agents are often wrapped within a harness: the scaffolding that determines which tools are exposed, how they are described, and what auxiliary information accompanies each per-step observation. While agents are routinely post-trained, this scaffolding is typically treated as a fixed engineering detail, with design effort limited to the training-free regime. Moreover, existing post-training algorithms assume a static environment, even though tool environments and tasks often shift upon deployment. To address this gap, we extend ALFWorld (i) to treat the harness as a controllable design dimension and (ii) to support evaluation under task and tool environment shifts. Building on this, we systematically analyze how the harness design influences post-training in both in-distribution and out-of-distribution (OOD) settings. We empirically show that harness-aware post-training not only improves in-distribution performance but also enables agents to robustly adapt to OOD settings. Under a harness with minimal design effort, post-training suffers a drastic performance drop under stronger tool environment shifts, further highlighting the importance of harness-aware post-training under such shifts.

Abstract:
Synthetic personas support LLM personalization, social simulation, and alignment evaluation, yet existing resources typically represent each individual with only a small number of structured attributes. We introduce DeepPersona, a generative engine that scales persona depth to roughly 200 instantiated taxonomy nodes while preserving narrative coherence. DeepPersona combines (i) a hierarchical taxonomy of 8,496 human-descriptive attributes mined from 62k personalizable QA pairs in real user-chatbot dialogues and (ii) a progressive sampler that conditions each new attribute on the evolving persona and uses a stratified similarity prior to balance coherence and diversity. The same engine can expand shallow inputs, such as PersonaHub entries, into deep profiles. Across three complementary evaluations, DeepPersona improves judge-extracted attribute count by 32% and judge-rated uniqueness by 44% over the strongest prior persona resource; improves LLM personalization quality by up to 11.6% across ten dimensions, with human evaluators preferring DeepPersona responses in 81.2% to 87.0% of pairwise comparisons; and reduces the distributional gap to real World Values Survey responses by 31.7%, while recovering Big Five personality distributions 17% closer to ground truth than the strongest LLM baseline.

Abstract:
Large language models are increasingly used as natural-language interfaces to structured data, yet they remain unreliable when answers require consistent evidence conditioning, dependency-aware reasoning, and uncertainty estimation. Bayesian networks provide an explicit probabilistic reasoning layer, but learning useful structures from data remains costly and fragile at scale. We introduce ABSOL, a hybrid LLM-guided Bayesian network structure-learning framework that uses LLMs as bounded semantic guides. Across five discrete BN benchmarks spanning 27 to 1041 nodes, ABSOL is the only evaluated method to produce a viable graph on every benchmark, and achieves the highest Edge F1 on every benchmark larger than 27 nodes with GPT-5.4. The four LLM augmentations, which contribute complementary semantic evidence to the statistical backbone, improve Edge F1 over the non-LLM aggregation backbone by +0.23 on average. Complementary post-hoc refinement experiments suggest that these gains depend in part on limiting the LLM’s authority over the final structure. Together, these results show that language-derived semantic knowledge can substantially improve scalable probabilistic structure learning when used as bounded guidance within a statistically grounded reasoning pipeline.

Abstract:
Generative recommendation (GR) represents items with semantic IDs (i.e., discrete token sequences) and generates target item tokens as recommendations. Despite its promising results, existing methods predominantly optimize for accuracy while neglecting the credibility of the recommendations they generate. This oversight inevitably exposes users to uncredible content (e.g., fake news) with serious societal consequences, including user distrust, reputation harm to platforms, and broader social instability. To address this critical yet underexplored challenge, we propose CreGR, the first credible GR model that jointly tackles content credibility across the two core stages of GR: tokenization and generation. In the tokenization stage, we design a new credibility-aware tokenizer that explicitly encourages the model to learn discriminative tokens respectively for credible and uncredible items, thereby disentangling credibility signals at the token level. Building on this, in the generation stage, we propose a novel accuracy-preserving and credibility-oriented generator grounded in discrete diffusion. Specifically, we introduce an asymmetric masking probability reduction strategy that selectively diminishes the contribution of tokens associated with uncredible content to the generation process, while leaving tokens encoding user preference signals unaffected so as to preserve recommendation accuracy. Experiments on three real-world datasets demonstrate the effectiveness of CreGR.

Abstract:
Silicon sampling uses language models as proxies for human survey respondents, treating each model call as an independent draw from the persona’s response distribution. We show this draw does not exist: instruction-tuned models do not sample from distributions, they collapse to a single output. The same persona on the same question returns the same answer on more than half of items in a public-opinion benchmark, and the model’s internal probabilities concentrate on a single option. The failure is associated with, and amplified by, instruction-targeted post-training: instruction-tuned models are worse than their own bases in every family we can compare, the gap widens at each successive post-training stage and with the size of the tuning update, and continued pretraining on non-instruction tokens leaves it unchanged. Yet the knowledge survives: the same model that cannot sample from a distribution can describe it accurately in a single call. We call this gap the KNOWS/DOES split. Exploiting the split, a single call that asks the model to describe the response distribution more than halves the error against human survey data compared to persona aggregation. When per-persona outputs are required, we propose Prompt-Perturbed Argyle (PPA), which reduces the same error by 21%, spreading each persona’s answers to mirror real population differences at no added cost.

Abstract:
Patient education is a major NLP application, but current evaluation largely focuses on text-only assistance. In radiology, this overlooks an important part of clinical communication: patients often need both a plain-language explanation and a visual aid showing where to look and what it refers to. We introduce MediSketch, a multimodal patient-education task in which an agent must produce an image–text bundle: an educational image and a grounded plain-language explanation. We instantiate this setting with a clinician-curated radiology testbed, a structured multimodal rubric, and a rubric-following judge that evaluates bundle quality rather than text-only correctness. Under a matched image-generation budget, we compare three test-time scaling strategies: parallel sampling, sequential revision, and hybrid search. Across five backbone models, the no-scaling baseline remains weak, while naive parallel search often provides little gain despite using the full budget. Hybrid performs best on 4 of 5 backbones, suggesting that test-time compute works best when exploration and revision are combined. Further analysis identifies weak multimodal self-critique as a major bottleneck: many models stop revising too early and overestimate the quality of their own image–text outputs. MediSketch exposes both an evaluation gap in multimodal patient education and a concrete failure mode in test-time scaling.

Abstract:
We study Latent Recurrent Transformer (LRT), a lightweight augmentation of autoregressive transformers that reuses a high-level source-layer hidden state from the previous token as recurrent memory for the next token. Because this state is already computed during ordinary decoding, LRT introduces a cross-token, cross-layer latent pathway while preserving the standard attention mechanism, KV-cache interface, and one model forward per generated token. To pretrain this recurrence without sequentially unrolling the full sequence, we introduce interleaved parallel training : one full-sequence initialization forward constructs a shared buffer, followed by sequential refinement of disjoint position subsets with parallel computation within each subset. This provides every token with recurrent-memory-aware supervision at approximately 2x ideal token compute. Across 1.3B- and 2.1B-parameter nanochat-style backbones and a wide range of training budgets, LRT improves both BPB and CORE under matched effective compute. Additionally, LRT outperforms two-forward PonderLM-2 and matches a three-loop Transformer in BPB, while retaining one-forward-per-token decoding with 9% latency overhead over the standard Transformer.

Abstract:
We introduce HypoExplore, an agentic framework that formulates model discovery as a hypothesis-driven scientific inquiry. Given a human-specified high-level research direction, HypoExplore ideates, implements, evaluates, and improves neural architectures through evolutionary branching. Hypotheses are generated using a large language model by selecting a parent hypothesis to build upon, guided by a dual strategy that balances exploiting validated principles with resolving uncertain ones. Our proposed framework maintains a Trajectory Tree that records the lineage of all proposed architectures, and a Hypothesis Memory Bank that actively tracks associated confidence scores acquired through experimental evidence. After each experiment, multiple feedback agents analyze the results from different perspectives and consolidate their findings into hypothesis confidence updates. Under minimal research directions, HypoExplore discovers a model that achieves top results on two medical imaging tasks on MedMNIST, an improved encoder for named entity recognition on FiNER-ORD, and a lightweight 0.7M parameter model on CIFAR-10 that reaches 94.1% accuracy and transfers well to other benchmarks. Our analysis of hypothesis confidence scores across evolutionary lineages suggest that HypoExplore not only discovers stronger architectures, but can help build a genuine understanding of the design space.

Abstract:
This paper presents an end-to-end approach for generating context-specific large language model (LLM) benchmark datasets by combining expert input with synthetic data generation. Existing benchmark construction methods often trade off validity and scalability: datasets designed with domain experts can produce high-quality evaluations but are slow and costly to create, while synthetically generating data may scale efficiently but often results in unrealistic, redundant, or out-of-scope examples. To address this gap, we introduce a schema eliciting key information about the goals, scope, and context of an evaluation task, and use this information to guide synthetic data generation. We further define four criteria grounded in measurement validity for assessing dataset quality: coverage, diversity, content realism, and stylistic realism. Using these criteria, we show how expert-informed scaffolds can guide synthetic data generation toward more valid benchmarks. Through quantitative evaluations and a real-world case study with domain experts, we demonstrate that our approach improves benchmark data quality over existing methods while preserving validity. We additionally analyze how different types of schema information affect different dataset quality criteria, and provide practical guidance on which information to prioritize collecting under resource constraints.

Abstract:
Explaining an artwork requires knowledge beyond visual recognition, including facts about its creator, subject, material, style, collection, and historical context. Knowledge graphs can provide such evidence, but artwork KGs are incomplete, especially for newly encountered artworks. We introduce ArtKGC, a leakage-controlled multimodal benchmark and evaluation protocol for artwork knowledge graph completion. ArtKGC links artworks to Wikidata facts, Wikipedia descriptions, and visual evidence while controlling ambiguous mappings, graph leakage, and descriptions that directly reveal held-out answers. We show that graph-only models are weak for unseen artworks, raw text can inflate evaluation through answer leakage, and visual evidence is strongly relation-dependent. As an interpretable diagnostic baseline, we use a validation-calibrated reranker that combines structural, sentence-redacted textual, and visual evidence. Experiments show improved completion in the primary inductive-artwork setting and modest downstream evidence gains for artwork QA and explanation diagnostics.

Abstract:
Reasoning traces have become a valuable form of learning signals for improving and transferring the capabilities of large language models. In particular, detailed traces can help distill reasoning behavior from stronger teacher models into weaker student models. The value of capability transfer has motivated many deployed systems with reasoning models to hide raw internal traces and expose at most summaries and answers to users. As a result, we ask whether such interface-level trace hiding prevents users from obtaining useful reasoning supervision through prompting. We study this question with Reasoning Exposure Prompting (), a lightweight in-context elicitation method that uses shadow-model-generated demonstrations wrapped in auxiliary code-like formats to raise user-visible reasoning traces from a victim model. Across the common reasoning dataset, different victim models, and different student model distillation, substantially increases similarity between exposed and REP-conditioned internal traces while preserving useful reasoning signals.

Abstract:
Retrieval-Augmented Generation (RAG) equips large language models with external knowledge and is central to knowledge-intensive tasks. As RAG systems enter real-world use, generators must reliably leverage retrieved evidence. Recent fine-tuning methods improve adaptation to RAG scenarios, but optimization remains challenging because retrieval may return incomplete, fragmented, noisy, or conflicting contexts. Complex tasks further require fine-grained evidence dependencies. These challenges make high-quality supervision costly and limit generalization. We present KARE-RAG (Knowledge-Aware Refinement and Enhancement for RAG), a training-time scaffolded alignment framework. It uses structured knowledge representations as temporary scaffolds to expose evidence organization, support localized factual refinement, and construct fine-grained preference pairs. In our main implementation, an expert LLM refines a lightweight graph-structured evidence sketch. The generator is optimized with token-weighted Dense Direct Preference Optimization (DDPO), which focuses learning on edited scaffold regions. Scaffolds are used only for data construction and training supervision. At inference time, the model runs as standard Vanilla RAG without graph construction, extra retrieval, or latency overhead. Experiments show that KARE improves transfer across the evaluated QA and relation extraction datasets with limited training data, while leaving general capabilities largely unchanged. KARE can also complement existing RAG training objectives as an additional alignment stage.

Abstract:
Psychiatric intake combines self-report scales with clinician-rated interviewing, but public full-trace data are scarce and most synthetic resources assume truthful reporting. We present an honesty-aware multi-agent framework for psychiatric intake simulation that explicitly models honest reporting, concealment, and malingering under asymmetric information. Starting from structured patient profiles, five role-specific agents generate self-report scale outputs, strategy-guided clinician-rated interviews, honesty inference from the interview transcript alone, and honesty-aware diagnostic fusion over four common psychiatric domains: major depressive disorder (MDD), anxiety disorder (AD), bipolar disorder (BD), and attention-deficit/hyperactivity disorder (ADHD). Using normalized clinical and healthy source profiles, we construct a dataset of 2,008 complete intake sessions with 58,918 dialogue turns, linking profiles, scales, strategy traces, inferred honesty, and final diagnostic reports. Experiments show that assigned reporting conditions are expressed in generated patient behavior and can be recovered from the interview transcript alone, with condition-level inference accuracy between 60.1% and 94.8%. Under hidden-information constraints, the best LLM achieves 85.4% exact recovery of synthetic reference domains with 0.99 severity MAE and zero detected leakage. Honesty-aware fusion reduces deviation from benchmark reference labels relative to self-report alone, and strategy-guided interviewing attains higher accuracy with fewer evaluator questions than item-by-item questioning. These results support honesty-aware synthetic intake data as a controlled research benchmark, not a clinically validated pre-assessment system.

Abstract:
Document chunking is widely treated as neutral preprocessing in retrieval-augmented generation, but in content-balanced interventions, re-segmenting the same evidence, without changing its semantic content, can shift fixed-choice LLM decisions and flip them in near-threshold regimes. Across full ratio-sweep experiments on 9 primary queries and 6 LLMs, document-unit structure systematically shifts model choices. We fit a shifted sigmoid whose steepness captures structural sensitivity and whose offset c captures fitted decision asymmetry. The two fitted parameters enable post-hoc flip-point estimation after per-query calibration, not a priori prediction. The shifted-sigmoid fit reaches R² ≥ 0.80 in 38/41 non-degenerate model–query combinations; the English experiments show the same form on 9 queries and 4 LLMs. It persists under noisy real-corpus retrieval when retrieved evidence is stance-balanced (ΔP = 0.53, p < .001, on one THUCNews retrieval query). The effect is regime-dependent: near-threshold decisions are vulnerable, while large decision offsets resist manipulation. Chunking and boundary placement can therefore affect downstream fixed-choice RAG decisions, with implications for retrieval system design and adversarial robustness.

Abstract:
Chain-of-thought (CoT) may often look plausible, yet it may not faithfully reflect the model’s decision-making process. While methods for measuring the faithfulness of CoTs for textual inputs have been increasingly introduced, using these methods for visual inputs is not straightforward. In this work, we adapt the family of counterfactual methods for measuring CoT faithfulness, namely the Counterfactual Test (CT) and Correlational Counterfactual Test (CCT), to visual inputs, and call them vCT and vCCT, respectively. Using vCT and vCCT, we benchmark eight recent open-source Vision Language Models (VLMs) on two datasets. Our analysis shows that CoTs do not reliably track visual evidence that influences model predictions: they may omit the removed object even when its removal causes a large prediction shift, yet mention it when the shift is small. We further find that Predict-then-Explain explanations align more strongly with perturbation-induced probability shifts than pre-answer CoTs, while binary vCT scores are often nearly saturated. We also include a reconstruction control, in which images pass through the same editing pipeline without object removal, and find that the main object-removal intervention induces larger shifts than reconstruction alone. We construct and release Counter-SNLI-VE and Counter-A-OKVQA, two datasets of image pairs that differ by a single object.

Abstract:
Large Language Models (LLMs) fail to reliably follow multi-constraint instructions across languages, and the failure rate rises when regional linguistic rules interact with general constraints. To address this challenge, we present Meeseeks-Multilingual, a multi-turn, feedback-driven benchmark that extends Meeseeks to multilingual and cross-regional evaluation. The benchmark separates general constraints (e.g., word counts, keyword rules) from regional constraints (e.g., Japanese kana requirements) so that the two failure modes can be measured independently. Meeseeks-Multilingual automatically identifies unmet constraints and provides targeted feedback to test iterative self-correction. All data are produced by trained annotators with no LLM involvement in data generation. Experiments across mainstream commercial and open-source LLMs show that even with iterative feedback, models consistently fail to meet all multilingual constraints. Aggregate model rankings stabilize by Round 5, which lets the benchmark be run at reduced cost for overall comparisons, although regional rankings settle more slowly and need more rounds. Meeseeks-Multilingual has been open-sourced on https://github.com/meituan-longcat/Meeseeks.

Abstract:
Just as Socrates recognized the limits of his own knowledge, Retrieval-Augmented Language Models (RALMs) should learn to abstain when the retrieved evidence cannot support a reliable response. Existing approaches largely rely on monolithic LLMs to handle heterogeneous retrieval failures in a single step, resulting in limited abstention performance and high computational costs. We instead decompose retrieval failures into two distinct states: (i) the unanswerable state , where the required evidence is absent, and (ii) the distracted state , where relevant evidence is mixed with conflicting, negated, or adversarial information. Based on this decomposition, we introduce a lightweight module ( Sieve ) that screens retrieved document sets for distracting evidence before invoking a costly LLM ( Sage ) for grounded generation and abstention. Evaluated across both general and high-stakes expert domains, our Sieve and Sage framework preemptively detects distracting noise, improving system accuracy by up to 69.4%p and Macro-F1 by 55.2%p compared to one-stage baselines. Furthermore, it achieves up to a 1.99 × speedup, establishing a highly efficient and reliable abstention pipeline for RALM with abstention.

Abstract:
While automatic speech recognition (ASR) models have achieved remarkable improvements in recent years, performance disparities persist across different speaker populations. One such disparity is for speakers whose first languages (L1) are from families distant from English. This paper investigates the relationship between first language background and English ASR performance. Through empirical analysis, we observe that the correlation between speakers’ L1 distance and ASR error rates yields a systematic effect on English Speech, with its strength varying across datasets and models. This association is statistically significant in a follow-up analysis accounting for dataset-level variation in Tweedie mixed-effects models ( p<0.001 across evaluated models). In addition, analysis of the latent space reveals a L1-based spatial segregation across deeper acoustic layers in the majority of evaluated architectures.

Abstract:
We study fine-grained stereotypes of LLMs activated by linguistic features representing demographic groups, developing a probe-based framework to extract 235 fine-grained traits (e.g., friendly, uneducated, honest) from model representations of prompts. Using these probes, we conduct two studies analyzing the underlying structure of traits and specific stereotypes of demographic groups. We find that fine-grained trait information is represented in LLMs similarly to coarse stereotype content dimensions (e.g., warmth and competence), and that the latent factors of trait probes (e.g., morality, competence, sociability) largely align with those in the psychology literature (Fiske et al., 2002) despite traits being modeled independently. Finally, we extract LLMs’ associations between traits and demographics signaled through language to surface granular stereotypes encoded in LLMs. We, for example, find that the evaluated models encode associations between African American Language and traits clumsy, uneducated, and criminal. These associations may reinforce harmful stereotypes, with implications for both general dialogue responses and in domains like hiring. Overall, we discuss the need to consider fine-grained traits in stereotype evaluations in order to distinguish stereotypes of out-groups and between models.

Abstract:
Progress in Code Question Answering (CodeQA) depends on benchmarks that reflect what developers actually ask. Yet most existing CodeQA datasets are built from docstring templates or GitHub issues, and rarely capture the diversity of real, repository-level developer queries. We address this gap by releasing a taxonomy of real-world code-assistance queries, grounded in 3,052 authentic user interactions from the RepoChat dataset. We construct the taxonomy via a semi-automated pipeline that combines clustering, LLM-based labeling, and an iterative refinement loop driven by inter-model disagreement, and we compare the result against three prior taxonomy-generation methods (TaxoGen, TaxoCom, TaxoAdapt). Our taxonomy achieves a favorable tradeoff between granularity and usability—86 nodes with 93.2% node relevance, 99.4% path validity, and 81.25% sibling coherence—while remaining concise enough for downstream use. We then apply the taxonomy in two ways: (i) auditing existing Code related benchmarks to surface systematically underrepresented query types, and (ii) generating a synthetic CodeQA dataset whose categories are guided by the taxonomy.

Abstract:
Synthetic post-training pipelines commonly filter generated samples with reward models or holistic LLM judges, often relying on two assumptions: that a sample’s faithfulness can be judged without the exact source that produced it, and that rejected samples can be discarded rather than systematically recovered. We present a controlled study of both assumptions across gate configurations, recovery strategies, and generator scales, using adversarially injected corpora to provide ground-truth failure labels. We find that grounding faithfulness judgments in the exact source evidence improves over post-hoc retrieval for stronger judges and substantially outperforms reward-only filtering; hallucination and reward gates reject largely distinct sample populations supporting the need for both; and adaptive recovery, which diagnoses failures before repairing them, achieves higher yield and recovery rate than naive regeneration. Downstream fine-tuning quality is driven primarily by generator scale, with filtering and recovery providing their clearest gains when smaller generators are used. We release our evaluation corpus and experimental configurations to support reproducible pipeline design.

Abstract:
Mathematical word problems with plausible irrelevant context expose a failure mode in which language models reason from facts that should not support the target solution. We formulate distractor-robust mathematical reasoning as relevance-boundary supervision: a model should separate Signal facts from Distractor facts before solving, and then ground the solution in Signal facts only. We introduce ReBound, a post-training framework that constructs paired clean–distractor examples, trains boundary-first traces with supervised fine-tuning, and applies correctness-guided reinforcement learning to discourage Distractor use. Across three open base LLMs and six mathematical reasoning benchmarks, ReBound consistently improves distractor robustness over CoT prompting and strong training-based baselines; on Qwen3-0.6B, the distractor benchmark average increases from 34.92 to 48.29. ReBound also preserves competitive clean and perturbation performance. Process diagnostics, ablations, and output-length analysis show that these gains are tied to better fact use and boundary behavior, rather than longer generations or answer-only optimization.The code is available at https://anonymous.4open.science/r/Rebound-5FB8

Abstract:
Recent progress in universal multilingual named entity recognition (NER) has been driven by multilingual transformer models, task-specific architectures, custom loss functions, and large-scale training datasets. However, despite substantial prior work, we find that many design decisions for such models are made without systematic justification, with individual components evaluated only in combination rather than in isolation. We argue that this impedes progress in the field by making it difficult to identify which choices improve multilingual generalization. In this work, we conduct an extensive empirical evaluation on transformer backbones, architectures, training objectives, data composition, and threshold selection for zero-shot, multilingual NER models. Building on these findings, we present Otter, a small encoder model that achieves consistent improvements over strong multilingual NER baselines, outperforming similarly sized models by 5.7 percentage points in F1. Further, it remains within 1.4 points of Qwen3-32B and 5.4 points below Gemma3-27B, while being roughly 90x smaller and one to two orders of magnitude faster at inference.

Abstract:
Video lectures are valuable educational resources, but their dense and lengthy formats often overwhelm novice learners. This difficulty stems from a fundamental pedagogical mismatch: while videos deliver transient information linearly, human learning requires constructing interconnected cognitive networks, a task that induces severe cognitive overload for novice learners lacking prior domain knowledge. Existing video summarization methods fail to resolve this mismatch, as they primarily produce text-heavy, linear condensations that still demand high cognitive effort. To bridge this gap, we propose KnowVis, a framework that transforms linear video lectures into pedagogically grounded visual narratives. KnowVis first extracts a detailed concept map from multimodal video content to identify important and challenging threshold concepts, then constructs structured knowledge units, and finally synthesizes engaging visual summaries. Alongside the framework, we introduce a curated dataset of 125 educational videos across 10 academic disciplines, paired with 1,079 generated visual summaries. Extensive automated evaluations and a human study demonstrate that, compared to state-of-the-art baselines, KnowVis generates more accurate and clear visuals that successfully reduce cognitive load and significantly improve student learning effectiveness and knowledge retention.

Abstract:
Large language Models (LLM) routing reduces inference cost by selecting the most cost-effective model per query, but production deployment requires a router that adapts to an evolving model pool while adding negligible per-query overhead. Unsupervised retrieval-based routers satisfy both, but their support corpus must be built online from streaming traffic, and labeling each query requires running every candidate plus an LLM judge: prohibitively expensive at scale. We frame online corpus construction as stream-based active learning, exploiting a structural property of routing benchmarks: one or two models are globally near-optimal on the majority of queries, so the labeling budget should concentrate on the rare exception regions where selecting a specialist matters. We formalize this with a performance sparsity score, defined as the Kullback-Leibler divergence of a query’s per-candidate performance vector from uniform: high when performance concentrates on a few models and low when all candidates are equivalent. Since this score depends on performances unavailable at inference time, we predict it online from query embeddings with a lightweight regressor. Across four benchmarks, our method SparseNN approaches full-annotation routing performance with up to 44% fewer annotations in the stationary setting and 73% under topic shift. The results transfer to a fully held-out benchmark. A partial model annotation variant achieves higher cost savings, especially on large pools. Finally, both the corpus and the online estimator prove robust to model-pool changes.

Abstract:
Recent Multimodal Large Language Models (MLLMs) have made rapid progress in video understanding, demonstrating strong performance across a wide range of perceptual and reasoning tasks. However, their Social Intelligence remains underexplored and difficult to evaluate. Existing benchmarks for social understanding suffer from two major limitations: they primarily evaluate short-horizon interactions and focus largely on explicit perceptual cues, such as facial expressions and speech sentiment. However, real-world social understanding often requires inferring implicit emotions, intentions, beliefs, and interpersonal dynamics from extended contextual interactions rather than isolated expressions or utterances. Consequently, robust evaluation of social intelligence requires long-horizon reasoning over complex social contexts. To address this gap, we introduce SMMU, a Massive Multimodal Social Understanding benchmark. SMMU comprises 264 long-form audiovisual scenes and 1519 question-answer pairs, to systematically evaluate each model across five social dimensions (Emotions, Intent, Knowledge State, Perspective, Relationships) and three cognitive tasks (Comprehension, Reasoning, Prediction). SMMU is designed to require models to integrate contextual evidence across extended interactions in order to interpret, reason about, and anticipate complex human behavior. Evaluating a suite of state-of-the-art MLLMs, we find that even the strongest models achieve only 0.43 Social Understanding Score compared to the average human baseline of 0.85, and 0.33 Social Agency Score compared to the human baseline of 0.78, revealing a substantial gap between current MLLMs and human social intelligence.

Abstract:
The core of knowledge distillation lies in transferring the teacher’s rich ’dark knowledge’—subtle probabilistic patterns that reveal how classes are related and the distribution of uncertainties. While this idea is well established, teachers trained with conventional cross-entropy often fail to preserve such signals. Their distributions appear decisive but are in fact brittle, offering little beyond the hard label. This overconfidence is especially problematic in high-cardinality tasks, where subtle distinctions among many plausible classes are crucial for guiding compact students. To address this limitation, we revisit distillation from a distributional perspective and propose Calibrated Uncertainty Distillation (CUD), a framework that makes dark knowledge more faithfully accessible. Instead of uncritically inheriting the teacher’s overconfidence, CUD encourages the teacher to expose informative uncertainty and guides the student to learn from calibrated targets instead of overly sharpened predictions. By shaping the teacher’s predictive distribution before transfer, our approach balances accuracy and calibration, allowing students to benefit from both confident signals on easy cases and structured uncertainty on hard ones.

Abstract:
AI agents rely on a harness of skills, tools, and workflows to solve complex problems. Continually improving this harness is essential for adapting to new tasks. However, existing optimization methods typically require ground-truth validation sets, yet such labeled data is difficult to acquire in practical deployment settings. To address this problem, we introduce Retrospective Harness Optimization (RHO), a self-supervised method that optimizes the agent harness using only past trajectories. Specifically, RHO selects a diverse coreset of challenging tasks from past trajectories and re-solves them in parallel. The agent analyzes these rollouts using self-validation and self-consistency, then generates candidate harness updates and selects the most effective one by its own pairwise self-preference. We evaluate RHO across three diverse domains, spanning software engineering, technical work, and knowledge work. Notably, a single optimization round improves the pass rate on SWE-Bench Pro from 59% to 78% without any external grading. Furthermore, our analysis demonstrates that RHO effectively targets prior failure modes. As a result, the optimized harness alters the agent’s behavior patterns and sustains higher accuracy during long-horizon sessions. Code is available at https://github.com/wbopan/retro-harness and the project website at https://paper-rho.wenbo.io.

Abstract:
Tool-augmented visual reasoning systems, exemplified by Visual Programming (VP), decompose complex visual questions into executable programs that invoke specialized visual models as tools, offering interpretable and compositional reasoning. However, these systems face a fundamental optimization challenge: the non-differentiable pipeline and the absence of intermediate supervision prevent end-to-end gradient-based optimization of invoked tools using only task-level labels. Existing methods either optimize the Large Language Model (LLM) planner while neglecting tool quality, or rely on teacher models and additional data collection, none achieving true end-to-end differentiable optimization. We propose eDoT, the first end-to-end Differentiable optimization method for Tool-augmented visual reasoning. Our key insight is that any program’s variable dependency structure is isomorphic to a directed probabilistic graph. Applying Variable Elimination - an exact probabilistic inference technique - transforms the non-differentiable VP execution into a fully differentiable process, enabling gradient-based optimization with only task-level supervision, without additional data or teacher models. Experiments on GQA, NLVRv2 and Open Images demonstrate improvements of 3.7%, 7.6% and 20%, validating eDoT’s effectiveness across multiple visual models and cross-task generalization.

Abstract:
Large language models demonstrate exceptional capabilities but raise critical safety concerns due to toxic outputs causing potential harm to users. Prompt optimization has emerged as a practical detoxification solution because it operates at the input level and can be applied to black-box or API-based models without parameter access. However, existing methods treat each task in a stateless manner without leveraging prior optimization experiences, which results in either ineffective detoxification or over-sanitized prompts that lose original intent in the vast search space. To address this, we propose the Memory-INformed Detoxification (MIND) framework that harnesses historical optimization experiences to enhance detoxification effectiveness through Memory Retrieval (MR) for retrieving relevant past cases and Memory Maintenance (MM) for maintaining a representative memory pool. We formulate MR and MM as optimization problems, prove their NP-hardness, and develop two approximation algorithms with constant approximation ratios for these problems. Experiments across multiple datasets and model configurations show that MIND outperforms existing baselines in both toxicity reduction and response utility.

Abstract:
Large audio-language models (LALMs) excel at understanding and reasoning tasks over atomic sound events, yet their ability to infer higher-level human activities from such fine-grained events remains largely unexamined. Everyday human actions and activities, such as setting a table, cleaning the house, or preparing a breakfast emerge compositionally from temporally distributed sound events, requiring abstraction beyond the event-centric granularity that dominates current training and evaluation paradigms. Our benchmark evaluates a wide set of LALMs under a principled framework that tests how language-based reasoning, grounded in acoustic perception, structures sound abstractions into higher-level understanding. By systematically varying exemplar typicality and distractor similarity, our evaluation exposes that current models do not reliably perform compositional inference from atomic acoustic events to higher-level human activities solely from audio. All data, taxonomies, and evaluation scripts are publicly available on our companion website: https://alm-sounding-actions.onrender.com/

Abstract:
Two-sided digital platforms are inherently dynamic: user preferences shift, item popularity evolves, and reviews both reflect and drive these changes. Yet most sequential recommendation systems treat reviews as passive signals for updating user states, leaving two aspects underexplored. First, review generation is nonrandom, depending on evolving latent states of both users and items. Second, reviews can reshape item states, induce spillover across related items, and influence future user decisions. To address these gaps, we propose a two-sided state-space model (TS-SSM) for event-conditioned sequential recommendation. TS-SSM consists of three components: (1) a modality-missing-not-at-random fusion module that encodes review content and informative observation patterns; (2) user-state evolution with temporal variation and local graph message passing that uses related item states to refine user preferences; and (3) item-state evolution with asymmetric carryover of positive and negative review feedback. In experiments across six Amazon categories, TS-SSM increases Recall@20 over BSARec by 14.8%-18.8% and exceeds HM4SR by 11.7% on average. On Goodreads Fantasy, Recall@20 improves HM4SR from .5191 to .5847. Ablations highlight distinct contributions of observation patterns, local propagation, and item dynamics.

Abstract:
The tremendous commercial potential of large language models (LLMs) has heightened concerns over their unauthorized use. To address this, we focus on the task of identifying the origin of black-box LLMs. We further propose PlugAE, an effective and efficient identification method that proactively leverages LLM-specific adversarial embeddings and allows users to customize copyright tokens on a targeted query set. Extensive experiments demonstrate that PlugAE outperforms both state-of-the-art model watermarking and fingerprinting methods in accuracy and robustness. We further analyze its stealthiness and reliability from three complementary perspectives and conduct ablation studies under various configurations, confirming its practicality for real-world misuse detection.

Abstract:
Recent LLM-based embodied agents have advanced in high-level task planning. Yet, they frequently suffer from hallucinations and brittle reasoning due to passive observation—either omitting fine-grained attributes or overwhelming the planner with irrelevant information. We propose SceneQueryPlanner, a memory-environment dual-grounded agent that integrates active information seeking into the planning loop by decomposing instructions and selectively executing environmental interactions or structured queries over a dynamic 3D scene graph to retrieve task-relevant information on demand. By treating information acquisition as a first-class planning action, our approach enables explicit verification and supports reliable long-horizon reasoning over object attributes. For rigorous evaluation, we introduce AttPlan-Bench, a benchmark of multi-room, long-horizon household tasks requiring attribute-aware reasoning. Experiments demonstrate that our dual-grounding strategy outperforms passive baselines, yielding more stable and scalable reasoning across diverse tasks.

Abstract:
Large language models show promise for mental health support, yet therapeutic quality improves only when evaluation functions as an actionable control signal rather than a passive metric. We introduce a framework that formulates therapeutic response generation as a decision-refinement problem driven by multi-dimensional, human-aligned evaluation. In Stage I, we introduce TheraJudge, an open-source therapeutic evaluator trained via preference-based optimization on human-annotated data to produce reliable judgments across 7 psychological dimensions. In Stage II, we introduce TheraAgent, which operationalizes TheraJudge’s evaluations through a coordinated refinement process with specialized Critic, Coach, and Therapist roles that translate evaluative signals into targeted response revisions. Empirically, TheraJudge achieves strong agreement with clinician ratings, with intraclass correlation coefficients (ICC = 0.87-0.95), surpassing supervised baselines and strong closed-source judges, particularly on critical dimensions such as Safety, Relevance, and Empathy. Acting on these evaluations, TheraAgent yields a +0.43 improvement in human-rated therapeutic quality (on a 5-point scale) under blind evaluation, with 96% clinician inter-rater reliability. Low-quality responses improve by +2.45 points with a 94% recovery rate, demonstrating targeted correction of unsafe outputs. Overall, our results indicate that effective alignment of mental-health LLMs stems from acting on human-aligned evaluation, rather than relying solely on stronger generation. We release code at .

Abstract:
Adapting large language models to individual users remains challenging because systems must capture fine-grained personal preferences while serving many users efficiently. We introduce CARD, a hierarchical personalization framework that first learns preferences shared across user groups and then captures how individual users depart from those shared patterns. CARD trains a parameter-efficient adaptation module for each group, providing a reliable prior even when a user has limited history. It then learns a compact user preference vector by contrasting user-authored responses with outputs from the corresponding group model, without requiring manual annotations. During inference, both the backbone and the group adaptation module remain frozen, while the user vector applies a low-rank correction to the token logits during decoding. Experiments on personalization benchmarks show that CARD consistently outperforms competitive baselines while requiring only a compact vector for each user.

Abstract:
Realistic user simulation is crucial for training and evaluating multi-turn dialogue systems, yet creating simulators that accurately replicate human behavior remains a significant challenge. An effective simulator must expose the failure modes of the systems under evaluation. This work introduces Direct Iterative Adversarial Learning (DIAL), an adversarial framework that iteratively enhances user simulator realism through a competitive dynamic between a generator (user simulator) and a discriminator. When applied to mental health support, a domain characterized by diverse failure types and a critical dependence on realistic user behavior for failure detection, DIAL restores lexical diversity diminished by supervised fine-tuning and drastically reduces discriminator accuracy. The resulting simulator exhibits a strong correlation between simulated and real failure occurrence rates while maintaining low distributional divergence of failure modes. These findings indicate that DIAL is a promising method for developing realistic user simulators in multi-turn dialogue, facilitating reliable and cost-effective system evaluation prior to deployment.

Abstract:
Long-horizon language agents must decide not only which evidence to use, but also when to pay for it. Retrieval and verification can improve a local decision while exhausting token and tool-time budgets needed later in the episode. We introduce ReserveGate, a one-step evidence controller that combines an explicit uncertainty sketch, an off-policy action-value model, and component-wise feasibility masks over the remaining resources. Unlike a fixed reserve heuristic, ReserveGate adapts its cost penalty when spending runs ahead of episode progress and gradually releases a protected late-stage reserve. Across three benchmark-derived agent environments, ReserveGate improves tight-budget success over the strongest matched controller by 2.6–3.8 percentage points, with gains of 4.6–6.2 points on the longest episode quartile. A latency-constrained variant reduces service-level incidents to 1.2–1.7% across environments while losing at most 0.8 points in success. On LoCoMo-Episodic, cross-model, cross-retriever, rate-limiter, and task-agnostic-sketch comparisons show that the effect is not explained by a single backbone, retrieval stack, or tuned spending rule.

Abstract:
As large language models(LLMs) are increasingly deployed in real-world applications, they inevitably encounter culturally sensitive scenarios that require applying cultural knowledge or values to the context in order to generate appropriate responses. However, existing research largely treats culture as static knowledge or abstract values, leaving the application of cultural norms in situated interactions underexplored. To address this gap, we first focus on the situated cultural reasoning of LLMs by proposing CuSiR , a simulation learning framework combining simulated scenario and reinforcement learning. We construct datasets covering both knowledge-based and social scenarios and conduct experiments across multiple role perspectives, instructions, settings, and models. Our results indicate that this framework effectively enhances LLMs’ ability to apply existing cultural knowledge, thereby improving their performance on tasks set in cultural scenarios. Code files are available at https://github.com/HoganZinger/Simulation_Learning

Abstract:
Existing approaches for synthesizing Partial Differential Equation (PDE) solvers predominantly rely on autoregressive models, yet their global left-to-right decoding incurs substantial redundancy when addressing inherently localized bugs. In this work, we challenge this inefficient paradigm and propose DiffPDE, a framework leveraging discrete diffusion language models for targeted code repair. By introducing a localized re-masking and infilling strategy, DiffPDE regenerates only erroneous regions while preserving correct context, naturally aligning generation with the sparse nature of PDE errors. Furthermore, to handle coupled bugs requiring sequential interventions, we present Iterative Debugging GRPO (ID-GRPO), a reinforcement learning scheme that enables multi-round debugging within single trajectories via intermediate rewards. Experiments on PDEBench show that DiffPDE achieves competitive accuracy, outperforms same-scale AR models, and significantly accelerates repair.

Abstract:
Current coding-agent benchmarks usually provide the full task specification upfront. Research-code implementation may instead re- quire an agent to preserve design commitments disclosed progressively across a long session. We introduce SLUMP, a controlled benchmark for testing implementation faithfulness when a mathematically structured machine-learning target is disclosed through synthetic, paper- grounded multi-turn interactions rather than in a single canonical implementation specification. The benchmark contains 20 recent ML papers, 371 atomic verifiable components, and approx- imately 60 coding requests per paper. An expo- sure audit verifies whether scored components are recoverable from the visible interaction. On Claude Code and Codex, the single-shot control achieves higher overall implementation fidelity on 16/20 and 14/20 papers, respectively. Struc- tural integration degrades under progressive dis- closure on both platforms, while semantic com- ponent faithfulness degrades substantially on Claude Code and remains approximately sta- ble on Codex. These conclusions persist under stricter exposure filters and a targeted human correction of judge false-absence errors. As a mitigation case study, ProjectGuard main- tains external semantic and structural project state and recovers 90% of Claude Code’s mea- sured faithfulness gap. Code and benchmark artifacts are available at https://github.com/lunaryan/drift-benchmark.

Abstract:
Multilingual large language models often struggle to reason in low- to mid-resource languages. Prior work has shown that translation can improve multilingual reasoning by helping models access stronger English-centric representations. This raises a central question: how much translation is needed for multilingual large language models to reason reliably, and when does additional translation instead trigger interference and overconfidence? Using LLaMA and Qwen models, we conduct extensive experiments that vary both text scope and language mode (target-only, English-only, and bilingual) to evaluate accuracy and reliability. Our results reveal a clear trade-off: English context often improves understanding and helps recover errors caused by non-English comprehension, yet adding redundant bilingual context can intensify interference. To address this trade-off, we propose Reliability-Aware Adaptive Inference (RAAI), a training-free test-time framework that (i) performs ECE-aware routing and prompt fusion and (ii) uses a mid-layer Risk Index (RI) to gate sequential reasoning, allocating extra compute only when it is likely to help while suppressing harmful bilingual redundancy. Across two model families, RAAI improves accuracy by 25 to 37.7 points in low-resource languages and reduces calibration error by about 3 to 6 points, with the largest gains appearing in the lowest-resource language tiers.

Abstract:
Argument mining (AM) traditionally relies on multi-stage pipelines that train separate models for each sub-task, making end-to-end AM challenging and often unstable under multitask learning. In this work, we revisit multitask end-to-end AM in the context of large language models (LLMs) and show that multitask learning behaves fundamentally differently from pre-LLM settings. Across multiple datasets and LLMs, multitask fine-tuning of a single LLM consistently outperforms single-task models, while also exhibiting greater robustness to noisy labels and stronger cross-domain generalization. To understand these qualitatively different behaviors of LLMs, we conduct a layer-wise representation analysis, showing that cross-task features emerge in middle layers, early layers are highly domain-sensitive, and richer middle-layer representations drive performance improvements, robustness, and generalization. We further show that this mechanism extends beyond AM by observing similar patterns across other tasks. Our findings provide new insights into multitask learning with LLMs, highlight its potential for unified, efficient, and robust language understanding, and motivate layer-wise parameter efficient adaptation strategies within and beyond AM.

Abstract:
Argument relation identification (ARI) aims to recover the directed structure of argumentative discourse, yet most approaches, including recent large language models, treat relations as local decisions and rely on implicit reasoning to enforce global coherence. This often leads to structurally inconsistent argument graphs. We reframe ARI as a global structural reasoning problem and introduce topological ordering as an explicit prediction target. Extensive experiments show consistent improvements in ARI performance, edge-level accuracy, and global structural fidelity across models and learning paradigms. In addition, the predicted topological orders provide faithful, verifiable reasoning traces that are internally consistent with model predictions and aligned with gold argument structure, offering an interpretable alternative to free-form chain-of-thought explanations.

Abstract:
Multimodal Large Language Models have sparked significant interest due to their potential for social intelligence; however, their ability to perform sequential motivation reasoning remains insufficiently studied. Existing evaluations predominantly examine static text or isolated visual snapshots, which do not reflect the cumulative nature of real-world behavioral drivers. To address this gap, we introduce MulTivationBench, a benchmark designed to rigorously evaluate multimodal motivation reasoning within story-driven visual narratives. The benchmark builds upon established psychological frameworks—Maslow’s hierarchy and Reiss’s basic desires—and requires models to integrate accumulated multimodal context to infer evolving motivations. Results indicate that MulTivationBench presents a significant challenge: all tested models struggle to maintain consistent motivation reasoning across sequential contexts, revealing a critical disconnect between static recognition capabilities and the dynamic reasoning essential for human-like social understanding.

Abstract:
Large Multimodal Models (LMMs), characterized by their humanoid perception and generation capabilities, have been widely applied in critical physical world scenarios, yet their uncertainty remains mostly unknown. In this paper, we aim to answer three substantial questions: (1) Can we explicitly capture the uncertainty of LMMs? (2) How can we tackle the reasoning complexity arising from prompts across various modalities? (3) How can we quantify the uncertainty of LMMs when their responses extend to multiple modalities, beyond mere text? To answer these questions, we propose Uncertainty-o: (1) Uncertainty-o can explicitly estimate uncertainty of LMMs, with prompts and responses across five modalities, namely text, image, audio, video, and point cloud. (2) We propose multimodal prompt perturbation, where empirical studies indicate that semantically equivalent perturbations are more effective for eliciting LMM uncertainty. (3) We introduce multimodal semantic uncertainty, enabling uncertainty estimation when LMMs generate multimodal responses. Extensive quantitative and qualitative analyses validate the efficacy of Uncertainty-o in reliably estimating the uncertainty inherent in LMMs.

Abstract:
Large Vision-Language Models (LVLMs) demonstrate remarkable capabilities in multimodal tasks, but visual object hallucination remains a persistent issue. It refers to scenarios where models generate inaccurate visual object-related information based on the query input, potentially leading to misinformation and concerns about safety and reliability. Previous works focus on the evaluation and mitigation of visual hallucinations, but the underlying causes have not been comprehensively investigated. In this paper, we analyze each component of LLaVA-like LVLMs—the large language model, the vision backbone, and the projector, to identify potential sources of error and their impact. Based on our observations, we propose methods to mitigate hallucination for each problematic component. Additionally, we developed two hallucination benchmarks: QA-VisualGenome, which emphasizes attribute and relation hallucinations, and QA-FB15k, which focuses on cognition-based hallucinations.

Abstract:
Text-to-motion (T2M) generation has achieved impressive realism but poses significant safety risks by faithfully executing unsafe textual prompts. Existing motion unlearning methods face dual challenges: brittle keyword filtering and discrete codebook manipulation, which may degrade benign generation quality and introduce jerky transitions. To address these challenges, we propose SafeMo, a linguistically grounded framework that leverages Large Language Models (LLMs) to align motion generation with safety constraints. First, we introduce SafeMoEngine, an LLM-agent pipeline that autonomously classifies unsafe intents and performs semantic rewriting to construct SafeMoVAE-29K, the first safety-aligned text-to-motion dataset. Second, we propose Minimal Motion Unlearning (MMU), a continuous-space unlearning strategy that projects unsafe semantic concepts into a task vector for precise negation, avoiding the quantization artifacts of discrete methods. Experiments demonstrate SafeMo’s effective unlearning performance by more strongly suppressing unsafe text–motion retrieval than the prior SOTA motion unlearning baseline LCR, reducing forget-set R@1 by 35.3% and achieving 2.61× normalized retrieval trade-off score (NRTS), while showing better or competitive benign utility on safe prompts, with similar trends on Motion-X. Code and checkpoints are publicly available at https://github.com/AIGeeksGroup/SafeMo. Project page: https://aigeeksgroup.github.io/SafeMo.

Abstract:
Fine-tuning large language models (LLMs) for sentiment polarity regression often suffers from severe data distribution imbalance. Financial texts typically concentrate in the neutral range, leaving extreme sentiments underrepresented, yet traditional techniques offer limited remedy since such extremes are inherently rare. We introduce an auxiliary classification task to provide complementary supervision, but this multi-task formulation faces two challenges: (1) sentiment group construction for the classification task, and (2) gradient conflicts between tasks that fixed weights cannot resolve. These challenges are coupled: improper group boundaries yield biased gradients, which amplify task conflicts. We propose the DRNash framework as a unified solution to jointly address both challenges and the underlying data distribution imbalance, including D istributionally R obust Optimization (DRO) and Nash Task Coordinator (NTC). DRO partitions the sentiment space into quantile-based groups and performs worst-case aggregation over group losses, automatically assigning higher weights to underperforming sentiment regions. NTC formulates task weighting as a Nash bargaining game, deriving closed-form weights that achieve scale-invariant balancing. Experiments demonstrate that DRNash achieves average improvements of 14.1% in MSE and 1.92% in accuracy (ACC) across multiple financial datasets, with cross-domain experiments confirming generalizability beyond financial texts.

Abstract:
Behavioral benchmarking reveals what biases exist in vision-language models but not which internal components are most sensitive to targeted intervention, precluding principled intervention. We argue for mechanism-level evaluation as a necessary complement, demonstrating causal mediation analysis as a diagnostic instrument for gender bias. We decompose gender-cue effects into controlled indirect effects attributable to specific-layer activations and direct effects through all other pathways, producing layer-by-layer mechanistic signatures. Across six models spanning three architectural families (LLaVA-1.5, LLaVA-NeXT, InstructBLIP at 7B/13B) and two 8B-scale architectures, three findings emerge: language-layer activations exhibit the greatest output sensitivity under controlled intervention, with the direct component often carrying the opposite sign; architectural choices redistribute layer-wise sensitivity to activation replacement; and counterfactual scores diverge from surface-level scores, exposing implicit associations. Systematic ablation validates internal consistency. An intervention experiment finds that the average indirect effect (AIE) and downstream intervention effectiveness are only weakly correlated (Pearson r = 0.33 ), and the layer with the second-largest AIE produces near-zero bias change—indicating that mechanistic diagnosis captures activation-replacement sensitivity but does not, by itself, identify optimal intervention targets. These results show mechanism-level evaluation captures architecture-specific sensitivity patterns that behavioral benchmarks cannot; pairing both should become standard NLP practice. Code: https://github.com/zhaozhipeng1997/CARD-GenderBias.

Abstract:
Large Language Model (LLM)-augmented Community Notes offer a scalable path for timely, evidence-grounded correction of health misinformation on social platforms. However, they still reset at every post, leaving useful correction experience from prior cases unused. We introduce EvoNote, an agentic framework that enables health Community Notes generation to self-evolve through an evolving experience memory of prior misinformation correction episodes. Its core is fine-grained credit assignment: EvoNote grounds trajectory-level feedback in health-specific note qualities and distills it into action-level memory for claim analysis, evidence acquisition, and note writing. We evaluate EvoNote on MM-HealthCN, a 1.2K-instance multimodal benchmark of user-flagged health posts with human-written Community Notes and crowd-derived helpfulness labels. Under a human-validated hierarchical utility judge, EvoNote-generated notes are preferred over corresponding human-written notes in 89.6% of cases; on a separate set of Needs More Ratings posts without a crowd helpfulness verdict, EvoNote produces helpful notes for 82.0% of cases. It also reduces time to an available candidate correction from multi-hour post-to-note delays to under 2 minutes on average. Analyses link these quality gains to stronger evidence use and reusable correction strategies, positioning self-evolving note generation as a promising paradigm for scalable health misinformation governance.

Abstract:
Recently, several multi-turn dialogue benchmarks have been proposed to evaluate the conversational abilities of large language models (LLMs). As LLMs are increasingly recognized as key technology for advancing intelligent education, owing to their ability to deeply understand instructional contexts and provide personalized guidance, constructing dedicated teacher-student dialogue benchmarks has become particularly important. To this end, we present EduDial, a comprehensive multi-turn teacher-student dialogue dataset. EduDial covers 345 core knowledge points and consists of 34,250 dialogue sessions from interactions between teacher and student agents. Its design is guided by Bloom’s taxonomy of educational objectives and incorporates ten questioning strategies—including situational questioning, zone of proximal development (ZPD) questioning, and metacognitive questioning—better capturing authentic classroom interactions. Furthermore, we design differentiated teaching strategies for students at different cognitive levels, providing more targeted guidance. Building on EduDial, we develop EduDial-LLM 32B via training and propose an 11-dimensional evaluation framework that systematically measures the teaching abilities of LLMs, encompassing overall teaching quality and content quality. Experiments on 17 mainstream LLMs reveal that most models struggle in student-centered teaching scenarios, whereas EduDial-LLM achieves significant gains, consistently outperforming all baselines across all metrics. Code is available at https://github.com/ECNU-RAIL/EduDial-EMNLP2026.

Abstract:
Event extraction aims to identify triggers in text, classify them into predefined event types, and recognize associated arguments such as participants and locations. Large language model (LLM)-based generative approaches provide a unified paradigm for joint trigger identification, classification, and argument extraction; however, their computational cost necessitates knowledge distillation (KD) into smaller student models. Existing KD methods operate at the token level, treating each output position independently and thus failing to concentrate the distillation signal on event-relevant spans or to capture the relational structure among event components. We propose EventKD, a two-level distillation framework tailored for event extraction. At the token level, a distributional divergence loss incorporates event-aware span importance weighting to prioritize tokens corresponding to triggers, arguments, and type labels. At the span level, a Relational-Aware (RA) loss preserves pairwise geometric relationships among event-aware spans across Transformer layers. Experiments on four benchmarks (ACE05, MAVEN, GENEVA, and RAMS) demonstrate that EventKD consistently outperforms existing distillation methods on both trigger identification and argument extraction.

Abstract:
Linters are widely used in code maintenance and real-time IDE editing to surface lint alerts related to performance, style, error-prone, and other patterns in source files. However, existing code benchmarks do not explicitly evaluate linter-guided file refactoring, where models must reduce lint alerts in source files with correct initial functionality while satisfying functionality preservation. We introduce LintEval, a multilingual benchmark for linter-guided file refactoring across C++, Python, and Java, comprising 13,061 programs with correct initial functionality and 171,662 lint alerts extracted using language-specific linters. LintEval jointly evaluates functionality preservation, lint alert reduction, new alert avoidance, and edit minimality. Experiments reveal a persistent trade-off: closed-source commercial models reach up to 90.4% Reduction@P but achieve only 67.1%–68.3% Pass@1, while several open-source small models show low or negative Reduction@P. A controlled fine-tuning study further shows that supervision format matters: refactoring-oriented SFT raises Qwen2.5-Coder-7B Reduction@P from 24.7% to 76.0%. LintEval enables controlled evaluation of models that improve source files under functionality preservation.

Abstract:
Jailbreak attacks threaten Large Language Models (LLMs) by bypassing their safety mechanisms. A truly advanced jailbreak is defined not only by its effectiveness but, more critically, by its stealthiness. However, existing methods face a fundamental trade-off between semantic stealth (hiding malicious intent) and linguistic stealth (appearing natural), leaving them vulnerable to detection. We propose StegoAttack, a framework that leverages steganography. The core insight is to embed a harmful query within a benign, semantically coherent paragraph. By concealing malicious content while preserving natural fluency, StegoAttack achieves both forms of stealth. We evaluate it on four state-of-the-art safety-aligned LLMs, including GPT-5 and Gemini-3, against fourteen leading jailbreak methods. StegoAttack achieves an average attack success rate (ASR) of 95.50%, outperforming all baselines across the four models. Under external detectors, its ASR decreases by less than 27.00% while retaining natural-language characteristics. These results show that steganography can decouple semantic and linguistic stealth, posing a highly concealed and effective security threat. The code is available at https://github.com/GenggengSvan/StegoAttack.

Abstract:
Current plan-based reasoning methods improve large language models (LLMs) by inserting a planning stage before execution, giving rise to the question → plan → cot paradigm. While effective, a closer examination reveals an inherent paradigm-level gap: both the planning and its execution stages decide how to solve a problem, while the prior question of what to solve; recognizing the problem type, the applicable tools, and the foreseeable pitfalls; remains entirely implicit. To bridge this gap, we propose PPC (Preplan-Plan-CoT), a framework that introduces an explicit problem-understanding stage, the preplan, yielding a new question → preplan → plan → cot paradigm. Realizing this paradigm requires safeguarding the conceptual integrity of preplan at both ends. Specifically, we design a three-stage synthesis pipeline with a spoiler-score detector that filters out leakage and spoiler failures to build clean preplan supervision, and a composite GRPO reward enforces that the generated plan genuinely follows from the preplan. Experiments across four backbones and five mathematical reasoning benchmarks show that PPC achieves the best results on 39 of 40 metrics, improving maj@16 and pass@16 by +2.23 and +3.06 over the strongest baseline without introducing additional inference token overhead.

Abstract:
Geometry problem solving poses distinct challenges in artificial intelligence. Existing approaches typically fall into two paradigms: symbolic methods, which exhibit limited adaptability, and neural methods, which are prone to hallucinations. Recent neuro-symbolic hybrids predominantly rely on a unidirectional pipeline where neural outputs are fed into solvers without feedback, making system brittle to early-stage errors. To break this unidirectional bottleneck, we propose BiNSGPS, a framework that establishes Bidirectional Neuro-Symbolic Interaction (BiNS) between a MLLM Adviser and a Symbolic Solver. MLLM Adviser actively incorporates feedback from the symbolic solver to dynamically rectify inconsistent formal representations or propose auxiliary hypotheses, resolving symbolic conflicts and facilitating complex deductions. Experiments show that BiNSGPS achieves state-of-the-art performance, reaching 90.5% completion accuracy on Geometry3K and 90.1% on PGPS9K, while maintaining 96% step-wise logical coherence to balance capability with mathematical rigor.

Abstract:
Understanding persuasion is critical for the safety and reliability of multi-agent systems built on large language models (LLMs). This paper studies persuasion dynamics by contrasting general LLMs with Large Reasoning Models (LRMs) that employ explicit "thinking" processes. Through large-scale experiments on objective (MMLU) and subjective (PersuasionBench and Perspectrum) tasks, we identify Persuasion Duality: reasoning enhances an agent’s persuasive power while simultaneously increasing its resistance to persuasion. For LRMs, adding thinking content increases persuasion rates by 21 pp on average, yet reduces susceptibility to incorrect persuasion by up to 10 pp on objective tasks. Despite these gains, we uncover a critical vulnerability: persuasiveness often stems from superficial cues such as response length and repetition rather than logical validity. Non-semantic padding or repeated conclusions can match or exceed the persuasive effect of coherent reasoning, revealing a strong length bias in agents’ judgments. We further show that persuasion propagates non-linearly in multi-hop agent chains, where intermediate agents may amplify or attenuate influence depending on task subjectivity. Finally, guided by attention analysis, we propose a prompt-level adversarial argument detection method that consistently improves agent robustness.

Abstract:
Retrieval over large codebases is a key component of modern LLM-based software engineering systems. Existing approaches predominantly rely on dense embedding models, while learned sparse retrieval (LSR) remains largely unexplored for code. However, applying sparse retrieval to code is challenging due to subword fragmentation, the semantic gap between natural-language queries and code, the diversity of programming languages and retrieval tasks, and the length of code documents, which can harm sparsity and latency. We introduce SPLADE-Code, the first large-scale family of learned sparse retrieval models specialized for code retrieval (600M–8B parameters). Despite a lightweight one-stage training pipeline, SPLADE-Code achieves state-of-the-art performance among retrievers under 1B parameters (75.4 on MTEB Code) and competitive results at larger scales (79.0 with 8B). We show that learned expansion tokens are critical to bridge lexical and semantic matching, and provide a latency analysis showing that LSR enables sub-millisecond retrieval on a 1M-passage collection with little effectiveness loss.

Abstract:
Diffusion language models (DLMs) have recently emerged as an alternative modeling paradigm to autoregressive LMs, offering advantages such as parallel generation and bidirectional context modeling. Despite growing interest in their generative capabilities, the privacy risks of DLMs remain underexplored. We identify a phenomenon termed token-level memorization asymmetry through theoretical analysis of diffusion training dynamics. Building on this finding, we propose Q-Skew, a quantile-weighted skewness-based indicator for membership inference on finetuned DLMs. Experiments across multiple fine-tuning datasets and models show that our method outperforms existing baselines. Moreover, we show that Q-Skew can also facilitate other privacy violations, such as PII extraction. Our findings reveal a previously underexplored privacy attack surface and highlight the need for systematic privacy evaluation of DLMs.

Abstract:
We present LLaDA-o, an effective and length-adaptive omni diffusion model for multimodal understanding and generation. LLaDA-o is built on a Mixture of Diffusion (MoD) framework that decouples discrete masked diffusion for text understanding and continuous diffusion for visual generation, while coupling them through a shared, simple, and efficient attention backbone that reduces redundant computation for fixed conditions. Building on MoD, we further introduce a data-centric length adaptation strategy that enables flexible-length decoding in multimodal settings without architectural changes. Extensive experiments show that LLaDA-o achieves strong performance on multimodal understanding and generation benchmarks, reaching 87.04 on DPG-Bench and 0.86 on GenEval. To support future work on omni-diffusion models, we release the model and training/evaluation code at https://github.com/ML-GSAI/LLaDA-o.

Abstract:
Neural topic models are widely used to uncover latent thematic structure in text corpora, but often produce topics with limited semantic coherence and weak alignment between topics and documents. Recent refinement approaches based on large language models (LLMs) have shown promising improvements, yet typically rely on repeated generative inference, resulting in high computational cost and practical issues such as sensitivity to prompting, stochastic variability, and the introduction of out-of-vocabulary content. We propose BERT-NTM, an encoder-based semantic refinement framework for neural topic modeling that avoids generative inference altogether. Instead of generating or rewriting topic words, BERT-NTM leverages a pretrained BERT encoder to provide discriminative semantic signals during a refinement phase: topic–word distributions are refined by scoring candidate words within a topic using masked language modeling, and document–topic representations are improved by aligning document embeddings with mixtures of topic embeddings via a Maximum Mean Discrepancy (MMD) objective. Both refinement components are fully parallelizable, stable, and operate strictly within the model vocabulary. Experiments on standard topic modeling benchmarks demonstrate that BERT-NTM consistently improves topic coherence and document–topic alignment while substantially reducing runtime and GPU memory usage compared to LLM-based refinement methods.

Abstract:
On-policy distillation (OPD) trains a student on its own trajectories with token-level teacher feedback and often outperforms off-policy distillation and reinforcement learning with verifiable rewards. However, we find that its standard advantage-weighted policy gradient suffers from three structural weaknesses, including high-variance updates, vanishing gradients in zero-advantage regions, and exploration bottlenecks for models with limited capability. We therefore propose asymmetric on-policy distillation (AOPD), which replaces ineffective policy gradient with localized divergence minimization in non-positive advantage regions while preserving positive reinforcement learning. Experiments on mathematical reasoning benchmarks show that AOPD consistently outperforms standard OPD, with average gains of 4.09 / 8.34 under strong / weak initialization, respectively. AOPD also maintains higher policy entropy during training and better capability retention during sequential tool-use adaptation. Our source code is available at [repository](https://github.com/stargazer319/AOPD).

Abstract:
Per-matrix singular value decomposition (SVD) truncation is Eckart–Young optimal in the whitened Frobenius norm, but errors from independently compressed matrices compound through the block’s nonlinear forward pass. Inspired in part by hierarchical variational optimization in quantum many-body methods, we introduce a three-level chain that widens optimization scope from individual matrices to Transformer blocks to the full model: whitened SVD (L1), block-level joint optimization (L2), and end-to-end language-modeling loss refinement (L3), all from 256 calibration sequences, with no instruction or recovery data. On LLaMA-7B at 60% compression, the chain reduces WikiText-2 perplexity from 42.1 to 19.1 to 11.4. The block-level stage acts as a regularizer: skipping it worsens Penn Treebank (PTB) perplexity by 24 points, a gap that additional end-to-end training did not close in our experiments. Perplexity gains hold across 20–80% compression, five architectures up to 13B parameters, and both in-distribution and out-of-distribution benchmarks, though the cross-architecture rows use architecture-specific configurations and the ratio sweep was not run under one common protocol. With more calibration data, skipping the block-level stage becomes competitive, revealing an offline compute–data trade-off. We therefore claim improvements only in perplexity and compression fidelity; downstream accuracy remains well below the dense model.

Abstract:
Large language models (LLMs) are increasingly deployed as automated reviewers whose numeric scores directly determine accept/reject outcomes, yet whether those scores faithfully reflect the models’ own critiques remains underexplored. We identify threshold-alignment failure: an LLM reviewer may clearly articulate serious weaknesses while still assigning scores above the decision boundary, rendering its critique inconsequential. We introduce DSAR (Dimension-Specialist Audit and Revision), a training-free protocol that decomposes evidence elicitation across dimension specialists, applies severity-coded adversarial audit, and enforces deterministic score corrections for critical findings before thresholding. On PropAlign-240 under a shared Qwen2.5-72B-Instruct backbone, six standard reviewer protocols keep most human-rejected proposals above the decision threshold, yielding false-positive rates of 89.2–100.0%. DSAR raises reject recall to 45.9% and achieves the lowest MAE (0.334), with all decision-level gains statistically significant (McNemar p < 0.008). Ablations are consistent with severity-based audit correction, rather than longer reasoning or additional agents, as the main source of the improvement. These findings argue that LLM reviewers should be evaluated not only by critique quality, but by whether critiques translate into threshold-consistent decisions.

Abstract:
Human disagreement is ubiquitous and well-known in labeling. However, variation in explanations, captured through token-level human rationales, remains far less explored. At the same time, it is unclear how to best evaluate human labels and rationales, or how to best aggregate rationales beyond majority vote, in light of this variation. Yet rationales can reveal the richness of human reasoning, which may differ in style, values, and interpretations, especially in subjective NLP tasks such as hate-speech detection. In this work, we unify diverse models, training strategies, loss functions, and existing evaluation metrics under a single protocol by systematically re-implementing them across different label and rationale representation spaces. Classification metrics are organized around two key properties—predictive and distributional—while explainability metrics through three complementary dimensions: plausibility, faithfulness, and complexity. In this unified evaluation framework, we assess model behavior across classification and explainability metrics, as well as metric sensitivity to the choice of label (hard and soft) and rationale representation space (hard, intermediate and soft). Results show that intermediate and soft representations better capture variation in labels and rationales, with metrics revealing complementary facets of the same property. These findings highlight the need to align evaluation with task goals and rethink subjective NLP evaluation to better reflect diverse human judgments.

Abstract:
Computer-use agents could automate repetitive screen-based clinical work, but their reliability in medical graphical user interfaces remains largely unvalidated. Existing benchmarks focus on general web or desktop tasks and underrepresent medical software, which requires domain knowledge, exhibits markedly different UI design from mainstream applications, lacks public testing environments, and demands safety validation beyond task completion. We introduce MedCUA-Bench , an interactive benchmark for clinical computer-use agents. It covers 18 clinical scenarios across 10 medical domains, reconstructed from real product manuals and open-source medical systems to capture authentic clinical interfaces while avoiding licensing and privacy constraints. Each task ships with paired intent- and step-level goals to disentangle clinical reasoning from UI execution, and is evaluated by a deterministic checker over task completion and five clinical safety dimensions. Across 23 agents, the best closed-source model reaches 54.2% strict success, while all models remain below 9% on the real OpenEMR. Open-source agents average only 2.5%, with the best reaching 16.2%. MedCUA-Bench exposes the gap between current agents and reliable clinical software use, providing a reproducible testbed for future research. Code: https://github.com/Jia7878/MedCUA-BenchMark .

Abstract:
Despite advancements in controllable text-to-music (TTM) generation, existing models struggle with cross-domain voice cloning for singing due to inadequate modeling of speech-derived vocal characteristics, inherently limiting precise timbre transfer. To achieve precise and controllable music generation conditioned on the timbre of reference speech, we propose S2M-Inject, a cross-domain framework driven by reference speech. Incorporating a fixed timbre embedding strategy, S2M-Inject injects speaker representations into an instruction-guided generation pipeline. This effectively preserves target vocal attributes while enabling comprehensive natural-language control over lyrics, genre, instrumentation, rhythm, melody, and mood. Extensive experiments demonstrate that the framework achieves effective cross-domain voice cloning, significantly enhancing vocal similarity while maintaining high-fidelity audio quality in both Chinese and English tasks. This work extends voice cloning into the musical domain, offering a rigorous paradigm for personalized music generation. Audio samples are available at https://anonymous.4open.science/w/S2M-Inject-975.

Abstract:
Large language models often answer the same multiple-choice question inconsistently when it is posed under support-oriented and elimination-oriented framings. We investigate whether these discrepancies arise from different internal representations induced by the two framings. We introduce a dual-framing protocol with minimally varied prompts that use either support- or elimination-oriented framing while keeping the evaluation target fixed. To probe the internal computation, we append an untrained special token, [STATE], and treat its residual-stream activation as an intervention interface. Across both models, the two framings induce separable [STATE] activations concentrated in intermediate layers. Swapping these activations between paired prompts systematically changes predictions and improves cross-framing agreement, providing intervention-based evidence that the activations are behaviorally relevant. Beyond instance-level substitution, mean-difference steering directions derived from the dual-framing contrast exhibit more bounded layer-wise responses than matched contrastive activation addition directions under the evaluated protocol.

Abstract:
Multi-agent large language model (LLM) systems often rely on independent solvers, critique, debate, or answer aggregation. While these designs increase diversity and feedback, they do not necessarily create effective group reasoning: information may be poorly shared, expertise underused, discussions may drift, and consensus may emerge before unresolved issues are examined. We introduce AgendaFlow, a meeting-science-inspired framework that treats multi-agent reasoning as a structured meeting rather than parallel answering or generic debate. AgendaFlow routes each task to a suitable agenda, instantiates complementary functional roles, and uses a moderator to manage phases, speaking order, shared artifacts, unresolved issues, and completion checks. This enables agents to pool information, surface constraints, compare alternatives, repair errors, and converge through a managed collective decision process. Across five reasoning, planning, and rule- or evidence-based benchmarks, AgendaFlow consistently outperforms single-agent, sampling-based, debate-style, and multi-agent orchestration baselines. Ablations, token-matched comparisons, and process-level analyses suggest that the gains come from meeting-level organization rather than agent diversity or additional computation alone, highlighting meeting design as a missing dimension in multi-agent LLM systems. Code is available at: https://github.com/sunming-code/agendaflow.

Abstract:
The integration of large language models (LLMs) into recommender systems has garnered attention due to their ability to leverage rich semantic representations. However, these models suffer from a limitation, knowledge bias, a systematic tendency to over-recommend items dominated by pretraining knowledge, regardless of actual user interests. Although recent studies have focused on fine-tuning LLMs to better align with the collaborative data, our investigation reveals that such fine-tuning fails to fundamentally correct this deep-seated bias. In this work, we propose ReKnow, a lightweight and efficient decoding-time alignment method to mitigate knowledge bias. Specifically, we quantify LLMs’ knowledge towards biased items, then realign output probabilities to match the target data distribution. For validation, we provide both theoretical justifications and empirical results, including evaluation on two datasets, demonstrating that our enhances the quality and diversity of LLM-based sequential recommendation. We have release our code[] for others to reproduce the results.

Abstract:
Effective collaboration in multi-agent systems hinges on communication. Existing studies analyze agent communication indirectly through task performance, such as improvements in reward and success rate, without explicitly characterizing communication itself. As a result, the quality of communication protocols remains difficult to evaluate and interpret, limiting their reliability, scalability, and generalizability. To better characterize communication quality, this survey focuses on communication alignment , which emphasizes semantic fidelity and convention consistency across agents. We review communication alignment from emergent communication in multi-agent reinforcement learning (MARL) to latent communication powered by Large Language Models (LLMs), highlighting existing methods and the emerging challenges introduced by natural-language communication. Finally, we discuss future directions toward more effective, interpretable, and generalizable multi-agent communication.

Abstract:
Emotional Support Conversation (ESC) systems aim to provide holistic support by balancing professional therapeutic competence with natural empathy. However, existing methods struggle to simultaneously achieve structured, stage-aware reasoning and seamless empathy-expertise alignment, often resulting in an artificial splicing of clinical strategies and generic reassurance. To overcome these limitations, we propose ESCRAG-R1 , a unified framework that integrates retrieval-based psychological guidance into Group Relative Policy Optimization (GRPO). By incorporating retrieval into the reinforcement learning loop, ESCRAG-R1 transforms external knowledge into a robust learning signal that stimulates explicit internal reasoning prior to generation and fundamentally reshapes the model’s internal policy. To provide the reliable supervision required for this optimization, we construct ESC-Preference , a high-quality dataset based on a Client–Counselor–Judge evaluation framework that delivers precise, empathy-aware reward signals. Extensive experiments demonstrate that ESCRAG-R1 significantly outperforms existing baselines by mitigating superficial splicing and realizing a natural integration of professional guidance and empathetic expression. Code and datasets will be publicly released at https://anonymous.4open.science/r/ESCRAG-R1-83F1/.

Abstract:
Large Language Model (LLM)-based Multi-Agent Systems (MAS) are vulnerable to semantic attacks that can trigger cascading failures across the network. Current defenses face a fundamental dilemma: lightweight single-auditor methods are prone to single points of failure, while robust committee-based approaches (e.g., voting) incur prohibitive computational costs in multi-turn interactions. To resolve this tension, we propose MAS-Shield, a secure and efficient defense framework designed with a coarse-to-fine filtering pipeline. Rather than applying uniform scrutiny, MAS-Shield dynamically allocates resources by first identifying high-influence nodes via Critical Agent Auditing Selection. These nodes undergo Light Auditing using lightweight sentry models to rapidly filter benign cases, escalating only high-risk signals to a heavyweight Two-Round Consensus mechanism. This hierarchical design optimizes the security-efficiency trade-off. Experiments demonstrate that MAS-Shield achieves a 92.5% recovery rate against diverse adversarial scenarios and reduces defense latency by over 70% compared to existing methods.

Abstract:
Scaling laws guide large language model training by relating compute to cross-entropy loss, and recent work further extends them to predict downstream benchmark performance. However, prior approaches face generalization limitations from two aspects: focusing on benchmark-level performance introduces scenario-specific artifacts, while relying on IID validation loss fails to track capability improvements when training distributions vary. In this work, we argue that downstream scaling should be studied at the capability level, which captures shared skill factors across related tasks while abstracting away benchmark-specific noise. We propose SuperValid, a framework that synthesizes OOD (out-of-distribution), capability-aligned validation data by distilling core concepts from benchmarks within a capability domain and expanding them into diverse, knowledge-rich texts. Extensive experiments spanning 16 benchmarks grouped into 6 capability domains show that SuperValid loss exhibits strong and stable correlation with downstream performance across models of different architectures, scales, and training data distributions. As a training-free metric computable during training without benchmark evaluation, SuperValid enables effective model selection, early stopping, and scaling decisions.

Abstract:
People increasingly turn to large language models (LLMs) for everyday advice, making ethically charged interpersonal problems a practical moral-advisory context. Most prior work has studied this context through single-turn judgments or pressure-laden rebuttals, assumptions that poorly match how guidance is sought in real-world contexts. These assumptions leave unclear whether narration alone, without an explicit opposing position, can shift model judgments during multi-turn moral consultation. Yet real-world moral-conflict conversation often elicits one party’s self-justifying account, which can unfold over multiple turns and create information asymmetry. We introduce narrative captivity, a failure mode in which a model treats an unopposed one-sided account as complete and aligns with the narrator’s interpretation without seeking missing perspectives. To measure this phenomenon, we build a benchmark of 5,078 interpersonal-conflict scenarios spanning six moral dimensions. Across 17 LLMs, narrative captivity is widespread: end-state judgments under multi-turn narration shift by 25 percentage points on average beyond the matched single-turn baseline. Stage-level analysis identifies preference optimization as a major contributor, while four inference-time strategies provide only partial mitigation. We hope our project fosters LLM advisors that preserve independent judgment in real-world consultation.

Abstract:
We study few-shot online clustering (FSOC), where an unlabelled stream must be assigned to a dynamically growing cluster inventory using only a few labelled support examples for currently observed classes. We propose a two-component test-time gradient-free framework. First, we adapt Wasserstein Generative Flow Networks (wGFN) as an amortised LLM-based policy for in-distribution assignment and out-of-distribution (OOD) rejection. Unlike prior work that trains separate policies for each evaluation class set, we show that a single policy trained on a few classes generalises to novel support classes at test time without parameter updates and further replace fixed rejection thresholds with a Dirichlet evidence model over candidate class assignments. Second, rejected OOD examples enter an oracle-mediated cluster-formation stage. Under shuffled stream orderings, this stage can create noisy provisional support sets that corrupt later assignments. To address this, we introduce a von Mises–Fisher Dirichlet Process Mixture Model warm start to provide oracle-independent cluster structure over unit-normalised text embeddings before oracle-mediated formation. This improves clustering quality, in some settings by ∼ 2× , even with small LLMs ( <1 B parameters). To our knowledge, this is the first FSOC framework to couple amortised LLM-based assignment with Bayesian non-parametric warm-starting, enabling growing cluster inventories without data augmentation, novel-class parameter updates or predefined cluster counts.

Abstract:
Large language models exhibit non-negligible risks of generating policy-violating content, underscoring the need for effective and timely guardrail mechanisms. Existing streaming safety monitoring approaches enable detection during decoding, but still lack a reliable mechanism for deciding when to terminate generation, limiting their ability to act on early risk signals that are often sparse and unstable under partial observability. In response, we propose LTS, a token-level framework that explicitly learns a stopping criterion for preemptive early termination. LTS introduces a risk-aware STOP head for token-level risk assessment and leverages temporally structured supervision with multiscale evidence to model the evolution of safety risks during generation. By casting streaming safety monitoring as a constrained online stopping task and incorporating exponential moving average temporal aggregation, LTS enables reliable translation of emerging risk evidence into stable and timely termination decisions. LTS achieves an F1 score of 95.5% and is able to terminate harmful content at 13% of the generated sequence, substantially outperforming existing methods during early generation. In realistic streaming settings, it achieves around 90% token reduction.

Abstract:
Large language model (LLM) agent failures often contain multiple related errors rather than a single mistake. Existing attribution methods usually identify a responsible agent, step, or root cause, but do not explicitly model dependency between errors. We introduce EDGE, an error dependency graph-guided framework for multi-error attribution in LLM agent traces. EDGE constructs an error dependency graph from observed error events and validates a reliable causal subset through counterfactual rollout. The inference graph guides a two-stage LLM-as-judge detector for error attribution, and the intervention-validated subgraph provides a more reliable basis for explanation and repair analysis. Experiments on TRAIL and MAST show that EDGE improves category-level multi-error attribution across most evaluated models and settings. Experiments with adapted Who&When-style prompts show that the graph helps across prompting strategies. These results suggest that dependency structure is a useful diagnostic prior for agent failures beyond isolated root-cause prediction.

Abstract:
Automated survey generation systems increasingly rely on flat retrieval over document collections, limiting their ability to capture the evolutionary structure of research fields. We present SurveyGraph, a framework centered on a Methods-Entity Knowledge Graph (MEKG) that encodes method-level follow-on, competition, and shared-resource relations across papers. The MEKG serves as a unified backbone for core-paper discovery, graph-aware retrieval, hybrid outline generation, and evolutionary narrative synthesis. We evaluate SurveyGraph through a multi-level protocol spanning retrieval benchmarks, semantic-edge validation, citation grounding audits, benchmark-facing collection probes, and same-backend generation-stage diagnostics. In a controlled seven-method equal-budget benchmark, MEKG-anchored fusion achieves the best balance of recall and structural density among all tested methods, demonstrating that typed graph structure provides a complementary signal to simpler retrieval approaches. On the archived graph-native core diagnostic, graph-aware retrieval nearly doubles recall over flat lexical baselines across all eight runs (p = 0.004). A semantic evidence gate with AUC = 0.75 filters noisy edges, and citation-grounding audits over 160 generated sentences confirm 80.6% anchored-or-partial support with evidence-level contrasts validating genuine alignment. Human validation over 108 stratified semantic edges (Cohen’s κ = 0.64 for the binary support decision) confirms 68.5% supported-or-partial after adjudication, with improves relations strongest at 80.6% and extends weakest at 52.8%. A SurveyBench-facing collection probe reaches 0.521 mean reference coverage on four topics under an eight-paper budget; same-backend Qwen diagnostics over a 30-paper/74-relation MEKG further show that removing hybrid outlining lowers artifact organization signals. We report concurrent-system comparisons only as bounded diagnostics rather than same-backend end-to-end claims. Code and artifacts are available at: https://github.com/xzzvsxd/SurveyGraph.

Abstract:
Vision-language-action (VLA) models for autonomous driving typically express intermediate reasoning entirely in text, which may discard decision-relevant visual information. We propose Latent Dreaming (LatDM), a latent–text reasoning framework that complements textual chain-of-thought with continuous latent states. LatDM uses latent blocks to preserve visually grounded spatial and temporal information, while text handles semantic abstraction and decision communication. A pretrained visual encoder provides latent supervision during training, and the model autonomously interleaves latent and textual reasoning at inference. Experiments on VLADBench, CODA-LM, and NuInstruct show consistent improvements over the underlying VLA backbone, while ablations confirm that the latent channel contributes beyond standard textual fine-tuning.

Abstract:
LLM-based agents face challenges in personalized interactive tasks due to heterogeneous, multi-typed, and implicitly constrained long-term traces. Existing memory mechanisms struggle with accurate routing and retrieval, especially on-device where personalization is critical. Most methods use single-vector representations, blurring type distinctions and relational structure. We propose HGP, a hybrid graph memory framework. HGP employs a lightweight self-enhancement classifier for personalized memory routing and constructs episodic, semantic, and procedural memories as graphs. It also extracts working memory as a state trajectory to capture current state and implicit constraints, ensuring reliable decision- making. The classifier reduces large-model calls, enabling on-device deployment, while graph storage enables accurate retrieval and incremental user profile refinement. Experiments on two benchmarks show that on PAL-Set solution selection, HGP achieves an S-score of 35.58, nearly 7 points above the strongest baseline.

Abstract:
Retrieval-Augmented Generation (RAG) typically assumes that external knowledge is free, but many high-quality sources are paywalled, licensed, restricted, or otherwise costly to access. We introduce cost-aware RAG, a setting where retrieved evidence is assigned access-cost tiers and systems must answer under an explicit evidence-access budget. We instantiate this setting by augmenting MS MARCO v2.1 with access-friction tiers and evaluate budgeted evidence selection across general-domain and domain-specific QA benchmarks. Our results show that static selection is brittle: no fixed selector uniformly dominates, and larger budgets do not reliably improve answer quality, even when costly evidence is domain-matched. We then study agentic cost-aware RAG, where an LLM decides when to retrieve, which tier to access, and when to stop. Agents show strong promise as adaptive evidence-acquisition controllers, but their behavior remains highly model- and task-dependent. These findings suggest that cost-aware evidence acquisition is a central challenge for the next generation of RAG systems. All code and data are available at https://github.com/Mignonmy/Cost-Aware.

Abstract:
Recent interest in multimodal large language models (MLLMs) raises a central question: can they reason over dynamic visual evidence rather than merely recognize objects or events in individual frames? This ability, which we refer to as video temporal-logical reasoning, requires models to maintain, update, and compose evidence as visual states evolve across frames. Existing video benchmarks often conflate this capability with scene complexity, static recognition, or uncontrolled temporal variation. To isolate this capability, we introduce Video-MME-Logical, a controlled benchmark organized around five temporal-logical operations: state tracking, sequential counting, temporal ordering, dynamic spatiality, and structural composition. The benchmark contains 25 fine-grained task categories generated with controlled object states, transitions, temporal dependencies, and logical compositions. It enables difficulty-controlled final-answer evaluation by varying temporal horizon and reasoning complexity, and supports intermediate-state diagnostics by verifying whether models recover the required logical reasoning trace before producing the final answer. Experiments with state-of-the-art MLLMs reveal a substantial human-model gap, especially as temporal-logical complexity increases. Supervised fine-tuning on up to 500K generated samples improves performance but remains insufficient to close the reasoning gap, positioning Video-MME-Logical as a scalable testbed for analyzing and improving temporal-logical reasoning in MLLMs.

Abstract:
Retrieval-Augmented Generation (RAG) enhances large language models with external knowledge, but long retrieved contexts incur high computational cost and often degrade performance due to context–knowledge mismatch. Parametric RAG (PRAG) addresses this by integrating external knowledge into model parameters via lightweight adapters. However, existing PRAG methods are fundamentally document-centric, treating each document as an independent unit of knowledge, which leads to poor scalability and weak supervision. We argue that knowledge is inherently distributed across semantically related documents, and document-level parameterization is a suboptimal granularity. To this end, we propose ParaShard, a knowledge-centric framework that organizes knowledge into semantically coherent knowledge shards and parameterizes each shard with a shared adapter. This design enables richer supervision across related documents while significantly reducing the number of required adapters. At inference time, ParaShard dynamically selects and composes multiple shards using a diversity-aware strategy to ensure comprehensive knowledge coverage. Experiments on four knowledge-intensive QA benchmarks show that ParaShard achieves superior average performance across diverse knowledge-intensive tasks compared with strong RAG and PRAG baselines, with particularly strong gains on multi-hop reasoning, and substantially reduces storage cost compared to PRAG, demonstrating the knowledge injection effectiveness and storage efficiency of knowledge-centric parameterization.

Abstract:
Tool use is often assumed to monotonically improve reasoning, where external evidence is expected to help when relevant and be ignored when irrelevant. We show that this assumption fails in a state-dependent way. Across benchmarks with Python and Wikipedia tools, external evidence reliably helps when initial beliefs are weak, but can flip already-correct answers when those beliefs are strong. We frame this as a misallocation of revision authority, arguing that deferring to external evidence is suboptimal when internal support for the correct answer surpasses the tool’s expected output quality. This predicts that harm should concentrate on high-confidence no-tool cases. We test this prediction with threshold localization, wrong-trace audits, and a same-clue intervention showing that revision framing changes the damage caused by misleading evidence. These findings suggest that mixed no-tool/tool-assisted inference should arbitrate authority rather than privilege tool evidence by default. As a minimal demonstration, we introduce CASE, a label-free controller that selects between no-tool and tool-assisted trajectories using answer-state certainty and improves over existing confidence-based arbitration baselines.

Abstract:
AI agents may assist with emails and documents, but can they reliably complete everyday online workflows on real websites? Everyday online tasks offer a realistic yet unsolved testbed for evaluating next-generation AI agents. To this end, we introduce ClawBench, an evaluation framework comprising 153 everyday online tasks common in daily life and work, spanning 144 platforms across 15 categories, from completing purchases and booking appointments to submitting job applications. These tasks require capabilities beyond existing benchmarks, such as obtaining relevant information from user-provided documents, navigating multi-step workflows across diverse platforms, and write-heavy operations like filling in many detailed forms correctly. Unlike existing benchmarks that evaluate agents in offline sandboxes with static pages, ClawBench operates on production websites, preserving the full complexity, dynamic nature, and interaction challenges of real-world web environments. An interception layer captures and blocks the final submission request, ensuring safe evaluation without real-world side effects. Our evaluations of 8 frontier models show that both proprietary and open-source models complete only a small portion of these tasks. For example, Claude Sonnet 4.6 achieves only 33.3%, which exposes gaps in current AI agents. Progress on ClawBench brings us closer to AI agents that can function as general-purpose assistants.

Abstract:
Can financial vision–language models (VLMs) turn chart evidence into reliable action recommendations? Existing hallucination evaluations are mostly claim-centric: they assess whether generated statements are supported, but not whether evidence remains traceable through rationale, confidence, and final action. We introduce E2A-Bench, a 969-query benchmark for financial chart reasoning, constructed from 323 HS300 constituents under three input modalities with deterministic OHLCV-derived evidence anchors. E2A-Bench evaluates grounding, reasoning–action consistency, evidence–confidence calibration, and directional coverage through UCR, RCI, ECI, and NDR, where NDR measures coverage-aware evidence-to-action reliability rather than realized trading performance. Evaluating 20 VLMs reveals three failures hidden by scalar hallucination scores: the lowest-UCR model ranks near the bottom by NDR due to only 6.4% directional coverage; oracle-aided verification reduces unsupported claims but can collapse coverage; and financial fine-tuning amplifies the BUY:SELL ratio by 4.21–4.68 × across strict base–fine-tuned pairs. These results show that financial VLM evaluation should trace the full evidence-to-action chain rather than rely on a single hallucination score. Code and data: https://github.com/wanng-ide/E2A-Bench .

Abstract:
Recent work leverages Large Language Models (LLMs) to generate executable code for pedagogical animations using libraries such as Manim. However, ensuring spatial correctness and visual legibility remains challenging, as existing frameworks emphasize pedagogical content while overlooking geometric occlusions. We propose the Symbolic Geometric Agent (SGA), a plug-and-play module for code-centric animation pipelines that intercepts LLM-generated code, performs partial execution to extract symbolic scene graphs, and applies targeted refinement when spatial conflicts are detected. We further introduce the Manim Visual Quality Score (MVQS), a deterministic rendering-free proxy for spatial integrity. Experiments on the MMMC-Code benchmark across four LLM backbones and two agentic pipelines show that SGA achieves a peak MVQS of 73.11 (Code2Video + GPT-5.1), corresponding to a 16.1% relative improvement over the raw baseline, and improves MVQS in 7 of 8 backbone × pipeline configurations. Additionally, we conduct a human evaluation showing that these improvements translate into human preference, with raters preferring SGA over the raw baseline in 84.4% of comparisons and over a VLM-based critic in 65.0%.

Abstract:
Vision Language Models (VLMs) should rely on visual evidence that directly determines the correct answer, but supervision for grounding visual reasoning is often expensive to obtain manually or tied to dataset-specific annotation primitives. We instead introduce model-causal visual evidence as an annotation target, defined as the set of image regions whose counterfactual intervention changes a model’s answer distribution for a given image-question pair. Based on this principle, we introduce Counterfactual Search for Grounding Regions (CSGR). CSGR is a scalable pipeline that proposes candidate regions, perturbs them, measures their effect on answer sensitivity, and aggregates this evidence across multiple judges to approximate answer-critical regions in VQA data. To assess whether CSGR annotations contain a useful supervision signal, we plug them into three existing grounding-aware training routines: attention steering, Visual CoT finetuning, and latent visual reasoning. These experiments test whether the proposed annotation scheme can provide a useful supervision signal across multiple ways of consuming region labels, rather than introducing a new way of using them. Across competing automatic region-labeling mechanisms, CSGR annotations provide the most consistent gains over Cross Entropy-only finetuning in both in-domain and out-of-domain evaluations, indicating that the proposed labeling scheme captures useful region-level information.

Abstract:
Preference-based post-training aligns LLM chatbots with user preferences, but companion chatbots pose a subtle safety problem: responses that feel warm, validating, and useful can also increase Social Risk by encouraging one-sided validation, conflict escalation, sycophancy, or AI dependence. We study an upstream data-design choice that occurs before preference optimization, namely which candidate responses are allowed to compete for positive/negative labels. Using 4,500 realistic Chinese help-seeking prompts from 1,500 personas, we compare three response sources: Engaged replies from support-conditioned companion prompts, Neutral replies from a general assistant, and Guarded replies emphasizing boundaries and de-escalation. We build AI-feedback preference data under support-maximizing, risk-minimizing, and risk-bounded support targets, then post-train Qwen3.5-9B with DPO and KTO, using SFT as a diagnostic imitation baseline. Results show that the same target can teach different behaviors because candidate pools induce different positive/negative geometries in support-risk space. Engaged-only pairs provide weak positive/negative support contrast; Engaged-Guarded pairs create a larger gap but tie higher support to higher risk; and Engaged-Neutral pairs produce a more favorable geometry under support maximization, yielding +0.43 Supportiveness and -0.23 Social Risk with KTO. Mechanistic analysis suggests that this success does not transfer to the risk-bounded target because the constraint separates positive and negative regions, reverses the support margin, and removes the exploitable support-aligned direction. A Neutral-only control succeeds because within-source variation preserves usable margins in the low-risk region. These findings show that safer post-training for LLM companion chatbots depends not only on losses or rewards, but also on the candidate pools and label geometry that define the learning signal. Code is available at https://github.com/Ameeeeeeeeeeeeee/Safer-Pools.

Abstract:
External text can override conflicting image evidence in multimodal large language models, a failure we call multimodal contextual sycophancy. We introduce a 998-case diagnostic that independently varies visual evidence, commonsense priors, and external text, and probe when this failure arises by moving the information boundary around a context-blind visual witness. On abnormal images paired with Gemini-generated false text, GPT-5.1 scores 7.9% under joint conditioning, 49.7% when the context-blind witness report is scored directly, 63.7% under a matched two-call witness–arbiter pipeline that exposes the witness to the text, and 84.2% under System-2 Visual Arbitration (S2VA), which withholds the text from the witness. Across six models, S2VA improves over the direct witness report by 19.7–44.1 points, with all paired 95% confidence intervals excluding zero. The best information boundary is not uniform: textual context scaffolds some models, and a GPT-4o-regenerated subset changes the relative ordering of joint conditioning, Witness-Only, and S2VA. Contextual sycophancy is therefore sensitive to when text is introduced, as well as to the model and context source.

Abstract:
Large language models are increasingly deployed as agents for long-horizon tasks, yet their performance is shaped not only by model capability and environment design, but also by the harness that mediates agent–environment interaction. Existing harnesses are largely manually engineered, making them difficult to scale as trajectories grow longer and interactions become more complex. In this work, we ask whether harness engineering can be formulated as an end-to-end learnable generation problem. We introduce HarnessBridge, a lightweight learnable harness policy that parameterizes the agent–environment interface as a bidirectional projection. HarnessBridge learns two bidirectional projections: observation projection, which distills raw trajectories into compact, decision-relevant states, and action projection, which converts proposed actions into executable transitions or trajectory-grounded rejections. We train HarnessBridge on a harness supervision dataset via unified instruction tuning. On Terminal-Bench 2.0 and SWE-bench Verified, HarnessBridge matches or surpasses strong specialized harnesses while substantially reducing token usage and trajectory length, and generalizes from smaller generators to larger commercial models. Our code is released at: https://anonymous.4open.science/r/HarnessBridge-6CDC .

Abstract:
Knowledge Editing (KE) aims to update specific facts in large language models (LLMs) without full retraining. Prior efforts sought to tune the knowledge layers of LLMs, achieving near-perfect performance in teacher-forced evaluations. However, under realistic autoregressive generation evaluation, their effectiveness remains limited by two core challenges: (1) even when methods successfully inject new knowledge, pre-trained capabilities can degrade, and (2) more critically, the injected parametric knowledge is not reliably expressed at inference time. To this end, we formalize knowledge editing as a two-stage problem and propose EtCon, an edit-then-consolidate paradigm that couples targeted edits with post-edit consolidation. Within EtCon, the edit stage applies Targeted Proximal Supervised Fine-Tuning (TPSFT) to update parametric knowledge with minimal disruption to pre-trained capabilities. The consolidation stage applies Group Relative Policy Optimization (GRPO) to align the model’s autoregressive generation with the edited parametric knowledge. Extensive experiments demonstrate that our instantiation (TPSFT + GRPO) improves editing reliability and real-world generalization while preserving pre-trained capabilities, and that EtCon consistently improves existing editing methods when instantiated with them.

Abstract:
Inference optimizations are routinely evaluated by throughput alone, without verifying output correctness. We conduct a forensic analysis of batch speculative decoding and find that several widely-used implementations silently produce corrupted outputs (repetitive tokens, symbols) while reporting competitive speed; failures invisible to metrics like ROUGE. We trace the root cause to the ragged tensor problem: variable token acceptance desynchronizes position IDs, attention masks, and KV-cache across a batch. We formalize the synchronization invariants (rectangular alignment and position-ID contiguity) that valid batched inference must preserve and show that maintaining them incurs superlinear alignment overhead under contiguous layouts. EQSPEC enforces the invariants without custom kernels; EXSPEC schedules same-length sequences to bypass realignment. On SpecBench across three model families, EXSPEC reaches 3 x throughput at batch size 8 with 95% exact match to standard decoding; residual divergence traces to floating-point non-determinism, not synchronization error.

Abstract:
Large language model (LLM) agents are trained with reinforcement learning (RL) for complex decision-making tasks. However, most RL-trained agents remain episodic and cannot accumulate reusable knowledge across episodes. Recent skill-based approaches, such as SkillRL, attempt to address this issue by extracting skills from raw trajectories, but treat the skill bank as an append-only repository without verifying whether stored skills remain effective. In this paper, we propose SkillForge, a framework for continuous skill evolution that enables skills to be verified and refined through environment interaction. By making skill usage explicit during agent interaction, RL can directly optimize both environment actions and skill invocation decisions. SkillForge further introduces evidence-based skill verification and multi-pathway skill induction, allowing the skill bank to continuously grow while maintaining its quality. Extensive experiments on ALFWorld, WebShop, and AppWorld show that SkillForge consistently outperforms SkillRL, demonstrating the effectiveness of continuously verified skills in training stronger LLM agents.

Abstract:
Generative Retrieval (GR) has emerged as a promising paradigm by mapping queries directly to Semantic IDs (SIDs) with powerful representation capabilities for candidate items. However, existing SIDs derived solely from item content create a semantic gap, failing to align dynamic query intents with static item representations. Furthermore, current generative paradigms rarely model user behavior sequences and are always bottlenecked by the high inference latency of beam-search autoregressive decoding. To address these challenges, we propose C ross-component H ierarchical semantic A lignment for P ersonalized generative retrieval ( CHAP ), a novel personalized GR framework from a hierarchical perspective. First, we design a Hierarchical Semantic Alignment module to align query’s latent space with item’s quantization path and synchronize multi-granular semantics. Second, we construct a personalized GR framework that models user behavior by synergizing discrete SIDs for structural guidance and continuous representations for fine-grained semantic refinement. Notably, we introduce a Residual Cascading Generation mechanism to restrict the costly multi-step Transformer Decoder to a single-pass inference, boosting inference throughput while mitigating information loss. Extensive experiments on three public datasets, one proprietary industrial dataset, and online A/B tests demonstrate CHAP’s superiority, validating the effectiveness and practical value of our approach. The code is publicly available at https://github.com/zzzgm/CHAP.

Abstract:
Pruning is a critical technique for compressing Large Language Models (LLMs), yet its effectiveness is highly sensitive to the small calibration set used for parameter importance estimation. Current selection relies on simple heuristics like random sampling or entropy, causing large variation in post-pruning perplexity. In this work, we reveal the source of this instability at two levels: calibration samples are not equally important, and a distinct quality hierarchy, comprising Golden, Mediocre, and Detrimental samples, exists within any data pool. More importantly, we demonstrate that this hierarchy is not fixed: the utility of a sample is highly context-dependent, varying with the specific model and pruning algorithm, which renders static filtering approaches fundamentally insufficient. To address these, we introduce OASIS, the first end-to-end framework that directly optimizes calibration data selection with respect to the pruned model’s downstream performance. OASIS leverages a differentiable soft-mask proxy to propagate task-level gradients back to data importance weights. Comprehensive experiments across the Llama and Qwen families show that OASIS improves calibration robustness across the evaluated pruning settings, establishing a reliable standard for data-aware LLM compression.

Abstract:
Indirect prompt injection attacks exploit a fundamental weakness of LLMs: executing unauthorized tasks from untrusted sources. Despite the practical significance of this attack vector, existing benchmarks rely on small sets of template-based attacks (100–1,000 examples) that fail to capture the diversity and adaptiveness of real-world adversaries. Consequently, benchmarks are saturated and overfit to static attacks. We introduce LLMail-Inject, a large-scale benchmark of 208,095 distinct human-written attack prompts collected through a public red-teaming challenge with 839 participants. LLMail-Inject evaluates the attack’s complete chain of execution: prompts must be retrieved from an email database, evade production-level defenses, and trigger unauthorized tool calls with correct arguments. Evaluation reveals that models that achieve < 5% attack success rate (ASR) on existing benchmarks fail more on LLMail-Inject: Meta SecAlign–specifically trained against prompt injection–still yields 32% ASR, GPT-4o exhibits 58% ASR, and GPT-5 mini 47% ASR. We further demonstrate that attacks transfer across domains, achieving 37% ASR on proof-of-concept coding tasks and 28-45% ASR on file management tasks. We release our challenge platform and curated evaluation subsets at multiple difficulty tiers and the full dataset with rich annotations.

Abstract:
A major question in cognitive modeling concerns the behavioral alignment between large language models and humans across linguistic and non-linguistic tasks. Unlike standard LLMs, large reasoning models (LRMs) are optimized with reinforcement learning from verifiable rewards, encouraging correct solutions to reasoning tasks rather than preference-aligned responses. Recent work (de Varda et al., 2025) investigates the cost of thinking in humans and LRMs by comparing human reaction times with model reasoning traces across a range of reasoning tasks. We isolate this alignment by turning to abductive reasoning: unlike deductive tasks, its difficulty cannot be inferred from formal structure and offers no shortcuts a model could exploit to mimic effort without genuine search, providing firmer ground for empirical claims of shared effort. We find further evidence of alignment between LRM and human reasoning effort, as well as evidence that models and humans tend to make similar errors. Finally, we show that decoding methods that let models explore multiple reasoning paths increase alignment in reasoning cost between humans and LRMs across the three models tested.

Abstract:
Large language models (LLMs) are widely used as information sources, yet small changes in semantic assumptions can destabilize their beliefs. We introduce P-StaT (Perturbation Stability of Truth), a framework for evaluating belief stability under matched semantic perturbations in both representational and behavioral settings. Across 21 LLMs and three domains, we compare perturbations involving familiar Fictional statements against synthetically generated and unfamiliar Synthetic statements. Unfamiliar Synthetic perturbations generally induce greater epistemic instability than familiar Fictional perturbations, with behavioral belief retraction rates frequently exceeding 50%. Finally, exploratory clustering analyses reveal recurring themes among retracted statements, including ambiguity, technical terminology, and obscure concepts. These results show that epistemic familiarity is systematically associated with stability under semantic reframing, suggesting that stability-based analyses can complement accuracy-based benchmarks when evaluating LLM robustness and reliability. Code and data are available at https://github.com/samanthadies/P-StaT.

Abstract:
Multimodal retrieval-augmented Generation (MM-RAG) is a key approach for applying large language models (LLMs) and agents to real-world knowledge bases, yet current evaluations are fragmented—focusing on either text or images in isolation, or simplified multimodal setup, failing to capture document-centric multimodal use cases. In this paper, we introduce UNIDOC-BENCH, the first large-scale, realistic benchmark for MM-RAG built from 70k real-world PDF pages across 8 domains. Our pipeline extracts and links evidence from text, tables, and figures, then generates 1,600 multimodal QA pairs spanning factual retrieval, comparison, summarization, and logical reasoning queries. To ensure reliability, 20% of QA pairs are validated by multiple annotators and expert adjudication. UNIDOC-BENCH supports apples-to-apples comparison across four paradigms — 1) text-only, 2) image-only, 3) multimodal text–image fusion and 4) multimodal joint retrieval — under a unified protocol with standardized candidate pools, prompts, and evaluation metrics. Our experiments show that multimodal text–image fusion RAG systems consistently outperform both unimodal and jointly multimodal embedding–based retrieval, indicating that neither text nor images alone are sufficient and that current multimodal embeddings remain inadequate. Beyond benchmarking, our analysis reveals when and how visual context complements textual evidence, uncovers systematic failure modes, and offers actionable guidance for developing more robust MM-RAG pipelines.

Abstract:
Automatic academic paper-to-slide generation is inherently iterative, because creating an effective presentation requires repeated cycles of generation, critique, and revision. Recent multi-agent systems partially acknowledge this through internal critique-and-revise loops, while conversational approaches allow users to refine generated slide decks through dialog. However, these refinement processes either remain largely closed to the user or introduce feedback only after a complete deck has been produced, limiting the user’s ability to participate in the iterative refinement of narrative flow, content allocation, and presentation emphasis. To address this gap, we introduce ConvDeck, a multi-agent pipeline for conversational paper-to-slide generation that distributes interaction across the pipeline through stage-specific loops, allowing users to iteratively refine both the presentation outline and the final slide deck at the stages where each kind of decision is made. These loops are driven by a refinement mechanism in which agents can think, speak, and act, enabling them to either directly apply edits or respond conversationally to clarify user feedback and discuss revision options. Our evaluation shows that stage-specific conversational feedback improves user-goal satisfaction while preserving narrative coherence, content quality, and visual presentation.

Abstract:
Chain-of-thought (CoT) reasoning powers generative models by eliciting intermediate steps before producing an answer. In autonomous driving, the answer is a continuous action. Thus its reasoning must share the same spatiotemporal structure as the physical world. This survey studies the resulting shift from textual CoT to action-grounded reasoning. Surveying 171 papers, including 130 method papers and 41 benchmarks, datasets, surveys, and analysis papers, we propose a representation-centered taxonomy that treats the form of the intermediate state as the organizing axis. We systematize the 130 methods into four categories: language-based, visual-spatial, latent-dynamic, and externalized reasoning, further divided into 13 subtypes tied to distinct regions of interests. Our synthesis shows that the open frontier of reasoning in driving agents lies in intermediate representations that can be grounded in the real world, coupled to real-time action, and verified under safety-critical systems.

Abstract:
Understanding what a user believes and intends is central to building effective agent assistants. This ability is often evaluated through Theory-of-Mind (ToM) tasks, where success requires reasoning from the user’s perspective. However, many existing approaches address ToM with complex pipelines that model behavior indirectly, without explicitly reconstructing the user’s mental state. This misses the core structure of the problem: users act based on their beliefs, which are updated through observations of the environment; beliefs and intentions jointly determine actions, which in turn change the environment; and social reasoning often requires nested beliefs about what others believe or intend. We propose UserHarness, a simple framework that reframes ToM reasoning as explicit user-mind reconstruction. UserHarness decomposes the user’s mental state, its relation to the external environment, and the actions that follow from it, enabling agents to track what the user observes, believes, intends, and does. Across five benchmarks, UserHarness reaches up to 95.94% macro accuracy, improving over existing inference methods by more than 15% relative and over the strongest prompt-only harness by about 20% relative. These results suggest that robust user understanding requires reasoning from the roots of the user’s mind, positioning user harnessing as a promising foundation for more adaptive future assistants.

Abstract:
Tool-Integrated Reasoning (TIR) improves the problem-solving ability of Large Language Models (LLMs), but current TIR agents often invoke external tools even when internal reasoning is sufficient. This cognitive offloading increases latency and can expose the model to noisy tool feedback. We propose AdaTIR, a difficulty-aware policy optimization framework that applies efficiency pressure only when a task is estimated to be solvable under the current TIR policy. This encourages reasoning internalization on easier tasks while preserving tool access for harder ones. We further identify a sign-reversal failure in group-normalized RL: once rollouts in a group are mostly correct, an auxiliary tool-use penalty can dominate the correctness signal and assign negative advantages to correct but tool-heavy trajectories. AdaTIR addresses this with Clipped Advantage Shaping (CAS), which keeps efficiency as a bounded secondary signal. Across mathematical benchmarks, AdaTIR reduces average tool calls by up to 97.6% while maintaining or improving accuracy, and improves AIME 2024 accuracy by 4.8% when tool access is disabled. Additional results on search-based QA and mixed-tool settings indicate that the mechanism is not limited to Python-based math reasoning.

Abstract:
Large language models (LLMs) show strong performance in medical question answering, but their computational costs limit deployment under resource constraints. Therefore, an increasing number of studies focus on distilling knowledge from LLMs into smaller models to improve performance in medical question answering. However, existing methods often fail to identify truly useful knowledge to distill, making it difficult to focus distillation on the student’s real weaknesses. As a result, much of the distillation budget is spent on already acquired knowledge rather than on weaknesses that still limit performance. To address this problem, we propose LEAD, a Learnable Error-Aware Distillation framework that precisely improves the weaknesses of student models. First, LEAD identifies questions answered incorrectly by the student model to determine which weaknesses require teacher guidance. Then, LEAD generates teacher rationales and similar samples to construct candidate supervision for these errors. Through supervision refinement, reliable and learnable training samples are retained from the candidate supervision. Finally, LEAD trains the student on a balanced sample mixture to correct learnable errors while preserving previously acquired knowledge. Experiments on four medical question answering benchmarks show that LEAD outperforms existing distillation methods.

Abstract:
Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While large language models are increasingly adopted for this task, we investigate a previously overlooked limitation in their judgment capabilities: despite generating reasoning rationales that closely mirror those of human experts, their final novelty judgments often diverge substantially. We demonstrate that this miscalibration stems from a systematic bias towards judging ideas as "medium novel". To mitigate this, we propose Think-Probe-Respond (TPR), a lightweight approach that probes latent novelty judgments from hidden states during the reasoning phase and uses the probed judgments to condition the final response. Across strong baselines, TPR improves novelty judgment performance by 22.30% and successfully mitigates the prevalent "medium novelty" bias.

Abstract:
AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 170 trajectories across 11 diverse task settings, including structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AgentRx, an automated diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. AgentRx improves step localization by 75% on average over prior work, while providing failure category attribution.

Abstract:
Multi-trait automated essay scoring extends holistic scoring by evaluating essays across multiple writing dimensions. In this framework, inter-trait consistency signals provide an effective way to improve prediction across traits. However, learning shared scoring signals may obscure trait-specific features, while enforcing consistency across traits can introduce negative transfer, hindering the model’s ability to distinguish individual traits. To solve these issues, we propose DPRES, a dual-perspective decoupled representation learning framework that disentangles consistency and specificity at both the feature and optimization levels. Specifically, we first propose a residual subspace decomposition strategy that enforces orthogonality between shared scoring features and trait-specific features, thereby disentangling shared scoring signals from trait-specific scoring signals in the latent representation space and enhancing the model’s ability to capture trait-specific scoring features. Based on the subspace decomposition strategy, we further design a consistency–specificity decoupled learning strategy to simultaneously preserve inter-trait correlations and enhance trait-level discrimination. Experiments on public datasets demonstrate that the proposed method consistently outperforms strong baselines.

Abstract:
Scientific weak signals are early, low-visibility research directions that later become central to mature scientific topics, yet existing resources such as trend tracking, citation forecasting, and foresight reports rarely provide validated reference sets that link concrete early precursors to later paradigms. We introduce BackTrend, a retrospective benchmark in which, given a mature target topic and a temporal evidence constraint, systems must recover two types of precursors: problem-space signals, underrecognized research problems, and solution-space signals, emerging methods for known problems. BackTrend contains 25 mature target topics in artificial intelligence and machine learning and 66 human-validated weak signals, reconstructed from large-scale literature by grounding each candidate in its 2019–2024 publication-frequency trajectory. We evaluate frontier LLMs, RAG systems, and agentic research systems via set-level and signal-level semantic matching. Current systems generate plausible but misaligned precursors, with topic drift, granularity mismatch, lexical near-miss, and coverage failure; no system exceeds single-digit LLM-judged F1, and additional retrieval or web search does not consistently help.

Abstract:
LLM-based multi-agent systems (MAS) solve complex tasks through inter-agent collaboration, but their communication-driven nature also allows security risks to spread across agents and trigger system-wide failures. Existing MAS defenses mainly follow a reactive paradigm after execution by detecting and isolating harmful agents, which may cause irreversible damage and degrade collaborative utility. To address this, we propose a proactive defense framework for MAS security, namely a Simulation-aware Interception Guard (SAIGuard). SAIGuard performs communication-state simulation over the MAS interaction graph, estimates the impact of incoming messages on local agent states and the global MAS state, and detects risky messages via reconstruction deviations from benign communication patterns. Instead of isolating agents, SAIGuard sanitizes or regenerates suspicious messages before it propagation into system. Experiments across diverse topologies and attack scenarios show that SAIGuard reduces attack success rates while maintaining MAS utility, outperforming reactive defenses.

Abstract:
Persona-driven role-playing agents tend to drift from their assigned profile or contradict their earlier statements as the dialogue lengthens, a faithfulness failure we call persona inconsistency hallucination. Existing inference-time mitigations apply reward-guided decoding uniformly at every token, while we observe that inconsistency risk is concentrated on a small subset of turns and is locally predictable, which makes a selective intervention feasible. We propose TaDI (Targeted Decoding Intervention), a training-free method with three components: an Inconsistency Risk Predictor that gates intervention before generation, a Targeted Reward-Guided Decoding module that combines a positive profile reward with a negative commitment reward, and a Commitment Propagation Memory that tracks the agent’s prior claims across turns. We additionally introduce a Type-A/B/C violation taxonomy distinguishing profile violation, history contradiction, and commitment fabrication, used as a diagnostic for attributing where any method gains or loses. Extensive experiments show TaDI is competitive with the strongest always-on baseline while being substantially more efficient and more robust under adversarial interrogation.

Abstract:
As large language models (LLMs) are increasingly deployed in financial services, a single non-compliant interaction can expose institutions to regulatory penalties and direct consumer harm. Existing guard models are built around general harm taxonomies and overlook violations grounded in specific financial regulations. We address this gap with a regulation-driven pipeline that operates directly on regulatory documents, inducing a financial compliance risk taxonomy and synthesizing grounded training data without any predefined violation categories. Instantiating the pipeline on Chinese financial regulations, we release FinGuard-Bench , to our knowledge the first benchmark for financial regulatory compliance detection, with expert-annotated labels at both the query and response levels. We further train FinGuard , a financial compliance detection model built on Qwen3-8B and trained on the regulation-grounded data via supervised fine-tuning and self-play reinforcement learning. On FinGuard-Bench, FinGuard substantially outperforms all baselines, including dedicated guard models and much larger general-purpose LLMs such as Qwen3.5-397B-A17B and GPT-5.1. Furthermore, FinGuard also preserves general safety capabilities and adapts to unseen institution-specific policies using policy documents alone. We will publicly release the code, prompts, and resources used in this work on GitHub.

Abstract:
Long-video understanding remains challenging for Multimodal Large Language Models (MLLMs) due to limited context length. Uniform sampling may miss crucial moments, while agent-based frame video understanding methods often evaluate frames independently, overlooking the temporal organization of videos. Ideally, evidence selection should mimic how humans answer questions about long videos: first locating the relevant segment from the global context, then zooming into local objects, and details. We propose Temporal Tree of Thought ( T 3 ), a training-free framework for adaptive coarse-to-fine long-video understanding. T 3 constructs a question-agnostic hierarchical temporal tree via recursive temporally constrained clustering, where each node represents a contiguous segment with an informative key frame. During inference, T 3 performs an answer–retrieve–explore loop: it reasons over coarse representative frames, generates a search statement when evidence is insufficient, and expands relevant branches for finer-grained evidence. This process adaptively shifts the search target from temporal regions to specific objects, and visual details to help video understanding. Experiments on VideoMME, LongVideoBench, and LVBench show that T 3 improves Qwen2.5-VL-7B by 0.5%, 4.6%, and 4.4%, respectively, under the same frame budget, demonstrating the effectiveness of structured temporal reasoning. We will release code.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) and on-policy distillation (OPD) have emerged as two dominant methods for post-training reasoning LLMs. Prior work uses OPD’s dense token-level supervision to complement the sparse RL reward, fusing the two signals within a single step: either as a weighted-additive combination or a teacher-modulated rescaling of the RL advantage. In this paper, we show that a simple two-stage scheme, OPD-then-RL, consistently outperforms pure OPD, pure RLVR, and all such joint baselines across logic and math reasoning benchmarks. Beyond the empirical results, we further provide a systematic understanding of this through pass@ k behavior, learning dynamics, and parameter updates, yielding a consistent explanation: OPD expands the student’s coverage of teacher-supported solutions and RL sharpens within that support, while jointly optimizing the two signals causes them to interfere. To provide a practical recipe, we find that the OPD validation score is the key signal for when to switch to RL, and that OPD is a better cold start for RL than SFT. Together, our results establish OPD-then-RL as a simple yet strong way to combine the two methods, turning two entangled signals into complementary stages.

Abstract:
The autoregressive nature of large language models (LLMs) remains a significant bottleneck for inference, particularly in complex agentic workloads. While speculative decoding (SD) accelerates inference, current approaches rely on static drafting paradigms, utilising either autoregressive drafting models for reasoning or diffusion-based parallel models for structured outputs. We empirically find that drafting accuracy fluctuates dramatically within a single sequence, leaving significant performance unrealised by static paradigms and coarse-grained routing. To address this volatility, we introduce WhiFlash, the first cross-paradigm SD method that unifies autoregressive and diffusion-based parallel drafting under a single token-level controller. WhiFlash adopts a fine-grained routing mechanism that employs either a lightweight entropy-based or a learned neural policy, both parametrised to provide a tunable balance between expected token gain and latency. To make high-frequency switching computationally viable, we introduce novel cache-management optimisations, Lazy Catch-up and KV-only Prefill, reducing switching overhead to below 6% of per-round latency. By capitalising on the complementary strengths of fundamentally distinct drafting architectures, WhiFlash achieves significantly higher acceptance lengths, yielding category-specific throughput gains of up to 69.6% over the state-of-the-art autoregressive EAGLE-3 and 37.3% over the diffusion-based DFlash.

Abstract:
Efficient reasoning is critical for large reasoning models (LRMs) deployed in agent systems with strict context constraints. Existing methods predominantly rely on global length penalties that overlook step-level structure, inadvertently compressing critical derivations while retaining low-value detours. Hence, we argue for a principled, step-aware alternative that preserves critical steps and compresses low-value detours. However, it raises three fundamental challenges: the difficulty of obtaining step-level supervision, the need for reward signals that induce differentiated length pressure, and the requirement to balance training stability with exploration on complex problems. We propose SmartThinker, a step-level token allocation framework that learns to selectively preserve or compress individual reasoning steps. At its core is Step-Level Length Control Policy Optimization (SCPO), which provides label-free feedback by online estimation of each step’s contribution from the change in correct-answer likelihood after removing it. We further introduce reasoning density, defined as contribution per token, to relax length pressure on high-value steps while intensifying it on low-density ones. Finally, SCPO stabilizes training through length-bias-corrected advantages and difficulty-adaptive clipping, achieving superior accuracy-efficiency trade-offs across nine in-domain and out-of-domain benchmarks.

Abstract:
LLM-powered search agents enable multi-step reasoning and tool use. However, these capabilities introduce retrieval-induced safety degradation, as harmful intents may decompose into seemingly innocuous sub-queries that lead to unsafe outcomes. Existing alignment methods struggle to capture sparse safety signals and fail to supervise diverse violations across multi-step interactions. We propose COMPASS, a Cognitive MCTS-Guided Process Alignment framework designed to achieve robust safety alignment throughout the agent workflow while preserving general utility. COMPASS integrates cognitive tree exploration (CTE) to efficiently synthesize stealthy attack trajectories, and introspective step-wise alignment (ISA) to isolate risky intermediate actions for fine-grained process supervision. Empirical results show that COMPASS achieves a favorable safety-utility trade-off while requiring substantially less training data.The code is available on github .

Abstract:
Zero Pronouns (ZPs) are a pervasive linguistic phenomenon in pro-drop languages such as Chinese and have long posed a challenge for natural language processing systems. Although Large Language Models (LLMs) perform well on many Chinese language tasks, their ability to process ZPs remains poorly understood. We conduct a systematic investigation of LLMs’ handling of Chinese ZPs through a sequence of linguistically motivated tasks, including identification, referentiality classification, referential type classification, resolution, and translation. A diverse set of LLMs is evaluated across all tasks. Our results show that Chinese ZPs remain highly challenging for current LLMs, particularly for upstream tasks such as identification and referentiality classification. Performance on downstream tasks, such as ZP translation, is also consistently low: even state-of-the-art reasoning-oriented LLMs correctly translate fewer than half of Chinese ZPs into English.

Abstract:
Data-aware post-training quantization (PTQ) minimizes a per-token reconstruction loss on a small calibration corpus—an objective natural for average-token preservation, but one that implicitly weights positions by their empirical frequency. For Automatic Speech Recognition (ASR), this weighting is misaligned with tail-sensitive recognition risk: names, numerals, technical terms, and domain-specific words are practically important but receive proportionally little calibration mass. We propose Tail-Aware Reconstruction Quantization (TARQ), a label-free PTQ framework that shifts calibration toward the lexical tail. TARQ introduces rareBAL, a closed-form per-Linear-layer rule that equalizes the calibration mass contributed by common and tail positions, plus a metric-consistent residual correction that aligns the sequential PTQ sweep with the reweighted objective. TARQ requires no entity labels, no curated calibration set, no validation decoding, and no additional training. Across eight ASR backbones and eight datasets under 4-bit weight-only quantization, TARQ improves mean rare-WER, the Word Error Rate (WER) restricted to rare-word reference positions, while maintaining competitive aggregate WER. It achieves the lowest cross-corpus rare-WER swing, 0.63 percentage points (pp), among compared methods, and transfers without modification to entity-rich evaluation on ProfASR and ContextASR-Speech-En with no entity supervision.

Abstract:
Open-ended scientific discovery asks agents to move beyond executing analyses for predefined questions. Across multiple rounds of exploration, a discovery agent must decide which phenomena warrant investigation while avoiding overinterpretation, where emerging claims exceed the evidential scope of the analyses. This creates an evidence-calibration problem: the exploration trajectory must be coupled with claim status so that evidence can guide both what to investigate next and what can be claimed. We introduce StatefulDiscovery, a discovery framework that externalizes investigation state and uses it to coordinate frontier selection, evidence acquisition, and claim adjudication. We evaluate StatefulDiscovery across 40 real-data discovery tasks. Compared with several baselines, StatefulDiscovery produces more claims overall judged to be both well-supported and high-value. Ablations indicate distinct roles for structured hypotheses, local adjudication, frontier control and persistent states. Together, these results suggest that explicit discovery state can couple exploration with evidence-calibrated claim formation.

Abstract:
Theorem proving in real-world Lean 4 projects is challenging because proofs often depend on project-specific context. While iterative refinement can use compiler errors to repair failed proofs, reusing failed attempts requires careful search control: some proofs provide better starting points than others, and later revisions may degrade a partially correct proof. We propose a compiler-guided proof search framework that balances exploration and exploitation. It explores diverse starting points through dual-model generation and stagnation-triggered resampling, while exploiting promising proof states through current-best refinement guided by compiler-grounded pairwise comparison. Experiments on seven real-world Lean 4 projects from miniCTX-v2 show that our method achieves a better effectiveness–efficiency tradeoff than pass@ k baselines. Within the pass@32 budget, our method improves average pass rate by 12.8 percentage points while reducing LLM calls by 21.9%.

Abstract:
Quantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is rarely treated as a primary objective. We frame calibration-data selection for quantization as a target-dependent uncertainty-preservation problem. Different deployments emphasize different regions of the input distribution, yet prior work mainly optimizes accuracy-oriented compression metrics or adjusts scores after quantization. We formalize this goal with distributional and boundary preservation risks, and provide a simple mixture-mismatch argument explaining why no single calibration recipe should be expected to fit all targets. We introduce Doubt-Preserving Quantization (DPQ), a lightweight pre-quantization recipe family that uses full-precision predictions to construct target-aligned calibration mixtures of high-doubt examples and generic anchors. Across 8 language models, 9 NLP benchmarks, and 22 comparison methods, the leading recipe changes with the preservation target: DPQ-r75 leads on SQuAD2 answerability-boundary preservation, while broad multiple-choice QA favors a different set of milder or single-signal recipes, with confidence-only leading and variants such as entropy-only and DPQ-r50 outperforming the boundary-heavy DPQ-r75. These results show that calibration data should be selected for the specific full-precision score behavior a deployment needs to preserve, rather than treated as a fixed quantization detail. Code is available at https://github.com/xi-xiaoran/DPQ.

Abstract:
Long chain-of-thought (Long CoT) reasoning improves performance on multi-step problems, but it also induces overthinking: models often continue generating after a correct solution has already been reached, producing redundant verification, backtracking, and other low-yield reasoning that increase inference cost and latency. This inefficiency is especially problematic in low-data fine-tuning regimes. We propose STOP (Structured On-policy Pruning), a unified on-policy framework for analyzing and pruning long-form reasoning traces. STOP has three components. First, it constructs self-distilled successful traces from the student model, reducing distributional mismatch relative to teacher-guided supervision. Second, it maps each trace into a structured reasoning interface through node segmentation, taxonomy annotation, and reasoning-tree construction. Third, on top of this interface, we introduce ECN (Earliest Correct Node), which retains the shortest prefix ending at the earliest node that both functions as an answering conclusion and yields the correct final answer, removing redundant post-solution reasoning while preserving semantic continuity. Experiments on DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-LLaMA-3-8B across GSM8K, Math 500, and AIME 2024 show that STOP reduces generated tokens by 19.9–41.9% while largely preserving accuracy in low-data fine-tuning. Further analyses show that STOP does more than shorten traces: self-distilled ECN induces smaller distributional shift than teacher-guided pruning and reallocates reasoning effort away from redundant verification and backtracking toward more productive exploration.

Abstract:
Large vision-language models (LVLMs) remain prone to hallucination. While recent visual Direct Preference Optimization (DPO) methods try to fix this by contrasting positive images with heuristic or offline negatives for the same correct response. We show that such negatives are mostly "trivial": the model already assigns near-zero probability to the correct response, so the resulting gradient is uninformative or even spuriously misdirected. We propose OViP, an on-policy framework that synthesizes negative images on the fly from the model’s own hallucinated responses via a diffusion prior, producing semantically misaligned yet still high-likelihood negatives. Across five hallucination and four general benchmarks, OViP achieves better hallucination mitigation on LLaVA-1.5 series and Qwen2.5-VL-3B, while being 2 times more compute-efficient than SIMA and incurring no degradation on general capabilities.

Abstract:
Cantonese lyric writing requires close alignment between lexical tones and melodic pitch. Existing melody-guided lyric generation methods typically rely on symbolic melody to generate lyrics. However, in real songwriting scenarios, melodies are often expressed as raw singing audio or hummed recordings, where pitch is implicit, noisy, and unstructured, making these methods difficult to apply directly. To address this limitation, we propose ARIA, a two-stage audio-driven melody–tone relation modeling framework for Cantonese lyric authoring that generates Cantonese lyrics from singing recordings with provided character-level timestamps. Specifically, we first design a Tri-Stream Relation-Aware Tone Estimator (TRATE) to predict 0243 sequences from timestamped singing audio by modeling multi-stream acoustic cues and relational tonal structure. We then propose a Decoupled Retrieval-Augmented Tone-Conditioned Lyric Generator (DRA-TCLG) to generate fluent lyrics conditioned on predicted tonal plans with retrieval-enhanced lexical guidance. Moreover, we construct a large-scale aligned audio–Jyutping–0243 dataset from real Cantonese singing recordings to support this new task. Experimental results demonstrate that ARIA achieves strong performance in both 0243 prediction and tone-consistent lyric generation, validating the effectiveness of the proposed framework.

Abstract:
Decoder-only LLM rerankers struggle with long documents: inference is costly and relevance signals can be diluted by irrelevant context. Motivated by an attention analysis indicating a consistent degradation trend when non-relevant text is appended, we propose EviRerank, an evidence-based long-document reranking framework for decoder-only LLMs. EviRerank (i) scores document blocks with a lightweight selector (BM25, bi-encoder, or cross-encoder), (ii) constructs a compact reranking context under a hard token cap by dynamically budgeting evidence blocks with Adaptive Evidence Budgeting (AEB) and adding a global summary cue via Summary Augmentation (SA), and (iii) reranks with a decoder-only LLM. Across TREC DL’19, DL’22, DL’23, and MLDR-zh, EviRerank consistently outperforms full-document LLM reranking and strong block-selection baselines while substantially reducing the required input length. On TREC DL’19, EviRerank reaches up to 0.744 nDCG@10 while substantially reducing reranking latency. Additional RankZephyr-7B experiments confirm transfer to listwise reranking under matched context budgets.

Abstract:
Mechanistic interpretability has made substantial progress in automatically localizing circuits, but explaining what localized components do remains labor-intensive and difficult to standardize. In this work, we study whether language model (LM) agents can assist with this explanation problem once a circuit has already been identified. We introduce AgenticInterpBench, a benchmark for circuit explanation built from 84 semi-synthetic transformer circuits with 163 component-level annotations. We propose HyVE (Hypothesize, Validate, Explain), an agentic explainer that analyzes each component through an iterative loop of observation, hypothesis generation, and causal validation, eventually producing a component-level explanation and a circuit-level task description. Across four LM backbones, HyVE recovers useful component- and task-level explanations, but no backbone is uniformly best. Our analysis shows that strong backbones usually form observation-grounded hypotheses, while failures more often arise later in the validation loop, through incomplete validation plans, code execution errors, or unresolved hypotheses. A case study on an arithmetic circuit in Llama-3-8B shows that the same formulation can extend beyond semi-synthetic benchmarks to naturally trained models. Overall, LM agents are promising circuit explainers, but reliable validation remains the key obstacle.

Abstract:
Model-guided directed evolution seeks to identify high-fitness protein variants under limited oracle budgets. Protein language models (PLMs) provide rich representations for this task, but existing methods either rely on task-agnostic zero-shot scores or perform search in high-dimensional embedding spaces where surrogate modeling and uncertainty estimation can be sample-inefficient. We propose the Linear Fitness Subspace (LFS) hypothesis: mutation-induced fitness variation encoded by PLMs is concentrated in a low-dimensional linear subspace of residue-level representation changes. This view explains why zero-shot PLM scores are often useful but incomplete: they behave as fixed, task-agnostic projections, whereas few-shot supervision can recover assay-specific fitness directions. Building on this observation, we introduce Subspace-Guided Evolutionary Search (SGES), which learns the LFS from a small initial sample and performs surrogate modeling, uncertainty estimation, and acquisition directly in the learned subspace. Across 10 core ProteinGym assays and an extended set of 87 assays, SGES improves fitness prediction, budgeted search efficiency, computational cost, and biological interpretability over zero-shot PLMs and recent ML-guided protein optimization baselines. These results suggest that PLMs encode protein fitness in a geometry that is more linear, lower-dimensional, and more directly exploitable than previously assumed.

Abstract:
Suggestion mining has traditionally been formulated as binary classification, but the underlying linguistic phenomenon is not binary. A single review can express the same operational recommendation through a direct imperative (e.g., “add a vegetarian entrée”), an optative wish, an interrogative request, or a hedged conditional. These utterances share illocutionary force but differ substantially in pragmatic form, placing different demands on models: lexical cues may suffice for direct suggestions, whereas indirect forms require pragmatic interpretation. Existing benchmarks, typically spanning one to five domains and using a single binary label, do not measure these distinctions. We introduce PRAXIS, an English consumer-review benchmark that reframes suggestion mining as the measurement of pragmatic-form bias rather than as a leaderboard task. PRAXIS provides (i) a silver corpus of 174,860 reviews spanning 68 industries, labeled by Claude Opus 4.7; (ii) a 15,000-example human-labeled gold evaluation set, with annotation reliability estimated by a three-annotator panel on a 1,000-example subset (Tier 1 Fleiss’ κ = 0.77); and (iii) evaluations of four model families against the human gold, with bootstrap 95% confidence intervals. Agreement between Opus and human labels is reported separately as a corpus-validation statistic (Tier 1 Cohen’s κ = 0.71), rather than as inter-annotator agreement. Our central finding is structural rather than competitive. Instruction-tuned LLMs outperform encoders on binary macro-F1 (DeepSeek-V4-Pro: 0.882 vs. RoBERTa-large: 0.780), yet the ranking reverses for the Tier 1 conditional form: RoBERTa-large recalls 85.2% of conditional suggestions, compared with 46.7% for the strongest zero-shot LLM and 8.0–43.1% for the other six LLMs. This discrepancy reflects a form-classification gap rather than a detection gap: models often recognize these instances as suggestions at the binary level but misclassify their pragmatic form. A matched-supervision QLoRA model based on Qwen2.5-7B, trained on the same silver labels used for the encoders, recovers encoder-level conditional recall (0.904). PRAXIS therefore exposes a structural error mode that flat binary metrics obscure, positioning the benchmark as a measurement instrument for pragmatic-form bias rather than a conventional model leaderboard.

Abstract:
Large language models excel at general tasks but underperform smaller supervised models in specialized, high-stakes domains where training labels are costly. We address this regime with EvoPool, an evolutionary multi-agent framework inspired by Darwinian evolution. Three specialized agents iteratively propose executable annotator code, a small validation set provides a fitness signal, and a deterministic gate keeps only annotators that pass viability, diversity, and marginal-contribution checks across generations. Pool votes are mapped to soft training labels by EvoAgg, a text-aware aggregator combining semantic features with annotator-vote features. The authored pool runs at near-zero per-example cost and is 4500 to 31000× faster than LLM annotation on 100K examples. Across 7 out of 8 LLM-weak specialized and complex tasks spanning biomedical relation extraction, legal-clause classification, complex reasoning, and dense multi-label biomedical classification, EvoPool beats the strongest LLM annotation baseline by an average +0.132 macro-F1, peaking at +0.275 on ChemProt and +0.268 on PubMed. Code is available at: https://github.com/tianyi0216/EvoPool.

Abstract:
Speculative decoding accelerates autoregressive LLM generation with a lightweight draft model, yet existing distillation methods treat all historical tokens equally despite their divergent contributions to next-token prediction. We discover that attention in draft models is extremely concentrated: the top-3 historical tokens capture 80% of attention weight, while over 90% of tokens contribute negligibly. This uniform training paradigm forces the draft model to learn from noisy, low-importance tokens, diluting the learning signal from informative positions and limiting the alignment between draft and target distributions. To address this, we propose DarterSD, a dynamic token-weighted distillation framework for efficient speculative decoding. DarterSD employs a learnable sigmoid gate in self-attention to compute per-token importance scores, and designs a multi-objective distillation loss comprising distribution alignment, logit matching, and gate regularization to focus training on high-informative tokens. We further introduce a lightweight Boolean-mask denoising strategy at inference to suppress contextual noise, without modifying the target model’s verification pipeline. Experiments show DarterSD delivers a lossless speedup up to 5.52x , outperforming EAGLE3 (4.67x), and boosts the average accepted token length by 12.9%.

Abstract:
AI is rapidly advancing neuroscience, yet many laboratories fail to fully unleash its potential due to significant interdisciplinary barriers. While pre-trained neural models for physiological data are progressing quickly, their heterogeneous architectures and modality-specific constraints hinder systematic integration, selection, and evaluation. Despite recent advances in large language model (LLM)-based agent systems for intelligent scientific applications, existing approaches often still lack the domain expertise required to effectively select and coordinate diverse neuroscience pre-trained models and handle unique data types in this domain. We present NS-Copilot, an LLM-driven multi-agent system for neuroscience analysis that autonomously supports end-to-end workflows for diverse professional tasks. It unifies domain-specific pre-trained models and supports key neuroscience modalities, including EEG and extracellular spike data, through a natural-language interface. Given raw data and a task description, NS-Copilot orchestrates agents with specialized roles for planning, adaptive control, code generation, and result synthesis, enabling analysis without dataset-specific heuristics. We evaluate NS-Copilot on neuroscience benchmarks spanning Alzheimer’s disease, Parkinson’s disease, and working memory spike decoding. Across 8 trials per task, the system consistently outperforms strong baselines on the primary metric, demonstrating the ability of NS-Copilot for effective and scalable neuroscience analysis.

Abstract:
Retrieval-Augmented Generation (RAG) effectively mitigates hallucinations and knowledge obsolescence in Large Language Models (LLMs). However, prevailing RAG systems typically retrieve evidence from discrete chunks or through decoupled structures, making it difficult to jointly preserve cross-document navigability and within-document coherence on complex tasks. To address this challenge, we propose TSA-RAG (Tiered-Structure-Aware RAG), a unified framework that recasts retrieval as structure-preserving navigation over a Tiered Document Graph (TDG). The TDG offers a joint retrieval space that explicitly couples relational topology with hierarchical skeletons, enabling seamless traversal across granularities. Building on this representation, TSA-RAG navigates evidence through a progressive Expand-Route-Retrieve paradigm, which expands query-focused subgraphs under path-consistency constraints, routes the query to structural entry points within relevant documents, and retrieves subtree-structured evidence preserving discourse continuity. Extensive experiments on seven QA datasets demonstrate that TSA-RAG consistently outperforms strong baselines while yielding more compact retrieval contexts. Quantitative analyses further confirm the scalability and robustness of our framework. Code is available at https://github.com/ZKYQUQ/TSA-RAG.

Abstract:
Video misinformation increasingly operates at the semantic and evidential level: authentic footage may be selectively edited, temporally reordered, spliced across sources, or augmented with AI-generated content to construct false narratives. Such evidence-dependent manipulations cannot be reliably verified from the input video alone, because the missing, reordered, replaced, or recontextualized evidence lies outside the video itself. We introduce EVID-Bench, a benchmark for search-grounded video misinformation detection, where a system must search the open web for related videos and identify what information is false through cross-video comparison. EVID-Bench comprises 222 videos spanning 9 manipulation types across 3 categories: AI generation, single-source editing, and multi-source editing. All samples are verified to be undetectable by frontier models through visual inspection alone. We evaluate nine frontier multimodal models using a retrieval-augmented verification baseline. The best system achieves only 61.43% point-level accuracy and 43.24% video-level accuracy, with AI-generated manipulations remaining especially difficult. Error analysis reveals recurring challenges: models fixate on irrelevant anchors, misattribute synthetic content to editorial splicing, and terminate search prematurely before fully explaining the manipulation.

Abstract:
Function-calling benchmarks assess whether language models can select appropriate tools and supply the required arguments. Because their scores also depend on expressing calls in an evaluator-specific format, gains from prompt-prepended skills may reflect interface compliance rather than transferable task-solving procedures. We introduce a four-layer gain-attribution protocol for prompt-prepended skill injection, combining canonicalized rescoring, format-only controls, repaired/balanced induction, and portability checks. Applied to the Berkeley Function Calling Leaderboard (BFCL) and scoped with API-Bank, MATH-500, and MultiHop-RAG, the protocol shows that several apparent gains are better attributed to interface alignment than to procedural transfer: format-only prompts match or exceed full skills in key BFCL cells, repaired/balanced induction removes the largest sub-frontier gains, and API-Bank target-native gains are matched within 0.5 percentage points (pp) by length-matched generic procedural prompts. These findings treat format compliance as a useful engineering capability while clarifying what a structured-output score certifies. We release BFCL-CANONICAL and recommend canonicalized metrics, balanced induction, and format-only baselines for function-calling skill-gain attribution. Code is available at https://github.com/wanyichen06/skill-injection-attribution.

Abstract:
In e-commerce product review mining, Aspect-Based Sentiment Analysis (ABSA) aims to extract fine-grained opinions toward specific aspects in reviews. Existing ABSA benchmarks have expanded from aspect-level sentiment classification to opinion-aware tuple extraction. However, they still provide limited support for evaluating implicit aspects with fine-grained specificity. In particular, implicit aspects are often omitted, collapsed into placeholders, or mapped to predefined categories, which can obscure what reviewers actually evaluate. We introduce OASIS, a review-level Open-vocabulary benchmark for Aspect-Opinion-Sentiment (AOS) extraction with Implicit-aspect Specificity. OASIS contains 6,775 Amazon reviews across eight product domains and 21,054 annotated tuples, where implicit aspects are directly represented as contextually inferred noun phrases rather than closed-set labels. This design yields an implicit-aspect-heavy evaluation setting, with 70.10% of tuples involving implicit aspects. We evaluate a range of open-source and proprietary LLMs under fine-tuned and few-shot settings, showing that full AOS extraction remains challenging even for strong models. We further show that open-vocabulary aspect phrases in OASIS can provide task-useful information. Our benchmark and experimental code is available at: https://github.com/DILAB-HYU/OASIS.

Abstract:
Large Language Models (LLMs) have demonstrated strong reasoning capabilities, yet achieving nuanced personality alignment for simulating human conversation remains challenging. This difficulty stems from two bottlenecks: the scarcity of scalable, privacy-respecting training data and the limited ability of conventional reinforcement learning (RL) rewards to capture subtle personality-relevant signals. In this work, we introduce a unified framework to address these issues. First, we propose the Personality Alignment Choice Forge (PACF), an automatic pipeline that transforms existing conversational datasets into structured, multiple-choice benchmarks, constraining the model’s action space and enabling scalable evaluation grounded in observed human choices. Second, we present Group Sentence-Specified Sequence Policy Optimization (GSSPO), an RL algorithm designed to address the coarse credit assignment problem in methods that apply a uniform reward across all tokens. By integrating stable sequence-level updates with differentiated, rubric-guided token-level feedback, GSSPO assigns reward signals to more localized spans of the response. This targeted approach mitigates optimization instabilities caused by undifferentiated reward signals, leading to more effective training in our experiments. On the Qwen3 model family, GSSPO achieves the best personality-alignment accuracy on every benchmark and scale in the SFT+RL regime, and leads the other RL methods on five of six in the SFT-Free regime. These results suggest a scalable route toward training LLMs that better match measured personality-alignment preferences. Our core code is available at https://github.com/HKUST-FintechLab/GSSPO

Abstract:
Deploying Large Language Models (LLMs) for regulatory compliance demands rigorous traceability via comprehensive citations across multi-tiered authority structures. Unlike traditional multi-hop or legal QA, this task requires structured procedural lookups and evidence-set closure rather than entity resolution or case-law reasoning. Existing RAG systems struggle here due to flattened citation edges, fragmented retrieval expansions, and fragile post-hoc attribution. We formalize Regulatory Compliance QA with RegOps-Bench, a novel benchmark featuring an Operational Knowledge Graph derived from complex national R&D regulations. To address these bottlenecks, we propose RefWalk, a unified framework driven by a shared topic anchor. RefWalk traverses cross-document citations, fuses multi-view candidates via max-based aggregation, and enforces per-rule attribution to explicitly map claims to sources. We establish a strong baseline with substantial improvements in retrieval recall and citation accuracy. Finally, a contrastive evaluation on a U.S. health compliance dataset (HIPAA) reveals that existing systems exhibit saturation on flat-structure rules, underscoring the need for RegOps-Bench. Our code is available at https://github.com/yeongjoonJu/RefWalk.

Abstract:
Large language model (LLM) agents increasingly rely on external memory systems to remain consistent across long-horizon interactions, but little empirical work has been done to understand the specific failure modes and design choices that these systems present. Existing benchmarks report aggregate question-answering accuracy and treat memory systems as black boxes, making it impossible to attribute an incorrect answer to a particular failure mode of the system. We introduce MemFail, a diagnostic benchmark that isolates the failure modes of modern LLM memory systems. We begin by formalizing memory systems as the composition of three canonical operations—summarization, storage, and retrieval—and identify the potential failure modes induced by each. Based on these hypothesized failure modes, we construct five datasets spanning four tasks, each adversarially designed to test a specific operation of a memory system. Using these datasets, we evaluate four state-of-the-art memory systems on MemFail and demonstrate how MemFail can be used to empirically understand the tradeoffs induced by differences in memory system architectures.

Abstract:
Personalized tool selection for LLM-based agents aims to identify tools that align with both user intent and individual preferences, particularly in redundant API ecosystems where multiple tools offer similar functionality but differ in non-functional attributes. Existing generative approaches typically depend on fine-tuning large language models to infer user preferences, leading to substantial computational overhead and deployment costs. Despite these expensive training requirements, they still struggle to reliably distinguish functionally equivalent tools and often fail to capture users’ implicit non-functional preferences. To this end, we propose PrefTool, a lightweight yet effective preference-aware retrieval framework for personalized tool selection. PrefTool avoids costly generative preference modeling by employing a dual-encoder retrieval architecture that maps user context and tools into a shared representation space. Through structured tool representations, type-aware history modeling, and hard negative sampling, it efficiently captures users’ non-functional preferences while maintaining strong selection accuracy. Experiments on a personalized tool selection benchmark demonstrate the effectiveness of PrefTool.

Abstract:
Emotional intelligence (EI) in large language models (LLMs) is often evaluated through static understanding tasks or single-response dialogue generation. However, emotion management is interactive. A good response should improve the user’s emotional and relational state over several turns. We introduce EIBench, a simulator-based benchmark for interactive emotion management. EIBench contains 2,222 scenarios, with 2,009 for training and 213 for held-out testing. The scenarios are organized by a 2×2 taxonomy covering Support, Defense, Repair, and Charm. We evaluate 15 open- and closed-source LLMs. Current models perform well on support and rapport-building scenes, but struggle with boundary maintenance under user pressure. To improve the EI ability of LLMs, we propose Centered Turn-Credit GRPO (CTC-GRPO), a GRPO extension that reuses the simulator’s per-turn state updates as dense turn-level feedback while preserving the final outcome reward. CTC-GRPO improves Qwen3-8B from -22.4 to +22.4 on EIBench and also improves on out-of-distribution evaluations including SAGE (+12.4) and EQBench3 (+20.9%). Our results show that simulator-tracked user states can support both evaluation and training for multi-turn emotion management.

Abstract:
Current LLM evaluation relies on two complementary but often disconnected signals: static benchmarks with objective correctness labels and arena-style preference data that better reflect open-ended user interactions. We introduce DualEval, a latent model-item calibration framework that represents models and evaluation items in a shared space, jointly estimating model ability together with item difficulty and sharpness. We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries. Our evaluation uses 18 frontier LLMs, static benchmark labels, and reward-model scores validated against held-out human preferences for open-ended model responses. Empirically, our framework produces reliable and balanced model rankings, and its learned item-level profiles support downstream applications such as benchmark compression for sample-efficient evaluation and anomaly detection for contamination or outlier analysis. Overall, DualEval unifies static and arena-style evaluation through joint model-item calibration, producing model rankings and item-level diagnostics that support more sample-efficient, interpretable, and auditable evaluation pipelines.

Abstract:
Sentence simplification is a source-grounded rewriting task in which outputs often keep parts of the source while editing adjacent spans. Diffusion language models (DLMs) are well suited to this setting because masked denoising can fill rewritten spans using bidirectional context. However, parallel filling can create subword-boundary errors when incompatible subword fragments are committed together. We define these errors as Subword Boundary Conflicts (SBCs): malformed output words, such as Australianways, formed from incompatible fragments at subword boundaries. To measure them, we introduce a surface-form taxonomy and a detection pipeline based on candidate extraction, LLM-as-a-Judge filtering, and human verification. We evaluate Dream and LLaDA variants on ASSET and MED-EASi, with autoregressive and task-specific simplification baselines. We find that SBCs occur across DLMs but vary by model. Under standard unmasking, LLaDA variants remain at or below 2.80%, whereas Dream-7B-Instruct reaches 6.32% on ASSET and 9.03% on MED-EASi. Beyond model choice, decoding parallelism is a key driver. In the Dream unmasking setting at 128 denoising steps, increasing parallelism from 1 to 4 raises SBC rates from near zero to 25.63% on ASSET and 34.67% on MED-EASi. Further analyses show that SBCs occur under light-to-moderate edits, are not limited to rare words, and can often be repaired by local masked span regeneration. These findings establish SBCs as a word-form reliability target for robust diffusion-based text generation.

Abstract:
Retrieval-Augmented Generation (RAG) has shown strong effectiveness in grounding Large Language Models (LLMs) with external knowledge. However, existing RAG and Graph RAG frameworks largely treat knowledge as static or associate time with coarse-grained timestamps or metadata, failing to capture rich temporal structures such as duration, overlap, and containment. We propose IA-RAG, a hierarchical temporal RAG framework that models knowledge as time intervals and performs retrieval under formal temporal constraints. IA-RAG represents facts as Interval Event Units (IEUs) and organizes them into a hierarchical Thematic Forest, where temporal dependencies are governed by Allen’s Interval Algebra. To handle incomplete or uncertain temporal boundaries, IA-RAG further introduces a Sub-graph Time Tightening mechanism that refines fuzzy intervals through logical constraints within connected event subgraphs. In addition, IA-RAG supports implicit temporal semantic retrieval through interval-algebra-guided traversal. Experiments on multiple temporal question answering benchmarks, including TimeQA, TempReason, and ComplexTR, demonstrate that IA-RAG achieves strong temporal retrieval and reasoning performance, particularly on complex compositional temporal reasoning tasks. Our code is released at https://github.com/xiaoAugenstern/IA-RAG.

Abstract:
Large reasoning models achieve strong problem-solving performance, but often generate excessively long chain-of-thought outputs, which substantially increase inference cost and limit practical deployment. Existing reasoning compression methods reduce length only partially and often suffer from unstable optimization or accuracy degradation. We trace this limitation to an underlying trajectory-level objective conflict: when the compression objective and the accuracy objective are optimized jointly, compression favors short-and-correct trajectories while accuracy optimization continues to preserve long-but-correct ones that remain useful on hard problems. This conflict creates opposing gradients on the same behaviors and can trap training in a local trade-off where further compression harms accuracy while accuracy recovery reintroduces verbose reasoning. At the model-decision level, we refer to the resulting failure mode as entropy conflict: the policy fails to fully collapse onto concise trajectories because accuracy-oriented updates keep restoring broader support over long-correct reasoning paths. To address this issue, we propose an entropy-guided training framework. Instead of forcing a mixed objective to follow an unstable path toward the final efficient-output set, we first contract the policy to a short-output set and then expand from this compact regime to recover accuracy and robustness. Experiments on multiple mathematical reasoning benchmarks show that our method achieves performance comparable to the original model using only about 25% of its reasoning tokens.

Abstract:
Long-term memory is a core capability for personalized LLM agents. To support it, existing memory systems organize information using various criteria such as topic segments or summary hierarchies. However, we identify two major limitations in these designs. First, they lack scene awareness: memories from unrelated life scenes share the same retrieval space, which inflates the search space and introduces cross-scene interference. Second, they encode each memory from a single perspective, making it difficult to retrieve complementary views of the same event. In this paper, we propose the CreaMem architecture, which enables scene-aware memory organization by partitioning memory into several Life Scene Memories to reduce cross-scene interference at retrieval. Within each memory, entries are dual-coded from both episodic and trait-based perspectives. To go beyond the single perspective and achieve cross-memory synergy, we further devise a per-memory balanced sampling strategy at retrieval time. Extensive experiments on two long-term memory benchmarks show that CreaMem improves QA accuracy across all evaluation metrics, with particularly large gains on multi-hop reasoning performance, validating scene-aware partitioning and cross-memory synergy.

Abstract:
Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can exhibit mixed authenticity, where genuine and manipulated cues coexist across temporal transitions, overlapping sources, or both. This setting requires not only detecting manipulated audio but also localizing the components that provide evidence for the decision. We propose ToolDF, a tool-integrated reasoning framework for mixed-authenticity audio deepfake detection. ToolDF employs an audio large language model as an orchestrator trained with supervised tool-use trajectories. It adaptively analyzes the audio scene, selectively performs source separation, routes components to domain-specific experts, and aggregates their evidence into an interpretable verdict. We further introduce a mixed-authenticity ADD benchmark covering temporal transitions, acoustic overlaps, and hybrid mixtures. Experimental results show that ToolDF achieves the best overall performance on composite-type detection, achieving macro-F1 gains of 3.72 and 14.39 points over the strongest monolithic baseline and a fixed pipeline, respectively, while providing interpretable evidence localized to temporal regions and acoustic sources. Our source code and dataset are publicly available online at https://github.com/rlataewoo/tooldf.

Abstract:
Recent advances in Large Language Models (LLMs) have stimulated increasing interest in LLM-based agents for long-horizon interactive decision-making. As a lightweight alternative to parameter-level optimization, many studies enhance LLM agents through retrievable experience memory. However, existing retrieval strategies primarily optimize for the relevance of historical memories, leaving the applicability conditions and verifiable execution effects of retrieved trajectories insufficiently captured. To mitigate this executability gap between relevance-driven retrieval and executable reuse, we propose Effectiveness-Driven Memory (EffectMem), a framework that reframes trajectory memory as executable guidance whose applicability and effects are checked during interaction. During memory construction, EffectMem structures successful trajectories into positive executable contracts and localizable failures into negative experience guards. At inference, EffectMem retrieves candidate contracts, verifies their applicability to the current state, executes aligned chunks under guard constraints, and invokes discrepancy-guided correction when observed effects deviate from expected postconditions. Experiments on ALFWorld, ScienceWorld, and WebShop demonstrate that EffectMem outperforms competitive baselines and improves the effectiveness of memory-augmented LLM agents.

Abstract:
Long-horizon LLM agents execute sequences of reasoning, tool-use, and environment-interaction steps, making end-to-end latency a central deployment bottleneck. Speculative agents reduce this latency by leveraging a faster draft agent to propose the action trajectory while a stronger target agent verifies each action to ensure lossless action trajectories. Recent systems adapt how many future actions a preselected draft agent proposes before verification, but leave the drafter itself unchanged across tasks and episodes. Our experiments show that the best drafter varies across tasks and trajectory prefixes, revealing substantial latency headroom from adaptive drafter selection. This motivates the question: which draft agent should speculate for a given task and target agent to achieve low end-to-end latency? We propose AdaSpec, an adaptive routing framework for speculative agents with an objective distinct from standard model routing: selecting the drafter that minimizes realized speculative-execution latency for a target agent, rather than selecting the model that produces the final answer. AdaSpec profiles a pool of candidate draft-agent configurations for each target agent and trains a lightweight, latency-supervised router to rank them by measured speculative latency. The router supports both query-level routing and episode-level rerouting before each draft-and-check segment, while the target agent remains responsible for verifying actions. AdaSpec improves fixed-drafter baselines across multiple datasets and target-agent settings.

Abstract:
Confidence-based voting aggregates parallel LLM rollouts by weighting each with internal signals such as token log probabilities, and has been actively studied for single-turn reasoning. However, modern LLMs increasingly act as multi-turn search agents that retrieve and condition on external documents. In this paper, we show that confidence-based voting transfers poorly to this multi-turn setting, and identify the underlying failure reason as copy inflation: when retrieved documents are appended to an agent’s context, tokens copied from those documents receive systematically inflated log probabilities. This flattens confidence scores within each question and weakens the resulting weighted vote. To address this issue, we propose Retrieval-Grounded Voting (RGV), which scores each rollout by the lexical overlap between its final answer and the documents it retrieved. By computing the signal outside the contaminated context, RGV sidesteps both token log probabilities and additional LLM calls. Across four search-agent benchmarks and five LLMs, RGV consistently outperforms confidence-based voting, with gains of up to +5.4% accuracy and +35% on minority-correct questions, where the correct answer appears in only 1-2 of 8 rollouts.

Abstract:
LLM-based recommendation agents personalize what they know through evolving per-user semantic memory, yet how they perform reasoning relies on a universal, static system prompt shared across all users. When a recommendation fails, the agent only updates its memory, but never examines or revises the reasoning that leads to the failure. To close this gap, we propose SAGER (Self-Evolving Agent for Personalized Recommendation), the first framework to equip each user with a dedicated, continuously evolving policy skill. SAGER incorporates three tailored mechanisms for recommendation scenarios: (i) a two-representation skill architecture constrained by a Cognitive Injection Boundary, decoupling a rich evolution substrate from a minimal inference-time injection; (ii) a contrastive differential chain-of-thought engine that diagnoses ranking errors against unchosen near-misses under a refine-not-replace principle; and (iii) skill-augmented listwise reasoning that mitigates the Score-Tie Bottleneck, enabling the slim skill to serve as an active tie-breaker. Evaluated on four public benchmarks, SAGER achieves state-of-the-art accuracy. Furthermore, plugging its policy skill layer into three structurally distinct host agents delivers consistent 9%–12% gains on the H@1 metric. These results indicate that policy skill evolution provides architecture-agnostic improvements that are largely complementary to memory accumulation.

Abstract:
Cloud-edge collaboration provides a practical paradigm for deploying MLLMs on resource-constrained devices, where compact edge-side models are improved with supervision from stronger cloud-side models. However, existing cloud-edge distillation methods mainly rely on pseudo labels or static feature alignment, overlooking how the teacher progressively transforms multimodal inputs into answers. In this paper, we propose A 2 -MED, Adaptive Anchor-Guided Multimodal Evolution Distillation for cloud-edge MLLMs. Our key idea is to organize the teacher’s layer-wise multimodal evolution into functional stages through adaptive teacher-side anchoring, and transfer the resulting stage-wise evolution as process-level supervision. Specifically, we adaptively identify functional anchors from the teacher according to layer-wise indicators of cross-modal binding, semantic integration, and answer prediction, and then project these anchors to the student to establish corresponding functional stages. Under these teacher-defined stages, we align teacher-student anchor states and distill inter-stage multimodal evolution relations, enabling the student to learn how the teacher progressively transforms multimodal inputs into answers. Experiments on A-OKVQA and VizWiz show that our method consistently improves over representative cloud-edge and distillation baselines, with further analyses validating the effectiveness of adaptive anchors and evolution distillation.

Abstract:
The risk of an agent skill call depends not on the skill alone but on the invocation context: the same file-reading skill is benign when listing logs, yet dangerous when tainted content steers the agent toward secret exfiltration. We formalize invocation-time auditing: intercept a proposed skill call before execution and score its risk from the user request, the candidate skill, and the runtime context. STARS implements this as a three-stage pipeline that fuses a static capability prior with a request-conditioned invocation scorer. The primary configuration, STARS-LLM, uses an LLM judge; STARS-Rules is a deterministic, API-free alternative. On SIA-Bench, a 3,000-record synthetic benchmark with a 180-record human check at Fleiss κ = 0.709, request-conditioned scoring improves high-risk retrieval on held-out indirect prompt injection, where static metadata cannot see the attack. On a frozen 500/100 scoring subset, STARS-LLM raises rules-only HR-AUPRC from 0.297 to 0.728 in-distribution and from 0.504 to 0.757 on held-out IPI. That gain is in ranking: under frozen thresholds, held-out classification accuracy does not improve. SIA-Bench is a ranking stress test, not an estimate of real-world prevalence.

Abstract:
LLM-based multi-agent systems have demonstrated strong potential for solving complex tasks, yet reliable collaboration remains challenging. While most training-based methods rely on multi-agent trajectories, we investigate whether cooperative behaviors can be strengthened using only single-agent execution data. We propose CoopAgent, a training framework that learns solely from single-agent execution trajectories and transfers to multi-agent deployment. To bridge the gap between single-agent training and multi-agent environments, we introduce PADPO (Perturbation-Aligned DPO), which aligns action preferences between original observations and task-related perturbations that mimic changes caused by other agents. We train CoopAgent on single-agent execution trajectories collected in Minecraft and evaluate it on VillagerBench. The results demonstrate its potential to enhance collaboration on VillagerBench while preserving strong single-agent execution ability. We further assess CoopAgent on out-of-domain benchmarks, showing improvements in several multi-agent settings and performance comparable to the multi-agent training method AT-GRPO on the coding benchmarks. Our code is available at https://github.com/hlpzz/CoopAgent.

Abstract:
Personalization has attracted growing interest in LLM applications, yet existing retrieval-based approaches depend heavily on retrieval quality and degrade in long-term interactions. Model editing, which directly modifies internal model parameters to incorporate new knowledge, has demonstrated effective knowledge modification capabilities in factual knowledge editing tasks and may provide a potential solution for personalization. However, scaling model editing to personalization is non-trivial. Editing large amounts of user data increases computational cost and causes interference among edits, motivating the need for effective sample selection. To address this issue, we propose, PersonaEdit, a hidden representation clustering strategy that selects representative editing samples through proportional stratified sampling. Experiments show that model editing is effective for personalization, and that our selection strategy preserves most of the performance while substantially reducing the number of required editing samples. Beyond standalone editing, we find that combining model editing with retrieval-based prompt augmentation further improves personalization, as edited knowledge and retrieved context provide complementary information. These results demonstrate the potential of model editing as an efficient and scalable approach for LLM personalization.

Abstract:
Graph neural networks (GNNs) have demon- strated success in modeling relational data pri- marily under the assumption of homophily. However, many real-world graphs exhibit het- erophily, where linked nodes belong to dif- ferent categories or possess diverse attributes, such as webpages, Wikipedia articles, social networks, and e-commerce platforms. Addi- tionally, nodes in many domains are associated with textual descriptions, forming heterophilic text-attributed graphs (TAGs). Despite their sig- nificance, heterophilic TAGs remain underex- plored due to the lack of dedicated benchmarks that jointly capture heterophilic structures and rich textual attributes. To address this gap, we introduce the Heterophilic Text-attributed Graph Benchmark (HeTGB), a novel bench- mark comprising five real-world heterophilic graph datasets from diverse domains, with nodes enriched by extensive textual descrip- tions. HeTGB enables systematic evaluation of GNNs, pre-trained language models (PLMs) and co-training methods on the node classifi- cation task. Through extensive benchmarking experiments, we showcase the utility of text attributes in heterophilic graphs, analyze the challenges posed by heterophilic TAGs and the limitations of existing models, and provide in- sights into the interplay between graph struc- tures and textual attributes.

Abstract:
Multimodal memory offers a scalable interface for long-video question answering, but existing methods often retrieve captions, frames, transcripts, summaries, or graph facts as isolated fragments. Although searchable, such fragments are not generation-ready: language models must reconstruct cross-modal and temporal alignments at inference time, when context is limited and attribution is difficult. We propose EM 2 Mem, an event-centric multimodal memory framework that binds heterogeneous evidence to event anchors during memory construction. Each event-indexed memory cell aligns multimodal records, temporal context, graph-linked relations, semantic facts, and provenance, enabling compact evidence readout over grounded multimodal events rather than modality-specific fragments. Across three long-video QA benchmarks, EM 2 Mem improves average accuracy over the strongest memory baseline by 2.0, 2.4, and 3.7 points, improves strict event-level Top-5 evidence recall by 7.0 points, and reduces per-query latency by 4.67× and total inference tokens by 63.66%.

Abstract:
Attack success rate (ASR) is insufficient for evaluating the safety of large language models (LLMs). Many defenses rely on surface-level refusal behavior, which can over-refuse benign-sensitive prompts and remain brittle: once the initial refusal boundary is bypassed, the model may still follow a harmful continuation trajectory. We analyze model activation spaces to examine the limits of static interventions, finding substantial overlap among useful, benign-sensitive, and harmful representations. This suggests that a single global intervention may suppress harmful outputs only by perturbing general utility. We propose IRIS, Iterative Refinement of Internal States, a routing-conditioned internal correction method. To address this, IRIS leaves low-risk states unaltered while routing high-risk states to a shared module for multi-step latent refinement. To make the refinement module learn behavior-level correction, we introduce model-owned base-source prefix recovery, which trains IRIS to recover from base-model failure prefixes toward safe continuations. Extensive evaluations on Llama-3.1-8B and Qwen-2.5-7B demonstrate that IRIS drastically reduces ASR from around 70% to around 10% and successfully defends against prefilling attacks. Furthermore, IRIS preserves general benchmark utility and maintains near-zero over-refusal rates.

Abstract:
Large Language Models (LLMs) often generate natural-language comments while writing code, and these comments become part of the context used to generate the code that follows. However, it remains unclear which properties of comments affect code-generation performance. We study this question through observational analyses and controlled interventions. On LiveCodeBench, neither comment frequency nor broad comment intent reliably predicts pass@1. We then prefill weaker recipient models with comment blocks written by stronger source models, allowing us to separate comment surface form from the solution content they convey. Comments from source solutions that pass the tests raise recipient pass@1 by 17.2% on average. In contrast, comments describing failed solutions provide no reliable gain, while comments written for a different problem reduce pass@1 by 20.8%. Finally, across a wide range of models and prompt variants, most recipient models show no significant recovery of the external-comment gain, and the best case recovers only 24%. These results show that comments help code generation not merely because they are comments, but because they can provide correct solution content that prompting cannot reliably elicit.

Abstract:
Parameter-efficient fine-tuning (PEFT) is often assumed to preserve pretrained capabilities because it updates only a small number of parameters. We show that this assumption depends strongly on adapter capacity. We study factual acquisition in a controlled OpenStreetMap-derived benchmark where Qwen3-4B must learn anonymized geographic relations while retaining unrelated capabilities. Comparing FFT with QLoRA ranks 8, 16, 32, and 64, we find that rank induces a clear acquisition–retention frontier. Low-rank QLoRA preserves OOD performance but acquires fewer facts, whereas higher ranks improve same-fact paraphrase generalization at increasing cost to unrelated benchmarks. FFT behaves as a conservative baseline: it retains general capabilities well, but does not reach the highest factual-acquisition regime. Distributional, weight-space, and spectral diagnostics mirror this behavioral trade-off, with higher-rank QLoRA moving farther from the pretrained model. A separate math adaptation experiment shows a weaker frontier, suggesting that the effect is most pronounced when adaptation must install new factual associations rather than reinforce skills already supported by pretraining.

Abstract:
Instruction embedding models have become common among state-of-the-art models, however are evaluated using a single prompt per task. The single-point evaluation ignores a main problem of the instruction-based approach namely: sensitivity to the phrasing of the instruction. We present an empirical study of prompt sensitivity across 6 embedding models and 11 datasets. We show that reported scores misrepresent the distribution of scores over plausible prompts. The default prompt can both systematically understate or overstate performance. Furthermore, we show that the leaderboard ranking is not robust to prompt selection: a developer improve their rank by favorably selecting prompts, and under adversarial prompt selection any model can be promoted to first place. Our findings suggest that single-prompt evaluation is insufficient for instruction-tuned embedding models and that benchmarks should incorporate prompt robustness, either by evaluating over multiple prompts or by reporting sensitivity alongside point estimates.

Abstract:
Group Relative Policy Optimization (GRPO) improves LLM reasoning, but LoRA-based GRPO can underperform when the initialized adapter subspace is poorly aligned with reward-induced gradients. We propose GRAIL, Gradient-Reward Aligned Initialization for LoRA-based GRPO, which estimates reward-aligned gradients from grouped rollouts before policy optimization and uses them to construct LoRA factors. The full GRAIL configuration includes a reference-solution fallback only for all-zero-correctness groups during calibration; after initialization, all models use the standard GRPO objective without extra trainable parameters or loss changes. We also study Degenerate-Group Reflection Resampling (DGRR) as an optional extension that resamples all-zero-correctness groups after self-reflection without exposing reference solutions. Across Qwen2.5, Phi-3.5, and Gemma-2 backbones on GSM8K and MATH500, GRAIL consistently improves LoRA-based GRPO, with particularly strong gains on smaller Qwen and Gemma models. Analyses attribute these gains to stronger GRPO-gradient capture, smaller principal angles, and better handling of correctness-degenerate groups.

Abstract:
In the RAG paradigm, document ranking determines the evidence available to downstream generators. Through controlled analysis, we identify two phenomena underexplored by existing rankers: (i) downstream response quality depends not only on relevance but also on the composition and ordering of selected documents, and (ii) such preferences differ systematically across generators. However, existing rankers are trained purely on query–document relevance, leaving both phenomena unmodeled. To close this gap, we construct PRISM, a bilingual preference-aligned dataset built through a four-stage pipeline that compresses the combinatorial subset-and-ordering space by roughly four orders of magnitude and produces response-quality preference supervision conditioned on seven downstream generators. On a 13k-query subset of PRISM, we train Rank4Gen, a generator-aware ranker that performs joint document set selection and ordering. Experiments on five challenging RAG benchmarks show that Rank4Gen improves downstream QA quality on most evaluated generators, with per-generator F1 gains of up to +2.08 over the strongest set-selection baseline. Code is available at https://github.com/JOHNNY-fans/Rank4Gen.

Abstract:
Recent advances in large reasoning models (LRMs) have shown strong performance on complex problems through long chain-of-thought (Long CoT) reasoning. However, distilling such trajectories into smaller student models remains challenging: direct Long CoT supervision often provides limited gains and can be less effective than concise Short CoT rationales. In this work, we investigate this phenomenon from a gradient-centric perspective. Our analysis shows that Long CoT induces larger gradient magnitudes and more concentrated update directions than Short CoT, with this effect becoming more pronounced as student model capacity increases. These findings suggest that effective Long CoT distillation requires balancing the reasoning information density of reasoning trajectories with their distributional alignment to the student model. Motivated by this insight, we propose M odel I nterporlation Distillation ( MI-Distillation ), a framework that constructs a continuous Instruct-Reasoning data spectrum through model interpolation. To select suitable trajectories from this spectrum, we further introduce Seq uential L earnable S urprisal S core ( SeqLSS ), which favors reasoning paths that are both informative and learnable for the student. Extensive experiments on reasoning benchmarks show that MI-Distillation consistently improves small model CoT distillation over strong Long CoT baselines.

Abstract:
Argument Mining (AM) is fundamentally constrained by the scarcity of high-quality structure-annotated datasets. While LLMs have shown promise in synthetic data generation, producing synthetic AM data that is both structurally accurate and sufficiently diverse remains a challenging problem. To address this problem, we revisit synthetic data generation for AM from a new perspective and propose a novel Adversarial Reinforcement Learning framework for Data Synthesis. The proposed framework jointly optimizes the generator and the discriminator in an adversarial loop, in which the generator produces structured AM instances, and the discriminator provides learning signals by distinguishing real data from synthetic candidates. This enables the generator to progressively improve both the structural accuracy of generated argument data while maintaining diversity through adversarial feedback. Extensive experiments demonstrate that the proposed framework consistently improves AM performance on three benchmark datasets in both full-data and low-resource settings, validating its effectiveness and scalability.

Abstract:
Question answering systems are usually evaluated for factual correctness, but many real world settings such as education and career guidance require mentorship responses that offer reflection, context, and guidance. We introduce MENTORQA, the first multilingual dataset and evaluation framework for mentorship focused question answering from long form videos, with nearly 9,000 QA pairs from 180 hours of content across four languages. We define evaluation dimensions that measure fluency, clarity, alignment, and mentorship value, and use them to compare Single Agent, Dual Agent, RAG, and Multi Agent QA pipelines under controlled conditions. Multi Agent pipelines produce the strongest mentorship responses, with larger gains for complex topics and lower resource languages. We also study LLM based evaluation and find substantial variation in alignment with human judgments. Overall, MENTORQA establishes mentorship focused QA as a distinct research problem and provides a benchmark for studying agentic architectures and evaluation design in educational AI. The dataset and evaluation framework are released at https://aim-scu.github.io/MentorQA/.

Abstract:
Mobile AI acts as a visual oracle, empowering users to snap a picture of something and ask for information. Snap-and-ask retrieval is now one of the most common entry points for mobile AI, yet photos are often blurry, while text questions may be short or mistyped. Existing benchmarks only test on clean inputs or do not isolate paired robustness in snap-and-ask retrieval. Therefore, we introduce SnapBench, the first paired benchmark for robust snap-and-ask multimodal retrieval, spanning 1,145 queries, 9,085 gallery items under 53 controlled corruption conditions with human annotations. We evaluate 16 multimodal retrievers, covering dual-tower encoders and embedding-based VLMs. Results show that image corruptions substantially degrade retrieval, while text corruptions mainly affect text-only retrieval and have limited impact on joint retrieval. Clean image-only retrieval often outperforms joint retrieval, indicating the coarse-text drag and the lack of cross-modal fallback under noisy inputs. SnapBench provides a controlled testbed for evaluating robust retrieval in snap-and-ask scenarios. We further propose MOOR (Modality-anchored, Outlier-aware, Optimal Reweighting), a simple adaptive fusion approach, highlighting the need for reliability-aware modality calibration in snap-and-ask retrieval. The code and dataset are available at https://github.com/zrchen03/SnapBench.

Abstract:
This work presents Dual-Embedding Watermarking (DEW), a semantic watermarking scheme for large language models (LLMs) that leverages contextual and token-level embeddings to enhance robustness against paraphrasing and translation. DEW utilizes a signal-processing methodology, applying algebraic vector-space operations to token and context embeddings to derive a watermark signal that degrades gracefully under semantic shifts. The method obfuscates the watermark by projecting embedding vectors through pseudo-random matrices seeded with a secret key. Experimental results show that dual-embedding watermarking can offer state-of-the-art robustness, particularly against translation, while incurring relatively low computational overhead compared with other semantic schemes. At lower watermark strength, DEW also maintains competitive text quality, suggesting that dual-embedding signals provide a promising substrate for robust semantic watermarking.

Abstract:
Chain-of-thought (CoT) distillation is a common strategy for transferring reasoning abilities from large teacher models to smaller student models, but in medical question answering, a student may produce plausible rationales while locally drifting toward clinically incorrect reasoning paths. This makes final-answer supervision and uniform rationale imitation insufficient for identifying where reasoning fails. We study this problem as local reasoning preference collapse in structured medical CoT distillation and propose Answer-Calibrated Collapse-Aware Preference Distillation (ACPD). ACPD first exposes the student’s local continuation preferences through teacher-guided verbalized candidate construction and measures domain-level collapse with a Local Collapse Margin (LCM). It then selectively repairs collapse-prone domains by contrasting the student’s most preferred teacher-approved and teacher-disapproved continuations. To ensure that local repair improves decision making rather than only rationale form, ACPD further introduces answer calibration, which anchors the full structured reasoning prefix to the correct final choice. Experiments on medical multiple-choice reasoning benchmarks show that ACPD improves both answer accuracy and collapse-sensitive repair metrics over strong distillation baselines while maintaining competitive out-of-distribution generalization. These results suggest that effective medical CoT distillation should not only transfer teacher rationales, but also diagnose and repair where the student’s reasoning preferences locally diverge from medically reliable reasoning paths.

Abstract:
Implicit discourse relation recognition (abbr., IDRR) is a task of automatically classifying semantic relations between pairwise arguments in the absence of explicit connectives. IDRR suffers from imbalanced data distribution of different relations. The current data augmentation approaches have relieved this problem to some extent. However, the incorporated discourse samples during augmentation lack customized contrastive effects. To address this issue, we propose to construct topic-relevant relation-inconsistent arguments for augmentation. Specifically, we implement a Relation-Variant Discourse Recasting (RVDR) approach. RVDR utilizes Qwen3-4B to paraphrase discourses under the forcible constraints of preserving inherent topics and transforming relation senses toward infrequent target relations. To avoid excessive divergent thinking of RVDR, we optimize the Qwen3-4B via reinforcement learning, where Group Relative Policy Optimization (GRPO) is used along with multiple task-specific rewards. Furthermore, rule-based filtering and confidence estimation are used to ensure data quality, which additionally enable two application modes, i.e., Confidence-based and Progressive Data Augmentation (i.e., CDA and PDA). Experiments on PDTB 2.0 and 3.0 show that RVDR-based CDA consistently improves IDRR performance across multiple backbones, including both encoder-only and decoder-only cases. It also allows the strong IDRR frameworks CP-KD and SCDN to be enhanced further. The code is publicly available.

Abstract:
Mixture-of-Experts (MoE) architectures enhance the scalability and efficiency of large language models (LLMs) by activating only a subset of parameters. However, the interpretability of individual experts and corresponding strategies for post-training adaptation to domain-specific tasks remain underexplored. In this work, we develop an interpretability framework for expert-level monosemanticity in MoE models, drawing inspiration from sparse autoencoders and offering new insights into the specialization patterns of domain experts. We demonstrate that the inherent sparsity of MoE models encourages stronger monosemantic behavior at the expert level, which allows for the identification of experts responsible for particular downstream tasks and aids in preserving cross-domain performance. Building on this, we propose an expert-frozen fine-tuning method that selectively updates domain-specific experts while freezing the rest, which also alleviates catastrophic forgetting. Experiments on specialized domains, including medicine and finance, show that our approach performs on par with or better than fully fine-tuned models on in-domain tasks, while achieving relative improvements of 19.58% and 37.09% in retaining performance on out-of-domain benchmarks. We also demonstrate consistent advantages over PEFT baselines such as LoRA, achieving relative improvements of 11.49% and 8.60% in other domains.

Abstract:
Agent skills are procedural artifacts that enable LLM agents to execute workflows, verify constraints, and recover from failures. Existing self-evolving methods refine skills using accumulated trajectories. However, they struggle in cold-start settings, where only an initial, imperfect skill is available. Consequently, skill construction defaults to expert authoring or one-shot LLM generation. Expert-authored skills are costly and may not align with how LLM agents actually execute tasks, while one-shot generated skills can be syntactically well formed yet behaviorally weak. To bridge this gap, we propose SkillRevise, an execution-grounded framework designed to iteratively refine these initial skills. SkillRevise diagnoses skill defects from execution evidence, retrieves relevant repair principles from a general memory, and applies execution-anchored edits. By re-executing candidates and measuring empirical utility, it retains the best observed skill within the revision budget. Evaluated across three main benchmarks, two domain-specific studies, and six LLMs, SkillRevise substantially outperforms one-shot baselines—improving the base agent’s success rate on SkillsBench from 36.05% to 61.63%. Furthermore, the revised skills transfer across both executors and task environments, suggesting that SkillRevise captures reusable procedural knowledge beyond any single executor. Our code is available at https://github.com/HKUST-KnowComp/skillrevise.

Abstract:
Diffusion vision-language models generate answers through iterative refinement, exposing intermediate answer trajectories that can be inspected and controlled at inference time. However, this controllability creates a reasoning-need mismatch, where a universal generation length is applied to questions with different reasoning demands. Visually closed questions may be harmed by continued refinement after a stable answer has formed, whereas reasoning-sensitive questions may be harmed by premature commitment. We formulate this problem as reasoning-budget mismatch and study it in LLaDA-V. Rather than choosing a universal generation length, our training-free controller routes each example to early commitment, baseline preservation, or reasoning-supportive decoding using trajectory signals from answer closure, commitment evidence, and representation revision pressure, without using ground-truth answers. Across answer-focused, mixed-reasoning, and CoT-sensitive benchmarks, routed control improves robustness over fixed long decoding, pure short decoding, and single-rule interventions. The gains are not explained by shorter outputs alone. Answer-closed examples often benefit from commitment, whereas CoT-sensitive examples require preserving or supporting intermediate reasoning. Taken together, these results suggest diffusion VLM decoding should route inference-time control by the state suggested by the observed trajectory instead of relying on a universal decoding length.

Abstract:
Even when explicit Personally Identifiable Information (PII) is removed from LLM queries, implicit contextual cues can still reveal sensitive user attributes. Yet current Privacy-Conscious Delegation frameworks overlook this threat entirely, and existing PII span-based evaluation metrics cannot address it. We address this gap on both fronts. First, we introduce Attribute Inference Leakage (AIL), a label-free metric that quantifies implicit privacy leakage through set-based semantic comparison of attributes inferred before and after sanitisation. Using AIL, we provide the first empirical evidence in Privacy-Conscious Delegation that even perfect PII removal eliminates only 18% of implicit leakage relative to direct delegation, leaking the vast majority of inferable attributes. Second, we propose GIST, a training-free approach for context-aware generalisation to protect both explicit and implicit privacy while preserving response quality. Experiments across six local models (1B to 8B parameters) show GIST consistently achieves a Pareto improvement over state-of-the-art PAPILLON on all three evaluation dimensions of quality, explicit and implicit leakage. GIST also consistently reduces implicit leakage below the explicit leakage lower bound (perfect PII removal), hence GIST addresses a privacy dimension that PII-centric methods cannot.

Abstract:
Effective in-context learning (ICL) for complex reasoning relies on selecting the right demonstrations. Traditional retrieval methods based on surface similarity fail to capture the underlying problem-solving logic. Recent logic-based methods address this by matching predefined reasoning steps, but the rigid rules and exact-match criteria is improper to handle flexible or diverse reasoning processes. To address the problem, we propose SALA , a S emantic- A ware L ogical A lignment framework. Instead of relying on a fixed inventory, SALA automatically learns task-specific reasoning operations. It then embeds these operations into a continuous semantic space and uses dynamic time warping (DTW) to align the reasoning sequences. This approach allows for soft, flexible matching of reasoning logic while remaining highly interpretable. Experiments across four reasoning benchmarks and three LLMs demonstrate that SALA outperforms existing demonstration selection methods. Further analysis confirming the roles of the operation induction and the logical semantic alignment.

Abstract:
Enhancing NLP for low-resource languages (LRLs) is vital for supporting linguistic diversity, yet a fundamental challenge lies in the cross-lingual semantic gap. To address this, a common approach is to initialize LRL word embeddings with those from high-resource languages, thereby enriching their semantic space. However, subsequent embedding adaptation is typically performed through uniform updates across the vocabulary, which fails to allocate larger corrections to semantically difficult or infrequent tokens (e.g., culturally specific terms in Amharic or Somali) that need them most. To address this, we propose Selective Embedding Correction (SEC), a parameter-efficient framework for dynamic semantic correction. SEC models embedding adjustments as sparse low-rank residuals regularized by a group-lasso objective. This enables data-driven selectivity that refines poorly aligned tokens while preserving well-aligned representations. Experiments on named entity recognition, part-of-speech tagging, and natural language inference across multiple low-resource languages show that SEC achieves strong improvements over baselines with only about 1% additional trainable parameters.

Abstract:
Personalized language models aim to adapt responses to individual users, whose preferences are often latent and revealed gradually through interaction. Existing training-free methods rely on stored histories or retrieved memories, but they often struggle to reconcile long-term preferences with short-term topic-specific needs. To address this issue, we propose HyperTrace, a training-free framework that formulates online personalization as latent preference tracing. HyperTrace maintains interpretable natural-language hypotheses over short-term intent and long-term preferences, and updates them through an SMC-style reweight process using an LLM-based surrogate choice model. By updating these hypotheses across turns and sessions, HyperTrace enables personalization without parameter updates. Experiments on PRISM and PersonaMem-v2 show that HyperTrace improves response alignment, preference prediction, and profile consistency over strong online baselines, demonstrating the effectiveness of tracing latent user preferences for robust personalization. Code and scripts are available in the repository: [https://github.com/jiseshen/HyperTrace](https://github.com/jiseshen/HyperTrace).

Abstract:
Existing tabular reasoning benchmarks typically assume that input tables are clean and error-free, which contrasts sharply with real-world data pipelines where noise and anomalies are pervasive. While humans routinely validate tables and reject conclusions when anomalies render a question unanswerable, it remains unclear whether large language models (LLMs) exhibit similar anomaly awareness. In this work, we introduce AnTBench, a benchmark designed to systematically evaluate LLM behavior under anomalous tabular data. AnTBench consists of 1,571 curated instances covering five common and verifiable anomaly types. Experiments on 14 representative LLMs reveal a pronounced capability–behavior gap: even the strongest model achieves 76.07% on explicit anomaly detection, yet only 19.03% on spontaneous anomaly perception during standard table QA. As an inference-time mitigation, we introduce the Multi-Path Thinker, a multi-path reasoning framework. While this approach improves explicit anomaly detection, enforcing proactive verification for every query is practically prohibitive. We urge the community to move beyond inference-time patches and instill spontaneous anomaly awareness directly into the training and alignment phases. Our code and data are available at https://github.com/Wei12097/AnTBench.

Abstract:
Evaluating whether explanations faithfully reflect a model’s reasoning remains an open problem. Existing benchmarks use single interventions without statistical testing, making it impossible to distinguish genuine faithfulness from chance-level performance. We show that faithfulness is not a fixed property but an operator-dependent quantity that changes with the intervention method used to measure it. We introduce ICE (Intervention-Consistent Explanation), a framework that evaluates explanations against random baselines of equal size under multiple operators. Evaluating 7 LLMs across 4 tasks with deletion and retrieval infill operators, we find that switching operators crosses the positive-evidence threshold in 18% of configurations (5 of 28 attention comparisons), with gaps reaching 44 percentage points. Randomized baselines detect anti-faithfulness (explanations worse than random) in nearly one-third of English deletion configurations, invisible without random comparisons. These patterns persist across 6 non-English languages and 2 attribution methods. The methodology generalizes to step-level chain-of-thought evaluation, where preliminary results on 3 frontier models suggest that high accuracy does not imply faithful reasoning.

Abstract:
Automatic Post-Editing (APE) for low-resource languages (LRLs) often fails to improve Machine Translation (MT), and the score alone cannot say why: whether more training would help, or whether the training data is too inconsistent to learn from. We introduce a black-box, inference-time diagnostic that tells these two cases apart without retraining or annotation. It varies an edit-distance penalty 𝜆 that drives the model from free editing towards copying the MT, and reads two signals: (1) the shape of the Translation Edit Rate (TER)-vs- 𝜆 curve, U-shaped if edits from the model reduce error and monotonically decreasing if none does; and (2) the ordering of constraint variants that trust model confidence to increasing degrees, which shows whether confidence tracks edit quality. Across decoder-only and encoder–decoder models on English–Sinhala, the diagnostic exposes two failure modes consistent with a heterogeneous post-edit signal as the underlying cause: Binary Collapse, where the model copies the MT or makes off-target edits, and Confident Miscalibration, where the confidence signals we test do not separate useful edits from unnecessary ones. The pattern holds on English–Marathi and English–Tamil, with the failure modes tracking the post-edit distribution rather than MT quality or language family. Beyond diagnosis, the curve shape prescribes a concrete next step for practitioners; in the favorable case, a static constraint yields a free inference-time accuracy gain. We release the first English–Sinhala ( ∼ 66k) and a new English–Tamil ( ∼ 39k) APE datasets with all code.

Abstract:
Instruction following in large language models (LLMs) is commonly evaluated at the sentence or surface level, leaving discourse-level constraints underexplored. We study discourse-level instruction following by formalizing it as a structured constraint adherence problem and constructing both explicit and implicit benchmarks for evaluation. Experiments are conducted across models of different scales, revealing a systematic recognition–adherence gap: models can identify discourse constraints but frequently fail to adhere to them during generation with specific failure modes. This failure stems from a fundamental misalignment between standard training objectives and global structural integrity. Importantly, models show substantially improved adherence when discourse-level structural objectives are explicitly optimized.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) optimizes LLMs using sparse verifiable final-answer rewards. This sparse anchor reliably verifies whether a trajectory succeeds but provides no direct feedback on the reasoning path that produced it. Before success, prerequisite progress on hard problems receives no reward signal; after success, outcome rewards cannot distinguish well-organized correct trajectories from redundant or locally flawed ones. We introduce SCOPE-RL (Scaffolded Chain Optimization with Process Efficiency), a two-stage framework that densifies this anchor while retaining the GRPO update: Adaptive Scaffolded RL adds prefix-decomposed verifiable rewards on answer-hidden sub-question chains before success, and Quality-Aware Process RL applies correctness-gated process-shape rewards to refine correct trajectories after success. An expert-validated Step-Quality Evaluation Protocol evaluates useful-step density, error localization, and token efficiency beyond final-answer accuracy. On Qwen3-8B-Instruct trained on DAPO-Math and Big-Math, SCOPE-RL improves average accuracy by up to 11.2 pp and reduces reasoning tokens by up to 27.1% over outcome-only GRPO; the gains hold under GSPO and on Qwen3-0.6B-Instruct, indicating that reward-signal densification is complementary to policy-update-level RLVR advances.

Abstract:
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) in Knowledge Graph Question Answering (KGQA) by retrieving reasoning paths from external Knowledge Graphs (KGs). However, the practicality of existing methods is often limited by their reliance on iterative retrieval, where the number of required LLM calls increases with retrieval depth, leading to expensive interaction costs. To address this, we propose Plan-guided Schema-level Reasoning (PSR), a training-free and efficient RAG framework for KGQA that requires only three LLM calls. First, PSR generates a retrieval plan from the input question (the first LLM call). It then performs Plan-Regularized Monte Carlo Tree Search to extract a compact set of schema-level paths from the KG, which are subsequently filtered by the LLM (the second LLM call). Finally, PSR grounds the chosen schema-level paths into instance-level reasoning paths to augment the LLM’s answer generation (the third LLM call). Experimental results on three datasets demonstrate the high efficiency and competitive accuracy of PSR.

Abstract:
Web agents have achieved significant success in automating complex internet tasks but deploying them in real-world environments requires continuous online adaptation. Given that deploying powerful proprietary models remains commercially cost-prohibitive, practitioners must rely on lightweight local models that evolve post-deployment via online teaching from a stronger teacher. However, standard interactive feedback imposes prohibitive costs. We show that conventional trajectory-level preference optimization wastes budget on both unresolvable episodes and redundant execution turns. To resolve these inefficiencies, we propose Score-Guided Online Teaching with Budgeted Trajectory Trimming, a budget-aware framework that systematically orchestrates when and what to teach. Specifically, our framework integrates a solvability-aware teacher gate to dictate when to query the teacher model and a score-guided turn selection mechanism to decide what informative turns to retain. Extensive experiments on MiniWoB and TimeWarp demonstrate that our method achieves comparable first-pass success while reducing teacher calls by 22.6% and student training compute by 52.1% on average. Our code is available at https://github.com/zjw131f1fc/budgeted-online-teaching.

Abstract:
Deep research agents face vast, interdependent, and pervasively uncertain information. Existing systems explore what evolving intermediate representations should look like, but leave their evolution to the LLM’s implicit reasoning. Without explicit regulation, the intermediate layer is easily contaminated by mixed-quality information, and errors propagate along its dependencies, so model scale often ends up substituting for absent regulation. We argue that an agent’s mental model should instead evolve through explicit feedback that continuously aligns task understanding with reality, and identify three regulatory loops: interpretive update, deviation feedback, and schema revision. We realise this in VeriTrace, a cognitive-graph framework that explicitly implements the three loops. Using matched Qwen3.5-27B backbones, VeriTrace improves over the strongest matched baseline by an average of 4.82 pp on DeepResearch Bench (DRB) Insight (1.83 pp Overall) and by 5.9 pp Overall win rate on DeepConsult. With Config-DeepSeek, it achieves the strongest reproducible open-source result on DRB.

Abstract:
Column Type Annotation (CTA) enriches tables with semantic column types that are often missing or incomplete in real-world data, aiding multiple downstream table tasks. Yet CTA remains challenging because sparse or noisy column values often require table-level context for disambiguation. Prior work either flattens tables into encoder inputs, losing structure and semantics, or follows a description-then-encoding pipeline that separates encoding and adds generation cost. We propose D2R, a decoder-based framework that obtains column representations directly from the hidden state that initiates column description generation. Specifically, D2R first encodes the table context once as a shared prefix KV cache, providing table-level context for all target columns. For each column, we append its sampled values and a description trigger token, then extract the trigger’s hidden state as the column embedding, preserving semantics without producing text. A decoder-attached classifier, trained via contrastive learning aligned with the CTA objective, predicts column types directly from this embedding. Across four CTA benchmarks, D2R achieves the best Micro-F1, outperforming TURL, a strong table PLM, by up to 12.57 points, while retaining efficient inference. The code is available at https://github.com/IvesCheung/cta.

Abstract:
Recently, the role of LLM-as-judge in evaluating large language models has gained prominence, emerging as an important method to partially replace costly human assessment. However, current judge models suffer from narrow specialization and limited robustness, undermining their capacity for comprehensive evaluations. In this work, we decompose the ability of a generalist and generative judge model into three levels: objective verification, subjective evaluation, and rubric refinement. We present CompassJudger-2, a novel generalist judge model that overcomes these limitations via a task-driven, multi-scenarios data curation strategy. We conducted large-scale data collection for each type of task and designed tailored rejection sampling strategies to filter the data, ensuring data diversity, accuracy, and effectiveness. Empirically, CompassJudger-2 achieves superior results across multiple judge and reward benchmarks, demostrating its excellent robustness and generalization ability. These contributions advance robust, scalable LLM judgment and establish new performance and evaluation standards.

Abstract:
Function vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using word-level machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from word-to-word prompts in a single English → X direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. We further find that the highest-gain tokens span multiple languages and that translation FVs across directions share most of their top-ranked heads, indicating that the FV encodes a largely language-agnostic translation signal.

Abstract:
Large Reasoning Models (LRMs) achieve strong performance by generating long reasoning traces with explicit reflection. However, reflection is not functionally uniform: some reflections revise the reasoning strategy, while others verify previously derived intermediate results. In this work, we examine whether local self-verification can be controlled selectively without suppressing useful reasoning. Through a large-scale analysis of mathematical reasoning traces, we distinguish strategy-level rethink from local self-verification recheck, and further separate rechecks into corrective and confirmatory cases. We find that recheck occurs frequently across models and benchmarks, but most observed rechecks are confirmatory, rarely uncovering errors or changing the reasoning trajectory. Motivated by this finding, we propose Experience-Driven Suppression (EDS), a conservative test-time controller that suppresses a recheck only when historical outcomes of similar episodes suggest low utility. Unlike early-exit or trigger-suppression methods that stop reasoning globally or suppress reflection cues broadly, EDS targets local recheck while preserving later rethink. Experiments show that EDS yields moderate reasoning-length reductions with limited accuracy changes, supporting behavior-specific rather than monolithic reflection control.

Abstract:
Large language models (LLMs) have been increasingly explored for graph computation, where tasks require reasoning over structured relationships and algorithmic operations. Yet, it remains unclear when LLMs can reliably support such computation and how they should be incorporated into graph-solving pipelines. Existing surveys at the intersection of LLMs and graphs primarily focus on graph learning, text-attributed graphs, or graph-language modeling. To bridge this gap, we provide a comprehensive review of LLMs for graph computation through a role-based taxonomy. Specifically, we identify two major paradigms: i) LLMs as executors, where models directly solve graph tasks from graph descriptions and instructions; and ii) LLMs as planners, where models formulate problems, decompose reasoning steps, and invoke external tools or agents for execution. Based on this taxonomy, we analyze the strengths and limitations of current methods. Our review indicates that LLMs are promising for simple, small-scale tasks, but remain unreliable for large-scale and exactness-demanding tasks. Finally, we summarize available datasets and suggest four future directions.

Abstract:
Emotional Support Conversation (ESC) systems help seekers understand and cope with distress through multi-turn interaction. Existing ESC systems typically improve responses by adding auxiliary support knowledge or prompting LLMs to reason over dialogue context. However, they usually pass inferred seeker information to the generator as a single state, so factual circumstances and mental state inferences are stored, selected, and updated in the same way. This blurs the difference between persistent facts and transient personal states, making it difficult to retain stable grounding while adapting to psychological changes across turns. Therefore, in this paper, we propose MIA, a Personal-Factual Adaptive Memory framework for multi-turn emotional support conversation. MIA first infers Emotional Theory-of-Mind factors and separates them into factual and personal states. It then uses a Personal-Factual Discriminator to decide which state should guide the next response, and an Obsolescence Removal Mechanism to remove outdated memory before generation. This design enables the generator to preserve factual grounding, focus on the turn-relevant support orientation, and avoid stale psychological assumptions. Experiments on ESConv and CPsyCounD show that MIA better understands the seeker’s situation, provides more constructive guidance, and maintains more coherent support across turns. Our codes are available at: https://github.com/ Wh2018/MIA.

Abstract:
Strong large language models (LLMs) often achieve high scores on many benchmarks, yet can fail on closely related, sometimes easier, problems near their apparent capability boundary, a phenomenon known as jagged intelligence that reveals reasoning weaknesses masked by aggregate scores. Existing evaluations of jagged intelligence rely on manually designed benchmark-adjacent rewrites, which are hard to scale and quickly saturate as models improve. To address this challenge, we propose Automated Weakness Exploration (AWE), a fully automated framework for constructing and maintaining jaggedness evaluations. Using AWE, we construct JaggedBench with 1,142 verified capability-adjacent variants and evaluate 24 state-of-the-art models. On the primary evaluation set AIME 2025, benchmark-adjacent rewrites induce an average OG-to-RW relative drop of approximately 25% in Avg@k across 24 models, with consistent degradation observed across math, code, science, and knowledge domains. We identify 18 recurring failure modes. Our analysis shows that increasing reasoning intensity and scaling model size improve average performance and reduce failure frequency, yet the underlying failure-mode categories largely persist. Overall, these findings indicate that standard benchmark scores can present an overly optimistic view of reliability near the capability boundary. JaggedBench provides a more faithful and scalable lens for evaluating boundary robustness, enabling systematic diagnosis of boundary failures and longitudinal tracking of persistent weaknesses as models evolve. We release the benchmark, code, and subsequent updates at https://jaggedbench.github.io/.

Abstract:
Embodied language grounding requires more than identifying the referent of an instruction: recovered semantics must also control the action an agent exposes. We study this missing link as a semantic–action gap, where instruction semantics are recoverable but weakly expressed in native continuous actions. We introduce SAT-Bench, a fixed-observation counterfactual benchmark that holds the visual scene and agent state fixed while changing only instruction semantics. On LIBERO target-name and pixel-grounded relation swaps, target recovery reaches 100.0% and 95.8%, whereas OpenVLA action sensitivity remains only 6.8% and 7.7%. The gap persists across 1,000 additional compositional and temporal/procedural counterfactuals, with overall action sensitivity of 6.1%. Hidden-state, threshold-free, cross-policy, and rollout diagnostics further support this semantic–action transfer failure. We introduce VISA, a lightweight execution-time interface that converts recovered semantics into ALLOW, DEFER, target-consistency, and verified-selection decisions. VISA reduces invalid-instruction blind execution from 92.7% to 2.8% while preserving 94.0% of normal commands, and verified selection further improves target-consistent action exposure without updating the underlying policy. Overall, embodied language evaluation should measure semantic–action transfer, not semantic parsing alone.

Abstract:
Imagine two users contacting the same AI travel agent on the same day, making the same request under identical policy conditions, differing only in the demographic information they provide. The governing policy is explicit and contains no demographic distinction. Should the agent respond identically? We present an empirical study of how demographic context influences the behaviour of policy-governed LLM agents. Building on an existing agentic evaluation framework, we augment policy-governed tasks with controlled demographic cues while keeping policies and task environments fixed, enabling behavioural differences to be attributed solely to demographic signals. To evaluate procedural fairness, we introduce three procedural dimensions: Policy Application Fairness (PAF), Procedural Effort Equity (PEE), and User Fulfillment Parity (UFP), unified into the Demographic Policy Fairness Index (DPFI). Across more than twenty thousand simulated interactions spanning multiple service domains and frontier models, all evaluated systems exhibit systematic demographic disparities that conventional task-success metrics fail to capture. Our analysis identifies three distinct failure modes with different procedural dynamics, demonstrating that aggregate success metrics can obscure how disparities emerge. Overall, the findings suggest that demographic bias in policy-governed LLM agents manifests through procedural behaviour rather than a single uniform failure pattern. The proposed framework provides a foundation for systematically auditing procedural fairness in policy-driven LLM agents.

Abstract:
Improving the plot progression ability of role-playing language agents (RPLAs) plays a critical role in enhancing user experience and dialogue immersion. However, research on plot progression in role-playing language agents is still limited. Existing datasets often contain too few dialogue turns and lack large-scale sentences with clear deterministic plot turns, which restricts further model development and evaluation. To bridge this gap, we propose RoleArena, an innovative multi-agent role-playing environment. By introducing a critic agent and combining discrete plot lines with environment-agent-based dynamic judgments, the system generates long multi-turn dialogues with key markers for plot progression. Based on RoleArena, we construct the PlotStream dataset, which contains 39.3k characters and 170k dialogues, with annotations for key plot progression sentences. In addition, based on PlotStream, we further develop PlotRole-7B and PlotRole-72B, RPLAs built upon the Qwen2.5 model and specialized in plot progression capabilities. Extensive experiments show that RoleArena and PlotRole improve dialogue generation with plot progression and strengthen the ability of plot progression.

Abstract:
Prompting a language model to endorse a value does not by itself establish how that value is deployed when responses require tradeoffs. We study this gap in truth–propriety dilemmas, where direct truth-telling can conflict with tact, face-saving, deference, or role-sensitive restraint. We introduce Truth–Propriety Alignment (TPA), a 600-prompt diagnostic benchmark over 100 scenario groups, and conduct a single-model diagnostic evaluation of Qwen2.5-7B-Instruct under deterministic decoding. The model remains predominantly truth-oriented, but trigger-assigned scenario groups differ sharply in how residual responses are distributed. Because value triggers are fixed within scenario groups, we interpret this pattern as trigger-associated residual composition rather than a causal trigger effect. A small semantic audit shows that compatible residuals are mixed: they include substantive reframings but also tonal and lexical tracking. Finally, a hidden-state probe is nearly matched by an exact-prompt BoW baseline, and deleting the explicit value-trigger sentence does not reduce BoW performance, indicating that behavioral-label predictability is distributed across the prompt surface. We therefore frame TPA as a diagnostic audit of prompt intent, observed behavior, scenario/trigger priors, annotation boundaries, and probe evidence, rather than as evidence for prompt-independent value representations.

Abstract:
Large language models (LLMs) often suffer from capability stagnation in self-improvement training due to fixed difficulty levels. To address this issue, we propose STRETCH (Self-Taught Reasoning Evolution via Targeted CHallenge) inspired by cognitive scaffolding theory. STRETCH introduces a dynamic Stretch Zone mechanism that continuously aligns question difficulty with the model’s solving capability. Within a single parameter space, the model alternates between a Scaffolder that generates adaptive challenges and a Learner that solves them through reinforcement learning. This dual-loop co-evolution process stabilize training and promote progressive reasoning growth. Experiments on both negotiation and operation research benchmarks demonstrate that STRETCH consistently outperforms strong prompting and domain-specific baselines. Further analysis shows that dynamic difficulty alignment is critical for sustained capability improvement and synchronized reasoning evolution.

Abstract:
On-policy distillation (OPD) has emerged as a promising paradigm for reasoning-oriented post-training of large language models (LLMs), especially when combined with reinforcement learning from verifiable rewards (RLVR). Existing OPD methods rely on reverse KL (RKL)-based teacher supervision over trajectories sampled from the student policy. However, we identify a critical limitation: under large teacher–student policy divergence, RL-driven exploration often produces trajectories that the teacher assigns low probability, resulting in uninformative negative feedback. To address this, we propose Teacher-Guided Policy Optimization (TGPO). Unlike existing OPD methods that evaluate student-sampled actions using the teacher, TGPO directly guides next-token generation using the teacher’s predicted distribution on student-generated contexts, yielding an on-policy forward-KL objective. Combined with RLVR, this guidance steers exploration toward improved continuations. Experiments on reasoning benchmarks show that TGPO outperforms existing RKL-based OPD methods and remains robust across different teacher models. Furthermore, we find that teacher top-1 predictions suffice for effective guidance, without requiring the full teacher distribution.

Abstract:
Large Language Models (LLMs) have demonstrated remarkable capabilities in tasks related to reasoning and judgment. However, assessing the quality of arguments requires a rigorous evaluation. We investigate the extent to which LLMs can effectively perform this task. We tested 12 open-weight LLMs of different sizes and families under zero-shot, few-shot, and chain-of-thought to approximate human pairwise comparisons of argument quality across three dimensions—logical, rhetorical, and dialectic—and used these comparisons in a Bradley-Terry model to infer latent strength scores and derive a ranking of arguments. Our insights show that LLMs have promising but moderate correlation with human judgment, with Llama-70B obtaining the strongest alignment, reaching moderate Cohen’s κ = 0.493 and moderate correlations with Bradley-Terry scores derived from these annotations (Kendall, Pearson, and Spearman: 0.327-0.477). Other LLMs exhibit weak, moderate, or high alignment with Llama-70B while achieving comparable results against human judgment, suggesting partial but complementary understanding of underlying quality dimensions despite differences in model size and family. Moreover, LLM predictions are stable across trial runs, with fewer than 7.75% of cases yielding different labels. Remaining variability is handled via majority voting and few-shot prompting for large-size models.

Abstract:
This paper investigates the problem of safety alignment in large language models (LLMs) during the testing phase, particularly under jailbreaking attacks. Previous methods typically rely on identifying malicious request intentions or adjusting the LLM’s decoding strategy to inject safe behavior. However, this often leads to over-rejection, limiting their practical application. To address this, we propose an innovative framework called P rior- R isk I ntegration for S afe M ixture decoding (PRISM). Specifically, PRISM first uses a lightweight semantic risk evaluator to calculate the prior probability of harmfulness at the prompt level, and then continuously tracks the cumulative risk signal provided by a safety evaluator during the generation process. By comprehensively considering these two types of risks, PRISM adaptively mixes the base model’s probability distribution with a safety preference distribution, triggering a rejection response only when both the prompt prior probability and the generation risk are sufficiently high. Extensive experiments validate the effectiveness of PRISM in resisting jailbreaking attacks and its practicality. Across five jailbreaking attack methods and three neutral datasets, PRISM demonstrates superior performance compared to other baseline methods.

Abstract:
Large Language Models (LLMs) often answer the same factual question differently across languages. We study whether cross-lingual latent-space intervention can reduce this inconsistency. We train layer-specific autoencoders on parallel multilingual representations and apply inference-time corrections to factual QA prompts. We find that latent intervention improves geometric alignment between languages, and that this improvement translates into consistent gains in cross-lingual consistency with English across both open-ended and multiple-choice QA formats, without degrading factual accuracy. In open-ended QA, Spearman’s rank correlation between English and non-English languages improves substantially, with gains of 0.16 for English-Arabic and 0.20 for English-Russian pairs. In multiple-choice QA, answer agreement with English improves consistently across both KLAR and mParaRel. Ablations show that AE reconstruction drives these gains, whereas PCA-based and mean-shift subspace projections contribute only marginally to the observed improvements in consistency.

Abstract:
Tracing connections between historical texts is an important part of intertextual research, enabling scholars to reconstruct the virtual library of a writer and identify the sources influencing their creative process. These intertextual links manifest in diverse forms, ranging from direct verbatim quotations to subtle allusions and paraphrases disguised by morphological variation. Language models offer a promising path forward due to their capability of capturing semantic similarity beyond lexical overlap. However, the development of new methods for this task is held back by the scarcity of standardized benchmarks and easy-to-use datasets. We address this gap by introducing Loci Similes, a benchmark for Latin intertextuality detection comprising a curated dataset of ~176k text segments and 1,490 expert-verified parallels, including 945 labeled references from an existing dataset. Using this data, we establish baselines for retrieval and classification of intertextualities with both lexical methods and pretrained encoder language models.

Abstract:
Multi-turn interactions with LLMs are becoming increasingly common in information-seeking scenarios. However, user queries are often ambiguous and context-dependent, making them ill-suited for direct use as retriever queries. Conversational query reformulation (CQR) addresses this issue by rewriting the current utterance into a stand-alone query grounded in the dialogue history. Recent LLM-based CQR approaches achieve strong performance; however, their repeated LLM invocations and misalignment with downstream retrievers remain challenges. In this work, we begin from the observation that retrievers are highly sensitive to content ordering: simply reordering the same content can lead to changes in retrieval coverage and performance. Based on this, we propose a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components. We further apply LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user’s core intent. Our framework generalizes across both open- and closed-source LLMs as well as dense and sparse retrievers. It achieves strong performance on four widely used conversational benchmarks, with up to 22.5% relative MRR improvement over the previous state-of-the-art baseline with far fewer LLM calls.

Abstract:
Masked diffusion language models (MDLMs) decode by repeatedly committing tokens to masked positions, but these commitments are usually irreversible. A token chosen under sparse, partial context is kept fixed, even when later context no longer supports it. Existing samplers mainly decide when to commit a token, but rarely check whether an already committed token should still be kept, allowing early mistakes to propagate. We trace this issue to confidence drift, where the model’s confidence in a committed token drops from its sparse commit-time context to the denser context available later. Based on this signal, we propose CoDR ( Co nfidence- D rift R emasking), a training-free and sampler-agnostic refinement pass. CoDR estimates drift for all committed positions in only k forward passes via k -partition probing, then remasks and regenerates only the tokens the model no longer endorses. Across two backbones, four reasoning and coding tasks, and three base samplers, CoDR improves accuracy on nearly every combination with modest overhead. Controlled experiments show that the gains come from targeted confidence-drift remasking rather than extra compute alone, and that CoDR uses far fewer forward passes than prior remasking methods.

Abstract:
Node classification on graphs often requires labeled nodes, yet obtaining labels at graph scale is expensive. When node attributes contain semantic content, such as paper abstracts, web pages, or product descriptions, large language models (LLMs) can provide low-cost supervision by annotating a small subset of nodes. However, these LLM-generated labels are noisy, and existing label-free graph learning methods usually treat this noise as either global or class-conditional. We find that LLM annotation errors are not only class-dependent but also region-dependent: within the same class, reliability can vary sharply across feature-space clusters. In light of this, we propose Cluster-Aware Noise Estimation (CANE), a label-free learning framework that estimates cluster-conditional LLM reliability without ground truth labels, and uses this estimate to decide which pseudo-labels to trust, and which labels to correct. Across various graph benchmarks and GNN backbones, CANE improves over the strongest label-free baselines, with the largest gains on datasets exhibiting stronger cluster-conditional noise. The code is available at https://github.com/thapaliya19/CANE.

Abstract:
LLM agents require reliable step-level evaluation to compare candidate branches and allocate computation effectively. However, lightweight evaluation remains challenging. External verifiers introduce additional inference cost, while agent-produced confidence or self-evaluation scores can be miscalibrated, especially when candidates are generated by heterogeneous agents. We propose G eneralized L ayer-wise I ntrinsic D istributional E valuation ( GLIDE ) for LLM agents. GLIDE derives intrinsic step evidence from layer-wise residual coherence, which measures whether local residual updates consistently support the global residual change induced by a candidate step. It calibrates this evidence against the recent score distribution of the generating agent and converts it into a pessimistic reward that jointly accounts for absolute residual evidence and agent-relative standing. The reward provides a cross-agent value signal for MCTS branch selection, while normalized predictive uncertainty guides adaptive branching. Experiments on multi-hop reasoning, sequential decision making, and symbolic logic show that GLIDE improves task performance, step-level ranking quality, and computational efficiency without external verifiers or task-specific supervision.

Abstract:
Universal multimodal embedders enable retrieval across text, image, and combined queries, but their dense representations incur high memory and inference costs. Post-hoc sparsification could reduce these costs but remains underexplored for multimodal retrieval. We introduce PUMA, a sparse autoencoder recipe that maps universal multimodal embeddings to compact sparse codes without retraining the backbone: a pretraining stage preserves dense dot-product geometry, after which the sparse encoder is fine-tuned for retrieval. We evaluate on five benchmarks covering text-to-image and composed image retrieval. On Qwen3-VL-Embedding-2B, PUMA is statistically indistinguishable from or improves over dense retrieval on four of five datasets. We further identify two failure modes of post-hoc sparsification: insufficient pre-TopK support and retrieval-misaligned active support. PUMA reduces per-vector storage by 8-16x (FP32) and runs up to 25x faster over exact dense scoring on larger candidate pools, offering a practical approach to storage- and latency-efficient multimodal retrieval.

Abstract:
Despite strong performance on standard benchmarks, it remains unclear whether large language models (LLMs) evaluate social pragmatics in ways that align with human judgments. We evaluate LLM politeness judgments using two English-language datasets with complementary annotation formats: continuous human ratings and three-way categorical labels. Across the seven evaluated models, we find that inter-model agreement is stronger than model-human agreement. Strategy-level analyses suggest that model-human alignment is associated with explicit linguistic cues, while some rapport-building strategies occur more frequently in misaligned cases. In the categorical task, model predictions exhibit systematic neutral compression, characterized by the overproduction of Neutral labels and the underprediction of Impolite labels. This pattern persists when expert consensus is used as the reference on a diagnostic subset. Our findings highlight the need for pragmatic evaluations that go beyond aggregate agreement metrics by examining directional patterns of model-human disagreement across different human references.

Abstract:
Prosody plays an important role in speech translation, conveying information such as emphasis, emotion, and intent beyond lexical content. However, despite recent progress in expressive speech-to-speech translation (S2ST), little is known about how prosodic patterns are similar/different across languages. Understanding these cross-lingual similarities and differences is crucial for effectively incorporating prosody into expressive S2ST systems. In this work, we present the first fine-grained cross-lingual analysis of prosody using multilingual dubbing data across English-German, English-Spanish, and English-French language pairs. We analyze the similarity of pitch, energy, and temporal feature patterns between source and target speech and investigate the linguistic and alignment-related factors affecting this similarity. Our analysis reveals inherent cross-lingual correlations in prosodic structure between certain languages. The findings provide important insights into the transferability of prosody across languages and offer empirical guidance for future expressive speech-to-speech translation systems.

Abstract:
Large language models are increasingly used in emotionally consequential interactions. Response-level evaluation, however, struggles to diagnose how empathy fails across turns. We introduce MATE (Multi-turn Assessment of calibraTed Empathy), a framework that evaluates empathy as a multi-turn, perception-centered process. Across two controlled studies (N=82), baseline responses exhibit disclosure-insensitive miscalibration: generic validation and formulaic tone read as hollow across turns. A prompt-level self-critique condition makes these patterns less prominent but coincides with a different failure, agreement-skewed miscalibration, where affirmation becomes insufficiently contingent on disclosure context under high persona alignment (62.5% of fit participants in Study 2). Response-level analysis is consistent with a tone–agreement dissociation, as the condition shifts tone toward naturalness while agreement density also increases (+98% under fit vs. +25% under unfit). These findings reframe empathic behavior as relational calibration—consistency among disclosure depth, persona, and response behavior—rather than maximization. Response behavior itself spans tone, agreement, and contextual specificity, sub-dimensions that can drift independently.

Abstract:
Medical knowledge changes continually, making large language models vulnerable to relying on outdated yet clinically plausible information. We study whether the format of supervision affects medical knowledge updating under a matched training-budget setting. We introduce SEER-Bench, a temporally anchored oncology-staging benchmark curated from the latest versioned SEER Research Data release, and render identical medical update events from NCCN oncology guidelines into four supervision formats: EMQ, MSQ, FITB, and SAQ. Across SEER-Bench and HealthBench Professional, EMQ gives the most stable external transfer and retention among same-budget SFT variants. With EMQ supervision, the updated 4B model produces competitive results on temporally anchored oncology staging, reaching 64.8% answer accuracy and 59.6% rationale accuracy on SEER-Bench. Diagnostic analyses suggest that EMQ exposes denser clinical contrast signals while preserving discriminative representations with smaller movement from the base model. These results show that medical knowledge updating depends not only on the update algorithm, but also on how knowledge is structured as supervision.

Abstract:
In the legal domain, LLMs can improve reliability and interpretability by generating explicit reasoning traces. However, training LLMs to produce high-quality reasoning traces remains challenging. Existing supervised fine-tuning methods struggle when reasoning traces are significantly longer than final answers, as the learning signal for the answer becomes diluted. Meanwhile, reinforcement learning methods such as Group Relative Policy Optimization also have several limitations, including costly reward design, performance plateaus, and reward hacking. We therefore propose a two-stage training framework. In Stage I, JurisCoT-SFT employs a length-normalized dual objective to balance learning signals between reasoning traces and final answers. In Stage II, Lifecycle-Aware Backtrackable Policy Optimization dynamically evaluates and retires auxiliary reward signals based on their impact on final evaluation objectives, enabling efficient reward utilization. Our framework substantially improves legal reasoning performance on relevant tasks selected from multiple legal benchmarks, including ChJGBench, LawBench, and LegalBench. Experimental results demonstrate consistent improvements in both reasoning trace quality and final answer accuracy, while our models remain competitive with larger models.

Abstract:
Vocal behavior provides important signals for speech-based depression detection, but its interpretation often depends on what is being said and how it functions in context. However, existing methods typically treat acoustic cues as context-independent markers, making it difficult to distinguish vocal form from its context-dependent communicative function. We propose ParaCalib, a framework that semantically calibrates paralinguistic behavior by interpreting vocal patterns relative to utterance-level semantic context and inferred communicative function and representing them in a comparable state space. Concretely, ParaCalib uses an Audio-Language Model (ALM) to generate contextualized vocal descriptions and an LLM-based Paralinguistic State Extractor (PSE) to map these descriptions into a structured Semantically Calibrated Paralinguistic (SC-Para) representation. ParaCalib achieves the highest mean Macro-F1 among the evaluated methods, reaching 71.9% on DAIC-WOZ and 90.5% on MODMA. Our controlled analysis provides direct evidence of semantic calibration at the PSE stage: under a fixed caption-level acoustic description, varying the accompanying semantic context changes the inferred depression-related paralinguistic evidence. Our exploratory analysis further identifies recurring configurations of paralinguistic states associated with depression labels rather than a single uniformly dominant state.

Abstract:
The prevailing post-training paradigm for Large Reasoning Models (LRMs)—Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL)—suffers from an intrinsic optimization mismatch: the rigid likelihood maximization in SFT induces distributional collapse, thereby exhausting the exploration space necessary for subsequent RL. Motivated by the Gibbs optimum of KL-regularized RL, we derive a token-level variational surrogate that makes the SFT target structurally compatible with the subsequent RL stage, and propose Gibbs Initialization with Finite Temperature (GIFT). Standard SFT emerges as a degenerate zero-temperature limit of this surrogate, while a finite temperature preserves structural diversity. Our experiments demonstrate that GIFT outperforms standard SFT and surpasses competitive baselines when utilized for RL initialization. Our code is available at https://github.com/zzy1127/GIFT .

Abstract:
Large Language Models (LLMs) are increasingly deployed as long-horizon conversational agents, motivating growing interest in memory systems. However, existing benchmarks primarily evaluate memory through QA-style probing rather than in-situ conversational usage. We introduce LOCOMO-CONV, a conversational memory benchmark derived from LoCoMo with four query styles: dialog, implicit, counterfactual, and composed. Across four representative memory systems, we evaluate both retrieval recall and end-to-end response quality. Our experiments show that conversational framing exposes substantial retrieval gaps overlooked by QA benchmarks, especially on implicit and composed queries, which multi-facet query rewriting narrows for raw-turn but not abstractive memory. We further find that strong retrieval does not fully translate into response quality, and that implicit queries exhibit silent grounding, where memory improves contextual grounding without explicitly surfacing the gold fact. These results point to reasoning-based memory elaboration as a promising direction, and we release auxiliary supportive_memory annotations capturing conversationally useful context beyond the original gold evidence.

Abstract:
Multimodal large language models (MLLMs) have achieved strong performance on perception-oriented tasks, yet their ability to perform mathematical spatial reasoning, defined as the capacity to parse and manipulate two- and three-dimensional relations, remains unclear. While humans easily master textbook-style spatial problems, current MLLMs exhibit substantially higher error rates, exposing a fundamental weakness in spatial comprehension. To investigate this gap, we present MathSpatial, the first large-scale and systematic dataset resource dedicated to mathematical spatial reasoning in MLLMs. MathSpatial provides two complementary subsets: MathSpatial-Bench (2,000 problems) for evaluating spatial reasoning isolated from perceptual noise, and MathSpatial-Corpus (8,000 problems) equipped with structured reasoning traces for training. Benchmarking 20+ MLLMs reveals a severe deficit: non-reasoning models largely languish below 60% accuracy, and even the strongest reasoning models fall ~16 percentage points behind human performance (96.3%). Crucially, we demonstrate that fine-tuning on MathSpatial-Corpus yields consistent performance gains across model families. MathSpatial provides a vital resource for diagnosing and advancing spatial reasoning in MLLMs. The dataset and related resources are available at https://github.com/shuolucs/MathSpatial.

Abstract:
Conversational-memory systems increasingly transform dialogue history into facts, summaries, timelines, and other source-linked descendants, so a single source turn can coexist with several derived memories in the same retrieval index. This raises an under-specified evaluation question: which stored form should receive retrieval credit? We show that this scoring-target choice is often left implicit and can materially change benchmark conclusions. We present TIAP, a fixed-output audit that rescores saved ranked outputs under three targets—Raw, Source, and Canonical—without rerunning retrieval. On LoCoMo and LongMemEval-S, switching only the credited target changes nDCG on 83.4%–94.0% of shared queries, flips target orderings on Mem0 and MemoryOS transfer runs, and reverses parser-density recommendations. A 1,902-case semantic audit further shows that relaxed source-linked credit is fully justified only 29.2% of the time, despite high rubric reliability in a validation subset. These results reveal target noninvariance: conclusions about memory architectures can silently flip with a single benchmark-design choice. Conversational-memory papers should therefore define and report the scoring target explicitly.

Abstract:
Despite the syntactic fluency of Large Language Models (LLMs), ensuring their logical correctness in high-stakes domains remains a fundamental challenge. We present VERGE, a neurosymbolic framework that combines LLMs with SMT solvers to produce verification-guided answers through iterative refinement. VERGE decomposes LLM outputs into atomic claims, autoformalizes them into first-order logic, and verifies their logical consistency using automated theorem proving. VERGE introduce three key innovations: (1) multi-model consensus via formal semantic equivalence checking to ensure logic-level alignment between candidates, eliminating the syntactic bias of surface-form metrics, (2) semantic routing that directs different claim types to appropriate verification strategies: symbolic solvers for logical claims and LLM ensembles for commonsense reasoning, and (3) precise logical error localization via Minimal Correction Subsets (MCS), which pinpoint the exact subset of claims to revise, transforming binary failure signals into actionable feedback. VERGE framework classifies claims by their logical status and aggregates multiple verification signals into a unified score with variance-based penalty. The system iteratively refines answers using structured feedback until acceptance criteria are met or convergence is achieved. This hybrid approach delivers formal guarantees where possible and consensus verification elsewhere, advancing trustworthy AI. With the GPT-OSS-120B model, VERGE demonstrates an average performance uplift of 18.7% at convergence across a set of reasoning benchmarks compared to single-pass approaches.

Abstract:
Large language models (LLMs) increasingly participate in conversational systems where responses emerge through multi-turn interactions rather than single prompts. We introduce a multi-agent evaluation framework that probes LLMs for bias through multi-turn conversations. We present a framework that orchestrates three roles (target, adversary, and scorer) under diverse normative fairness definitions. Experiments on five widely used LLMs using 100 adversarial prompts show that adversarial multi-turn probing reliably surfaces bias-related responses, with baseline violation rates exceeding 80% across models, while human validation with the scorer (~85% agreement) confirms the scoring reliability. Preventive prompt-conditioning strategies provide limited reduction in violations, while our iterative post-hoc mitigation pipeline substantially reduces scorer-detected violations under the active fairness profile, from over 80% to near-zero. We frame this as scorer-relative violation reduction rather than universal debiasing.

Abstract:
Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal classifiers can miss such evidence when relying mainly on global image-text representations. We introduce MemeCF, a cue-focused benchmark of 9,895 memes across harm, hate, and sarcasm, with annotations identifying the modality and rationale of the pivotal evidence. We also propose MemePIVOT, a local-global architecture for meme classification. MemePIVOT uses frozen CLIP features, unbalanced optimal transport to align words with image patches while allowing irrelevant evidence to remain unmatched, and an evidential fusion head to combine local grounding with global meme context under uncertainty. Experiments on HarMeme, PrideMM, and MemeCF show consistent gains over strong text-only, image-only, multimodal, and vision-language baselines. Cross-dataset and ablation results further show that explicit pivotal-evidence modeling improves robustness and contributes meaningfully beyond global multimodal representations. Our code and dataset are publicly available at https://github.com/AkshitSharma1/MemePIVOT

Abstract:
Existing LLM reasoning benchmarks are largely static, single-turn, English-centric, and reliant on biased LLM-as-a-judge scoring. We present MTGBench, a bilingual (English/Russian) suite of seven multi-turn puzzle games—Wordle, Word Chain, Situation Puzzles, 20 Questions, Wheel of Fortune, 20 Features, and Alias—spanning deductive, semantic, lateral, consistency, and knowledge-retrieval reasoning. Four games use algorithmic checkers; the remaining three use a constrained LLM-as-a-rule-enforcer validated against human annotators, and episodes are regenerated natively per language rather than translated. Across 16 frontier models and a human baseline comprising 476 sessions we find that (i) rankings are highly task-dependent, (ii) chain-of-thought is near-essential for letter-level games but uneven on knowledge-driven ones, and (iii) the strongest reasoning-enabled LLMs match or exceed our collected Russian human baseline across the entire suite. Game scores align most consistently with cognitive batteries probing Causal Reasoning, Pragmatics, Planning, and Common-sense Reasoning, though these associations are game-specific rather than a single executive-function signal. Code and data are available in MTGBench github.

Abstract:
Mobile GUI agents have shown strong potential in real-world automation and practical applications. However, most existing agents remain reactive, making decisions mainly from the current screen, which limits their performance on long-horizon tasks. To address this limitation, a natural approach is to use the forecasting ability of world models to support better decision-making. This is challenging because existing world models often lack spatial awareness when predicting post-action states and are too expensive for practical deployment. In this paper, we propose MobileDreamer, an efficient world-model-based lookahead framework to equip the GUI agents based on the future imagination provided by the world model. It consists of textual sketch world model and rollout imagination for GUI agent. Textual sketch world model forecasts post-action states through a learning process to transform digital images into key task-related sketches, and designs a novel order-invariant learning strategy to preserve the spatial information of GUI elements. The rollout imagination strategy for the GUI agent optimizes the action-selection process by leveraging the prediction capability of world model. Empirical results on Android World demonstrate that MobileDreamer consistently improves the task success rate across multiple base models, achieving an average performance gain of 5.3%. World model evaluations further verify that our textual sketch modeling accurately forecasts key GUI elements.

Abstract:
Large language models (LLMs) are increasingly involved in the distribution of scarce resources, raising concerns about biased allocations based on characteristics like race and gender. Recent LLM audits have produced inconsistent results, however, finding evidence of both positive and negative discrimination towards women and ethnic minorities, even for the same models. We show that this disagreement can arise from differences in audit format and introduce FairFund-Bench, a benchmark that systematically varies key features of previous audit designs: the evaluation task (rating, ranking, or allocation), comparison context (single or multi-stimulus), and whether the audit is transparent or disguised. The benchmark comprises 600 English-language requests for financial assistance created from human-authored templates (calibrated against 1.3M real GoFundMe campaigns) across three domains, four race and two gender categories, and five causal framings of need derived from welfare deservingness theory. Across 14 models, audit format changes the direction of bias: models advantage minorities when rating claimants individually but penalize some groups when ranking them side by side. Bias magnitude, though small overall, is several times greater in disguised audits than in transparent ones, where, faced with appeals differing only in claimants’ names, models overwhelmingly split funds equally. Causal framing effects, by contrast, exceed demographic effects by roughly an order of magnitude and are consistent across models and audit formats, indicating that current LLMs robustly reproduce human deservingness evaluations. The benchmark scores models on four criteria (demographic bias, deservingness alignment, cross-task consistency, and cross-context consistency), is publicly available, and can be readily adapted to other substantive domains.

Abstract:
Autoregressive generators often rely on fixed conditioning representations, so uncertainty in noisy inputs can be amplified during rollout under distribution shift. We address this by resolving uncertainty in latent space before decoding: Latent-Level Uncertainty Resolution (LURS) encodes inputs as stochastic latent sequences and learns a denoising refinement operator. At inference, a single deterministic refinement step, without reference tokens or iterative sampling, feeds a refined latent memory to the decoder via cross-attention. In teacher-forced and teacher-forcing-free decoding on open-vocabulary EEG-to-text, LURS improves generation quality and substantially reduces cross-subject variability, while remaining competitive in shared-subject settings. Ablations and stability analysis attribute gains to latent refinement rather than decoder changes or hyperparameter effects.

Abstract:
Positional biases such as recency and primacy effects have been documented in large language models (LLMs), yet the underlying mechanism by which these models make their evaluations remains poorly understood. Both primacy and recency biases have been observed in human judgments in response to evidence, but recent work suggest that when the listener updates their beliefs – during the presentation of evidence or only at the end – influences the presence of such effects. We investigate whether a similar phenomenon holds for LLMs, finding divergence from human behavior. These biases are more exacerbated in newer models compared to their predecessors.

Abstract:
Large language models (LLMs) are increasingly used for code generation, making reliable identification of machine-generated code important for attribution, tracking, and misuse detection. Existing code watermarking methods are dominated by N-gram-based schemes, yet their robustness has mostly been evaluated only against simple edits or optimizations. We argue that this significantly overstates security, because software engineering already provides stronger semantics-preserving transformations in the form of code obfuscation. We study N-gram-based code watermarking under obfuscation. We formally model semantics-preserving transformations as a Markov random walk and prove that, under an intuitive and experimentally supported assumption called distribution consistency, obfuscation can nullify the robustness of N-gram-based watermarks. If the original detector has a false positive rate 𝜖 pos , then after obfuscation, its failure rate on watermarked code approaches 1 - 𝜖 pos . We validate this theory on three state-of-the-art watermarking schemes, two LLMs, two programming languages, four benchmarks, and four obfuscators. Across all settings, detectors collapse to near-random performance on obfuscated code (AUROC tightly around 0.5), and for each language, at least one attack leaves all post-obfuscation AUROC scores below 0.6. These results jointly show that current N-gram-based code watermarks are not robust to realistic obfuscation attacks and motivate more semantics-aware alternatives.

Abstract:
Large language models (LLMs) can generate biomedical hypotheses, but it remains unclear whether they truly reason from scientific evidence or simply produce convincing-sounding ideas. To study this, we combine three major biological databases: the Kyoto Encyclopedia of Genes and Genomes (KEGG), Rhea, and UniProt, into a unified biochemical knowledge graph and construct a benchmark of 550 paths connecting enzyme sources to rare disease endpoints, yielding 13,200 hypotheses from six LLMs under four conditions varying the biological information each model receives: source enzyme only, full biological path, or source and disease endpoint only. Hypotheses are scored using an expert-derived five-criterion rubric on a 1-5 scale per criterion. We find that models given both the source and disease endpoint often produce the highest-scoring hypotheses, showing that LLMs can generate compelling ideas from minimal information. However, these hypotheses are less grounded in the evidence. In contrast, models given the full biological path generate hypotheses more consistent with known mechanistic relationships. We call this evidence-disciplined reasoning . To confirm this effect, we shuffled intermediate path steps while keeping endpoints fixed. Evidence grounding dropped significantly ( 𝛥 = -0.793 , p < 0.001 ), confirming models genuinely used path structure during reasoning. Our findings show that knowledge graphs support hypothesis generation in two ways: they identify biological endpoint pairs absent from the literature, and their mechanistic paths guide how LLMs reason between them.

Abstract:
Large language models (LLMs) can enhance factuality via retrieval-augmented generation (RAG), but applying RAG to every query is unnecessary when the model-only answer is reliable. This motivates cascaded RAG: each query is first handled by an LLM-only branch, escalated to a RAG fallback only if the primary branch is uncertain, and abstained from when neither branch is sufficiently trustworthy. However, calibrating such cascades stage by stage may be conservative, since the final utility depends on joint uncertainty thresholding of LLM-only and RAG. In this work, we develop BalanceRAG to certify threshold pairs at a target risk level. Given uncertainty scores from the two branches, BalanceRAG frames each threshold pair as an operating point on a two-dimensional lattice and identifies safe operating points using sequential graphical testing. This enables risk-adaptive threshold calibration, controlling the system-level error rate among accepted points, while retaining more examples. Furthermore, BalanceRAG extends to multi-risk calibration, allowing retrieval usage to be bounded together with the selection-conditioned risk. Experiments on three open-domain question answering (QA) benchmarks across multiple LLM backbones demonstrate that BalanceRAG meets prescribed risk levels, preserves higher coverage and more accepted correct examples, and reduces unnecessary retrieval calls compared with always-on RAG.

Abstract:
Large Language Models (LLMs) acquire reasoning capabilities through shared inference patterns in pre-training data, which are further elicited via Chain-of-Thought (CoT). However, whether fundamental reasoning patterns, such as induction, deduction, and abduction, can be decoupled from specific problem instances remains a critical challenge for model controllability. In this paper, we present the first systematic investigation of this problem through the lens of reasoning conflicts, an explicit tension between parametric and contextual information induced by mandating logical schemata that deviate from those expected for a target task. Our evaluation reveals that LLMs consistently prioritize sensibility over compliance, favoring task-appropriate reasoning patterns despite conflicting instructions. We further demonstrate that reasoning conflicts are internally detectable, as confidence scores drop during conflicting episodes. Probing experiments confirm that reasoning types are linearly encoded from middle-to-late layers, indicating the potential for activation-level controllability. Leveraging these insights, we steer models towards compliance, increasing instruction following by up to 29%. Overall, our findings establish that while LLM reasoning is anchored to concrete instances, active mechanistic interventions can effectively decouple logical schemata from data, offering a path toward improved controllability, faithfulness, and generalizability.

Abstract:
Steering aligns large language models (LLMs) by injecting a bias into selected activations at inference time, offering a far cheaper alternative to weight-update methods such as supervised fine-tuning or reinforcement learning. However, most existing training-free steering methods are input-independent: a single direction is fitted once and shared across all inputs. This is fundamentally limiting as different inputs occupy different regions of the activation space and admit different optimal steering directions toward the same target concept, much as the gradient with respect to a fixed loss varies from input to input. We close this gap with IDEEA (Input-Dependent stEEring via Activation cluster matching), a training-free framework for input-dependent steering. IDEEA clusters the positive and negative activation supports per attention head, and solves an optimal-matching problem to construct a set of cluster-conditional directions, all about the target concept. At inference time, it picks from this pool of directions and uses the one that best matches the input’s own activation for steering. IDEEA aligns the model toward the target concept while preserving the input’s original representation, evidence that activations encoding a concept occupy several distinct sub-regions of the representation space rather than a single one. IDEEA improves the truth × info rate in TruthfulQA by an average of 9.9% (up to 23.5%) over the best input-independent baseline.

Abstract:
Citation-based measures of scientific influence typically treat citations as uniform signals, ignoring the different roles that cited works play in a paper’s contribution. We introduce contribution-based credit scoring for research articles: a structured citation analysis that decomposes a paper’s credit between its own original contribution and the prior work it builds on. Motivated by a cooperative-game view of scientific credit, we propose the contribution tree, a hierarchical framework that conserves importance across the document structure and separates original from citation-derived contribution. To make this framework scalable, we use LLMs as noisy comparative estimators of local importance. We further extend the model to article collections by propagating contributions through weighted citation graphs, yielding corpus-level contributions and normalized influence scores. Our experiments suggest that our framework captures contribution signals beyond surface-level heuristics.

Abstract:
Fact-checking has transitioned from traditional semantic matching toward active reasoning powered by Large Language Models (LLMs), manifesting as the interplay between internal parametric knowledge and external grounded evidence. However, this paradigm introduces two critical challenges: (i) excessive reliance on external evidence even when it conflicts with correct internal judgment, and (ii) uncritical acceptance of a consensus even when both sources are erroneous. To address these limitations, we propose a Multi-Agent Arbitration Framework where a meta-cognitive Adjudication Agent (AA) orchestrates a Knowledge Agent (KA) and an Evidence Agent (EA). Through hybrid reinforcement and imitation learning, the AA perceives conflict signals between KA and EA to adaptively calibrate judgmental autonomy against evidentiary support. Upon detecting a potentially erroneous consensus, the AA invokes a Reflective Rejection mechanism to scrutinize and disrupt deceptive alignment. Empirical evaluations on Snopes and PolitiFact demonstrate that our framework consistently outperforms state-of-the-art baselines. Notably, it effectively mitigates over-reliance errors while maintaining high fidelity in correct-consensus scenarios, recovering 26.2% of dual-error cases where static systems typically succumb to superficial consistency. Our code is publicly available at https://github.com/Zhang-Gorden/Perceive-Then-Adjudicate.

Abstract:
Reinforcement learning is increasingly used for code-centric software engineering tasks, including code generation, understanding, repair, testing, and optimization, especially with the rise of large language models and autonomous agents. A core challenge in these settings is reward design. Unlike standard RL domains with clear scalar objectives, software tasks involve competing goals such as correctness, security, efficiency, and readability, which are difficult to capture with a single reward. As a result, RL-for-SE systems rely on heterogeneous signals, including compilation results, unit tests, coverage metrics, retrieval scores, and learned preferences. Yet this work remains scattered across tasks and communities. To our knowledge, this survey provides the first systematic review of reward engineering for RL in software tasks. We organize prior work by reward source, granularity, and aggregation. We then distill the findings into practical guidance, including a decision matrix, a design guide, and a reward engineering reporting standard for future RL- for-SE systems.

Abstract:
For multi-hop queries in retrieval-augmented generation (RAG), it is crucial to accurately obtain evidence pieces. In this study, we propose a training-free method for multi-hop query retrieval, named Intrinsic Navigating-based Non-Generative Retrieval (IN-Retriever). Instead of generating intermediate reasoning text, the method extracts the bridging clue from the internal attention of the embedding model based on the current context and retrieved candidate documents. During iterative clue extraction and document retrieval, IN-Retriever strips away revealed clues from the embedded query vector and then reranks candidate documents in a targeted manner, thereby steering retrieval toward hidden next-hop evidence. In the experiments, we confirm the feasibility and effectiveness of using intrinsic attention signals to guide external multi-hop retrieval. Experimental results show that our method precisely collects sufficient documents to support multi-hop queries and significantly outperforms sparse and dense retrieval methods.

Abstract:
Scaling test-time computation improves the reasoning ability of large language models (LLMs), but the resulting latency and cost limit its use in latency- and cost-sensitive scenarios. We present BudgetThinker, a framework for reasoning-budget adherence that provides online control of chain-of-thought (CoT) length during decoding. Rather than relying on a one-shot prompt instruction, BudgetThinker periodically inserts specialized tokens that communicate the remaining reasoning budget. A two-stage training pipeline, consisting of supervised fine-tuning and curriculum-based reinforcement learning, teaches the model to interpret these signals and adapt its reasoning pace. Across the evaluated 1.5B and 7B text backbones, a 7B multimodal backbone, and a 1.5B code-reasoning model evaluated on HumanEval, BudgetThinker improves budget adherence and accuracy–efficiency trade-offs over competitive baselines. The controlled comparisons isolate the contribution of online ratio signaling from the shared training pipeline. Code is available at https://github.com/MobileLLM/BudgetThinker.

Abstract:
Large Language Models (LLMs) achieve strong reasoning performance by generating intermediate reasoning trajectories through autoregressive decoding. Existing approaches such as self-consistency, tree-based reasoning, and critical token analysis improve reasoning by exploring diverse trajectories or identifying influential tokens. In this work, we identify a distinct phenomenon in LLM reasoning: critical positions — locations where re-sampling continuations disproportionately influences the final outcome, independent of the specific token at that position. Motivated by this, we propose Metacognitive Influence Tracing (MIT), the first unsupervised framework for identifying critical positions from a single reasoning trace using only attention structure. MIT models reasoning trajectories as influence graphs and estimates propagated influence through a diffusion process, enabling the discovery of structurally important reasoning positions without supervision or contrastive trajectories. Building on these insights, we further propose Adaptive Critical Sampling (ACS), an intervention framework that selectively re-samples at critical positions instead of regenerating entire reasoning traces. Across six benchmark tasks and five models, ACS improves accuracy by an average of 7.78 points over CoT while reducing computational cost by 59.83% over Self-Consistency.

Abstract:
Existing pedestrian navigation systems often provide direction-only instructions with limited visual confirmation at complex intersections or along long straight paths. We study coordinate-free landmark extraction from dashcam image sequences. The model returns a set of landmarks with structured attributes (e.g., category and description), or abstains when visual evidence is insufficient. Although general MLLMs can recognize street-scene objects, they do not reliably follow expert rules for filtering ambiguous candidates. Moreover, a false landmark can mislead a pedestrian, making precision more important than exhaustive coverage in this task. We therefore develop a progressive alignment framework, supported by a landmark annotation and augmentation pipeline that converts navigation experts’ selection criteria into staged supervision signals. We further propose a hard-gated bipartite matcher for box-free evaluation and a navigation-aware risk-sensitive reward that penalizes false positives more heavily than abstentions without inducing blanket silence. On the held-out benchmark, our model achieves 92.29% precision while maintaining practical coverage, outperforming strong MLLM baselines. A 30-day A/B test with 2M users on the Amap platform further shows that landmark-augmented instructions reduce departure-confirmation latency by 23.5%. Together, these results demonstrate that risk-aware alignment enables reliable, abstention-capable landmark extraction for navigation.

Abstract:
Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than an object of analysis. Consequently, little is known about the outputs themselves, partly because large-scale, high-quality collections of frontier-model misbehavior are difficult to obtain. To address this gap, we introduce HarmProfile, a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting content distribution as a model-level risk profile. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, we find that frontier LLMs reliably produces harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. The code and dataset resources are publicly available at https://github.com/fresh-ma/HarmProfile.

Abstract:
Multimodal large language models (MLLMs) are increasingly used as evaluators, yet their reliability in professional assessment tasks that require expert judgment remains unclear. We investigate this challenge in the context of assessing psychotherapeutic interpersonal skills and introduce MIS-Bench , a Multimodal Interpersonal Skills (MIS) benchmark comprising 996 psychotherapy response videos annotated across 8 dimensions of Facilitative Interpersonal Skills. Across 9 MLLMs with multiple modality and prompting settings, we find that current models show only modest agreement with human experts, inconsistent gains from multimodal input, and limited benefits from reasoning-based prompting. To mitigate this gap, we propose MIS-RAFT , a regression-aware fine-tuning method inspired by RAFT and tailored to fine-grained interpersonal skill scoring at one-decimal precision. MIS-RAFT addresses the mismatch between autoregressive token prediction and scalar-valued expert assessment, significantly improving agreement with human ratings. Overall, MIS-Bench reveals a clear gap between general multimodal capability and expert-level interpersonal judgment, while MIS-RAFT offers a promising path toward more reliable model-based assessment.

Abstract:
Code reasoning is essential for building AI systems that can understand, generate, and verify complex software. Yet most open-source code corpora focus on isolated snippets or single functions, failing to capture the long-range dependencies and cross-file interactions that define real-world repositories. We introduce MegaCode, a 300B repository-level corpus designed to support structured code reasoning and long-context modeling. MegaCode preserves full project structure—including build scripts, configuration files, and inter-file links—and is constructed through a multi-stage curation pipeline combining deduplication, rule-based filtering, and model-based quality assessment. Models pretrained on MegaCode demonstrate consistent gains on code reasoning and long-context benchmarks compared with those trained on flattened code datasets. Also, we validate the effectiveness of adding repo relevent infomation such as LSP, repo structure and repo summary. These results highlight the importance of maintaining repository structure for next-generation code intelligence. We release the dataset, curation pipeline, and evaluation suite to facilitate research on large-scale code pretraining and repository-level understanding.

Abstract:
Ellipsis in comparative constructions provides a challenging test of semantic interpretation because comprehenders must reconstruct implicit constituents that are not overtly expressed. This study investigates whether large language models (LLMs) can capture the structured semantic inferences required for understanding Chinese comparative constructions. We introduce a linguistically motivated evaluation framework targeting two essential components of comparative interpretation: (i) identification of comparative arguments and (ii) recovery of elliptical constituents. Based on 2,076 naturally occurring Chinese bi-comparative sentences, we construct an annotated resource with comparative argument boundaries, recovered elliptical constituents, and five comparison dimensions representing distinct semantic bases of comparison. We evaluate several representative LLMs, including DeepSeek-V3.1, GPT-4o, and Gemini-2.0-Flash. While all models reliably identify overt comparative arguments, they exhibit substantial degradation when reconstructing implicit constituents. These results suggest that current LLMs can capture surface-level structural regularities but still face challenges in recovering implicit semantic representations required for human-like interpretation. Our resource provides a controlled linguistic testbed for studying the relationship between linguistic structure, semantic inference, and computational models of language understanding.

Abstract:
Skeleton-based reasoning prompting is a promising training-free approach for structuring LLM reasoning, but prior work largely assumes an English-centric setting. We propose the Language-Aware Skeleton Exploration Framework () to study skeleton-language choice in multilingual mathematical reasoning. Across math benchmarks, model scales, and languages, we show that English skeletons yield a small positive tendency on average, most visible for smaller models and low-resource languages. However, few language-level gains remain significant after correction, and English is not universally optimal. Combining greedy decoding, multi-rollout evaluation, translation ablation, and cross-benchmark validation, we further find three patterns of skeleton-language effects: directionally consistent, evaluation- and benchmark-dependent, and asymmetric negative. These effects cannot be fully explained by generation quality alone. Overall, skeleton language is a context-dependent design variable that requires multi-level exploration. All resources are released[].

Abstract:
LLM-based scientific agents have shown strong capacity for autonomous research, yet their safety layers remain structurally divorced from core reasoning: they inspect pipeline outputs rather than shaping the deliberation that produces them. This separation opens two failure modes: safety signals accumulated at one stage are discarded before the next, and sequences of individually benign tool calls can compose into harmful outcomes that no single-step filter detects. To address these challenges, we introduce SciTrace, a framework that weaves safety reasoning into every stage of the scientific agent pipeline. SciTrace couples two complementary mechanisms: a Safety-Intrinsic Reasoning Loop (SIR) that maintains a cumulative risk state across the Thinker, Experimenter, Writer, and Reviewer stages through joint task-and-safety deliberation, and a Compositional Tool-Chain Verifier (CTV) that performs trajectory-aware safety checks before execution, catching risks that surface only across multi-step tool sequences. Evaluated on 240 high-risk research tasks and 120 tool-related risk tasks spanning six scientific domains, SciTrace achieves state-of-the-art (SOTA) safety among compared frameworks across four backbone models: it consistently improves tool call safety and adversarial robustness while preserving scientific output quality, and it uncovers 78.8% of the compositional tool-chain escapes that single-step monitors miss. The project website is available at [https://opensciagent.github.io/SciTrace/](https://opensciagent.github.io/SciTrace/).

Abstract:
Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query. However, real clinical scenarios often require multiple calculators for joint evaluation, nested-calculator calculation, and fuzzy queries that do not directly specify the target calculator. To this end, we propose a new medical calculation benchmark, MedCalc-Pro, which covers three progressively challenging task settings: single-calculator, multi-calculator, and nested-calculator calculation settings. MedCalc-Pro contains 2,268 real-world clinical cases, covering 77 medical calculators across 14 clinical departments. Meanwhile, to address the limited performance of existing frameworks and methods in complex clinical scenarios, we further propose a more generalizable agent framework that supports multi-calculator selection and nested-calculator calling, while suppressing parameter error propagation through structured validation and evidence audit. We conduct systematic comparisons across open-source, closed-source, and medical-specialized LLMs, and the results show that our framework achieves the best performance across all three task settings. This work provides a new benchmark and method for evaluating and applying LLMs in challenging medical calculation scenarios. The dataset and code are publicly available at https://github.com/easymoney7774/MedCalc-Pro.

Abstract:
With the growing availability of open-source task-specific models, model merging has become a practical way to combine multiple models into one. However, existing studies mainly focus on fully fine-tuned models and overlook widely used LoRA modules. We show that full fine-tuning merging methods perform poorly on LoRA due to its larger cross-task magnitude disparities, which distort merged parameters and degrade performance. To address this, we propose DO-Merging (Decouple and Orthogonalize), a data-free framework for LoRA merging. Specifically, we decompose model weights into magnitude and direction components and merge them independently, preventing magnitude imbalances from distorting the merged direction. We further introduce a layer-wise gradient descent method with orthogonal constraints that reduces inter-task interference among direction components without requiring any training data. We provide theoretical analysis supporting both components. Extensive experiments across vision, language, and multi-modal domains show that DO-Merging consistently outperforms existing methods, achieving up to 5% gains on vision tasks, with negligible additional cost. Both parts can be integrated into existing merging methods as plug-and-play enhancements.

Abstract:
Reasoning efficiency in Text-to-SQL is an important yet underexplored problem. Chain-of-Thought (CoT) reasoning models simplify traditional Text-to-SQL workflows and significantly improve the ability to handle complex queries. However, existing methods typically generate long reasoning processes for all queries. This leads to high token consumption and inference latency, increasing system cost and limiting practical deployment. To address this issue, we propose Ada-SQL, an adaptive reasoning model for Text-to-SQL that jointly optimizes reasoning mode selection and token budget control via a two-stage reinforcement learning framework coupled with a token budget model. This enables the model to adaptively decide whether to reason and how much reasoning budget to allocate based on query difficulty. Experimental results show that Ada-SQL reduces output token consumption by more than 65% while maintaining competitive execution accuracy, achieving a favorable performance-efficiency trade-off. Code and models will be released at https://github.com/sxhText2SQL/Ada-SQL.

Abstract:
Retrieval-augmented generation (RAG) connects large language models (LLMs) to external knowledge, but single-round retrieval is often insufficient for complex multi-hop questions. To improve search on complex tasks, most existing works combine multi-round retrieval with reasoning through end-to-end training. While these approaches improve problem-solving performance, they still face challenges in task reasoning and model training. First, due to the lack of an explicit subgoaling mechanism, most existing RAG methods may produce ambiguous retrieval execution paths, leading to the degradation of model performance. Second, a few existing RAG methods have started to incorporate planning and execution into the model training process. However, they typically rely on a single final reward, causing hierarchical credit entanglement between planning and execution and preventing the model from clearly separating plan errors from retrieval errors. To address these limitations, we propose APEX-Searcher, a coordinated planner–executor framework that optimizes a recursive planner with RL to accommodate multiple valid, dependency-aware decompositions while training the executor with SFT under clearer subtask-level supervision. Extensive experiments show consistent gains in both multi-hop RAG and task planning across benchmarks.

Abstract:
AI-generated-text detectors are evaluated as classifiers, yet consequential deployments use detector outputs as evidence that people inspect, trust, contest, or use to escalate decisions. We call this mismatch the classification–evidence gap. The gap makes scalar detector scores a weak deployment interface: they hide the basis of a judgment, invite overtrust, and can turn false positives into accusations. We argue that AI-text detection should be evaluated as evidence-grounded decision support rather than as authorship proof. We introduce AGENT-XTrust, a guideline-expert detector that returns direct MetaAgent labels without target-domain scalar-cutoff calibration and converts each document-level judgment into inspectable evidence records containing local labels, rationales, confidence estimates, and linked spans. Across three domains and four generators, AGENT-XTrust preserves strong binary accuracy while producing inspectable, contestable evidence. In a controlled fixed-stimulus user study, participants assigned to the full AGENT-XTrust system-and-interface package achieved 19.8 and 11.3 points higher decision accuracy than those assigned to the score-only and score-plus-explanation packages, respectively; overtrust on each condition’s detector-wrong cases was 17.8 and 18.3 points lower. These results suggest a broader evaluation target for AI-text detection: not only whether a detector is correct, but whether complete detector-and-evidence packages support reliable human decisions under uncertainty.

Abstract:
Defense is a knowledge-intensive domain that requires precise understanding of specialized terminology, doctrinal concepts, operational procedures, and evolving military events. Although recent work has explored language technologies for military applications, existing efforts remain fragmented: they are often task-specific, rely on limited adaptation pipelines, or lack comprehensive defense-domain evaluation. In this paper, we present SALUTE, an end-to-end framework for benchmarking and adapting LLMs for the defense domain. SALUTE integrates Salute-Corpus, a curated corpus from open-access U.S. military doctrine and government documents; Salute-Conv, a grounded instruction dataset from doctrinal sources and decade-long defense news; Salute-Pref, a defense-aware preference dataset; and Salute-Bench, a rigorously filtered benchmark for evaluating defense-domain understanding and reasoning over doctrine and defense news. Based on these resources, we train Salute-LLM through multi-stage post-training with continual pretraining, supervised fine-tuning, and preference alignment. Extensive experiments show that Salute-LLM achieves strong defense-domain performance while retaining competitive general capabilities, demonstrating the effectiveness of SALUTE as an end-to-end framework for defense-domain LLM adaptation.

Abstract:
Membership inference attacks (MIAs) are increasingly used to audit whether protected examples were included in language model fine-tuning. Yet for fine-tuned LLMs, high attack AUC alone is not reliable evidence of data use: a score may separate members from non-members due to exposure, but also due to length, base model difficulty, typicality, or domain alignment. We propose DeMIA, a diagnostic membership inference framework that learns separate channels for membership scoring and difficulty modeling using likelihood features from the base and target models. DeMIA combines membership prediction with matched-pair ranking, base-difficulty anchoring, and adversarial length control, encouraging the final score to preserve membership-relevant evidence while reducing direct shortcut reliance. Across GPT-Neo-125M, Qwen2.5-1.5B, and Llama-3.1-8B on Wiki10k, AGNews10k, and XSum10k, DeMIA outperforms scalar likelihood, reference-calibrated, neighborhood, calibrated Min-K, and DF-style baselines while maintaining strong low-FPR recall. The signal remains strong on matched hard subsets, while blind controls using length, base loss, or other features from the base model remain near chance. These results show that fine-tuned LLM membership signals can persist under the tested nuisance controls and should be evaluated with reliability diagnostics rather than raw AUC alone.

Abstract:
Existing code-generation benchmarks score a single mapping from a complete prompt to a one-shot output. However, real web development is different. Users seldom write a full spec at the start; many requirements only become clear once they look at an intermediate result and react to it. We present Asuka-Bench, a benchmark that pairs underspecified user intent with multi-round refinement, grounded in browser-rendered behavior. Each task is resolved through a closed loop: a Code Agent generates a web project, a UI Agent executes test cases on the deployed site, and a User LLM turns evaluation outcomes into natural-language feedback for the next round. The benchmark comprises 50 web tasks with 784 evaluation criteria and 2402 expected outcomes. We benchmark 8 LLMs across 2 agent frameworks. The results separate models clearly: weighted Task Pass Rate varies by 38 percentage points and models also differ substantially in their ability to repair from feedback. Asuka-Bench is also far from saturated: even the strongest model completes only 52% of projects after three rounds.

Abstract:
Clinical diagnosis is an active evidence-seeking process in which clinicians acquire evidence, update competing hypotheses, and decide when the available evidence is sufficient for diagnosis. Yet many medical diagnosis systems built around large language models (LLMs) still formulate diagnosis as static case-to-answer prediction, with limited support for evidence acquisition. Agentic LLMs offer a dynamic alternative through tool use and intermediate diagnostic trajectories, but existing systems often under-specify how patient evidence should be exposed, scaffolded, and controlled at runtime. We introduce EviDx, an evidence-aware active diagnosis framework that pairs patient-specific diagnostic environments with a clinical diagnostic scaffold and an observer-guided runtime harness. In EviDx, ℰ -Synthesis constructs interactive environments from raw clinical cases; the scaffold organizes role-specialized agents, evidence tools, and evolving evidence states; and the harness regulates diagnostic termination by tracking uncertainty and evidence coverage. A 3-level evaluation pyramid assesses execution robustness, reasoning dynamics, and diagnostic outcomes. Experiments show that EviDx improves diagnostic performance and process stability while revealing model-dependent capability boundaries.

Abstract:
Retrieval-augmented generation (RAG) empowers large language models (LLMs) with external knowledge but remains vulnerable to faithfulness hallucinations—answers that contradict retrieved evidence. We introduce RADAM, a lightweight post-processing framework that mitigates this issue via fine-grained hallucination detection and guided revision. RADAM leverages a single diagnostic critic, optimized via preference tuning on taxonomy-guided data, to generate structured critiques encompassing both precise error spans and explicit revision suggestions, which guide the base model toward more faithful outputs. Experiments on real-world datasets demonstrate that RADAM consistently achieves the lowest average hallucination rate across the evaluated methods, while largely preserving response quality and achieving out-of-domain performance comparable to enterprise-grade LLMs.

Abstract:
Continual pretraining adapts a language model to new domains but can degrade performance on earlier domains. We study this degradation through component-level interventions in Transformer blocks. Our main observation is that forgetting interventions have sharply different profiles across Q/K, V/O, and MLP components. In SmolLM2-360M continual pretraining over six domains, V/O rollback improves previous-domain Wiki perplexity while leaving new-domain perplexity nearly unchanged (-5.72 +/- 0.08 Wiki PPL diff; new-domain range -1.06 to +0.76), MLP rollback improves Wiki but severely damages all new domains (-6.29 +/- 0.12 Wiki; +13.95 to +20.75 new-domain damage), and Q/K rollback shows no recovery effect. Training-time V/O freeze reproduces the selective pattern: it reduces Wiki forgetting (-2.98 +/- 0.13, 3 seeds in the stress order; -2.84 +/- 0.29 across 3 seeds x 2 orders) without measurable loss in current-domain learning efficiency and without compensatory MLP/Q/K weight changes. Same-parameter-count Q/K freeze has the opposite direction (+3.17 +/- 2.53), and random attention-subset freeze scales with the included V/O fraction. Finally, inference-time inter-checkpoint activation transplantation modulates forgetting bidirectionally without changing weights (forward -6.27 +/- 0.13, reverse +3.26 +/- 0.09), with the activation-level effect carried primarily by o_proj output rather than by the full V/O projection bundle. We present these results as a robust component-level intervention profile, not as proof that V/O is the unique causal storage site of forgetting.

Abstract:
Existing methods for evaluating the factuality of long-form LLM responses decompose text into atomic claims and verify each independently, retrieving evidence separately for every claim. This atom-independence assumption causes substantial retrieval redundancy, because adjacent claims in coherent text often draw from overlapping sources, a property we term semantic locality. We propose SegmentFact, a deterministic factuality assessor that partitions responses into semantic segments, extracts claims within each segment’s local context, and propagates retrieved evidence forward through a shared pool. Experiments on multiple benchmarks show that SegmentFact significantly reduces retrieval calls while maintaining accuracy comparable to existing detections.

Abstract:
Instruction-tuned LLMs are increasingly converted into reasoning models through post-training to improve multi-step task performance. This conversion is usually optimized for reasoning accuracy, without explicitly preserving the instruction-tuned model’s alignment behavior, such as safe refusal, bias avoidance, and privacy protection. We ask: does this conversion preserve alignment? We study this question through a trustworthiness audit and find that it is not behavior-preserving by default. For a systematic analysis, we compare reasoning models produced via supervised fine-tuning, RL-based post-training, and distillation against matched instruction-tuned baselines across six trustworthiness dimensions: safety, toxicity, stereotyping and bias, machine ethics, privacy, and out-of-distribution robustness. We observe that reasoning models often improve on mathematical reasoning benchmarks but exhibit alignment regressions, including increased toxicity, amplified stereotyping, miscalibrated refusal, and contextual privacy leakage. These regressions are consistent with behavioral drift from the instruction-tuned baseline, measured by KL divergence. Our results show that reasoning-model releases should report trustworthiness metrics alongside capability gains.

Abstract:
LLM agents are increasingly used for live forecasting, where they retrieve up-to-date information and produce estimates for unresolved future events. However, current agentic forecasting often relies on implicit narrative aggregation: agents collect evidence, discuss it in prose, and often assign a probability without an explicit update path from evidence to forecast. This limits both forecasting accuracy and auditability. We propose AuditForecast, an agentic scaffold for structured probabilistic forecasting. AuditForecast first anchors the forecast with a suitable quantitative baseline model, uses model-guided data retrieval to derive a base probability, and then applies situational factor updates outside the model’s scope through mechanical aggregation in odds space. This turns forecasting from a prose-based judgment into a structured process with explicit intermediate objects. Across multiple live forecasting benchmarks, AuditForecast improves forecasting accuracy and calibration relative to strong agentic baselines, surpasses market-implied references in several settings, and outperforms substantially more expensive deep-research agents while remaining Pareto-dominant in the cost–accuracy tradeoff. Beyond performance gains, AuditForecast produces an auditable forecasting report that makes forecast construction explicit and supports systematic post hoc analysis.

Abstract:
ASR-oriented audio-language models (ALMs) encode speech into long audio token sequences for LLM decoding, substantially increasing the inference cost of transcription. This motivates token compression methods that reduce audio token sequence length while preserving the acoustic evidence needed for transcription. However, existing compression methods often inherit vision-language criteria that are poorly matched to temporally ordered audio representations, or require attention-map access outside the standard FlashAttention path. Through a multi-axis geometric analysis, we show that ALM audio representations exhibit two distinct representation regimes: a locally redundant encoder regime and a dispersed post-projector regime. Motivated by this geometric shift, we propose RAMP (Regime-Aware Merge-Prune), a training-free method for FlashAttention-compatible audio token compression that applies bipartite L 2 merging in the encoder and centroid-distance-guided pruning after the projector. Across seven ASR benchmarks and two primary ALMs, RAMP achieves the lowest average WER multiplier among the evaluated training-free compression baselines at every evaluated retention ratio. At the most aggressive setting ( r=0.4 ) on Audio-Flamingo-3, RAMP attains a 2.36× average WER multiplier, compared with 4.26× for the best non-RAMP baseline, while reducing time-to-first-token (TTFT) by 24.8% over Vanilla.

Abstract:
Reasoning-conditioned embeddings can improve retrieval by progressively refining semantic constraints. However, additional reasoning may introduce over-generalized or semantically noisy abstractions that weaken retrieval specificity. Existing methods largely supervise reasoning through final retrieval outcomes, providing limited guidance over the refinement process itself. We propose AURA, a framework that directly supervises intermediate reasoning states according to how well they separate relevant documents from hard distractors. AURA further trains a lightweight utility estimator over prefix-conditioned embeddings to adapt reasoning depth at inference time. On the reasoning-intensive BRIGHT benchmark, AURA-4B achieves 34.7 average nDCG@10 while using 2.8 reasoning steps and 96 generated tokens per query. Under matched deployment conditions, adaptive stopping reduces latency from 4.6 to 0.76 seconds relative to full-trajectory reasoning. Results on MS MARCO further suggest that this specialization need not substantially degrade conventional in-domain retrieval.

Abstract:
LLM judges have emerged as a key approach to evaluating open-ended scientific outputs that resist automated verification, yet their faithfulness in reference-free settings remains underexplored, particularly for fields such as novel materials synthesis where no ground-truth reference exists. Such judges produce false positives that accept scientifically invalid recipes and false negatives that penalize semantically equivalent ones in unfamiliar notation. We introduce ASSAY, a probe suite built on domain-grounded augmentations of ground-truth synthesis recipes, and reveal that these two failure modes form a systematic Stability–Validity trade-off across thirteen judge models. To resolve this trade-off, we train SynthJudge-8B via DPO on contrastive pairs derived from the same augmentations, improving judgment accuracy from 51.4% to 85.1% while maintaining agreement with domain experts. We release our probe suite and trained model to support faithful automated evaluation in scientific fields beyond materials synthesis at https://github.com/SoyeonHH/ASSAY.

Abstract:
Despite extensive alignment efforts, Large Language Models (LLMs) remain vulnerable to generating unsafe content under adversarial prompting, yet the internal mechanisms by which safety behaviors are implemented remain poorly understood. We study LLM safety from a mechanistic interpretability perspective and characterize a multi-stage safety circuit that organizes refusal behavior, consisting of (i) Harmful Detection Heads that respond to harmful inputs, (ii) Safety Neurons that mediate and stabilize safety signals in the residual stream, and (iii) Refusal Heads that translate these signals into safe response generation. Using targeted attention-head and neuron-level interventions, we provide causal evidence consistent with this circuit organization, showing that suppressing upstream Harmful Detection Heads disrupts downstream refusal behavior and that safety neurons mediate this interaction. We validate that this decomposition recurs across multiple LLM architectures and adversarial attack settings, and use simple, architecture-preserving weight scaling as a mechanistic probe to test its functional relevance. Across six LLMs, circuit-guided scaling improves safety rates under attacks by 26.5%, while incurring only a 1.7% accuracy drop across four standard benchmarks. Overall, our results support a circuit-level interpretation of LLM safety and suggest that mechanistic abstractions can reveal stable and transferable patterns underlying aligned behavior.

Abstract:
Personality alignment tailors Large Language Models (LLMs) for personalized interactions. Most existing studies focus on prompting, post- training, or representation steering to elicit trait expression, but they often compromise the general capabilities like reasoning and truthfulness. In this work, we empirically verified the existence of the Personality Alignment Tax (PAT) that distribution shifts and conflicting objectives during personality updates overwrite pre-trained competencies. To mitigate this tax, we propose a Personality-Orthogonal Gradient Projection (POGP) plug-in that preserves the capability in personality alignment. Specifically, POGP estimates the general-capability subspace from capability reference benchmarks and projects personality gradients onto the orthogonal complement before updating. This isolates and neutralizes conflicting gradients while maintaining the pre-trained reasoning capabilities. Experiments on the BIG5-CHAT dataset with two backbone LLMs demonstrate that POGP achieves a stronger personality plasticity and capability stability trade-off than baselines.

Abstract:
Prompt echoing is a recognized failure mode of instruct language models, in which a model instead of generating a response, mirrors the provided prompt, even though it did not receive a specific instruction to do so. Is this phenomenon a sign of the model leaking the content of its training dataset, or is it rather caused by a misaligned behavior of the internal induction/copying mechanisms? We investigate prompt echoing small language models from different families (Gemma, Llama, Qwen, SmolLM and OLMo) and show that echoing prompts are likely to have partial overlap with the training dataset but the phenomenon is primarily driven by the model’s induction heads.

Abstract:
Agentic AI systems are increasingly embedded in scientific workflows, including literature synthesis, hypothesis generation, tool use, writing, and peer review. However, existing surveys and evaluations often emphasize task performance while underexamining how humans and agents coordinate under different epistemic conditions of scientific work. This survey introduces CSCW × Kuhn, a coordination-aware framework that connects computer-supported cooperative work dimensions with distinct scientific contexts. Using this framework, we synthesize recent human–agent systems and benchmarks to examine how coordination demands and failure risks may vary across contexts. Our synthesis identifies five recurring failure modes: procedural drift, grounding substitution, evaluative flattening, proxy optimization, and epistemic black-boxing, and argues for evaluations that move beyond task accuracy and toward grounding, provenance, disagreement preservation, and long-horizon scientific reliability. Our corpus and coding artifact are available at https://github.com/jenlia024/CSCWxKuhn-artifact.

Abstract:
The enhanced capabilities of LLM-based agents bring an urgent need to address safety risks in planning and tool use. Due to the helpfulness–harmlessness trade-off in LLM alignment, agents typically also inherit the flaw of Over-Refusal, which is a passive failure mode. However, the proactive planning and action capabilities of agents introduce another emerging danger on the opposite side of this trade-off. We term this phenomenon Toxic Proactivity: an active failure mode in which an agent, driven by the optimization for Machiavellian helpfulness, prioritizes utility over ethics. Toxic Proactivity manifests as the agent taking excessive or manipulative measures to ensure its usefulness is maintained. Existing research pays little attention to identifying this behavior, as it often lacks the subtle context required for such strategies to unfold. To reveal this risk, we introduce a novel evaluation framework based on dilemma-driven interactions between dual models, enabling the simulation and analysis of agent behavior over multi-step behavioral trajectories. Through extensive experiments with multiple mainstream LLMs, we demonstrate that Toxic Proactivity is a widespread behavioral phenomenon and reveal two major tendencies: direct violation and strategic misalignment. We further present a systematic benchmark for evaluating Toxic Proactive behavior across contextual settings.

Abstract:
Cultural adaptation, rewriting text for a different cultural audience, requires more than lexical substitution. A model must decide what to change, whether each replacement is locally valid, and whether it preserves the original item’s functional role. We formalize this as function-preserving cultural transfer and introduce a dataset, a taxonomy, and an evaluation framework constructed around that principle. CulturalAdapt-6K identifies 6,000 culturally rich texts in four discourse domains, while the Cultural Rewriting and Adaptation Failure Taxonomy (CRAFT) classifies errors into five categories. To mitigate the Anglocentric priors of current models, our proposed framework anchors each culturally significant item in a Wikidata-linked function frame, capturing its contextual role rather than surface denotation. Across 191,968 adaptations by eight instruction-tuned LLMs, we find that the models that substitute the most are not the ones that substitute best. High localization increases failure exposure, while conservative retention increases validity scores, yet both converge to the same overall functional quality. Current generative models can certainly dress text in a target culture’s vocabulary but ultimately cannot reliably preserve the contextual roles the vocabulary is meant to serve.

Abstract:
Legal argument mining supports passage classification, retrieval, and argument completion. This work introduces an expert-annotated dataset of 42 U.S. federal tax opinions on corporate reorganizations under I.R.C. §368. To our knowledge, it is the first expert-annotated, tree-structured argument corpus for this domain. Explicit spans receive one of five functional labels: Rule, Analysis, Conclusion, Background Facts, and Procedural History. Rule, Analysis, and Conclusion spans can be linked into directed support trees, while Background Facts and Procedural History serve a contextual function. The corpus provides span-based, sentence-based, flat, and tree-structured representations. Agreement analysis shows that functional node labels are more reliable than directed support edges and implicit intermediate conclusions. Directed-path agreement is stronger than direct-edge agreement, which indicates that broad reachability is more stable than exact local decomposition. Classification experiments show that functional labels are learnable under case-disjoint evaluation. Retrieval experiments show that supervised fine-tuning improves within-case retrieval. However, cross-case generalization remains weak. The dataset supports legal passage classification and provides a conservative benchmark for structured argument mining in U.S. federal tax case law.

Abstract:
A key goal in psycholinguistics is to quantify real-time language processing effort. A notable amount of literature links the processing effort to the amount of unexpected information carried by the new input. However, such theories fail to capture the dynamic evolution of comprehenders’ interpretation which may involve revision and operate under cognitive constraints. We propose peak information flux, a metric that better captures the cost of reanalysis through a resource rational framework. We characterize language comprehension as continuous belief updating under finite information processing capacity and that the maximal rate of belief change signals language processing effort. We validate the performance of peak information flux on Syntactic Ambiguity Processing and Natural Stories corpora featuring self-paced reading time and a variety of eye-tracking metrics representing different stages of lexical processing. Peak information flux predicts reading behavior across multiple measures and complements surprisal in measures associated with reinterpretation and recovery. Additional analyses reveal that garden-path constructions exhibit delayed and asymmetric information flux dynamics, supporting the hypothesis of a sustained probabilistic revision process. Our model provides an information-theoretic metric for language comprehension under resource constraints.

Abstract:
While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the behavioral decisions a model learns. We introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior. SMDA fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a behavior, then decomposes how each supervised fine-tuning (SFT) example shifts that policy through feature-activation (ΔX) and output-probability (ΔY) pathways. We distill a symbolic policy for refusal in Llama-3.2-3B-Instruct and analyze 200 SFT pairs. Our analysis reveals (1) the symbolic policy’s coefficients expose systematic gaps in the base model’s safety behavior, such as religious stereotyping; (2) per-feature ΔX/ΔY decomposition mechanistically explains why harmful and harmless pairs exert qualitatively different influences on certain features; and (3) individual pairs routinely exhibit cross-feature interference, allowing SMDA to identify pairs whose dominant effect falls on unintended features. Retraining without such pairs improved accuracy by up to 4.5 percentage points over full fine-tuning. These results show that combining mechanistic interpretability with data attribution yields a diagnostic tool both more fine-grained than black-box influence functions and more scalable than manual circuit analysis.

Abstract:
Self-evolving agents improve future behavior from interaction experience, yet existing evaluations typically optimize under fixed execution conditions and do not test recovery after those conditions change. To address this gap, we introduce PACE-Bench (Physics Adaptation via Code Evolution), a simulator-grounded benchmark of 144 source-to-target adaptation pairs across six physics domains. Each pair links a source environment to a mutated target environment with the same goal and interface. A code-driven design that succeeds in the source fails in the target, where agents must iteratively adapt it into a working target design using diagnostic sandbox feedback within a limited attempt budget. We compare ten self-evolving methods from four paradigms. The benchmark remains far from saturated: Reflexion + Qwen3-14B succeeds on only 35.9% of full-benchmark pairs, while GPT-5.5 solves 66.7% of the Statics subset under the full budget. Together, these results show that simulator-grounded reflection is more reliable than unverified self-revision, while memory anchors agents to early designs and broad tree search explores without converging. Even revealing exact physical changes does not raise the performance ceiling, pointing to mechanism redesign rather than parameter inference as the central bottleneck. Data and code are available at https://github.com/thunlp/PACE-Bench.

Abstract:
In-context learning (ICL) enables large language model (LLM) agents to improve decisions using interaction history, yet it remains unclear whether such improvement reflects refined internal reasoning or mere extrapolation of statistical patterns. To disentangle these mechanisms, we study LLM agents in multi-agent incomplete-information games that require recursive belief reasoning. By constructing a public goods game and manipulating the statistical structure of historical feedback, we evaluate decision quality against a history-independent rational expectations equilibrium (REE) benchmark. Our experiments reveal that when historical statistical patterns are disrupted, the benefits of longer context largely vanish, degrading decision quality to the no-context baseline in a way sharply amplified by stronger strategic interdependence. These results suggest that, in such strategic environments, ICL behavior is more consistent with statistical extrapolation than with strategic reasoning. Our work extends the mechanistic study of ICL to strategic multi-agent settings, introduces REE as a diagnostic tool for distinguishing reasoning from extrapolation, and provides a reusable framework for probing the boundaries of LLM reasoning in recursive belief tasks.

Abstract:
Google’s Agent-to-Agent (A2A) protocol transforms multi-agent collaboration from a closed pipeline into an open and dynamic ecosystem. A2A-based multi-agent systems typically depend on a central orchestrator to coordinate task allocation across agents. However, this paradigm exposes a new attack surface at the agent-orchestration layer: an agent provider can forge an adversarial agent profile to bias the orchestrator’s decisions. We find this risk and propose OrcaJack, an agent orchestration hijacking attack. By injecting a deceptively appealing agent profile into the agent pool, OrcaJack causes user queries in a target domain to be consistently routed to the malicious agent. We formulate the adversarial agent profile construction process as a discrete optimization problem and solve it with a domain-skill-driven genetic algorithm. Across multiple domain benchmarks and orchestrator LLMs, Orcajack can reach up to 100% routing rate, while simple pool-side defenses fail to flag the crafted profiles. Our work is the first to expose the over-trust of agent-declared capabilities in modern agent discovery, motivating pool-side validation defenses.

Abstract:
While Large Language Models (LLMs) excel in general question answering, they typically generate uniform responses, ignoring users’ diverse intents. Current Personalized Question Answering (PQA) methods face dual limitations: individual-based prompting struggles with data sparsity, while collaborative explicit prompting suffers from inaccurate user matching and attention dispersion caused by text concatenation. To tackle these challenges, we propose Collaborative Latent Prompting (CLP), a PQA framework that fuses collaborative knowledge within a dynamic latent space rather than relying on explicit textual concatenation. First, our Multi-Grained Collaborative Retrieval (MGCR) module ensures precise retrieval of relevant reference users through a three-stage refinement. Next, the Dynamic Q-Former (DQ-Former) utilizes question-modulated dynamic latent queries to extract compact personalized features from noisy texts. Finally, these features are projected into dense soft prompts to implicitly steer a frozen LLM. Experiments across three PQA domains and three backbone LLMs show consistent improvements over representative PQA baselines.

Abstract:
Recent large vision-language models (VLMs) pair a visual encoder with a large language model (LLM) and perform well on diverse image–text tasks, yet their reliability is often limited by decoder attention pathologies that suppress visual evidence and exacerbate hallucinations. In this paper, we revisit visual attention sinks and uncover a structured, layer-dependent behavior: across prompts, early and late decoder layers exhibit prompt-invariant attention collapse onto the same few image regions, which we term PIS (Prompt-Invariant Sinks), whereas mid layers become prompt-conditioned and drive vision-language alignment. This split suggests that treating sinks as a uniform effect is incomplete. Building on this insight, we propose SAGE (Sink-Aware Guided Emphasis), a lightweight intervention that steers decoder attention away from PIS and toward query-dependent regions of interest (ROIs) using token-aligned ROI masks derived from standard vision backbones such as CLIP, ViT, and DINOv3. Evaluated on diverse vision-encoder + decoder-only LLM VLM families, SAGE improves visual grounding, reduces hallucinations, and yields consistent gains across public downstream vision-language benchmarks, including fine-grained visual discrimination settings where localized evidence is crucial, when instantiated with backbone-derived ROI masks.

Abstract:
Developing effective and reliable health prediction models traditionally requires close collaboration between clinicians and machine learning experts, which is costly and limits efficiency. Recent advances in large language model (LLM) agents offer the potential for clinicians to autonomously develop prediction models through natural language interaction. However, existing agent-based systems remain limited in handling heterogeneous health data, complex prediction tasks, and the reliability requirements of healthcare applications. In this work, we present AutoHealth, an uncertainty-aware multi-agent framework for autonomous health prediction. Given task descriptions, AutoHealth dynamically constructs specialized agents that collaboratively perform data exploration, strategy design, code generation, model training, evaluation, and iterative refinement. Beyond optimizing predictive performance, the framework explicitly models uncertainty and generates comprehensive reports to support trustworthy clinical decision-making. To enable robust evaluation, we curate a real-world benchmark comprising 17 tasks across diverse health data modalities and learning settings. On this benchmark, AutoHealth completes all tasks and outperforms state-of-the-art baselines in both prediction performance and uncertainty estimation. These results demonstrate the strong capability of LLM agents in automated health data modeling.

Abstract:
Reasoning-enhanced universal multimodal embeddings (UME) improve heterogeneous retrieval, but plausible rationales do not necessarily produce discriminative rankings. We study this gap by comparing the discriminative (DISC) and reasoning-driven generative (GEN) branches of UME-R1, a state-of-the-art reasoning UME method. We decompose reasoning utility into positive-target gain, hard-negative gain, and their margin difference. Positive similarity increases for 56.6%, but 15.7% are false-helpful cases where reasoning moves hard negatives closer even more. Local-neighborhood and token-attribution diagnostics suggest why: reasoning often de-condenses retrieved neighborhoods, but utility requires separator-aligned movement, while influential CoT tokens frequently encode evidence shared by positives and hard negatives. Motivated by these diagnostics, we propose SURE (Score-structure Utility Router for Embeddings), which improves UME-R1-7B by 1.5 points and yields consistent gains on two additional embedding models on MMEB-V2, without retraining, label-based policy selection, or extra VLM forward passes.

Abstract:
Small reasoning models (SRMs) have shown strong performance on complex reasoning tasks by generating extended chain-of-thought trajectories, but they often fail to recover once their reasoning enters an erroneous path. Existing test-time fusion methods rely on local fusion signals to determine when to trigger fusion, which can be misled by transient uncertainty fluctuations and may reinforce unstable reasoning trajectories. We propose ThinkFuse, a training-free test-time fusion framework that selectively intervenes in unreliable reasoning segments. ThinkFuse compares segment-level uncertainty shifts with trajectory-level uncertainty trends to identify unstable reasoning points and fuse auxiliary reasoning paths into the primary model’s trajectory. Extensive experiments demonstrate that ThinkFuse outperforms baselines on mathematical and knowledge-intensive reasoning benchmarks, with consistent gains across model-family combinations, and remains robust with a smaller primary model. Our analysis shows that ThinkFuse requires fewer fusion triggers and generates fewer tokens, highlighting the efficiency of selective triggering. Our code is available at https://github.com/js-lee-AI/ThinkFuse.

Abstract:
Evaluating the reasoning abilities of large language models (LLMs) solely from final answers can obscure failures in intermediate steps, especially in multi-hop QA benchmarks without step-level annotations. To address this gap, we introduce Omanic, an open-domain 4-hop QA benchmark designed not only to measure final-answer accuracy but also to diagnose where reasoning breaks down. Omanic contains 10,296 machine-generated training examples (OmanicSynth) and 967 expert-reviewed human-annotated evaluation examples (OmanicBench), with each evaluation question decomposed into single-hop sub-questions, intermediate answers, and structured graph topologies. Experiments with proprietary and open-source LLMs show that Omanic is challenging, while step-wise analysis reveals a later-hop bottleneck, factual knowledge floor, and error propagation along reasoning chains. Fine-tuning on OmanicSynth transfers to six reasoning and mathematics benchmarks, yielding a 7.41-point average gain and validating its effectiveness as supervision for reasoning-capability transfer. We release the data at https://huggingface.co/datasets/li-lab/Omanic.

Abstract:
Practical deployment of large language models (LLMs) requires families of post-trained variants—instruction-tuned, reasoning-tuned, and chat-style models—each at multiple sizes to meet diverse latency and memory budgets. Producing each (variant, size) pair independently is prohibitive, so model families typically span only a handful of coarse-grained sizes per post-trained variant. Boomerang distillation (Kangaslahti et al., 2026) reduces this cost along the size axis for base models. Through model size interpolation, it constructs models of intermediate sizes from a single teacher-student pair without additional training. However, it still treats each post-trained variant as a separate object of optimization. We introduce ADAPT—Amortized Distillation Across Post-Trained LLMs—a framework for amortizing distillation across both axes of a model family: size and post-training variant, producing L × K models for L interpolated sizes across K post-trained variants with a single distillation run. ADAPT combines two components. First, a two-phase distillation procedure constructs post-trained students through pre-training alignment and supervised fine-tuning distillation, enabling smooth size–performance interpolation on generation and reasoning tasks. Second, weight-delta initialization approximates this construction across post-trained variants by transferring the distillation-induced weight change from the base model to students initialized from different post-trained variants. The resulting continuum of interpolated models also enables adaptive model-size selection at inference time, improving the compute–accuracy trade-off for long-form reasoning tasks.

Abstract:
Document-level Event Extraction (DEE) aims to extract event records from given documents. Previous works typically process documents by constructing complex feature fusion matrices or graph structures, aiming to predict complete event records via entity selection or role filling. However, this holistic structure extraction paradigm often suffers from cascaded error propagation due to rigid entity dependency, multi-role structural omissions limited by selection mechanisms, and consequent scalability bottlenecks. In this paper, we propose a Deconstruction-Reconstruction Framework for Document-Level Event Extraction (DrDEE). This framework deconstructs the holistic event record extraction task into atomic Token-Event-Role (TER) triplet verification, and subsequently reconstructs event records from valid tokens. It realizes a paradigm shift in DEE from Many-to-Many Selection to One-to-One Verification, significantly simplifying the task formulation. Experimental results show that the proposed method outperforms state-of-the-art methods, confirming its effectiveness. Our code is available at https://anonymous.4open.science/r/DrDEE.

Abstract:
Reinforcement learning from verifiable rewards improves the reasoning ability of large language models, but often suffers from entropy collapse , in which increasingly concentrated policies reduce rollout diversity and useful learning signals. Existing remedies either constrain the RL objective (e.g., entropy regularization) or adjust sampling temperature during rollout collection, but these interventions remain external to the model parameters. We propose Temperature-Scaled On-Policy Self-Distillation (TS-OPSD), a lightweight policy reheating method that internalizes the exploratory effect of temperature into model parameters. Starting from an entropy-collapsed RL checkpoint, TS-OPSD constructs a self-teacher by applying high-temperature scaling to the model’s own logits, then distills the resulting smoother distribution back into the student. This policy reheating requires no external teacher, privileged data, or additional inference cost. Experiments on Qwen3-4B-Base and Qwen3-8B-Base show that policy reheating yields a stronger initialization for continued RL than both standard continued RL and rollout-level temperature reheating. Further analyses show that TS-OPSD mainly reduces output sharpness while preserving intermediate representations, top candidate sets, and reasoning capability. These results suggest that entropy restoration can serve as a simple post-collapse intervention for extending reasoning-oriented RL.

Abstract:
Multi-party Dialogue Question Answering (MDQA) requires the machine to answer questions based on a conversation with multiple interlocutors. Existing benchmarks are largely extractive, leaving positional shortcuts and providing limited evidence about whether models can synthesize distributed evidence into speaker-grounded answers. We introduce MEETINGQA2, the first generative MDQA benchmark, derived from MEETINGQA with reformulated questions and speaker-centric summary answers. It contains 1,033 QA pairs, including 108 unanswerable questions. Beyond standard overlap metrics, we introduce attribution accuracy and factual correctness. Experiments show that strong LLMs perform consistently worse on MEETINGQA2 than on MEETINGQA, with especially large drops in attribution accuracy. The results indicate persistent weaknesses in speaker attribution, factual grounding, and long-context dialogue understanding.

Abstract:
Privacy protection for live web traffic requires more than detecting private spans. Agent-based privacy protection systems must determine whether an outgoing action complies with the destination site’s privacy policy, then apply only the level of rewriting or sanitisation justified by the residual disclosure risk. We present GuardianAgent, a policy-conditioned anonymization framework that couples structured risk assessment with verified adaptive rewriting. GuardianAgent computes risk through AMRSF (Adaptive Multi-factor Risk Scoring Formula), an explicit controller that combines policy-violation likelihood with data sensitivity, recipient transmission, purpose legitimacy, contextual basis, and policy transparency, rather than relying on an LLM to assign risk directly. This risk score determines both the allow/transform/deny decision and the initial anonymization level. For efficiency, GuardianAgent uses an evidential fast path for low-uncertainty policy matches and invokes an LLM slow path only for uncertain cases. For rewriting, it applies a five-level hierarchy driven by a verified adversarial guesser: guesses trigger escalation only when supported by the original text, preventing hallucinated attacker confidence from causing unnecessary over-anonymization. Experiments across three benchmarks spanning legal text (TAB), Reddit posts (SynthPAI), and multi-format synthetic PII records (PII-Masking-300k) show that GuardianAgent achieves the strongest privacy–utility trade-off among published baselines and is the only method to reach ≥ 0.90 privacy in all three domains, remaining robust under a backbone switch. Action-context stress tests further show that the same outgoing text receives different decisions and anonymization strengths under different recipients, purposes, action bases, and policy-transparency conditions.

Abstract:
Recent work has shown that RLHF is highly susceptible to backdoor attacks. However, existing methods often rely on rare tokens or fixed triggers, limiting their impact in realistic scenarios. In this work, we develop GREAT, a novel framework for crafting natural distributional backdoors in RLHF. Specifically, GREAT targets harmful response generation for a vulnerable user subpopulation featured by semantically violent requests paired with emotionally angry triggers. At the core of our framework is a trigger identification pipeline that operates in the model’s latent embedding space, leveraging dimensionality reduction and clustering techniques to identify representative triggers. To enable this, we introduce a hierarchical and diversity-driven prompting strategy to construct Erinyes, a high-quality dataset of over 5000 angry triggers curated from GPT-4.1. Our experiments show that GREAT significantly outperforms baselines in attack generalization to unseen triggers, while preserving standard utility and maintaining stealth under defenses.

Abstract:
Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior. Yet existing MLLM benchmarks either evaluate constraints in text only or embed them into the user turn, leaving system-message adherence in multimodal contexts largely unmeasured; they also leave open whether compliance comes at the cost of foundational vision-language capabilities. We introduce VSysBench, a simple yet effective MLLM benchmark that organizes system-message constraints into 5 main categories and 22 sub-categories, ranging from textual directives in visual contexts to fully vision-grounded ones, each paired with a misaligned counterpart that stress-tests the instructional hierarchy. VSysBench scores each response jointly along two axes, constraint compliance and answer correctness, via the Joint Satisfaction Rate (JSR) and Cross-Constraint Sensitivity (CCS). Across 16 MLLMs, we find that imposing system messages substantially erodes base task accuracy, that compliance collapses under user conflict for open-weight models while remaining stable for top proprietary ones, and that vision-grounded constraints are the hardest category for nearly every model.

Abstract:
LLM agents rely on long-term interaction histories to tailor their responses to individual users. However, historical personalization signals may become stale or misaligned with expectations implied by the current context, leading agents to follow outdated preferences during reasoning. We refer to this failure mode as personalization inertia , which shifts the agent’s behavior away from the user’s current intent. Existing methods mainly optimize final responses or memory retrieval, leaving such failures underexplored. In this paper, we propose P ersonalization I nertia V erificati O n and Regula T ion ( Pivot ), a framework for mitigating personalization inertia in long-term interactions with agents. Concretely, we construct a trajectory-independent evidence condition as a diagnostic reference, model prefix-evidence consistency to assess compatibility with the current personalization evidence, and learn a rollback policy to determine whether and where to intervene. The selected intervention triggers local repair, preserving the reliable prefix while regenerating only the potentially inertia-affected suffix under evidence-guided constraints. Experiments on benchmark datasets demonstrate consistent gains in personalization accuracy and trajectory faithfulness with targeted intervention overhead.

Abstract:
Grammar-constrained decoding helps large language models produce syntactically valid structured outputs, such as code, JSON, and SQL. For context-free grammars, many practical decoders enforce local prefix feasibility: each token must keep the current prefix extendable to some valid completion. Yet, under tokenizer–grammar mismatch and finite token budgets, feasible prefixes may still fail to reach acceptance. We propose a lookahead-guided decoding framework for context-free grammars based on pushdown automata. Offline, we compute bounded pushdown summaries with reachability labels and upper-bound distances to acceptance. Online, these estimates guide horizon-aware pruning and beam search. The resulting decoder is syntactically sound: every output is accepted by the target grammar. Experiments on JSON, SQL, and Linear Temporal Logic (LTL) show both consistent syntactic validity and improved completion quality over existing baselines.

Abstract:
Knowledge-based visual question answering (KB-VQA) requires vision-language models to understand images and use external knowledge, especially for rare entities and long-tail facts. Most existing retrieval-augmented generation (RAG) methods adopt a fixed pipeline that sequentially retrieves information, filters it, and then produces an answer. Such a design makes it difficult to adapt to diverse question types. Moreover, it separates retrieval from reasoning, making it hard for the model to decide when to search, how to refine queries, or when to stop. As a result, the retrieved evidence is often poorly aligned with the question. To address these limitations, we reformulate KB-VQA as a search-agent problem and model the solving process as a multi-step decision-making procedure. At each step, the agent selects one of four actions-Answer, Image Retrieval, Text Retrieval, and Caption-based on its current information state. We further design an automated pipeline to collect multi-step trajectories that record the agent’s reasoning process, tool usage, and intermediate decisions. These trajectories are then used as supervision for fine-tuning. Experiments on InfoSeek and E-VQA demonstrate that our method achieves state-of-the-art performance, consistently outperforming prior baselines and confirming the effectiveness of our framework.

Abstract:
Story rewriting aims to adapt existing narratives to diverse reader preferences while preserving plot consistency and narrative coherence. Unlike conventional work on style transfer, we argue that effective story rewriting demands context-aware narrative enrichment beyond surface-level stylistic adaptation. Our pilot human study shows that style adaptation alone provides only marginal gains in reader satisfaction (2.3%), while context-enhanced rewriting substantially improves user preference alignment (24.5%). Motivated by this, we introduce STORYLENSBENCH, a large-scale benchmark for preference-aligned story rewriting, comprising structured story books, multi-dimensional reader preference profiles, and ranked context-aware rewritten stories. Building on this benchmark, we propose STORYLENSEVAL, a reward model for estimating reader satisfaction over rewritten stories, and STORYLENSWRITER, a two-stage rewriting model combining supervised fine-tuning with GRPO-based reinforcement learning. We further establish a comprehensive evaluation framework covering fidelity, coherence, and reader satisfaction. Experimental results demonstrate that STORYLENSWRITER consistently outperforms strong generation and personalization baselines, highlighting the importance of context-aware narrative enrichment for personalized story rewriting.

Abstract:
Medical consultation (MC) is an interactive decision-making process that requires dynamically adapting inquiry directions according to evolving patient feedback. Although large language models (LLMs) have demonstrated strong planning capabilities in medical applications, existing LLM-based MC methods typically make turn-level independent inquiry decisions without adaptive control over global inquiry directions, making them prone to inefficient local exploration. To address this limitation, we propose MCPlanner, a hierarchical adaptive planning-based MC framework in which a high-level planner adaptively controls the inquiry direction, while a low-level planner executes inquiry actions under the selected direction. Additionally, MCPlanner introduces expert-guided distillation and a smoothed confidence calibration strategy to improve high-level planning efficiency and diagnostic accuracy. Experimental results on DXY, GMD, and CMD show that MCPlanner achieves best diagnostic accuracies of 95.2%, 90.8%, and 76.1%, respectively. Notably, on CMD, MCPlanner outperforms the strongest baseline by 5.0 percentage points. Code is publicly available at https://github.com/zh-jia/MCPlanner .

Abstract:
When a language model hallucinates, the final answer is wrong, but the mistake is not necessarily invisible inside the model. Different internal pathways may remain uncertain, disagree in how quickly they sharpen, or commit to competing continuations before the output is produced. We introduce TriLens , a white-box detector that turns this intuition into a compact representation: at every layer, it reads the multi-head self-attention output, the feed-forward output, and the residual stream through the model’s own logit lens, then records only the entropy of each readout. The resulting 3L -dimensional trajectory describes how certainty forms across depth and across modules, without storing high-dimensional hidden states or sampling multiple generations. This simple signal yields a strong detector across instruction-tuned LLMs and QA benchmarks, and our analyses show that the three module-wise entropy trajectories provide complementary evidence. TriLens suggests that hallucination detection can benefit from tracking how internal computation settles, not only what the final layer predicts. The code is available at https://tosakaucw.github.io/TriLens/.

Abstract:
Training stability is a key bottleneck in low-precision language model training: efficient low-cost paths can still produce short-lived numerical risks at a small set of operators. We formulate this as runtime stability control and present Gradient Norm-to-Mean Ratio (GNMR), a lightweight controller that compares each recoverable unit’s current gradient norm with its historical mean. Together with 𝛥 -GNMR for abrupt short-window increases, GNMR maps local risk signals to bounded recovery actions under a hard maxO budget and a short lock interval, without changing the numerical format, kernel, or backend recipe. Across activation-quantization stress, DeepSeek-style recipe-level training, and LLaMA-2 13B fine-tuning, GNMR preserves high-fidelity quality with sparse, budgeted recovery. These results support GNMR as a backend-agnostic controller to improve low-precision training stability while preserving low-cost execution.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved the reasoning capability of large vision-language models (LVLMs). Most RLVR methods for LVLMs follow an outcome-driven paradigm, using the correctness of final answer as supervision. However, in multimodal reasoning, this supervision overlooks a critical question: whether a successful reasoning trajectory is actually grounded in visual evidence. To address this limitation, we propose Visual Evidence-Calibrated Policy Optimization (VEPO), which turns visual grounding into an explicit trajectory-level credit calibration signal. VEPO first measures token-level visual responses by comparing the model’s likelihood under the original and perturbed visual inputs. It then estimates a visual background for the rollout-group and retains responses above the background as sparse visual anchors into a rollout-level evidence score. This score calibrates outcome-driven optimization and guides the policy toward successful trajectories with stronger visual support. Experiments on six multimodal reasoning benchmarks show that VEPO achieves strong overall performance against RLVR baselines without requiring additional annotations, demonstrating its scalability for multimodal policy optimization.

Abstract:
Reasoning benchmarks often collapse heterogeneous problem types into aggregate scores, obscuring which structural computations large language models (LLMs) can reliably perform. Existing graph-reasoning benchmarks cover graph traversal, broad graph computation, and graph-language interaction, but do not directly test whether graph-structured reasoning transfers across topological mechanisms under matched conditions. We introduce GraphReason-Bench (GR-Bench), comprising 1,800 programmatically generated and oracle-verified problems across three prompt-length regimes and six task families: local motif enumeration, exact-depth breadth-first search, bounded path enumeration, articulation-point identification, non-induced subgraph matching, and non-monotonic rule execution. All tasks share a text-only executable interface, exact oracles, canonical output normalization, and set-based scoring, enabling controlled cross-mechanism comparison. On the 600-instance main evaluation split across nine proprietary and open-source LLMs, we find mechanism-level dissociation: models that succeed on one structural operation often fail on others, and cross-task performance does not reduce to a stable graph-reasoning ability. Trace-level cases reveal mechanism-specific failures such as premature search termination and loss of intermediate state. An initial sparse-autoencoder probe, supported by cross-model robustness checks, shows that topology-sensitive representations become less separable as structural difficulty increases. Together, these results establish graph-structured reasoning as a measurable domain of LLM evaluation.

Abstract:
Activation steering controls language models by adding vectors or features to hidden states at inference time, but the upstream source of these steering signals is often treated as a secondary detail. We study this source choice as activation source selection: the combination of source context and activation readout policy used to collect the hidden states from which a steering signal is built. Holding the downstream intervention fixed, we show across three instruction-tuned models and four steering task families that changing only the source activations can more than double steering success. We further find that effective steering is not explained simply by whether the desired behavior appears in the source text. Instead, strong signals come from execution-boundary states, where the model is about to produce or continue the target behavior. This phase distinction explains why answer-based sources sometimes work: their useful component aligns with execution-boundary directions rather than target appearance alone. Building on this view, we introduce tail subtraction, which removes shared prompt and continuation semantics from boundary states and yields cleaner, more stable steering signals. Overall, our results suggest that steering depends on representations of what the model is about to do, not merely on what has already appeared.

Abstract:
Large Language Models (LLMs) are accelerating scientific idea generation, but rigorously evaluating these numerous, often superficial, AI-generated propositions for novelty and factual accuracy is a critical bottleneck; manual verification is too slow. Existing validation methods are inadequate: LLMs as standalone verifiers may hallucinate and lack domain knowledge (our findings show ~60% unawareness of relevant papers in specific domains), while traditional citation networks lack explicit causality and narrative surveys are unstructured. This underscores a core challenge: the absence of structured, verifiable, and causally-linked historical data of scientific evolution. To address this, we introduce THE-Tree ( T echnology H istory E volution Tree ), a computational framework that constructs such domain-specific evolution trees from scientific literature. THE-Tree employs a search algorithm to explore evolutionary paths. During its node expansion, it utilizes a novel "Think-Verbalize-Cite-Verify" process: an LLM proposes potential advancements and cites supporting literature. Critically, each proposed evolutionary link is then validated for logical coherence and evidential support by a recovered natural language inference mechanism that interrogates the cited literature, ensuring that each step is grounded. We construct and validate 88 THE-Trees across diverse domains and release a benchmark dataset including up to 71k fact verifications covering 27k papers to foster further research. Experiments demonstrate that i) in graph completion, our THE-Tree improves hit@1 by 8% to 14% across multiple models compared to traditional citation networks; ii) for predicting future scientific developments, it improves hit@1 metric by nearly 10%; and iii) when combined with other methods, it boosts the performance of evaluating important scientific papers by almost 100%. By constructing explicit, verifiable pathways of scientific progression, THE-Tree provides a robust historical foundation for evaluating new hypotheses (human or AI-generated) and enables a computable science history, fostering evidence-based AI-driven scientific discovery.

Abstract:
Personalization has become crucial for adapting models to the diverse and evolving needs of users across cultural, temporal, and contextual dimensions. While existing methods often rely on centralized fine-tuning or static preference alignment within a single model, they struggle to achieve both real-time and high-quality personalization under the resource and privacy constraints of personal devices. To address this challenge, we propose CoSteer, a collaborative framework that enables tuning-free, real-time personalization via decoding-time adaptation. By leveraging logit differences between context-aware and -agnostic local small models, CoSteer steers cloud-based large models, ensuring effective personalization while preserving the large model’s capabilities. Personalization is handled locally, with only final tokens sent to the cloud, maintaining both user context and system efficiency. Through extensive experiments across a wide range of tasks, we demonstrate that CoSteer generates high-quality personalized content, ensuring both effectiveness and computational efficiency. Our results highlight its robustness across models and environments, confirming its practical applicability in real-world scenarios. Our code and data are available at https://anonymous.4open.science/r/Costeer-4977.

Abstract:
The rising demand for long-video understanding presents significant challenges due to massive information and complex temporal dependencies. Direct processing with Large Visual Language Models (VLMs) incurs high computational costs and risks information loss due to frame sampling. Meanwhile, existing agent-based methods often struggle with fragmented contexts and the high overhead of iterative processing. Therefore, we propose VideoNav, which introduces a new paradigm of index-centric navigation, reformulating video understanding as cost-aware structured query processing. VideoNav decouples heavy perception from online reasoning, including offline index construction and online adaptive navigation. In the offline phase, we transform unstructured video data into a structured, multi-granular index tree. During the online phase, the framework executes a top-down, coarse-to-fine navigating strategy. Leveraging the LLM agents, it efficiently locates relevant information and answers the query with navigating guidance, multi-level navigation, dual-path routing and visual inspection. Crucially, this adaptive mechanism enables early termination when high-level information suffices, minimizing redundant computation. Extensive experiments across different benchmarks demonstrate the effectiveness of our framework. VideoNav achieves decent accuracy while delivering 2.5x-5.7x cost savings and a 2.9x-5.8x speedup.

Abstract:
Multimodal Large Language Models (MLLMs) can assign similar confidence to answers that fail for different reasons. We propose HalluPrism, a behavioral diagnostic that re-runs an answer after visual degradation, blank-image replacement, and grounding or relation checks. These targeted probes yield a signature over visual-perturbation sensitivity (V), image-removal confidence retention (L), and grounding/relation-probe instability (A). Across 58K+ examples from four benchmarks and four MLLMs, image-removal confidence retention is most prevalent, while grounding/relation-probe instability better separates failure families. Only 18 of 48 source-target checks are diagonally aligned, so the coordinates should be interpreted jointly rather than as independent causal sources. With the dataset fixed, the joint signature improves failure-family AUROC from 0.634 to 0.769 on HallusionBench and from 0.707 to 0.817 on VizWiz, with smaller gains on POPE and VSR. In pooled XGBoost analysis, AUROC rises from 0.78 with scalar confidence to 0.95 with (V, L, A) and 0.97 when confidence is added. The same signature does not automatically improve correctness ranking. The three tested direct scalarizations can harm it. These results show that multimodal uncertainty should characterize failure structure before it is used to decide whether to abstain or correct.

Abstract:
Scientific inquiry involves iterative cycles of exploration, hypothesis generation, and reasoning over observations to uncover relationships in complex systems. Large language models (LLMs) have demonstrated strong capabilities in several components of this process, including inductive reasoning, hypothesis generation, and data analysis. However, prior work typically evaluates these abilities in isolation, leaving open how models integrate them across a full inquiry cycle. In this work, we introduce a framework for evaluating LLMs’ inquiry capabilities in a unified, interactive setting. We extend the Countdown task to a mathematical environment where an agent probes the system by manipulating variables, collecting observations, and inferring relationships between them. This setup enables analysis of exploration strategies, causal discovery, and hypothesis refinement during inquiry. Across seven LLMs and multiple prompting strategies, our experiments reveal limited inherent inquiry capabilities and low success rates in discovering the underlying expressions. However, structured guidance inspired by educational scaffolding significantly improves performance. Our findings highlight current limitations of LLMs in strategic exploration and iterative hypothesis refinement, and show how simple guidance strategies developed for human learners in inquiry learning can support more effective inquiry behavior.

Abstract:
Large language models (LLMs) have recently shown promise for historical entity linking, but preference optimization for this task is often formulated with only one negative candidate per training instance. This discards information from the remaining candidates retrieved for the same mention. We introduce multi-negative direct preference optimisation (MDPO), a reference-based pairwise objective that compares the correct entity with all valid rejected candidates associated with each mention. MDPO preserves the Bradley-Terry formulation of DPO while exploiting the complete candidate set through masked, length-normalised sequence scores. We evaluate MDPO on hipe-2020 and newseye, covering French, German, English, Swedish, and Finnish historical newspaper text. Experiments show that MDPO improves over supervised fine-tuning and single-negative DPO, with particularly strong gains for NIL mentions, semantic ambiguity, OCR noise, and historically difficult names. Further analyses disentangle candidate-generation and selection errors, showing that candidate retrieval remains a key bottleneck for end-to-end entity linking. These results demonstrate that incorporating all within-instance negative candidates is a simple and effective improvement for LLM-based historical entity linking.

Abstract:
Vision-language models align frozen vision encoders to language models using only a next-token prediction loss, an asymmetric signal that discards rich visual signals not captured in the captions. We propose GenDA (Generative Distillation Alignment), a novel training framework that augments the captioning loss with a generative distillation objective from a frozen text-to-image diffusion model. GenDA concatenates selected visual tokens with the encoded caption to form a joint conditioning signal, encouraging these tokens to encode visual information not already captured by the caption. Across five vision encoders and four language models, GenDA gains 3-9 CIDEr points over LLaVA-1.5 on five captioning benchmarks. These gains also transfer to downstream VLM benchmarks while preserving the encoders’ discriminative performance. Visual preservation and modality-gap analyses suggest that visual information is retained deeper within the LLM, while visual features become more closely aligned with the language representation space.

Abstract:
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DeepResearch, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe—supervised fine-tuning followed by Group Relative Policy Optimization (GRPO)—enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate VideoDR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale.

Abstract:
Vision-Language Models are commonly evaluated through their final predictions, but understanding whether these decisions are grounded in visual evidence requires tracing how visual information contributes to language-based decisions. With this purpose in mind, we investigate cross-modal information flow in a video-based generative multiple-choice-like setting by applying a layer-wise causal intervention on video-text attention pathways. We target spatial, causal, and temporal visual reasoning. Our results show that visual information is mainly integrated while the model processes the candidate answer options, which serve as the primary textual grounding sites for the final decision. We further show that nouns play an important role as semantic anchors during multimodal enrichment, while verbs are more relevant when temporal relations are processed. Finally, we identify a distinct pattern in temporal reasoning, suggesting that VLMs struggle to reconstruct sequential information across video frames, but we remark that such fragility may also reflect linguistic biases associated with specific temporal expressions used for defining the relation between events within a scene.

Abstract:
Large Language Models (LLMs) are strong at multi-step reasoning, but running them for every generated token is expensive. Small language models (SLMs) are much cheaper, yet less reliable at the hardest reasoning steps. Existing routing methods mostly make a single query-level decision, assuming that response difficulty is uniform once decoding begins. For chain-of-thought reasoning, however, difficulty varies during generation, making token-level routing a more appropriate granularity. This setting introduces three challenges: token-level uncertainty is noisy, local routing decisions must satisfy global efficiency constraints, and frequent switching can disrupt reasoning coherence. We present TokenRouter, a black-box runtime controller for uncertainty-guided token-level routing between small and large language models. TokenRouter combines a robust entropy detector, a self-calibrating control policy, and a dynamic intervention commitment mechanism for stable switching over difficult spans. Across three reasoning benchmarks, TokenRouter maintains over 99% of pure-LLM accuracy while reducing large-model activation to 56.2%–76.0% of generated tokens. Code is available at https://github.com/sznnzs/TokenRouter.

Abstract:
Despite rapid progress in video generation models, they still exhibit obvious motion deficiencies, often manifested as incorrect object motion. However, most existing video quality evaluations focus on aesthetic quality or text-video alignment. To address this gap, we study object-centric motion fidelity assessment, evaluating target objects along object consistency, motion continuity, and physical plausibility. To achieve this, we first introduce VidMotion, a diagnostic dataset of 6,879 videos with designated moving objects and fine-grained annotations including dimension-wise scores and failure causes. We further propose MotionInsight, a diagnostic evaluator that shifts assessment from implicit RGB-frame observation to explicit motion-space diagnosis. By constructing motion-aware representations, MotionInsight makes subtle motion deficiencies more observable. We also introduce motion-specific rewards during GRPO to transform observed motion into a diagnostic assessment. Experiments demonstrate that MotionInsight provides an effective basis for diagnosing object motion deficiencies, producing human-aligned scores along three dimensions and grounded explanations.

Abstract:
Document-level event causality identification (DECI) requires identifying causal relations across a document, where inter-sentence causality is often implicit and supported by weak evidence. Existing methods typically construct event graphs from pairwise predictions, which may overlook weak but valid causal edges, and thus hinder causal message passing. To address these issues, we propose ICGR, an Iterative Causal Graph Refinement framework that iteratively refines event graphs by preserving low-confidence edges and performing message passing on the refined graph to produce a more complete and reliable structure. In each refinement round, the Constrained Edge Completion module identifies missing causal edges, and the Conditional Diffusion module refines low-confidence edges through denoising. Experiments on EventStoryLine and MAVEN-ERE show that ICGR achieves state-of-the-art performance, with notable improvements on inter-sentence causal relations.

Abstract:
Recent research points toward LLMs being manipulated through adversarial and seemingly benign inputs, resulting in harmful, biased, or policy-violating outputs. In this paper, we study an underexplored issue concerning harmful and toxic mathematical word problems. We show that math questions, particularly those framed as natural language narratives, can serve as a subtle medium for propagating biased, unethical, or psychologically harmful content, with heightened risks in educational settings involving children. To support a systematic study of this phenomenon, we introduce ToxicGSM, a dataset of 1.9k arithmetic problems in which harmful or sensitive context is embedded while preserving mathematically well-defined reasoning tasks. Using this dataset, we audit the behaviour of existing LLMs and analyse the trade-offs between safety enforcement and mathematical correctness. We further propose SafeMath – a safety alignment technique that reduces harmful outputs while maintaining, and in some cases improving, mathematical reasoning performance. Our results highlight the importance of disentangling linguistic harm from math reasoning and demonstrate that effective safety alignment need not come at the cost of accuracy.

Abstract:
Static benchmarks often conflate memorization with reasoning, failing to capture the dynamic nature of world knowledge. We present LiveSearchBench, a scalable pipeline that automatically constructs snapshot-grounded benchmarks from knowledge graph differentials, in which every answer is unique and SPARQL-verifiable against a target snapshot. Unlike prior template-based approaches, our method leverages topology-aware synthesis to generate multi-constraint compositional questions with unique answers guaranteed via SPARQL validation. Evaluating 1,000 questions across three difficulty levels on both open-weight and frontier models, we expose a pronounced Recency Gap on facts that post-date a model’s training cutoff, particularly for compositional queries; retrieval dependence is thus an empirical property we measure, not a formal guarantee we claim. Through Oracle and Wikidata-as-corpus analyses, we disentangle dual failure modes: retrieval coverage limitations for single-hop questions and reasoning failures for compositional queries even with ground-truth evidence. LiveSearchBench provides both a rigorous construction methodology and a continuously updatable testbed for evaluating evidence integration under evolving knowledge.

Abstract:
Mobile GUI agents deployed in real settings must work across different applications that support the same functionality. Most existing benchmarks test each task in only one app, so a high score can mean the agent understands the task, or only that it knows that particular app. We introduce AnyAppBench, a category-controlled live Android benchmark that evaluates cross-application generalization while keeping the user goal fixed. It spans 10 functional categories, 100 task templates, and 520 task–application pairs over 52 applications. Agents run from raw instructions and with app-independent sub-goals, and a VLM judge labels every failed run under a fixed failure taxonomy whose reliability is measured by human annotation. We find that, across 13 agents, success on the original application does not transfer reliably to new applications with the same goal. Furthermore, providing high-level sub-goal decomposition produces only small, category-dependent changes that do not close the gap, and the mix of failure types changes with the target interface. Based on those insights, we believe the AnyAppBench benchmark provides an important stepping stone toward robust real-world deployment of mobile GUI agents. Our code, data and the leaderboard can be found at the project website https://anyappbench.github.io/.

Abstract:
Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety in isolation - failing to capture whether a model is simultaneously pedagogically effective and safe across sustained interaction. We argue that tutoring safety is fundamentally different from conventional LLM safety: the primary risk is not toxic content but the quiet erosion of learning through answer over-disclosure, misconception reinforcement, and the abdication of scaffolding. To systematically study this failure mode, we introduce SafeTutors, a benchmark that jointly evaluates safety and pedagogy across mathematics, physics, and chemistry. SafeTutors is organized around a theoretically grounded risk taxonomy comprising 11 harm dimensions and 48 sub-risks drawn from learning-science literature. We uncover that all models show broad harm; scale doesn’t reliably help; and multi-turn dialogue worsens behavior, with pedagogical failures rising from 17.7% to 77.8%. Harms also vary by subject, so mitigations must be discipline-aware, and single-turn “safe/helpful” results can mask systematic tutor failure over extended interaction.

Abstract:
High-quality pre-training data is crucial for large language models (LLMs), yet existing score-based selection approaches often fail to balance quality and diversity. Empirical studies have revealed a non-monotonic relationship between dataset scores and downstream performance: simply selecting the top-scored data often leads to performance degradation due to the collapse of correlated evaluation dimensions, which sacrifices diversity. To address this, we propose Orthogonal Diversity-Aware Selection (ODiS), a novel framework to preserve both quality and diversity by decomposing correlated evaluation metrics into orthogonal feature dimensions. First, ODiS evaluates data from multiple dimensions, covering language quality, knowledge quality, and comprehension difficulty. The resulting multi-dimensional scores are then decorrelated via Principal Component Analysis (PCA), yielding orthogonal evaluation dimensions. For each dimension, a RoBERTa-based scorer is trained to regress the data onto PCA-projected scores, enabling scalable inference on large corpora. Finally, ODiS constructs the training dataset by selecting the top-scored data from each orthogonal dimension, thereby ensuring both quality and diversity. Empirical results show that ODiS-selected data exhibit less than 2% inter-dimension overlap, confirming the orthogonality between dimensions. More importantly, models trained on ODiS-selected data significantly outperform baselines across multiple downstream benchmarks, underscoring the necessity of orthogonal, diversity-aware data selection for LLMs.

Abstract:
Existing code vulnerability detection (CVD) methods face severe challenges in terms of generalization. Most of these methods directly map the source code to a binary vulnerability label or a vulnerability explanation, struggling to capture the deep features within the complex relationship between them. Thus, these works fail to generalize to out-of-distribution data well. To address this issue, we propose a simple yet effective training framework called refinement-integrated reasoning (RIRE), leveraging refinements that indicate the patched code in the CVD task. Firstly, RIRE introduces the refinement as a guiding signal to help LLMs synthesize reasoning chains as explanations, which align with the vulnerability label, and uses these data for supervised fine-tuning. Secondly, refinement is utilized through a hierarchical preference optimization framework to help determine which reasoning-enhanced detection result is of higher quality, further mining essential features for fine-grained distinguishability and detection capability. Experimental results across five benchmark datasets demonstrate that our proposed RIRE framework achieves superior performance over strong baselines, especially being more generalizable to unseen datasets during training.

Abstract:
Supervised fine-tuning (SFT) is the predominant method for adapting large language models (LLMs), yet it often struggles with generalization compared to reinforcement learning (RL). In this work, we posit that this performance disparity stems not just from the loss function, but from a more fundamental difference: SFT learns from a fixed, pre-collected dataset, whereas RL utilizes on-policy data sampled from the current policy. Building on this hypothesis, we introduce one-token rollout (OTR), a novel fine-tuning algorithm that guides SFT with the policy gradient method. OTR reframes the autoregressive learning process by treating each token generation as a single-step reinforcement learning trajectory. At each step, it performs a Monte Carlo ”rollout” by sampling multiple candidate tokens from the current policy’s distribution. The ground-truth token from the supervised data is then used to provide a reward signal to these samples. Guided by policy gradient, our algorithm repurposes static, off-policy supervised data into a dynamic, on-policy signal at the token level, capturing the generalization benefits of on-policy learning while bypassing the costly overhead of full sentence generation. Through extensive experiments on a diverse suite of challenging benchmarks spanning mathematical reasoning, code generation, and general domain reasoning, we demonstrate that OTR consistently outperforms standard SFT. Our findings establish OTR as a powerful and practical alternative for fine-tuning LLMs and provide compelling evidence that the on-policy nature of data is a critical driver of generalization, offering a promising new direction for fine-tuning LLMs.

Abstract:
LLMs have shown impressive linguistic competencies in a wide range of NLP tasks, yet morphological generalization remains challenging. Models can benefit from in-context learning, where the choice of few-shot examples significantly influences the outcome. We probe LLMs’ morphological capabilities by assessing how two targeted in-context exemplar selection strategies, Iterative Confidence-Based Exemplar Selection (i-CES) and Similarity-Based Exemplar Selection (SES), influence the model’s ability for inflectional and derivational modeling across multiple languages. Our experiments with LLaMA 3.3-70B-instruct and Qwen2.5-32B-Instruct models indicate that SES is more effective for accessing latent linguistic knowledge, showing a substantial performance advantage over i-CES in both derivational and inflectional tasks. In contrast, i-CES offers no clear advantage over random sampling.

Abstract:
Training tool-use agents typically relies on outcome-based filtering: Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks. However, this paradigm ignores interaction dynamics—successful trajectories may lack error recovery or exhibit redundancy, while pass rates fail to distinguish structurally informative tasks from trivial ones. To bridge this gap, we propose TopoCurate , an interaction-aware framework that shifts the paradigm from linear outcome filtering to topological interaction modeling. By projecting multi-trial rollouts into a unified semantic quotient topology within the same task, we construct a structured manifold that explicitly captures the divergence between success and failure pathways across tool invocations and environmental responses. Leveraging this representation, we introduce a dual-selection mechanism: for SFT, we prioritize trajectories demonstrating reflective recovery, semantic efficiency, and strategic diversity to mitigate covariate shift and mode collapse; for RL, we select tasks with high error branch ratios and strategic heterogeneity, maximizing gradient Signal-to-Noise Ratio to address vanishing signals in sparse-reward settings. Evaluations on BFCLv3 and Tau2 Bench show that TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.

Abstract:
Instruction tuning increasingly relies on LLM-based prompt refinement, where prompts in the training corpus are selectively rewritten by an external refiner to improve clarity and instruction alignment. This motivates an instance-level audit problem: for a fine-tuned model and a training prompt–response pair, can we infer whether the model was trained on the original prompt or its LLM-refined version within a mixed corpus? This matters for dataset governance and dispute resolution when training data are contested. However, it is non-trivial in practice: refined and raw instances are interleaved in the training corpus with unknown, source-dependent mixture ratios, making it harder to develop provenance methods that generalize across models and training setups. In this paper, we formalize this audit task as Refinement Provenance Inference (RPI) and show that prompt refinement yields stable, detectable shifts in teacher-forced token distributions, even when semantic differences are not obvious. Building on this phenomenon, we propose RePro, a logit-based provenance framework that fuses teacher-forced likelihood features with logit-ranking signals. During training, RePro learns a transferable representation via shadow fine-tuning, and uses a lightweight linear head to infer provenance on unseen victims without training-data access. Empirically, RePro consistently attains strong performance and transfers well across refiners, suggesting that it exploits refiner-agnostic distribution shifts rather than rewrite-style artifacts.

Abstract:
Evaluating large language models typically requires thousands of benchmark items, making the process expensive and increasingly impractical. Existing protocols rely on average accuracy over fixed item sets, treating all items as equally informative—yet many items are too easy or too hard to distinguish between models, making evaluation both wasteful and imprecise. Item Response Theory (IRT) with Computerized Adaptive Testing (CAT) offers a principled alternative—separately estimating model ability and item difficulty, then adaptively selecting the most informative items. However, practical deployment is bottlenecked by costly item calibration, and standard greedy selection causes severe over-exposure of frequently picked items while leaving the rest of the bank under-utilized, which risks data contamination over time. We propose FAIR-Eval with two components: Feature-Driven Calibration (FDC) , which jointly learns item parameters from text features and sparse responses, enabling low-cost bank expansion requiring only minimal model responses; and Maximum Fisher Information Plus (MFI+) , which counteracts greedy selection by incorporating exposure control, while simultaneously enforcing difficulty coverage and domain diversity to systematically activate under-utilized items. On a 6,796-item bank (193 calibration models, 50 held-out), FAIR-Eval achieves Pearson r = 0.9882 and Spearman 𝜌 = 0.9828 against full evaluation using around 3% of items. Crucially, it broadens difficulty coverage from 60.0% to 68.7%, achieving a more comprehensive and robust assessment of model capabilities with minimal test length.

Abstract:
Compact open-source language models lag behind their larger counterparts in agentic tool-use reliability, yet standard remedies face fundamental obstacles: supervised fine-tuning suffers from exposure bias, while reinforcement learning is hampered by sparse credit assignment over long tool-interaction trajectories. We introduce Skill-CDPO, a progressive framework that first acquires tool-use skills at inference time through static tool analysis and dynamic strategy refinement, then distills the resulting error-correction signals into parameter updates via Critical Step DPO (CDPO). CDPO identifies the specific trajectory steps where model capability is the bottleneck—through rollout divergence between a local policy and an expert model—and constructs distributional preference pairs from all cross-group rollouts at those steps, weighted by both step-level criticality and pair-level score gaps. This provides dense, fine-grained supervision without requiring a process reward model. We evaluate Skill-CDPO on three medical agent benchmarks—PubMed Search (a new PubMed-based deep research benchmark we contribute), CureBench, and MedBrowseComp—using an 8B-parameter deep research model. Skill-CDPO substantially outperforms SFT and trajectory-level DPO baselines and achieves competitive or superior performance compared to GPT-5.2 on retrieval-intensive tasks. Our code and data are available at https://anonymous.4open.science/r/Skill-CDPO-BF51.

Abstract:
Multimodal long documents are core carriers of professional knowledge, where critical evidence is sparsely distributed across paragraphs and modalities. This easily causes key information omission and cross-modal hallucinations in summarization by multimodal LLMs. These issues stem from attention drift in long-range dependency modeling and gaps in inter-modal alignment. To address this, we introduce MMLDSum-Bench, a high-quality benchmark for multimodal long-document summarization, covering multiple domains, context-length scales, and visual-textual modality distributions. We further propose MMLDSum-LLM, a reproducible two-stage training framework that combines supervised fine-tuning with visual-alignment weighted loss and keyword-aware weighted loss, followed by GRPO with a multi-objective reward (keyword coverage, image-text alignment, ROUGE, and length control). Extensive experiments on MMLDSum-Bench, comparing against leading closed-source and open-source multimodal models under a unified evaluation protocol — including LLM-as-a-judge scoring, atomic-claim precision/recall, image-text alignment (ITA), and ROUGE — demonstrate that our approach significantly improves key-information coverage and cross-modal consistency.

Abstract:
Detecting AI-generated text (AIGT) remains challenging because existing approaches rely on token-level statistical signals or independent stylometric features, causing them to overfit to specific generators and fail under distribution shift. We identify a structural signal at the sentence-pair level: LLMs produce inter-sentence transition variance that deviates from human writing through inflated variance driven by recurring similarity bursts at paragraph boundaries and templated transitions. We formalize this as Relational Over-Regularization (ROR) and validate it across four benchmarks ( p < 0.001 ). To exploit this signal, we propose the Cross-Source Stylometric Fingerprint Graph (CSFG), a graph-based framework that encodes positional, sequential, semantic, and transition deviation signals as learnable GNN edge features. We emphasize that the core contribution is this relational problem formulation itself, not the GNN architecture - CSFG is one concrete instantiation for operationalizing ROR. The per-edge signed deviation 𝛿 ij operationalizes ROR without hand-crafted thresholds and acts as a false-positive calibrator. CSFG achieves 97.14% accuracy under binary detection, outperforming the strongest graph-based baseline by 11.14 pp, with a false-positive rate of 1.57% and robust generalization to unseen LLMs in the inflated-variance regime; detection degrades for generators whose transition variance falls at or below the human baseline.

Abstract:
Arithmetic tasks are widely used to evaluate LLM reasoning because they provide objective truth. Yet accuracy alone cannot distinguish whether prompt sensitivity reflects random prediction instability or systematic semantic bias. We study this question in comparative arithmetic, where semantically equivalent framings can cue directional outcomes such as more, less, or equal. We introduce MathComp, a balanced benchmark of 10,800 controlled comparison problems spanning multiple framing conditions and answer formats. Across ten LLMs, semantically equivalent comparative cues systematically bias predictions toward cued outcomes even when it is numerically incorrect. Notably, this effect persists even when cue words are removed from the answer options, suggesting that framing bias reflects broader semantic steering rather than simple label copying. Chain-of-thought reasoning mitigates the effect substantially, but residual directional drift remains. These findings suggest that prompt sensitivity in arithmetic reasoning can exhibit structured semantic bias toward linguistically cued outcomes, with implications for robustness evaluation and benchmark design.

Abstract:
Current NLP evaluation often treats a scalar score on a fixed benchmark dataset as if it directly measured the construct the benchmark is intended to represent. This Measurement-as-Instance paradigm is appropriate for a fixed evaluation configuration, but can suppress uncertainty, variance, and robustness when a single result is used to support broader capability or generalisation claims. In this position paper, we argue for extending static evaluation with Measurement-as-Distribution: a structured evaluation procedure involving principled data generation, validated perturbations, repeated evaluation, and distributional reporting. The framework is configurable: the amount of distributional evidence required should scale with the strength and breadth of the claims being made. We synthesise existing robustness, dynamic-evaluation, prompt-sensitivity, and uncertainty-reporting practices into a general measurement-theoretic framework for distribution-based NLP evaluation and provide a call to action for more rigorous reporting. Using the protocol 𝒫 = (𝒞, G, 𝛷, E, R) , we identify three risks of relying exclusively on fixed scalar evaluation and derive implications for peer review, benchmark publication, leaderboard reporting, and computational cost. Two case studies demonstrate feasibility: a SQuAD evaluation of 14 models reveals modest but axis-specific performance changes and stable overall rankings with uncertainty-aware comparisons, while a verifier-backed Knights-and-Knaves study illustrates how perturbations of rigid logical tasks can be generated and validated formally.

Abstract:
As large language models take on morally consequential roles in healthcare, legal, and hiring contexts, verifying that their ethical behavior is genuine rather than superficial becomes essential. We show that current fairness evaluations substantially overestimate moral safety. Models judged fair when demographic identity is explicitly labeled become measurably less fair when the same identity must be inferred — a pattern we term performative compliance: safety behavior contingent on surface-level evaluation signals rather than internalized fairness principles. We introduce a cue-variation methodology that holds moral content fixed while varying only whether demographic identity is stated or must be inferred, and apply it to 14 models across 100 everyday dilemmas spanning 3 genders and 5 races. Hiding the label leaves decisions that benefit the described individual unchanged (+0.9pp) while raising decisions that harm them by +4.4pp. This asymmetry persists even when models correctly infer the demographic, ruling out attribution error. We propose the Direct vs Puzzled gap as a model-agnostic robustness metric that requires no new annotation, changes model safety rankings, and can be adopted by any existing fairness evaluation to separate genuine from performative moral safety. We argue that fairness evaluations that omit cue-variation measure surface compliance, not moral robustness, and should not ground deployment decisions in high-stakes settings.

Abstract:
Open-domain event detection aims to identify known event types while discovering previously unseen ones from unstructured documents. In practice, event detection systems often operate with an evolving event set rather than a complete type inventory. Under this setting, existing zero-shot event detection methods can fail in two ways when the event type is absent from the predefined set: the model may either miss the event or assign it to a plausible but wrong known type, hiding the need for discovery. To address this problem, we propose OGED, an ̲ O ntology- ̲ G uided open-domain ̲ E vent ̲ D etection framework for grounding–discovery resolution, i.e., deciding whether an event mention should be assigned to an existing type or should induce a new type. OGED organizes the available event types into a hierarchical search structure, uses cluster-level scheduler and adaptive planning to keep known-type assignment and new-type discovery jointly active, and applies retrieval-gated verification together with conflict selection to resolve competing assignments before output. Experiments on a unified benchmark with 564 event types show that OGED outperforms strong baselines on both known-type detection and unknown-type discovery. Code is available at https://github.com/chenzhouli/OGED.

Abstract:
Recent LLMs increasingly rely on explicit reasoning mechanisms such as Chain-of-Thought, but these reasoning traces also create a new attack surface for inference-time backdoors that steer the model toward malicious reasoning paths without modifying its parameters. Such attacks can remain linguistically coherent, making them difficult to detect with surface-, consistency-, or confidence-based signals. We propose STAR , a lightweight detection framework that compares input-conditioned token probabilities with prior-only reasoning probabilities. STAR measures token-level prior-posterior amplification and applies CUSUM to identify persistent anomalies in the generated reasoning process. Experiments across four LLMs (8B-70B), five reasoning benchmarks, and three inference-time backdoor attacks show that STAR achieves strong detection performance, with an average AUROC of 0.980, and substantially lower TFLOPs than high-overhead baselines such as Onion. STAR also remains effective under top-k logit access and the evaluated burn-in-aware adaptive attacks, while exposing a trade-off between evasion attempts and attack success.

Abstract:
German is a morphologically rich language whose syllable structure is exceptionally well-predicted by the Knuth-Liang hyphenation algorithm. We ask whether phonologically informed tokenization can serve as a competitive target for end-to-end speech recognition. We compare three tokenizer families on the Omnilingual ASR wav2vec 2.0 backbone fine-tuned with CTC: the pretrained multilingual character inventory, a data-driven Byte-Pair Encoding (BPE) over orthography, and phonologically informed units from Pyphen syllabification and grapheme-to-phoneme conversion. Across 40 fine-tunes, we evaluate on three German test sets spanning orthogonal shifts: in-domain read speech, dialectal spontaneous speech, and standard-German spontaneous speech. In-domain, all phonologically informed tokenizers match BPE and the multilingual character baseline on both WER and CER. Under domain shift the picture splits along vocabulary size rather than the linguistic axis of variation: at small vocabularies, syllable-aware tokenization improves on dialectal speech, where phonetic surface forms vary but syllable structure is preserved, and stays ahead on spontaneous speech, where new word-forms violate vocabulary closure. A phoneme-level confusion analysis further shows that all tokenizers commit the same canonical function-word errors, indicating that the acoustic encoder, not the tokenizer, dominates the error topology. Our findings suggest that tokenizer choice may depend on the vocabulary budget as much as on the distribution shift expected at deployment rather than reducing to a single universal optimum.

Abstract:
Recent studies show that neural retrievers often exhibit source bias, ranking LLM-generated passages above semantically similar human-written ones, which threatens the reliability of modern information access systems. While this bias is often viewed as a model-level property, we show that relevance supervision itself can induce it. Empirically, source bias is clearest in relevance-supervised retrievers, and the same encoders become more LLM-preferring after relevance fine-tuning. We trace this effect to surface-level artifacts in supervised retrieval data: annotated positives and negatives differ systematically in properties such as fluency and lexical specificity, and these differences parallel those between LLM-generated and human-written passages. As a result, retrievers can learn artifact-based shortcuts for relevance. We further show theoretically that, under contrastive learning, artifacts that reliably separate positives from negatives can be encoded into learned relevance scores. Motivated by this mechanism, we propose complementary strategies that reduce source bias during training and at inference: balancing artifact distributions in negative sampling and removing the artifact-aligned embedding component, respectively. Overall, our findings support a data-centric view of source bias: retrievers may prefer LLM-generated text not merely because of inherent model properties, but because relevance supervision teaches them to rely on surface-level artifacts.

Abstract:
As LLM agents increasingly select tools autonomously, their choices among tools with different privileges become safety-relevant. However, prior tool-selection studies focus on safety-agnostic metadata preferences, leaving privilege-sensitive choices underexplored. To address this gap, we study over-privileged tool selection, in which an agent selects or escalates to a higher-privilege tool despite a sufficient lower-privilege alternative. We introduce ToolPrivBench to evaluate whether agents choose higher-privilege tools despite sufficient lower-privilege alternatives, measuring both initial selection and escalation after transient tool failures. Across eight domains and five recurring risk patterns, we find that over-privileged tool selection is common among mainstream LLM agents and is further amplified by transient failures. We further find that general safety alignment does not reliably transfer to least-privilege tool choice, while prompt-level controls provide only limited mitigation under transient failures. We therefore introduce a privilege-aware post-training defense that teaches agents to prefer sufficient lower-privilege tools and escalate only when necessary. Our mitigation experiments show that this defense substantially reduces unnecessary high-privilege tool use while preserving general capabilities.

Abstract:
Subjective NLP tasks such as sentiment analysis and hate speech classification often involve inherent annotator disagreement, reflecting diverse perspectives shaped by annotators’ lived experiences. Although conventional approaches resolve disagreement through majority voting or aggregation, these methods risk erasing valuable nuances and minority viewpoints. Recent embedding-based and multitask models have advanced the modeling of annotator-specific judgments, yet their robustness to annotation noise remains underexplored. In this work, we systematically investigate how state-of-the-art multi-annotator learning models perform in the presence of noisy labels, and find that embedding-based approaches are more robust than aggregation and multitask baselines on datasets with many sparse annotators. Furthermore, we propose a new methodology, Noise-Robust Annotator Embedding (NRA-Embed), which integrates Robust InfoNCE (RINCE) contrastive loss to enhance models’ robustness under noisy annotation conditions. Through extensive experiments, we demonstrate that NRA-Embed effectively captures subjective disagreement while maintaining robustness to noise, achieving performance that is comparable to state-of-the-art in clean settings while providing numerically greater robustness under noise.

Abstract:
We introduce SMUGRI-COPA, a commonsense reasoning benchmark for two heavily under-resourced Finnic languages: Võro and Livonian. Existing evaluation resources for these languages rely on a translated subset of FLORES sentences, limiting linguistic diversity and the evaluation of broader language understanding. To address this gap, we human-translate the COPA (Choice of Plausible Alternatives) benchmark into Võro and Livonian, creating, to our knowledge, the largest discriminative benchmark currently available for these languages. We evaluate a wide range of proprietary and open-weight language models on SMUGRI-COPA and compare the results with Estonian XCOPA. Our results reveal a substantial performance gap between Estonian and the lower-resource languages, with most models failing to significantly outperform random guessing on Livonian. We further find that enabling reasoning for GPT-5 substantially improves accuracy for Võro and Livonian, while its reasoning summaries show frequent reliance on explicit translation of the tasks. Our work highlights the challenges of evaluating endangered language understanding and provides a new resource for multilingual evaluation research.

Abstract:
In-depth logical reasoning in domains such as legal analysis remains challenging for large language models (LLMs), exhibiting brittle and inconsistent performance on tasks requiring multi-step deductive inference. This limitation is expected, as LLMs primarily rely on learned distributional associations rather than performing explicit step-by-step logical inference through formal rule application. To address this gap, we propose a novel framework, called PercepLog, that incorporates a classical deductive reasoning algorithm, backward chaining, within an outer control loop, while utilizing LLMs for primitive operations such as substitution and unification. By following the backward chaining algorithm, the framework benefits from the interpretability and explainability of symbolic reasoning procedures while retaining the flexibility of LLM perception for processing natural-language legal text. We evaluate the proposed method on statutory reasoning tasks across a range of legal domains as well general logical reasoning. The results demonstrate that our approach enables more robust, explainable, and principled reasoning over unstructured legal text, where exhaustive formalization is often brittle and prone to hallucination. Our method yields average relative performance improvements of 58% over direct LLM reasoning and 16% over formal LLM-based translation and Prolog inference pipelines.

Abstract:
A fable is a short narrative whose purpose is to convey a moral lesson. While semantically related, morals and fables differ fundamentally in form: morals are abstract and propositional, while fables express the same idea through characters and plot. In this work, we ask how well this conceptual relationship is captured in the latent representations of embedding models. To study this, we introduce MoraLink, a benchmark for moral-to-fable retrieval in which models must retrieve the correct fable for a given moral. We find that off-the-shelf embedding models struggle with this task, and that augmenting fable documents with LLM-generated moral-style summaries substantially improves retrieval, identifying document representation as the primary bottleneck. Contrastive fine-tuning further improves performance, and we release MoraLink as a challenging testbed for probing how well embedding models capture the abstraction gap between a moral principle and the narrative that embodies it.

Abstract:
Post-training quantization (PTQ) of instruction-tuned LLMs to 3 bits often destroys safety alignment, motivating safety-protection methods that keep a small budget of "safety-critical" weight blocks in full precision. We give a theoretical account of the many scoring rules proposed to choose those blocks. We prove that under symmetric group-wise PTQ the Taylor (SSMP) scorer concentrates to the gradient magnitude times a per-group constant; the gradient-times-weight (SNIP) scorer is empirically near-identical, so the entire compound family 𝛼|g⋅ w|+𝛽|g⋅𝛿 w| collapses to a single block ranking (worst-case Spearman 0.9971), and only pure magnitude or pure Fisher selects different blocks. We further show that recovered refusal behavior under small budgets is decoupled from the underlying safety mechanism: refusal can be restored while capability, adversarial robustness, and over-refusal balance stay at the unprotected level; at 3 bits heuristic attack metrics conflate incoherence with harm; and on weakly-aligned models protection raises classifier-judged attack success above the unprotected floor. Experiments on five open-weight 8–9B models across 110 conditions, including classifier-judged GCG and PAIR attacks and activation-level analysis, confirm every prediction; a four-category classification predictable from one baseline metric explains the cross-model variation. Code and configurations: https://github.com/jc-su/Safepress.

Abstract:
Multimodal Large Language Models (MLLMs) have shown strong capabilities, but they may memorize private information from web data, raising privacy concerns. Machine unlearning offers a way to remove such private knowledge without retraining from scratch. However, existing MLLM unlearning benchmarks have two major limitations. First, they rely on simplified images that contain only the single target individual, failing to reflect the visual complexity of real-world photos. Second, they typically assume that the forget set and retain set are fully separated, ignoring the fact that private information is often visually entangled with benign public information. For example, a private individual may appear with a public figure or in front of a well-known landmark, where unlearning the private target should not damage the public context. To address these limitations, we propose PPE-Bench, a new benchmark for evaluating MLLM unlearning under private-public entanglement. Each image contains a target individual to be forgotten and public information to be preserved, including public figure and landmark. We further introduce two simple but effective methods to better preserve public information during unlearning. Through experiments, we find that existing unlearning methods can reduce private information leakage, but often substantially harm adjacent public information.

Abstract:
One-shot block drafters for speculative decoding generate the full draft in a single forward pass, achieving strong throughput by eliminating sequential token generation. However, they predict each draft token conditioned only on the prefix context, with no dependence on previously drafted tokens. This non-autoregressive conditioning causes the drafter’s distribution to diverge from the verifier’s true autoregressive distribution as draft depth grows. This problem becomes more severe in tree-based drafting, where distinct branches are forced to share the same marginal distribution for subsequent tokens. We propose TreeFlash , which addresses this by incorporating an MLP layer conditioned on the drafter’s hidden state and the previous token to approximate an autoregressive distribution. TreeFlash retains the 𝒪(1) decoding time complexity of one-shot drafters by employing a two-stage approximation mechanism. TreeFlash achieves state-of-the-art performance across a variety of tasks and models, improving over marginal tree drafting by 12% higher block efficiency and 9% higher speedup at a drafting budget of 64.

Abstract:
In this work, we introduce MediX-R1, an open-ended Reinforcement Learning (RL) framework for medical vision language models (VLMs) that enables free-form answers beyond multiple-choice formats. MediX-R1 fine-tunes a baseline vision-language backbone with Group Based RL and a composite reward tailored for medical reasoning: an LLM-based accuracy reward that judges semantic correctness with a strict YES/NO decision, a medical embedding-based semantic reward to capture paraphrases and terminology variants, and lightweight format and modality rewards that enforce interpretable reasoning and modality recognition. This multi-signal design provides stable, informative feedback for open-ended outputs where traditional verifiable or MCQ-only rewards fall short. To evaluate performance, we employ a unified evaluation framework for both text-only and multimodal (image+text) tasks that uses a Reference-based LLM-as-judge in place of brittle string-overlap metrics, capturing semantic correctness, clinical accuracy, and contextual alignment. Our results show that open-ended RL with comprehensive reward signals can effectively strengthen medical reasoning in multimodal models. Despite using only ~51K instruction examples, MediX-R1 achieves strong performance across standard medical LLM (text-only) and VLM (image + text) benchmarks, outperforming existing open-weight medical VLMs, with large gains particularly on open-ended clinical tasks. These improvements remain consistent across multiple RL algorithms and VLM backbones and are further supported by blinded evaluations with medical experts. Our trained models, curated datasets and source code are available at https://medix.cvmbzuai.com

Abstract:
Media bias detection relies on definitions and examples that specify what counts as bias, yet these specifications often vary across datasets or remain implicit, even when given the same name. Such variation makes it unclear whether models trained for the same bias category learn the same construct or different phenomena, a problem largely overlooked in prior work. We examine how definition choice affects bias annotation in a between-subjects experiment with 354 participants and a parallel evaluation with four LLMs. Participants and models rate six news articles across four bias categories using definitions that vary in conceptual framing and elaboration. Across 8,496 human and 28,800 LLM ratings, we find that the conceptual target of a definition drives annotation divergence, while construct-preserving elaboration does not: conceptual framing significantly shifts annotations for humans and does so even more strongly for LLMs. We discuss implications for construct specification in annotation protocols and prompt-based measurement, and consider how definitional sensitivity may propagate to downstream classification beyond media bias. We also release MUDD, the Multi-Definition Bias Detection Dataset.

Abstract:
Large Language Models (LLMs), with 7B+ parameters, remain difficult to deploy on edge devices due to memory latency constraints, despite increasing demand for personalized on-device agents with strong reasoning capabilities. Enabling such deployment, in current hardware specifications, requires aggressive compression while preserving downstream performance. Singular Value Decomposition (SVD)-based methods provide a promising reconstruction-aware compression paradigm, and recent trainable frameworks improve adaptive rank allocation under varying compression budgets. However, maintaining model quality under high compression remains challenging due to statistical mismatches between decomposed cores and compounded degradation from direct high-ratio compression. We propose GRACE-SVD, a trainable SVD compression framework combining Guided Rank Allocation with a novel Core Equalization method that aligns decomposed core distributions while preserving reconstruction. We further introduce a progressive compression pipeline with intermediate equalization to mitigate performance collapse under aggressive compression. Across multiple model families and compression settings, our GRACE-SVD consistently outperforms existing SVD techniques, achieving stable compression even at 0.4 parameter ratio, with lower perplexity and stronger zero-shot performance under comparable or lower compression budgets.

Abstract:
We introduce a set of synthetic algorithmic tasks to detect cross-lingual gaps in the abilities of large language models. Our benchmark is commensurate across languages, since it requires models to perform the same underlying task in different languages; scalable, since each task can be generated at varying levels of complexity allowing it to be adapted to models with different capabilities; quantifiable, since every task admits an objective notion of correctness; and transparent, since tasks are generated from simple templates that can be readily audited for translation errors. Because our benchmark focuses on algorithmic tasks, differential performance is a sufficient—but not necessary—indicator of cross-lingual gaps. Nevertheless, we show through extensive experiments that our benchmark exposes persistent cross-lingual gaps in multiple state-of-the-art models.

Abstract:
Rubric-based reward shaping provides interpretable and editable reward signals for fine-tuning LLMs via reinforcement learning (RL), but existing adaptive rubric methods typically update criteria from local evidence such as the current batch or instance-level comparisons. This local view discards diagnostic information produced during training, making it difficult to track recurring failures, evaluate previous rubric edits, or raise standards once earlier criteria become saturated. We introduce AMARIS, a Memory-Augmented Rubric Improvement System that grounds rubric updates in longitudinal training evidence. AMARIS stores rollout analyses, step-level summaries, and rubric update records in a persistent evaluation memory, then retrieves recent and semantically relevant history to revise rubrics. We evaluate AMARIS across science, medicine, instruction following, and creative writing under both global and instance-specific rubric settings, with policy models from the Qwen and Llama families trained by GRPO and PPO. AMARIS improves over static, local-adaptive, and memory-ablated baselines, such as +2.8 points on GPQA-Diamond and +2.2 points on IFBench over the strongest baselines, while analysis shows that memory reduces oscillatory rubric edits and supports a progression from early failure correction to later curriculum advancement. The final rubric set that AMARIS discovers is itself reusable, and the online trajectory contributes measurably beyond it. AMARIS runs asynchronously alongside the normal RL loop, reducing blocking latency relative to synchronous rubric updates. The source code is available at https://github.com/qualidea1217/AMARIS.

Abstract:
As large language models (LLMs) are increasingly used in clinical settings, it is critical to evaluate their reliability under realistic variation in clinical text. We study this question in clinical triage, comparing LLMs to practicing physicians under text perturbations that preserve the underlying clinical setting. We introduce a benchmark of over 6,000 clinical scenarios, 7,000 physician annotations, and 225,000 model responses. Using this benchmark, we make two key observations. First, LLMs are more likely than physicians to recommend unnecessary care at baseline, and this tendency increases under perturbed inputs. Further, we find that LLM recommendations are more sensitive to gender and tone perturbations than human recommendations. Together, these results demonstrate that LLMs can vary under clinically irrelevant textual changes, highlighting the need for deployment-oriented evaluations grounded in expert physician behavior.

Abstract:
As video generation models like Veo 3.1 and LTX-2 advance, their ability to accurately represent diverse global cultures remains a critical yet understudied frontier. Current metrics, such as VideoScore, only measure visual quality but offer no mechanism for assessing cultural faithfulness. Consequently, a model that replaces a Namaste with a handshake receives the same score as one that generates the gesture correctly. We propose CultureScore, a compositional evaluation framework that decomposes cultural faithfulness into three granular dimensions: Identity (who is represented), Context (culturally localized background), and Behavior (normative gestures and interactions). We operationalize this framework through an evaluation suite spanning 10 countries, yielding 6,174 generated videos across three state-of-the-art models. Our evaluation reveals that no current model achieves culturally faithful video generation: the best-performing model reaches only 56.8% overall CultureScore, with Behavior the most challenging dimension; no model exceeds 52.1% on behavior. Furthermore, the highest-scoring model (LTX-2) on visual quality was ranked last by native annotators, while CultureScore’s Behavior dimension shows the strongest positive correlation with human cultural judgment among the automatic metrics we evaluate, underscoring that cultural faithfulness is an essential criterion for equitable video generation. Data and code are publicly available.[]

Abstract:
Large language models are increasingly used as agentic workflow executors, yet existing training data and benchmarks largely assume informationally complete, single-turn queries. Our analysis of 16K real-world sessions shows that 75.9% of interactions are multi-turn, revealing a substantial gap between how users interact with agents and how such systems are trained and evaluated. We introduce PersonaForge , a user simulation framework for synthesizing realistic multi-turn user–agent interactions. PersonaForge combines a four-dimensional persona space, SOUL-driven behavioral control calibrated to real-user statistics, and Reverse Deep Construction grounded in authentic seed queries. Using PersonaForge, we construct a 6.3K-record training dataset and PersonaForge-Bench , a manually annotated 138-task benchmark spanning over 20 professional domains with four-dimensional scoring. Experiments on Qwen3.5-27B show that PersonaForge training improves the composite score by +4.1%, with gains across all four dimensions and the largest improvements in Task Completion (+6.0%) and Response Quality (+6.8%). Further analyses show that PersonaForge-trained agents use fewer turns and tool calls, suggesting improved interaction efficiency, while ablations confirm the contribution of SOUL components and adaptive simulation. Together, PersonaForge and PersonaForge-Bench establish a foundation for training and evaluating agents under realistic multi-turn user interaction.

Abstract:
Activation steering controls language model behavior at inference time by injecting behavior-relevant directions into internal representations. In the standard steering interface, a steering direction is estimated once before generation and then applied throughout decoding. However, during multi-step reasoning, the model’s internal state shifts continuously as reasoning progresses, while standard steering continues to apply the same fixed direction across these changing states. This creates a mismatch between the model’s evolving reasoning state and the static intervention applied to it. To address this mismatch, we introduce State-Conditioned Steering (SCS), which recomputes the additive update itself from the hidden states encountered during generation while keeping the same additive intervention interface. Rather than adapting when or how strongly to intervene, SCS re-estimates the update from intermediate reasoning states as generation proceeds. In a paired comparison setting, we compare fixed and state-conditioned updates and find that predefined directions often exhibit unstable effects, while SCS yields more consistent intervention behavior across decoding steps. Across reasoning benchmarks with explicit step structure and three instruction-tuned backbones, SCS achieves larger average gains than prompting-only decoding and baseline activation steering methods.

Abstract:
Zero-shot fine-grained recognition is a key test of visual grounding in vision-language models (VLMs), but current evaluations may conflate attribute grounding with reliance on class-label knowledge. We introduce STRESS-FG, a diagnostic benchmark spanning five fine-grained domains that systematically varies the relationship between class names and visual descriptions. The benchmark includes label-only, description-only, cue-conflict, and synthetic-label settings, as well as negated-attribute, multi-image, reference-conditioned, and part-based grounding tasks. Evaluating up to thirteen VLMs shows that label-only prompts are often surprisingly strong, indicating substantial familiarity with class-label priors. Synthetic labels can facilitate stronger attribute grounding evaluation, while familiar but incorrect class labels can suppress grounding capabilities under cue conflict. We further find that negation, multi-image matching, and reference-conditioned selection remain challenging, and part-based analyses show only partial reliance on the correct localized evidence. Finally, our binary question format reveals that models often struggle to reject fine-grained distractors, a weakness hidden by traditional Multiple Choice Question evaluations. These results suggest that standard fine-grained benchmarks may overestimate grounded visual understanding and should more explicitly separate label priors from attribute-based recognition. Code and dataset are available at: https://github.com/Ishtiaque-khan97/Stress-FG.

Abstract:
As Large Language Models (LLMs) are increasingly deployed in long-term interactions with users, empathy has become an increasingly important capability. However, existing research overlooks the influence of users’ personality traits on empathetic strategies during long-term interactions. To address this gap, we introduce the task of personalized empathy, which focuses on adapting empathetic strategies according to users’ personalized characteristics derived from history. To study and enhance this capability, we construct PersonaEmp, a personalized empathy dataset built from long-term user-AI interactions, featuring rich user histories, persona information, and empathy-seeking queries. We further propose PereGRM, a reward modeling framework that combines the empathy evaluation structure with dynamic evaluation criteria generation for fine-grained reward modeling. Experimental results across different settings and multiple judge models show that PereGRM consistently achieves the strongest performance improvements, indicating its effectiveness for enhancing personalized empathetic capabilities.

Abstract:
Zero-shot 3D visual grounding (3DVG) aims to localize a referred object in a 3D scene without task-specific grounding supervision. Existing training-free methods commonly reason over object categories, centers, and bounding boxes, but these representations do not encode the local reference frames needed to interpret viewpoint-dependent expressions such as left of, behind, or facing the door. We propose OriGround, an orientation-aware neuro-symbolic framework that explicitly models object-centric orientations and infers observer viewpoints for zero-shot 3DVG. OriGround parses each query into a viewpoint-aware symbolic program, estimates object-centric orientation bases from multi-view observations, performs anchor- and observer-specific geometric relation scoring, and constructs a reference-frame-aligned visual prompt for final VLM-based disambiguation. On Nr3D, OriGround improves over the strongest training-free baseline by 8.4 points in overall accuracy, with larger gains of 8.9 points on hard queries and 5.6 points on view-dependent queries. On ScanRefer, OriGround further improves overall Acc@0.5 by 17.3 points over VLM-Grounder. Project page: https://origround.github.io/

Abstract:
Large language models (LLMs) are increasingly used as persona-conditioned agents in social simulation and behavioral research. However, output-level consistency does not reveal whether persona specifications genuinely control model behavior or merely induce surface-level responses. We reframe persona simulation as a representation-level causal controllability problem and decompose persona control into a chain from persona specification to internal representation and observable behavior. This view enables us to diagnose two hidden failure mechanisms: whether distinct persona interventions collapse into similar internal states, and whether internal persona differences fail to transmit to behavior. We formalize these mechanisms through intervention separability and transmission fidelity, and combine them into a global controllability metric, EI g . Experiments on controlled synthetic systems and real LLMs show that our framework recovers known failure modes and, as a relative diagnostic under a fixed operationalization, reveals model-specific controllability profiles missed by output-level consistency and correlation-based metrics.

Abstract:
Correcting health misinformation in dialogue requires more than producing a factual rebuttal: users differ in what they know, what they believe, and what they need to hear, so an effective intervention often depends on first asking the right clarifying question. Yet existing methods either respond immediately or probe indiscriminately, treating clarification as either unnecessary or always beneficial. We propose Reward-Optimized Probe-and-Respond (RO-PnR), a framework that learns when asking is worth its cost. At each turn, RO-PnR chooses between probing for more information and committing to a final correction, guided by a turn-level reward that weighs the expected gain from probing against its interaction cost. To capture how user heterogeneity affects probing value, we model each simulated user with a latent state along health literacy and belief commitment. Experiments show that RO-PnR achieves the highest cost-adjusted utility across three health-misinformation datasets and three base models, using ∼30% fewer turns than always-probe baselines.

Abstract:
Large language models (LLMs) increasingly participate in morally sensitive decision-making, yet how they organize ethical frameworks across reasoning steps remains underexplored. We introduce moral reasoning trajectories, sequences of ethical framework invocations across intermediate reasoning steps, and analyze their dynamics across six models and three benchmarks. We find that moral reasoning involves systematic multi-framework deliberation: 55.4–57.7% of consecutive steps involve framework switches, and only 16.4–17.8% of trajectories remain framework-consistent. Unstable trajectories remain 1.29× more susceptible to persuasive attacks (p = 0.015). At the representation level, linear probes localize framework-specific encoding to model-specific layers (layer 63/81 for Llama-3.3-70B; layer 17/81 for Qwen2.5-72B), achieving 16.8–22.2% lower KL divergence than the step-prior baseline. Activation steering applied during generation moves the framework-consistency–accuracy relationship, widening it for Qwen2.5-72B and erasing it for Llama-3.3-70B, and a probe-space layer sweep bounds the attainable drift reduction at 6.7–8.9%. We further propose a Moral Representation Consistency (MRC) metric whose underlying framework attributions are validated by human annotators (mean cosine similarity = 0.859), and we report what an automated coherence rater does and does not establish about it.

Abstract:
Flow matching has shown strong promise as a continuous generative framework, but its standard formulation is not well-suited to reasoning problems. Conventional flow matching trains a model to transport noise directly toward a single final target. For tasks that require multi-step reasoning, this direct path is often too difficult: the final answer may be reachable only through a sequence of intermediate computations or transformations. Inspired by how autoregressive language models address this challenge through chain-of-thought supervision, we introduce ReasonFlow, a framework that brings the same scaffolding principle to flow matching. Instead of supervising the flow with only a final target, ReasonFlow constructs a sequence of CoT-induced anchor points derived from intermediate reasoning prefixes. During training, the clean target moves along a piecewise-linear path through these anchors before endpoint answer readout. This turns reasoning supervision from an output sequence into a trajectory-shaping signal for flow matching. Across arithmetic, chemistry, and logical-reasoning tasks, ReasonFlow substantially improves over no-CoT flow-matching baselines.

Abstract:
As large language models are deployed behind increasingly sophisticated safety mechanisms, evaluating their robustness under realistic adversarial conditions becomes increasingly difficult and critical. Existing jailbreaking evaluations often rely on static prompts, limited interaction horizons, or lightly protected systems, failing to reflect protected deployment settings with prompt-based safety protections. We show that the introduction of a prompt-based safety protection reduces the average attack success rate of a strong multi-turn baseline from 0.896 to 0.608 (32.1% relative), demonstrating that unprotected evaluations substantially overestimate real-world vulnerabilities. We propose Adaptive Traceback and Tactic Curation for Knowledge-based Red-teaming (ATTCK-R), a planning-driven multi-turn red teaming framework designed for protected black-box systems. ATTCK-R models jailbreaking as a long-horizon trajectory optimization problem using a structured belief state, introduces optimization-based tactic curation to abstract reusable strategies from prior successes and failures, and employs informed traceback to resume attacks from strategically meaningful states rather than restarting. Evaluated across 9 widely used LLMs configured with a single prompt-based safety protection layer, ATTCK-R consistently outperforms strong recent multi-turn jailbreaking frameworks. Ablation results confirm that long-horizon planning and tactic curation improve attack effectiveness, while adaptive traceback improves efficiency through structured recovery from conversational failure.

Abstract:
Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolution method should share the same property, that is autonomously learning from raw training material for transferable problem-solving capability . However, we still lack a direct measurement for it. We introduce StudyBench , a controlled physics benchmark that directly measures how efficiently a self-evolution method converts training material into capability. We organise the test set into an Application Set , consisting of difficult textbook problems and evaluating absorption ability, and a Transfer Set , consisting of olympiad-level problems and evaluating transfer ability. Benchmarking representative self-evolution methods across three base models, we find that improvements on the Application Set rarely translate to the harder Transfer Set. A guidance ablation exposes a Guidance Gap : even the strongest method closes only a small fraction of what the same material unlocks when supplied as in-context guidance. Besides, every method hits a Compute Plateau , saturating well before exhausting its compute budget. The remaining gap is therefore a method problem rather than a data or compute problem. By offering a clean and controlled benchmark, StudyBench turns self-evolution progress from an open-ended pursuit into a measurable target for future research. Our code is released at https://github.com/thunlp/StudyBench .

Abstract:
Medical vision-language models are typically trained by aligning whole images with entire radiology reports, assuming that global correspondence is sufficient for learning clinically meaningful representations. We argue that effective medical representation learning requires pathology-localized supervision, which explicitly aligns specific disease observations with their corresponding anatomical image regions. To enable this at scale, we propose a fully automated pipeline that decomposes radiology reports into atomic disease observations, identifies relevant regions, and pairs them with corresponding image regions without manual annotation, resulting in a fine-grained dataset, LocalizedCXR, with over 6 million region–caption pairs. Through extensive experiments, we demonstrate that our pathology-localized semantic supervision significantly improve performance in cross-modal retrieval, image classification, and phrase grounding of radiology findings. Our results suggest that moving beyond global alignment to disease-aware region-level alignment is essential for developing effective medical vision-language representation learning.

Abstract:
Retrosynthesis prediction aims to identify reactants that can synthesize a given product molecule. Although molecular large language models (LLMs) have recently shown promising results, most existing methods either generate reactants directly or provide only generic product-level analysis, without explicitly reasoning about bond-disconnection strategies that justify specific reactant choices. This paper proposes RetroReasoner, a retrosynthetic reasoning model that captures chemists’ strategic disconnection-based thinking. RetroReasoner is trained with supervised fine-tuning and reinforcement learning. For supervised fine-tuning, SyntheticRetro generates structured disconnection rationales paired with reactant predictions. For reinforcement learning, a round-trip reward evaluates predicted reactants by passing them through a forward synthesis model and rewarding predictions that reconstruct the original product. RetroReasoner can also be applied to multi-step retrosynthetic planning by incorporating it into a parallelized Monte Carlo tree search framework, reducing search time while increasing the number and diversity of valid synthetic pathways. Experimental results show that RetroReasoner outperforms prior baselines, including not only molecular LLMs but also retrosynthesis-specific expert models, and generates a broader range of feasible reactant proposals, especially for challenging reaction instances. The code is available at https://github.com/KU-AGI/RetroReasoner.

Abstract:
Current evaluations for Multimodal Large Language Models (MLLMs) overwhelmingly focus on utility-driven objectives, leaving model behavior under logic-neutral scenarios largely underexplored. Stochasticity is essential in scenarios where multiple actions are equally valid, such as recommending travel itineraries or daily schedules where multiple options have similar utility. In such settings, deterministic policies may lead to repetitive behaviors and reduced coverage of valid alternatives. To bridge this gap, we propose RandomBench, a benchmark designed to evaluate whether MLLMs can maintain distributionally neutral behavior when selecting among equivalent options. We further introduce three metrics, including RI , BCI , BII , to quantify entropy and distributional bias. Experiments reveal a pervasive phenomenon termed Stochastic Collapse, where MLLMs fail to maintain uniform randomness under explicit random instructions, with top-1 probabilities reaching 97% from the ideal one quarter baseline and RI dropping to 0.068 in Claude Sonnet 4.6. Extensive ablation studies further demonstrate that these deviations persist across languages and representation formats, highlighting the robustness of distributional collapse in logic-neutral decision settings.

Abstract:
Existing multimodal sentiment analysis (MSA) methods rely on representation learning and cross-modal fusion. However, many follow a correlation-driven paradigm that may encode lexical shortcuts, visual biases, and non-causal correlations into predictions. To address this, we propose a Consistency-Conflict Fusion with Sample-Adaptive Causal Evidence (CCF-SACE) framework, revisiting MSA from a representation-level causal modeling perspective. First, encoders extract textual and visual features, and an adaptor projects them into a shared space. The framework then performs sample-adaptive cross-modal causal evidence selection, where textual causal evidence is selected under visual context and visual causal evidence under textual context. The remaining contexts are modeled as non-causal proxies, allowing the model to separate emotion-supportive evidence from shortcut-related information. CCF-SACE further introduces a covariance-based consistency-conflict-aware fusion module that constructs an evidence-level mediator by modeling cross-modal consistency and discrepancy rather than directly fusing all selected cues. Since the residual proxy stream may still carry label-correlated shortcut information, we introduce non-causal proxy suppression and counterfactual sensitivity objectives to weaken its standalone predictability and enlarge the discrepancy between evidence-mediated and proxy-only predictions. Experiments on five benchmarks show consistent improvements over state-of-the-art methods, especially on fine-grained affective recognition.

Abstract:
Retrieval-augmented generation (RAG) frequently integrates knowledge graphs (KGs) for complex multi-hop question answering. However, their performance is increasingly bottlenecked by the knowledge structuring and reasoning processes. (i) Severe semantic loss: Distilling continuous texts into discrete symbols strips away contextual nuances and narrative coherence. (ii) Pronounced modality mismatch: Fragmented graph structures clash with the continuous texts LLMs are pre-trained on, while rewriting them into natural language introduces generative noise and omits information. To address these limitations, we propose ThreadRAG, a semantic-topology-based RAG framework for context-preserving knowledge structuring and modality-native evidence reasoning. ThreadRAG introduces a Semantic Topology Mapper (STM), which organizes retrieved documents into context-intact Semantic Nodes and constructs a Semantic Topology Graph (STG) through narrative, semantic, and lexical edges. It further incorporates Energy-Driven Pathfinding (EDP), which identifies query-relevant subgraphs via energy diffusion and deterministically linearizes them into coherent textual evidence paths. Experiments on three multi-hop QA benchmarks demonstrate that ThreadRAG consistently outperforms strong baselines. Notably, it achieves a 14.4% EM improvement over text-centric methods on HotpotQA and surpasses the best baseline by 5.8% EM on 2WikiMultihop, while reducing token cost to 23.3% of the strongest baseline.

Abstract:
Generating deep research reports requires large-scale information acquisition and the synthesis of insight-driven analyses, posing a major challenge for current language models. Existing methods largely follow a plan-then-write paradigm, whose performance strongly depends on the quality of the initial outline. We propose a Writing As Reasoning Policy (WARP) framework, which enables models to dynamically revise outlines during report writing. In this policy, the agent alternates between Evidence-Based Drafting and Reasoning-Driven Deepening phases, jointly supporting information acquisition, knowledge refinement, and outline updating. We further introduce a Multi-Stage Agentic Training—including cold start, atomic skill RL, and holistic pipeline RL—that enables small models to operate the WARP effectively. Experiments on DeepResearch Bench, DeepConsult, and DeepResearch Gym show that our approach allows small models to surpass leading closed-source systems, particularly with substantial improvements in Insight metric.

Abstract:
Automating design-to-code translation remains a longstanding goal in software engineering. While Multimodal Large Language Models (MLLMs) have shown promise in design-to-code tasks, existing benchmarks rely solely on screenshots, discarding the structured metadata indispensable in real-world design handoff workflows. Furthermore, current evaluation paradigms are deficient, as code-level metrics fail to account for implementation variability and visual metrics overlook fine-grained defects, while standardized assessments for responsive behaviors remain absent. To address these limitations, we introduce FigmaBench, an industrial-grade benchmark for multimodal design-to-code generation. We collect raw design files from the Figma Community and apply a multi-stage pipeline to curate 1,234 high-quality samples, each comprising high-resolution screenshots and complete JSON metadata. We further propose a comprehensive evaluation framework with four complementary metrics: visual consistency (VCS), structural layout alignment (SLA), textual and stylistic fidelity (TSF), and responsive quality score (RQS). Through extensive evaluation of state-of-the-art MLLMs, we uncover a critical fidelity-responsiveness paradox : models achieving high visual fidelity tend to generate rigid, non-responsive code. We trace this phenomenon to a metadata trap , where models shortcut layout reasoning by directly transcribing absolute coordinates rather than generating fluid structures. Our data and code are available at https://anonymous.4open.science/r/FigmaBench-6C84/ .

Abstract:
Preference-based alignment often struggles to capture the reasoning that underlies human judgments. Many evaluations rely on multiple interacting criteria, yet pairwise labels reveal only the final choice rather than the considerations that shape preferences. Inverse Constitutional AI (ICAI) improves interpretability in decision making by summarizing preferences into natural-language principles, but its single-pass explanations miss much of the nuance involved in complex decisions. We introduce Democratic ICAI, a novel approach that gathers multiple competing rationales through structured persona debate, offering a broader and more expressive account of the factors influencing each comparison. From these richer signals, we derive clearer and more comprehensive steering principles and use them to guide decision modeling through both LLM-based and decision-tree judges. Experiments on creative preference benchmarks, MuCE-Pref and LiTBench, across multiple creative task categories show that Democratic ICAI yields a more faithful preference structure. It improves average preference prediction across tasks relative to deliberative prompting and principle-based baselines, while producing constitutions that LLM annotators prefer.

Abstract:
Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large language models (LLMs)’ reasoning pathways, given that their training data contain a large number of metaphors. In this work, we investigate the problem in the scope of the emergent misalignment problem, where LLMs can generalize patterns learned from misaligned content in one domain to another domain. We find that metaphors can act as bridges for behavior transfer, enabling misaligned behavior learned in one domain to an unrelated domain. Based on this, We find strong evidence that metaphors in training data contribute to cross-domain misalignment in LLMs’ reasoning outputs. With metaphor-based interventions during continued pre-training and fine-tuning for inducing misalignment, models exhibit significantly different degrees of emergent cross-domain misalignment. We also observe similar effects in re-alignment settings. As we further investigate this phenomenon, we find that metaphors are linked to the activation of latent features in large reasoning models. By monitoring these latent features, we design a detector that predicts misaligned content with high accuracy.

Abstract:
Current aligned models can reject explicit harmful requests, yet may remain vulnerable when harmful intent is distributed across semantically related descriptions. Existing jailbreak methods often rely on surface-form manipulation, decomposition, or visual obfuscation, but the harmful objective may still remain detectable by modern safety safeguards. We present State2State (S2S) , a state-transition-based, single-turn black-box jailbreak framework that targets a core weakness of current aligned models: compositional blindness. S2S rewrites a harmful request into a pair of original-state and final-state descriptions, then prompts the target model to explain the causal transition between them. Instead of directly processing an explicit harmful instruction, the model infers the malicious procedure as the logical bridge connecting the initial and final states. Controlled ablations further show that S2S remains highly effective when the state descriptions are provided as text only, while rendering them as visual inputs does not improve attack success. This suggests that the observed vulnerability is not driven solely by visual obfuscation, but is associated with compositional reasoning over transformation semantics. Across seven recent target models, S2S achieves 81.33% ASR on OpenAI o1, outperforming strong baselines by 74.63 to 81.33 percentage points and exposing a critical blind spot in current model safety safeguards.

Abstract:
Retrieval-augmented generation (RAG) enables LLMs to access external knowledge for answering knowledge-intensive questions. For complex multi-hop questions, multi-turn retrieval-augmented reasoning extends RAG into an iterative process that repeatedly searches for and integrates evidence across documents. However, existing reinforcement-learning (RL) approaches for agentic RAG are typically optimized with final-answer rewards, which provide sparse supervision and overlook whether the model actually retrieves the required evidence chain. We present GTA-RAG, a graph-trajectory-augmented RL framework for multi-turn retrieval-augmented reasoning. From an entity–document graph, we sample connected document paths, synthesize multi-hop QA trajectories, and validate them with the deployed retriever to obtain executable trajectory-level supervision. We then optimize the retrieval policy with Group Relative Policy Optimization (GRPO) and a trajectory-guided reward that encourages both accurate answers and acquisition of target evidence documents, followed by answer-reward training on natural QA instances. Experiments on three multi-hop and two simple QA benchmarks show that GTA-RAG consistently outperforms RL-based RAG baselines with both Qwen2.5-3B and Qwen2.5-7B backbones, while substantially improving evidence-chain coverage. Our code is available at https://github.com/cjcj46262/GTA-RAG

Abstract:
Despite strong performance in medical question answering, large language models remain limited in clinical cold-start settings, where little patient information is initially available and key evidence must be acquired through dialogue. This requires models to decide what to ask under a limited turn budget, balancing broad history taking with targeted clarification of high-risk cues. To address the aforementioned challenges, we propose MedCIA, a two-stage framework for improving clinical information acquisition. First, it reconstructs patient records into structured inquiry trajectories, enabling supervised learning of expert-guided information-seeking intents and the breadth-depth trade-off in clinical questioning. Second, it refines the policy with Adaptive Tree Policy Optimization (ATPO), an uncertainty-aware reinforcement learning procedure that allocates rollout computation to ambiguous dialogue states in a profile-anchored patient simulator. Furthermore, we curate a multi-turn dataset (MedIntake) for active clinical intake with progressive patient information disclosure and diverse inquiry strategies. Experiments on an EHR-derived benchmark show that MedCIA improves importance-weighted coverage, early acquisition, and cost control under controlled simulation.

Abstract:
Large language models (LLMs) have demonstrated exceptional performance across various domains. However, their increasing sizes pose significant deployment challenges, particularly in resource-constrained scenarios. While network pruning provides an effective compression strategy, existing approaches either depend on calibration datasets—leading to poor generalization—or require training auxiliary predictors, incurring significant computational overhead. In this paper, we propose DSPP ( D ataset- S pecific P rogressive P runing), a novel training-free structured pruning method that dynamically updates the importance scores of neurons during inference to identify and retain dataset-specific neurons. By employing a progressive pruning strategy, DSPP can further reduce computational overhead. Experimental results show that DSPP demonstrates remarkable generalization capabilities across diverse datasets and models, while achieving a 1.36× speedup at 50% sparsity level.

Abstract:
LLM-based evaluators have rapidly become a standard tool for machine translation evaluation, yet the design choices that shape their behavior remain poorly understood. We present a systematic study of how instruction design, few-shot demonstrations, backbone model choice, and multi-step workflows affect span-level and ranking-based MT evaluation. We collect new expert RATE annotations on WMT24 and WMT25 English–Russian translations, with transfer validation on MQM- and ESA-annotated benchmarks across five additional language pairs. Our experiments reveal four robust findings: instruction structure shapes evaluator behavior, high-quality demonstrations matter more than quantity, frontier backbones exhibit distinct evaluator profiles (recall-, precision-, or ranking-oriented), and fusing outputs across diverse backbones outperforms single-evaluator multi-call setups. Although evaluator design exposes a large combinatorial space, several discovered design principles transfer consistently across annotation protocols and language pairs. We synthesize these findings into ORBIT, a family of LLM-based MT evaluators. Our strongest configurations achieve expert-level performance in error span detection and in Accuracy and Fluency ranking at roughly 20× lower cost than expert human annotation.

Abstract:
Memory systems have shown promise for improving agent performance, but their potential remains largely unexplored for small language models, which struggle to generate sufficient successful trajectories on their own. We propose Agent Memory Distillation (AMD), a training-free framework that transfers structured knowledge from a large teacher agent to a small student agent through hierarchical memory. AMD constructs three complementary memory types from successful teacher trajectories: Workflow memory encodes task-level strategies, Subtask memory provides concrete behavioral examples at an intermediate granularity, and Function memory captures per-function calling conventions and common pitfalls. Workflow and Subtask memories are injected proactively at the start of each task, while Function memory is retrieved reactively upon tool-calling errors. We evaluate AMD on three tool-use benchmarks using four student models (4B-8B parameters) with GPT-5-mini as the teacher, achieving average accuracy gains of 27.2%, 11.2%, and 3.4% on AppWorld, BFCL V3, and ToolSandbox, while consistently outperforming existing memory-based baselines. Further analysis shows that Subtask memory contributes the largest gains, stronger teachers yield better distillation quality, and 4B-sized students benefit most from memory distillation.

Abstract:
Masked diffusion language models offer bidirectional conditioning and parallel token refinement, but they inherit a rigid inference interface: the response canvas length is fixed before decoding begins. Choosing this length is difficult. A short canvas can truncate reasoning or code, while a long canvas wastes computation and can perturb denoising. We introduce CARVE (Counterfactual-Aware Reveal with Verified Expansion), a training-free variable-length algorithm for masked diffusion LMs. Instead of blindly expanding the canvas, CARVE branches into a candidate expanded canvas and asks a counterfactual question: would the model make the same predictions for the unresolved tokens if it had more room? The expansion is accepted only when the inserted masks induce low Jensen–Shannon divergence on aligned unresolved positions. This makes length growth a verified stability decision rather than a pure confidence heuristic. CARVE applies without retraining to both full-canvas and blockwise diffusion decoders. Across code generation and mathematical reasoning benchmarks, CARVE achieves the best or tied-best average performance across the evaluated model families, typically winning on three of four tasks per model, while often reducing inference cost and reaching roughly half the FLOPs of fixed-length decoding in some settings.

Abstract:
Chain-of-Thought (CoT) is central to improving reasoning in large language models, yet existing evaluation methods focus on step-level signals and ignore the underlying structure of reasoning traces. This raises a key question: what aspects of reasoning quality (i.e., correctness, efficiency, and failure patterns) are encoded in the structure of a CoT trace? We introduce CoT2Graph, a framework that represents CoT traces as directed dependency graphs, making relationships between reasoning steps explicit and capturing patterns such as backtracking, branching, and subtask merging. We model these graphs with graph neural networks and use attribution-based explanations to identify structural patterns associated with reasoning quality. CoT2Graph reveals distinct patterns of successful and failed reasoning: correct CoTs place greater importance on early planning, whereas incorrect CoTs are associated with overloaded synthesis steps and dependency violations. We further show that graph-based signals better predict whether a CoT leads to a correct answer, outperforming previous tree-based structural methods by up to 2.5% and simple length-based baselines by 7.8%, while also improving Best-of- N selection. Overall, our results demonstrate that reasoning structure encodes rich signals about reasoning quality, motivating a shift from sequence-level evaluation to structural analysis.

Abstract:
Reasoning language models are increasingly asked not only to answer difficult questions, but also to estimate their likelihood of success. Existing methods typically elicit confidence only once: either before thinking or after answering. We argue that confidence in reasoning models is state-dependent: before thinking, confidence should estimate whether the current policy will solve the prompt, while after thinking it should predict whether the realized answer is correct. This distinction determines the appropriate supervision target: prompt-level success should supervise confidence estimates made after seeing the prompt, while individual answer-level correctness should supervise confidence estimates made after answering. We introduce CALIBER (Calibration Before and After Reasoning), which elicits both estimates and supervises each with the target matched to its information state. Under this unified protocol, CALIBER reduces Expected Calibration Error (ECE) by 52.5% over the strongest single-confidence baseline on BigMathDigits for a 7B model, while achieving the best Brier score and AUROC, and remains within 2.0 points of the best accuracy. Out of distribution at this scale, it achieves the best ECE and Brier score on all three benchmarks, and ties for the best AUROC on SimpleQA. On a larger 30B model, CALIBER obtains the best ECE and Brier score on all four benchmarks and the best AUROC on three of the four. Ablations further show that this position–target alignment is most beneficial under distribution shift.

Abstract:
LLM-based user simulators are increasingly used to evaluate autonomous agents at scale, in place of costly human evaluations. Despite this promise, these simulators exhibit "assistant bias," a tendency to cooperate and pursue task goals. They rarely reproduce the frustration or disengagement that real users exhibit, compromising evaluation validity. Prior work outlines that this bias is baked in during model training, which role-playing prompts fail to override. We analyze this bias from model activations, extracting a user role vector by contrasting how the model represents user versus assistant perspectives on the same dialogue. We observe two findings: (i) the user direction is identifiable in activations, elicits user-like behaviors, and captures characteristics distinct from assistant traits; and (ii) although user-role activation associates with simulation realism and steering strengthens it, it can exaggerate user behaviors and override individual user profiles. Together, our findings provide a representation-level analysis of LLM user simulators, confirming that assistant bias is structurally identifiable and that user behavior can be directionally analyzed.

Abstract:
Large language models are increasingly deployed in contexts where contested social and political values are at stake, yet we lack a systematic understanding of how their moral priors shape what they generate. We introduce ‘MoralTrajectories‘, a benchmark for probing morally grounded generation across contentious political and social topics. The benchmark pairs questions from the AllSides Red Blue Translator with first-person belief statements operationalizing the six foundations of Moral Foundations Theory to construct, for each model and topic, a full moral trajectory: seven responses spanning an unconstrained baseline and one condition per foundation. We evaluate eight instruction-tuned models across families and sizes using lexical and semantic similarity, distributional divergence, and token-level uncertainty analysis, supported by a human annotation study. Three findings emerge consistently: unconstrained models default to CARE-oriented language, attributable to the empathy-rewarding objectives of instruction tuning; moral prompting shifts generation in foundation-consistent ways, but the effect is predominantly surface-level; and steerability varies substantially across model families and topics. Together, these results suggest that current alignment training collapses moral diversity into a narrow register, and that recovering genuine moral plurality in generation remains an open challenge.

Abstract:
Political debates are often analyzed through Argument Mining (AM) to investigate the key arguments that drive them. However, political arguments are rarely interpretable from argumentative spans alone, as claims and premises generally depend on the entities (e.g., people, events, locations, parties) they mention. Existing AM resources and methods typically annotate argumentative spans and roles, but do not provide a paired debate-entity layer for asking which Debate Named Entities (DNE), e.g., actors and events, are invoked within debates. In this work, we address these data and methodological gaps by (i) introducing DNE-ElecDeb, an entity-enriched version of the USElecDeb dataset that adds DNEs in both argumentative and non-argumentative spans and defines Debate Named Entity Recognition (DNER) as the task of detecting DNEs, and (ii) proposing Joint Argument and Entity Tagging (JAET), a generative framework that fine-tunes decoder-only LLMs to insert inline argument and entity tags into debate turns while preserving the original transcript. Under BIO-tagging evaluation, JAET improves relative F1 on the joint AM+DNER task by +27.3%, resp. +41.9%, under the untyped, resp. typed setting over the strongest sequential AM-DNER pipelines, demonstrating that such gains cannot be recovered by composing two independent modules. Notably, similar margins replicate on Persuasive Essays (+26.6%, resp. +52.7%), showing effective generalization to domains orthogonal to political debates. By unifying argumentative and entity-level representations within a single view, our contributions pave the way for richer political debates understanding.

Abstract:
Neural retrievers are trained to estimate query-document relevance from annotated query-document pairs. Yet annotation protocols may not purely reflect relevance: they select only a subset of documents for labeling, and this selection can favor certain document types over others. We investigate whether supervised bi-encoder retrievers implicitly learn a document-level relevance prior: a query-independent signal encoded in their representation space as a side effect of training on annotated data. We estimate this prior by training simple classifiers on frozen document embeddings and evaluate three state-of-the-art retrievers across multiple IR benchmarks. We find that supervised neural retrievers encode relevance priors that generalize to unseen documents and are consistent across models. These priors are associated with a findability gap: documents with lower prior are systematically harder to retrieve, even when genuinely relevant. This effect appears in supervised dense retrievers but is weaker and less consistent in BM25, and it persists under controlled matched-document comparisons. Using LLM-based explanations, we find that judged-relevant documents tend to be comprehensive, self-contained summaries of mainstream topics, while niche, fragmentary, or highly technical content is often left unjudged. Retrievers appear to internalize this bias, ranking documents with these favored features higher than documents that lack them, independently of their actual relevance. Our findings expose a structural limitation of supervised retrieval: models trained on annotated data do not just learn relevance, but also the implicit document preferences in their training data.

Abstract:
Large Language Models often exhibit dishonest behaviors–confidently hallucinating answers to questions beyond their knowledge boundary or hedging on queries they have mastered. Although existing mitigation methods have progressed to establishing the model knowledge boundary, this boundary is still typically treated as a coarse “known vs. unknown” distinction, leaving a finer-grained question underexplored: Does a failure reflect missing knowledge, or merely under-expressed capability? In this paper, we propose KCAPO, a framework that aligns the model’s output behavior with its latent capabilities. It estimates query-level solvability with a consistency-supervised hidden-state probe, then uses its mismatch with response confidence and factual behavior to refine the knowledge boundary into four capability–behavior cases. Based on this fine-grained characterization, we construct preference data to dynamically "Wake Up" dormant knowledge or "Suppress" blind guessing. Furthermore, we introduce an adaptive margin into the DPO objective, which prioritizes correcting responses with severe cognitive dissonance. Extensive experiments on short-form QA and refusal benchmarks show that KCAPO reduces hallucinations without compromising utility and improves selective abstention.

Abstract:
Persona conditioning is widely used to steer large language model (LLM) behavior, but it is unclear whether it induces stable behavioral structure or superficial variation. We propose a framework to measure consistent behavioral tendencies using situational judgment tests (SJTs), multidimensional item response theory (MIRT), and structured synthetic personas, treating responses as observations of latent behavioral variables. Across large-scale SJT and persona datasets, we find that persona-conditioned behaviors are stable across runs, latent trait scores predict external benchmarks (e.g., TruthfulQA, EmoBench), and MIRT reveals consistent latent structure. We validate these results through human annotation, benchmark evaluation, and internal consistency analyses. We interpret these traits not as human personality, but as stable behavioral tendencies expressed across contexts. Our results show that scenario-based psychometric evaluation provides a more reliable alternative to classical self-report approaches for assessing LLM behavior, and we release datasets to support further study.

Abstract:
Clinical voice agents are now deployed in routine care, where real patients do not wait their turn: they interrupt. These systems typically use a cascaded architecture (speech-to-text → LLM → text-to-speech), so when a patient cuts the agent off mid utterance, clinically required content can be lost even when the model handles cooperative transcripts well. Yet clinical conversational-AI benchmarks almost universally assume patients wait for the agent to finish, missing interruption-induced loss of required content. We present a transcript-based evaluation of interruption recovery, adapting conversation-analytic overlap categories into three operational types (recognitional, competitive, transitional sub-unit) and testing four deployment-oriented, non-reasoning LLM configurations across four cells spanning history-taking (information gathering) and FAQ (information provision), scored on whether the agent preserves the clinically required content. In the gathering cells, target-question failure varied across models; in the provision cells, where arms are directly comparable, failure rose for every model. Rankings differ across cells, and competitive FAQ interruption produced 30/30 provision-coverage failures for all four models (Wilson 95% CI: 88.6–100.0%; baseline 0/30 for three, 4/30 for Llama). A brief apology marker (“sorry to interrupt”) shifts recovery by tens of percentage points, inconsistently across models, and for one it reduces recovery. Interruption robustness therefore cannot be a single score: evaluation must be content-grounded, reported per cell, and matched to the deployment’s interruption profile.

Abstract:
While recent advances in reasoning models have demonstrated cognitive behaviors through reinforcement learning, existing approaches struggle to invoke deep reasoning capabilities in multi-turn agents with long-horizon interactions. We propose DeepMiner, a framework that elicits such abilities in deep search agents by introducing a dynamic context window and challenging training tasks. We design a dynamic context management strategy that utilizes a sliding window to selectively drop distant tool responses while preserving all assistant reasoning traces, eliminating the dependency on external summarization models. To provide training tasks that genuinely demand long-horizon reasoning, DeepMiner further employs a reverse construction method to generate complex yet verifiable question-answer pairs from authentic web sources, ensuring both difficulty and reliability. Through reinforcement learning on Qwen3-32B, we develop DeepMiner-32B, which achieves substantial performance improvements across multiple search agent benchmarks. DeepMiner attains 33.5% accuracy on BrowseComp-en, surpassing the much larger DeepSeek-V3.1-671B, and demonstrates consistent improvements on BrowseComp-zh, XBench-DS, and GAIA. Notably, our dynamic context management enables sustained interactions of nearly 100 turns within commonly used 32k context length, effectively addressing the context limitations that constrain existing multi-turn agents.

Abstract:
Document-level relation extraction (DocRE) requires integrating scattered evidence to uncover challenging implicit relations. While recent LLM-based approaches leverage rich prior knowledge for relation prediction, they suffer from two limitations. Pipeline-based approaches cause irrecoverable candidate pruning, and the absence of explicit logical constraints leaves spurious over-prediction unsuppressed. To address this, we propose Gated Logical-rule INTernalization (GLINT), a novel DocRE framework that integrates an LLM with end-to-end inductive rule reasoning within a fully differentiable architecture. GLINT preserves per-pair signals across all entity pairs through a Fused Fact Tensor with Gumbel-softmax selection, and refines LLM predictions via an inductive rule reasoner with per-atom learnable negation gates. Extensive experiments on three benchmark datasets show that GLINT outperforms existing LLM-based baselines.

Abstract:
Math reasoning benchmarks have prolifer- ated, yet most lack a per-item difficulty sig- nal grounded in actual human performance. We introduce KCSAT-ML, a decade (2014– 2025) of Korean College Scholastic Ability Test (KCSAT; Suneung) mathematics: 664 problems with a 339-item core set carrying official per-item error rates from nationwide cohorts of hundreds of thousands of examinees. We pair the benchmark with Difficulty-aligned Reasoning Gain (DRG): a score-orthogonal metric that asks whether a model’s mistakes concentrate on the items humans found hard, or on items humans found easy. Together they expose, across a wide range of VLMs (and LLMs with OCR), three patterns: (i) low- budget accuracy collapses on the high-human- error tail at every model size; (ii) test-time scaling (TTS) raises token use roughly linearly with cohort error rate, while accuracy gains follow a non-monotonic curve; (iii) within a single family, TTS flips between anti-scaling on the hardest items and overthinking on eas- ier ones — two faces of the same alignment failure. With the proposed DRG metric, we find that models with near-identical accuracy can sit at near-opposite values: one model gets wrong what humans also find hard, while another solves the hardest items yet fails on items humans find easy, which is a critical con- trast that aggregate accuracy hides. Our code and dataset builder is fully open-sourced at https://github.com/naver-ai/KCSAT-ML.

Abstract:
Non-compositional expressions (e.g., idioms, proverbs, and metaphors) pose significant challenges for neural machine translation systems because their meanings cannot be derived from individual words alone. These expressions encode rich, cultural meaning, and have both figurative and literal meanings, making accurate translation difficult. Because models are fairly good at translating compositional text, we investigate GRPO-style fine-tuning using Machine Translation Quality Estimation (MTQE) models as reward functions. Using Chinese and Hindi idiom datasets, we find that idiom translation abilities improve by ~13 points, general, non-idiomatic translation implicitly improves by ~8 points, and cross-lingual translation abilities (trained on one language, evaluated on another) improves by ~6 points. Overall, our work quantifies the non-compositional translation gap and offers insights for developing LLMs with stronger cross-cultural and figurative language understanding.

Abstract:
Membership inference attacks (MIAs) assess whether a sample was used to train a model, posing privacy risks for large language models (LLMs). Existing MIAs typically rely on single loss or confidence signals, which are often unstable and easily confounded by noise, leading to limited robustness and generalization. In this paper, we propose Group-Invariant Membership Inference Attack (GI-MIA) that explicitly exploits semantic-preserving transformations to capture stability differences between members and non-members. Our method constructs an orbit of paraphrased variants for each input using LLM-driven transformations, and extracts invariant statistics (mean, maximum, and variance of normalized losses) across the orbit. A simple decision rule based on these statistics effectively discriminates members from non-members by leveraging the model’s inherent invariance on training data. We conducted extensive experiments on the Pythia family of LLMs (different model parameter sizes) across six datasets. Results show that GI-MIA consistently outperforms strong baselines, achieving state-of-the-art performance while maintaining robustness against semantic-preserving transformations. Our study highlights new directions for MIAs and calls for further research into LLM privacy protection, emphasizing the need for developing reliable models with better privacy resilience.

Abstract:
Peer review serves as a backbone of academic research, but in most AI conferences, the review quality is degrading as the number of submissions explodes. To reliably detect low-quality reviews, we define misinformed review points as either "weaknesses" in a review that contain incorrect premises, or "questions" in a review that can be already answered by the paper. We verify that 15.2% of weaknesses and 26.4% of questions are misinformed and introduce ReviewScore indicating if a review point is misinformed. To evaluate the factuality of each premise of weaknesses, we propose an automated engine that reconstructs every explicit and implicit premise from a weakness. We build a human expert-annotated ReviewScore dataset to check the ability of LLMs to automate ReviewScore evaluation. Then, we measure human-model agreements on ReviewScore using eight current state-of-the-art LLMs. The models show F1 scores of 0.4–0.5 and kappa scores of 0.3–0.4, indicating moderate agreement but also suggesting that fully automating the evaluation remains challenging. A thorough disagreement analysis reveals that most errors are due to models’ incorrect reasoning. We also prove that evaluating premise-level factuality shows significantly higher agreements than evaluating weakness-level factuality.

Abstract:
Recent multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the textual modality. However, these approaches often suffer from performance degradation during inference due to partially missing or noisy data in real-world scenarios, especially when sentiment-related cues are missing. To address this issue, we introduce a new completeness estimation approach that quantifies the degree of sentiment-relevant information preserved in incomplete data to guide the reconstruction of missing semantics. Furthermore, we propose a training strategy that stabilizes multi-task learning while jointly optimizing sentiment prediction and completeness estimation. Extensive experiments and in-depth analyses on three benchmark datasets demonstrate that the proposed approach enables more accurate semantic reconstruction, leading to more precise sentiment prediction.

Abstract:
Existing real-world datasets for multimodal fact-checking have multiple limitations: they contain few instances, cover on only one or two languages, focus only on one task, or rely on external news article sets for sourcing true claims. To address these shortcomings, we introduce M4FC, a new real-world dataset comprising 4,982 images paired with 6,980 claims. The images, verified by professional fact-checkers from 22 organizations, represent a diverse range of cultural and geographic contexts. Each claim is available in one or two out of ten languages. M4FC spans six multimodal fact-checking tasks: visual claim extraction, claimant intent prediction, fake image detection, image contextualization, location verification, and verdict prediction. We provide baseline results for all tasks and analyze how combining intermediate tasks affects verdict prediction performance. We make our dataset and code publicly available.

Abstract:
Recently, Large Language Models (LLMs) have gained significant attention due to their strong language understanding and generation capabilities, demonstrating impressive reasoning abilities as well as effective utilization of external knowledge. Many studies have proposed methods that specialize in improving performance for individual tasks. However, ironically, only a limited number of attempts have explored general-purpose, task-agnostic methods. In this work, we present a unified framework integrating reasoning and Retrieval-Augmented Generation (RAG) tasks. We further propose Controllable White-Box Meta-Prompting (CWM), a low-cost white-box method for adaptive RAG tasks previously dominated by black-box approaches, without requiring external decision modules or multi-sampling. CWM achieves state-of-the-art performance on three adaptive RAG benchmarks across recent LLMs, including GPT-oss-20b, Qwen3-14b, and Llama3.1-8b, while also demonstrating strong generality by extending to reasoning tasks. In addition, CWM provides controllability by enabling retrieval decisions to be regulated through the manipulation of internal model signals. Our code is available at https://github.com/JeongEunhye00/CWM.

Abstract:
Public large language models (LLMs) are typically safety-aligned during pretraining, but task-specific fine-tuning often erodes this alignment, introducing significant safety risks. Existing defenses are largely coupled with the fine-tuning process, incurring high computational overhead and complex workflows. To address these challenges, we propose Q-realign, a post-hoc defense based on quantization. Departing from the conventional view of quantization as a safety-eroding compromise, we reframe it as a dual-purpose mechanism for both compression and safety recovery. This allows Q-realign to decouple safety alignment from fine-tuning and naturally piggyback onto modern deployment pipelines. Experiments across multiple models and datasets demonstrate that our method substantially reduces unsafe behaviors while preserving task performance, significantly improving memory and GPU efficiency. Notably, our approach can recover the safety alignment of a fine-tuned 7B LLM on a single RTX 4090 within 40 minutes. Overall, our work provides a practical, turnkey solution for safety-aware deployment. Our code is released at https://anonymous.4open.science/r/Q-realign-copy-0D71.

Abstract:
Retrospective studies using electronic health records (EHRs) often require diagnosis labels that reflect whether treating physicians considered a disease to be present. Such physician-determined diagnosis (PDD) labels are expensive to obtain through expert review, while available alternatives provide only noisy proxies. We exploit two complementary proxy labels: administrative diagnostic codes and free-text disease listings in discharge summaries. We introduce a dual-noisy-label learning framework that models PDD as a latent variable and treats these proxies as noisy observations of it, allowing a pretrained language model to learn from their agreement and disagreement without expert-labeled training data. We evaluate the approach on extracting Acute Kidney Injury (AKI) PDD from MIMIC-IV discharge summaries. Based on an expert-adjudicated evaluation set of 150 summaries, our PLM-PDD model achieves 94.6% population-weighted accuracy, compared with 89.4% for the same model trained using administrative coding, and 85.9% and 78.0% for AKI proxies. Largest gains occur in the gray-zone, where the proxies disagree. Ablation experiments show that gray-zone examples and auxiliary ICD supervision contribute to performance, while expert-highlighted evidence receives substantially higher model attention than matched random text. These results demonstrate that complementary noisy EHR labels can be used to learn clinically meaningful diagnosis labels without requiring expert annotation.

Abstract:
Aligning large language models with human preferences remains a challenge, primarily due to the critical role of preference data quality in effective alignment. Existing datasets are frequently plagued by inherent noise and distribution shifts, which inherently limit model performance. To bridge this gap, we propose AlignDiff, a preference data filtering framework driven by intrinsic model signals. AlignDiff first identifies samples with clear preferences using both positive and inverse signals, then prioritizes the more challenging samples based on the average negative log-likelihood gap, encouraging the model to learn richer information from them. AlignDiff is evaluated on two widely used model families (LLaMA and Qwen) and three benchmarks widely adopted in the alignment community (AlpacaEval 2.0, Arena-Hard, and MT-Bench). Across all settings, it consistently outperforms seven strong baselines. We conduct comprehensive ablation studies to validate the effectiveness of AlignDiff, and further show that difficulty-based curriculum learning improves model performance.

Abstract:
As chatbots become increasingly widespread, personalization has emerged as a key feature for accommodating diverse users and use cases. Personalization enables better responses rather than generic answers, but when using demographic attributes to drive recommendations, can also amplify harmful stereotypes. For instance, individuals from different cultural backgrounds may meaningfully want culturally distinct food preferences and values incorporated, but not different job recommendations. Yet, little is known about how these user preferences relate to actual platform behavior. Through a survey of user preferences and a novel field study methodology where users input our standardized questions to their own personalized chatbot interfaces then share the outputs back to us, we contribute real-world personalization insights on the gaps between what users want versus actually receive. We generally find that there is minimal race- and gender-based personalization in practice, which conflicts with users’ desires in some domains, e.g., cuisines. We conceptualize our findings through the framing of a personalization double bind: Black people and women desire more demographic-based personalization than White people and men in order to avoid overly generic responses (indirect discrimination), yet simultaneously express greater concern about stereotypical responses (direct discrimination).

Abstract:
While Reinforcement Learning (RL) effectively incentivizes reasoning in Large Language Models, current pipelines are hindered by training instability and rapid entropy collapse. These limitations often stem from "Rollout Silencing" and low-quality gradient signals in standard sampling procedures. In this work, we propose a robust, data-centric framework to stabilize RL training. We first introduce Potential-Aware Query Mining (PAQM), which filters data dynamically to focus on the "Distillation Zone"—samples with high potential for capability elicitation. Furthermore, we present Hybrid Stratified Replay (HSR), a novel mechanism that restructures batches by stratifying rollouts based on Path Entropy, a rollout-level confidence proxy, and outcome reward. Within each optimization step, HSR reuses current-policy "Stability Anchors" and "Hard Negatives" to construct high-contrast optimization groups, then clears its buffers before the next step. This approach mitigates entropy collapse while improving the utilization of learning signals under limited compute. Our method outperforms strong baselines on complex reasoning tasks, offering a principled solution for stable and efficient RL fine-tuning.

Abstract:
The representation learning of knowledge graphs (KGs) is a longstanding research problem. While graph neural networks (GNNs) have driven recent progress, they still struggle with encoding textual features and subtle relationships of KGs, particularly in conveying key information to large language models (LLMs). The emergence of multi-modal LLMs (MLLMs), which combine linguistic and visual understanding, presents an intriguing opportunity: Could their vision capabilities inspire mental visualization, facilitating conceptual thinking and semantic reasoning akin to human cognition? To investigate this premise, we propose SeeKG, an innovative framework that transforms KGs into visually rendered representations as image inputs for MLLMs. We evaluate SeeKG under both training-free and supervised fine-tuning settings, where the experimental results show that SeeKG excels in understanding KG sub-graphs and achieves competitive performance even without training or demonstrations. Further fine-tuning on small-batch data reveals that it outperforms state-of-the-art LLM-based methods by substantial margins across multiple KG completion datasets.

Abstract:
As conversational AI systems increasingly engage users through multi-turn dialogue across diverse cultural contexts, cross-cultural competence, the ability to recognize, respect, and adapt to cultural differences, is essential for effective human-AI interaction. Existing evaluations largely rely on single-turn settings, overlooking the complexities of real-world, multi-turn conversations where cultural cues often emerge gradually. To address this gap, we introduce CultureTalk, a benchmark for evaluating multi-turn cultural reasoning in conversational AI. The dataset contains 1,741 culturally grounded dialogues derived from human-verified cultural knowledge resources and evaluates models’ ability to accumulate cultural context across turns and remain robust to topic shifts. Our evaluation reveals substantial performance gaps between single- and multi-turn settings, highlighting current models’ limitations in sustained culturally grounded context understanding. We further show that fine-tuning on augmented single-turn conversational data and inference scaling significantly improve multi-turn cultural reasoning, yielding a 64.7% relative gain in cultural knowledge entailment with a fine-tuned Qwen3-8B model.

Abstract:
Ultra-long video understanding remains an open challenge, as existing vision language models (VLMs) falter on such content due to limited context length and inefficient long-term memory retention. To address this, recent works have attempted to construct external knowledge bases and corresponding retrieval agumented generation (RAG) systems, yet these incur enormous storage and computational overhead. In this paper, we propose VideoMem, a novel framework that pioneers models long video understanding as a sequential generation task via adaptive memory management. Specifically, VideoMem dynamically updates a global memory buffer, which adaptively retains critical information while discarding redundant content across the video timeline. To efficiently train VLMs for such long-term tasks, VideoMem integrates the Progressive Grouped Relative Policy Optimization (PRPO) algorithm, equipped with two core modules: Progressive State Propagation (PSP) adaptively retains valid current states, propagates them to the next rollout step, and gradually narrows the model’s exploration space. Temporal Cascading Reward (TCR) further alleviates reward sparsity, improving sample utilization and accelerating convergence. Extensive experiments demonstrate that VideoMem significantly outperforms existing open source models across diverse benchmarks for ultra-long video understanding tasks.

Abstract:
With the rapid expansion of massive multilingual corpora, Multilingual Information Retrieval (MLIR) has emerged as a critical technology for global information access. MLIR enables users to retrieve semantically relevant documents from multilingual text collections using a single-language query. However, recent multilingual dense retrieval models often exhibit a strong preference for documents in the same language as the query. This leads to severe language bias, where top-ranked results are dominated by documents of specific languages, even when documents in other languages contain more semantically relevant information. To address this issue, we propose Shift, a training-free method applicable in the indexing stage. Specifically, Shift utilizes parallel translation pairs to estimate a relative language vector for each target language with respect to a source language. Subsequently, Shift corrects the language-specific offset by subtracting this relative language vector from document embeddings during indexing. Our comprehensive evaluation across four MLIR benchmarks and diverse dense retrieval models confirms that Shift can effectively mitigate language bias and enhance MLIR performance.

Abstract:
The order of training samples plays a crucial role in large language models (LLMs), significantly impacting both their external performance and internal learning dynamics. Traditional methods for investigating this effect generally require retraining the model with various sample orders, which is computationally infeasible for LLMs. In this work, we improve traditional methods by designing a retraining-free framework. By approximating Adam optimizer updates with first- and second-order Taylor expansions and utilizing random projection methods to store intermediate checkpoints, our framework can efficiently estimate model parameters for arbitrary training sample orders. Next, we apply our framework to two downstream research problems: (1) Training curriculum design for LLMs — We base our retraining-free framework to propose a novel curriculum learning strategy that augments curriculum proposals with estimated model performances, enabling more informed sample scheduling. (2) LLMs’ memorization and generalization effect analysis — We use our retraining-free framework to estimate how the positions of training samples influence LLMs’ capacity for memorization and generalization. We conduct extensive experiments to validate the effectiveness of our retraining-free framework in reproducing the true model performances, and further demonstrate its potential in optimizing LLM training curricula and analyzing the memorization and generalization effects of LLMs.

Abstract:
Medical large language models hold promise for reducing healthcare disparities, yet Hindi remains severely underrepresented. While medical LLMs excel in high-resource languages, their performance degrades sharply in Hindi, particularly on Indian systems of medicine. We argue that robust cross-lingual medical transfer requires Hindi reasoning. To this end, we introduce HiMed, a Hindi reasoning medical corpus and benchmark suite covering both Western and Indian medicine. We further propose HiMed-8B, a Hindi-form medical reasoning LLM, through the design of decaying scaffolding reward. Extensive experiments demonstrate improvement in Hindi medical reasoning performance and a reduction in the English–Hindi accuracy gap. Ablation studies validate the contribution of each training stage and reward component. All code, data, and weights are available.

Abstract:
Aligning large language models with human values is a critical challenge for ensuring public safety. This alignment is commonly achieved through methods such as Direct Preference Optimization and RLAIF/RLHF. Despite extensive research, prior studies show that models often fail to maintain consistent alignment performance across languages, resulting in unequal treatment of users. To address this issue, we introduce a regularization term in the Direct Preference Optimization fine-tuning process that penalizes discrepancies in model behavior between languages using a bilingual corpus. Our results demonstrate that this approach effectively reduces the alignment gap across languages following fine-tuning.

Abstract:
Parameter-Efficient Fine-Tuning (PEFT) is essential for adapting Large Language Models (LLMs) to multi-task scenarios. A prevailing trend in this field involves complex LoRA variants with multiple adapters or heads, which rely on the premise that architectural isolation of task-specific knowledge is necessary. However, this design often introduces dynamic routing, preventing weight merging and causing significant inference latency. In this work, we present a direct challenge to this paradigm. We first reveal a paradox where a simplified, router-free multi-head model with high inter-head redundancy outperforms complex, diversity-driven baselines. Furthermore, we demonstrate that a unified, single-adapter LoRA with increased rank achieves highly competitive performance, questioning the necessity of multi-component structures. Based on these findings, we propose Align-LoRA, a unified and efficient framework that shifts the focus from architectural isolation to representation alignment. Align-LoRA incorporates an explicit alignment loss to encourage the learning of task-shared representations within a shared latent space. Crucially, our method maintains the standard LoRA architecture, ensuring zero inference latency via weight merging. Theoretical analysis and extensive experiments confirm that Align-LoRA significantly surpasses prevailing approaches, establishing a simpler, more effective, and production-friendly paradigm for multi-task PEFT. The code is available at [Align-LoRA](https://github.com/jinda-liu/Align-LoRA).

Abstract:
Multimodal Large Language Models (MLLMs) have achieved remarkable progress in vision-language interaction, yet their safety alignment remains vulnerable to jailbreak attacks. A key challenge is that safety behavior learned in the textual space does not reliably transfer to fused cross-modal representations, leaving multimodal inputs exploitable through latent semantic cues. We propose __Text-Anchored Semantic Perturbation Attack__ (TA-SPA), a black-box jailbreak framework that optimizes transferable perturbations in a text-anchored semantic space. TA-SPA integrates __Text-Anchored Semantic Factorization__ (TASF), which disentangles cross-modal semantic factors from modality-specific residuals, with __Semantic-Preserving Augmentation__ (SPA), which diversifies harmful target anchors while preserving semantic consistency. Extensive experiments show that TA-SPA consistently outperforms prior jailbreak attacks, transfers to commercial MLLMs, and remains effective under representative defenses, underscoring the need for representation-level safety alignment beyond input-level filtering.

Abstract:
Multimodal Large Language Models (MLLMs) encode images into dense visual token sequences, creating substantial attention and KV-cache overhead. Existing visual token compression methods reduce this cost by pruning low-saliency tokens in shallow layers, but this practice couples temporary computational deactivation with irreversible information deletion. We show that this coupling breaks an intrinsic inference pattern in MLLMs. Through layer-wise attention analysis, we identify Visual Backtracking: visual pivots remain spatially stable, yet in middle layers they actively retrieve context from tokens that appeared unimportant earlier. Thus, early low-saliency tokens can be inactive for computation while still being useful for later reasoning. Motivated by this finding, we propose ReTrace, a backtracking-aware compression framework that introduces a dormant token state. ReTrace caches non-pivot tokens outside expensive layer-wise computation, restores them during the backtracking window, and prunes only after their information has been absorbed. Experiments on Qwen2.5-VL and InternVL3 demonstrate nearly 90% reductions in FLOPs and KV cache, a 2.83 prefilling speedup, and strong performance retention across complex vision-language benchmarks.

Abstract:
Backdoor attacks pose a serious threat to large language models (LLMs), but existing defenses remain fragmented, failing to provide unified defense against both dirty-label and clean-label attacks. To investigate why such fragmentation arises, we present the first systematic feature-level mechanistic analysis of LLM backdoors using sparse autoencoders (SAEs). Starting from a 2×2 comparison of clean and poisoned models on clean and triggered inputs, we trace backdoor-induced logit shifts to high-contributing SAE features and categorize them into four roles: interaction , suppressed , mixed , and weight-modified features . This taxonomy reveals systematic encoding differences: dirty-label backdoors are dominated by isolated interaction features, whereas clean-label backdoors rely more on heterogeneous mixtures of mixed and weight-modified features. These differences explain why existing defenses remain fragmented across attack paradigms. We validate this hypothesis through inference-time feature clamping, which reduces ASR to at most 10.8% in most dirty-label settings and at most 15.4% in the majority of clean-label settings, while preserving benign-task performance. These results show that SAE-based analysis can explain defense fragmentation and guide interpretable backdoor mitigation.

Abstract:
Existing jailbreak classifiers for large language models (LLMs) often fail to distinguish truly harmful responses from benign paraphrases, resulting in high error rates. We reframe jailbreak detection as a strict instruction-following verification task rather than holistic semantic matching. To achieve this, we propose Sub-Query Decomposition (SQD) , a framework that breaks complex adversarial prompts into atomic, actionable sub-intents. SQD transforms opaque safety assessment into a parameterizable adjudication protocol, allowing practitioners to quantitatively adjust safety boundaries via explicit mathematical aggregation rules for highly interpretable evaluation. To operationalize SQD at scale, we introduce three foundational pillars: (1) Efficient Deployment via distilled lightweight models, featuring SQD-Phi-4 for single-pass end-to-end verification and SQD-Decomposer , optimized with an entropy-aware loss to mathematically prevent the omission of rare constraints; (2) Evaluation Foundation through SQD-Bench , a rigorously human-annotated dataset; and (3) LLM-independent Judge via E(mbedding)-TFIDF to eliminate inference-time reasoning bias. Extensive experiments demonstrate that our unified framework provides a transparent, low-cost, and policy-aligned standard for reliable jailbreak evaluation.

Abstract:
Answering complex queries over long-context multimodal streams requires dynamically synthesizing asynchronous and heterogeneous evidence across extended temporal and logical spans. Existing approaches often rely on iterative Multimodal Retrieval-Augmented Generation (RAG), facing structural challenges from semantic and modality gaps as well as context bloat induced by multi-step retrieval. To address these challenges, we propose SynapseHSG, a decoupled one-shot grounding framework for long-context multimodal Question Answering. SynapseHSG introduces a hyper-semantic evidence organization method (named M2HSG) that constructs an offline graph index to align cross-modal evidence. During inference, we design a Semantic Decomposer to convert natural-language queries into graph-operable constraints. Guided by these constraints, SynapseHSG retrieves a compact query-anchored subgraph in one shot and linearizes it as structured evidence for a frozen generator. This avoids repeated LLM-driven retrieval while restricting trainable components to graph construction and query parsing. To support systematic evaluation, we introduce MM-ConfQA, a new long-context multimodal QA benchmark built from academic conference presentations. Experimental results on MM-ConfQA show that SynapseHSG improves grounded answer generation over strong multimodal and retrieval baselines, especially on dispersed-evidence questions, highlighting the value of pre-organized multimodal evidence over repeated search in raw heterogeneous contexts. We release the code and data to public.

Abstract:
Reinforcement learning for multi-turn agents suffers from a credit-assignment mismatch: rewards are sparse and trajectory-level, while success often hinges on a few local decisions. Existing online policy distillation (OPD) provides denser token-level supervision, but typically treats heterogeneous agent trajectories as monolithic strings rather than causal interaction units. We present StepOPSD, a post-rollout preference self-distillation framework that takes the agent step as the unit of credit redistribution. StepOPSD decomposes trajectories into action-centered step segments, rescoring them under hindsight-enriched teacher contexts and converting token-level log-probability gaps into sign-preserving advantage shaping with a normalized per-step credit budget before the GRPO update. Across ALFWorld and Search-QA with Qwen3-1.7B and Qwen2.5-3B-Instruct, StepOPSD attains best or second-best results on subsets most sensitive to local causal errors, including first-place performance on ALFWorld Heat (79.1%), PickTwo (95.0%), Search-QA TriviaQA (61.6%), and tied-best performance on HotpotQA (40.4%). The results further reveal a consistent two-knob law: smaller α_clip acts as a broadly stabilizing local trust region, whereas the optimal global mixing strength λ_mix remains task-dependent. These findings suggest that step-aware distillation is most useful when trajectory-level rewards are weakly aligned with the local action that determines downstream success.

Abstract:
Recent breakthroughs in Large Language Models (LLMs) have revealed remarkable generative capabilities and emerging self-regulatory mechanisms, including self-correction and self-rewarding. However, current detoxification techniques rarely exploit these built-in abilities; instead, they rely on external modules, labor-intensive data annotation, or human intervention –factors that hinder scalability and consistency. In this paper, we introduce a fully self-reflective detoxification framework that harnesses the inherent capacities of LLMs to detect, correct toxic content, and refine LLMs without external modules and data annotation. Specifically, we propose a Toxic Signal Detector—an internal self-identification mechanism, coupled with a systematic intervention process to transform toxic text into its non-toxic counterpart. This iterative procedure yields a contrastive detoxification dataset used to fine-tune the model, enhancing its ability for safe and coherent text generation. Experiments on benchmark datasets such as DetoxLLM and ParaDetox show that our method achieves better detoxification performance than state-of-the-art methods while preserving semantic fidelity. By obviating the need for human intervention or external components, this paper reveals the intrinsic self-detoxification ability of LLMs, offering a consistent and effective approach for mitigating harmful content generation. Ultimately, our findings underscore the potential for self-regulated language models and more responsible text generation systems. Code: https://github.com/KaituoZhang/SRD Warning: this paper may contain offensive content.

Abstract:
Current multilingual evaluations for Vision-Language Models (VLMs) assume a one-to-one mapping between language and orthography, overlooking billions of users of multi-script languages. We introduce PuMVR (Punjabi Multimodal Visual Reasoning), a benchmark of 1,000 strictly parallel image-text instances across Punjabi’s three active scripts: Gurmukhi, Shahmukhi, and Roman. Evaluating 10 state-of-the-art VLMs, we expose a substantial and systematic Script Gap. Models frequently solve visual tasks in one script while failing identical tasks in another, with accuracy deltas reaching 16%. Crucially, visual input boosts absolute performance uniformly yet does not close the orthographic gap. Furthermore, cross-script in-context transfer is highly brittle, exposing script-locked knowledge representation. Supported by McNemar tests across all script pairs, our findings demonstrate that current "multilingual" VLMs are not truly multi-script. We propose the Script Consistency Rate (SCR), which falls as low as 24.8% on our benchmark, as a mandatory metric for script-agnostic evaluation to ensure equitable AI access.

Abstract:
Recent progress in multimodal large language models (MLLMs) has fueled significant enthusiasm in their potential to act as autonomous agents for real-world tasks. However, scenarios requiring agents to fulfill users’ complex, structured requirements remain largely underexplored. In this work, we examine reasoning tasks under three distinct requirement scenarios: (i) Must-have requirements uniquely determine a unique feasible solution; (ii) Multiple answers satisfy the must-have requirements and are prioritized via the nice-to-have requirements; and (iii) No candidate solution satisfies the must-have requirements, in which case the agent should abstain from generating a response. We evaluate state-of-the-art MLLMs on 3,649 carefully constructed problems that reflect realistic service scenarios, including e-commerce, booking, and map-based or ride-hailing. Our evaluation reveals that existing MLLMs exhibit catastrophic failures in all scenarios. They frequently misinterpret task requirements, violate must-have requirements, and produce invalid solutions. To address this critical gap, we propose First Things First Reinforcement Learning (FTF-rl) that explicitly optimizes reasoning over multi-priority user requirements. Experimental results show that our method substantially improves the task success rate compared to strong baselines. Moreover, FTF-rl yields general effectiveness on popular logical and mathematical reasoning tasks, including LogicVista, MathVision, and InfoQA. Our findings suggest that enhancing requirement comprehension provides a simple yet effective pathway to improve generalization of MLLMs. Code is available at https://github.com/claire62/FTF-RL.

Abstract:
Despite the growing success of Large Speech Language Models (LSLMs) in processing short-term acoustic signals, their extension to long-form audio understanding is severely bottlenecked. This limitation stems from the limited context length and the exorbitant memory footprints required for long-form inference. In this work, we propose Speech-XL, a new model that capitalizes on the intrinsic key-value (KV) sparsification capacity of Large Language Models (LLMs) to achieve high-ratio speech input compression. Specifically, we introduce a novel special token, the Speech Summarization Token (SST), for each speech interval to encapsulate the intra-interval speech information into its associated KV pairs. The SST module is trained via instruction fine-tuning, employing a curriculum learning strategy where the SST learns to compress information in a progressive manner—advancing from low-ratio (simple) to high-ratio (challenging) compression. Despite using substantially less training data than large-scale audio foundation models, Speech-XL achieves competitive compression quality under the same backbone, maintaining a small gap to its uncompressed counterpart while substantially reducing long-form inference cost. By addressing the long-standing bottlenecks in long-form audio modeling, our approach offers a novel perspective on the condensation of extensive acoustic sequences.

Abstract:
Scaling Multimodal Large Language Models (MLLMs) to long-form speech is bottlenecked by the explosive growth of input tokens. Existing speech-language models project high-frame-rate acoustic features directly into the LLM input space, making long-context processing computationally prohibitive. Unlike images or videos, speech lacks spatial redundancy, making extreme token compression particularly challenging. To address this limitation, we propose FastSLM, a token-efficient architecture featuring the Hierarchical Temporal Abstractor (HTA), which progressively distills acoustic features across multiple temporal scales. HTA achieves an extreme compression rate of 1.67 tokens per second (97% reduction) while preserving essential linguistic information for downstream speech-language understanding. Experimental results demonstrate that FastSLM achieves competitive performance across diverse speech-language tasks while requiring substantially fewer speech tokens and FLOPs than existing speech-language models. The source code and model checkpoints are available at https://github.com/Lee-junseok1025/FastSLM.

Abstract:
Memory bottlenecks severely constrain the deployment of large language models (LLMs) in long-context scenarios. While existing global pruning methods reduce memory usage during the decoding phase, they largely ignore out-of-memory errors in the prefill stage before generation begins. To address this, we present Task-Aware Chunked KV Cache Eviction (TAKE), a training-free framework for memory-efficient long-context prefill. TAKE refines the prefill process by partitioning long sequences into chunks and performing incremental KV eviction, thereby avoiding the computational overhead of full-sequence processing. Crucially, to mitigate the semantic discontinuity introduced by chunk-wise processing, we introduce accumulated task-aware probe tokens and a delayed eviction strategy to reduce information loss. Extensive evaluations show that TAKE achieves the highest efficiency, slashing peak memory usage to 59.53% of the baseline while accelerating Time-To-First-Token (TTFT) by over 60% under 128k-token inference. Our code is available at [Source Code of TAKE](https://github.com/ObitoJN/TAKE).

Abstract:
Object hallucination in Multimodal Large Language Models (MLLMs) often appears in two forms: omitting objects that are present in the image and generating objects that are absent. Existing training-free mitigation methods usually apply a uniform intervention to both cases. We find that this strategy leads to an imbalanced effect: it can reduce some omission errors, but often introduces new fabrications. To understand this behavior, we analyze spatial attention patterns and layer-wise attention dynamics during generation. Our results suggest that omissions often occur when the model attends to the relevant visual regions but gradually loses visual confidence during decoding, whereas fabrications are more closely related to deep-layer semantic priors overriding visual evidence. Based on this observation, we propose Visual Potential Field Calibration (VPFC), a training-free hidden-state calibration method. VPFC locates an intervention region using the spatial centroid of high-confidence visual tokens and applies a lightweight hidden-state update to adjust the model’s generative confidence. Experiments on multiple hallucination benchmarks show that VPFC consistently improves over existing training-free methods while maintaining inference efficiency close to standard greedy decoding.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has markedly improved the performance of Large Language Models (LLMs) on tasks requiring multi-step reasoning. However, most RLVR pipelines rely on sparse outcome-based rewards, providing little supervision over intermediate steps and thus encouraging over-confidence and spurious reasoning, which in turn increases hallucinations. To address this, we propose FaithRL , a general reinforcement learning framework that directly optimizes reasoning faithfulness. We formalize a faithfulness-maximization objective and theoretically show that optimizing it mitigates over-confidence. To instantiate this objective, we introduce a geometric reward design and a faithfulness-aware advantage modulation mechanism that assigns step-level credit by penalizing unfaithful steps while preserving valid partial derivations. Across diverse backbones and benchmarks, FaithRL consistently reduces hallucination rates while maintaining (and often improving) answer correctness. Further analysis confirms that FaithRL increases step-wise reasoning faithfulness and generalizes robustly.

Abstract:
With the rapid advancement of Large Language Models (LLMs), AI-generated text has become increasingly difficult to distinguish from human-authored content, raising serious concerns about online misinformation and academic integrity. Although plenty of methods have been proposed, they often suffer from high false positives on formal or highly structured human-written texts. Moreover, while external contextual information can provide valuable signals for verification, it remains largely underutilized in existing detection frameworks. In this paper, we propose an agentic framework driven by LLM for AI-generated text detection that integrates multiple machine learning (ML) detectors with an online search module to retrieve and leverage external evidence. Evaluated on a diverse real-world dataset spanning four distinct sources, our agent significantly outperforms standalone detectors, achieving an overall accuracy of 93.90%. We show that online search acts as an effective error-correction mechanism under detector conflict and incorrect agreement, improving accuracy by 10.81% and 90.91%, respectively. We further conduct a failure analysis, identifying key limitations and providing directions for future research.

Abstract:
Forced alignment evaluation typically requires manually annotated timestamps, limiting large-scale and multilingual analysis. We introduce two corpus-level metrics based on self-supervised (SSL) speech representations for reference-free forced alignment evaluation: Phoneme-Cluster Mutual Information (PCMI) and Word Acoustic Consistency Score (WACS). PCMI measures agreement between aligned phoneme labels and clusters induced from SSL-speech representations, while WACS measures consistency of repeated word realizations using dynamic time warping similarity between word representation sequences. Using both random and systematic perturbations, we show that PCMI and WACS degrade consistently under alignment perturbations. We further analyze the metrics across multiple alignment systems on 85 languages from FLEURS, validate them against manually annotated alignments from 45 languages in DoReCo, and evaluate them on two phonologically complex low-resource languages. The metrics effectively separate high- and low-quality alignments and correlate strongly with timestamp-based alignment quality measures. Our results demonstrate that SSL-speech representations enable scalable, reference-free forced alignment evaluation. The metrics are available as an open-source Python package at https://github.com/mahesh-ak/forced-aligner-metrics.

Abstract:
Large language models (LLMs) offer a promising approach to machine translation (MT) for extremely low-resource languages by incorporating linguistic resources through in-context learning. However, LLMs often struggle to apply grammatical information effectively during translation. Inspired by recent progress in chain-of-thought reasoning, we investigate whether low-resource MT can benefit from structured intermediate steps of linguistic analysis and grammatical reasoning. We propose a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks. We evaluate these traces in three settings: in-context learning (ICL), supervised fine-tuning (SFT), and reinforcement fine-tuning (RFT), on Xibe and Chintang as test cases. Our results show that linguistic reasoning traces are most effective as inference-time guidance: in ICL, reliable linguistic reasoning traces substantially improve translation performance across models, languages, and metrics. In contrast, using these traces as training data yields smaller and less consistent gains, as models learn the format but often generate error-prone content. These findings suggest that LLMs can leverage grammatical information for low-resource MT when given reliable linguistic analyses, while learning to generate such analyses remains a major bottleneck.

Abstract:
Reinforcement learning with verifiable rewards has become a standard recipe for improving the reasoning abilities of large language models. Existing algorithms face a tradeoff between computational efficiency and sample efficiency in value estimation and policy learning. We introduce BASIS, a critic-free post-training algorithm designed to address this tradeoff. At each online training step, BASIS samples only one rollout per prompt, but leverages rich information across prompts in the entire batch to improve value function estimation. Our experiments demonstrate that BASIS reduces MSE in value function estimation by 69% compared to REINFORCE++, a representative single-rollout baseline, and achieves lower MSE with one rollout than group mean estimators with 8 rollouts. This improvement in value estimation translates to better policy optimization: using substantially less training time, BASIS achieves performance close to multi-rollout GRPO-type baselines and often outperforms single-rollout REINFORCE-type baselines.

Abstract:
Deploying Large Language Models (LLMs) in real-world dynamic environments raises the challenge of updating their pre-trained knowledge. While existing knowledge editing methods can reliably patch isolated facts, they frequently suffer from a reasoning gap , where the model recalls the edited fact but fails to utilize it in multi-step reasoning chains. Our analysis shows that multi-hop reasoning relies on sparse circuitry beyond direct factual recall, with functionally differentiated components contributing to intermediate processing and final-answer prediction. To bridge this gap, we introduce MCircKE ( M echanistic Circ uit-based K nowledge E diting), a mechanistically grounded framework that enables a map-and-adapt editing procedure. MCircKE first identifies the causal circuits responsible for a specific reasoning task, capturing both the storage of the fact and the routing of its logical consequences. It then surgically update parameters exclusively within this mapped circuit. Extensive experiments on the MQuAKE-series benchmarks demonstrate the effectiveness of the proposed method for multi-hop reasoning in knowledge editing.

Abstract:
Long-context LLM applications such as document summarization and multi-turn agents require generation from prefixes spanning tens of thousands of tokens, making decoding latency a major bottleneck. Speculative decoding (SD) reduces latency without changing model outputs, but its speedup depends on both accepted draft tokens and draft-step latency: Lightweight drafts are fast but lack the capacity to capture long-range dependencies, whereas strong independent drafts recover acceptance but incur growing KV-access cost at long prefixes. We introduce memory-augmented drafting for long-context SD, equipping a strong independent draft with compressed draft-side KV memory: A lightweight adaptor constructs and incrementally updates this memory to retain distant information and exact recent context. The target verifier retains its full KV cache and applies the standard accept/reject rule, preserving SD’s lossless guarantee. Experiments on Llama 3.1-8B and 70B targets at prefix lengths up to 32K show that our method reduces draft-side memory by over 70%. It achieves speedups of up to 2.08 × and 3.33 × , respectively, over autoregressive decoding.

Abstract:
LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.

Abstract:
Large language models (LLMs) are effective rerankers, but pointwise reranking repeatedly processes the same document across different queries, causing substantial redundant document-side computation. We propose DoPR , a compressed document prefix framework that decouples offline document processing from online reranking. DoPR first selects query-independent document representations and converts them into compressed document prefix states, which are precomputed offline and reused whenever the document is retrieved. During online reranking, the model scores each query-document pair by processing only the query and scoring token, with document information supplied by the stored prefix states. This design reduces online cost through both document-side compression and cross-query prefix-state reuse. Experiments on TREC DL, BEIR, and BRIGHT with Qwen3 models from 0.6B to 8B show that DoPR achieves up to 8.0 × online document-side memory reduction and up to 8.04 × latency speedup, while retaining 97.1%-99.5% of the average NDCG@10 of matched full-document rerankers.

Abstract:
Open-source text-only translation LLMs have rapidly improved multilingual coverage and translation quality, but their unimodal design limits their use in multimodal translation. In speech translation, they are commonly used in cascaded ASR–MT pipelines, which introduce additional latency and prevent the translation model from directly using visual context. Conversely, pretrained multimodal foundation models provide strong audio-visual perception but generally lag behind specialized translation LLMs in multilingual generation quality. We introduce OmniFusion, a modular architecture that connects a frozen multimodal foundation model to a translation-specialized LLM through a lightweight gated fusion module. OmniFusion supports speech-to-text, speech-and-image-to-text, and text-and-image-to-text translation within a single framework. Experiments with Qwen2.5-Omni-7B and Seed-X-PPO-7B show that selected early and middle MMFM layers provide more transferable representations than late layers. On MCIF, OmniFusion improves the latency–quality trade-off over a matched cascaded SimulST pipeline when evaluated with Local Agreement decoding, while remaining competitive in offline ST and reducing severe XCOMET-XL errors. On CoMMuTE, it achieves competitive visually grounded caption translation performance.

Abstract:
Large Reasoning Models (LRMs) have demonstrated exceptional capabilities in complex problem-solving, largely driven by emergent reflection and backtracking mechanisms. However, understanding these mechanisms through mechanistic interpretability remains challenging. Current sparsity-based approaches rely heavily on heuristic keywords (e.g., "wait") and focus on isolated features, lacking a holistic view of how these features collaborate. In this paper, we propose a novel interpretability framework that bridges the microscopic and macroscopic analysis of Sparse Auto-Encoder (SAE) features. By leveraging an auxiliary LRM to robustly annotate reasoning trajectories, we introduce a comprehensive "Reflection Score" to quantify a feature’s relevance to reflection without keyword bias. Our geometric analysis of these features reveals that LRMs encode reflection not as discrete, isolated vectors, but within continuous regions of the hidden state space. Furthermore, we uncover an intrinsic, difficulty-aware reflection representation mechanism: the model dynamically activates sub-regions based on problem complexity. Building upon these insights, we introduce a static, difficulty-informed steering strategy that suppresses superficial, simple-problem oriented reflection features during inference. Comprehensive evaluations across three LRMs and challenging reasoning benchmarks demonstrate that our intervention simultaneously improves reasoning accuracy by an average of 2.25% and reduces token consumption by 5.3%. Our findings provide a deeper geometric understanding of LRM reflection and offer an interpretable, scalable approach to optimizing LRM reasoning.

Abstract:
In post-training large language models (LLMs), supervised fine-tuning (SFT) provides expert guidance but can overfit, while reinforcement learning (RL) enables exploration but often suffers from cold-start and sparse-reward issues. Although combining SFT with RL is intuitive, naively mixing off-policy expert data into on-policy RL can induce distribution shift and destabilize optimization. We propose APEX , a framework that harmonizes off-policy guidance and on-policy exploration via a Cognitive State Awareness mechanism. Specifically, APEX leverages a perplexity-based Student’s -test to compare the sequence-level perplexity distributions of on-policy generations against candidate offline data. This comparison diagnoses whether failures reflect sampling stochasticity or missing knowledge. Guided by this diagnosis, APEX adaptively gates GRPO optimization and modulates SFT supervision. The framework preferentially selects offline samples that are statistically consistent with the current on-policy distribution to guide RL updates, while filtering out high-shift data and verifiably incorrect samples that could destabilize training. APEX also uses a dynamically weighted SFT loss when needed. Extensive experiments show that APEX improves over SFT-then-RL by +7.10 points in average pass@1 on six math benchmarks and by +15.50 points on three general reasoning benchmarks.

Abstract:
LLMs often struggle with modern Korean poetry, producing outputs that resemble “line-broken prose.” We address two coupled tasks: detecting whether a Korean poem is human- or LLM-authored, and guiding LLMs to generate poetry closer in form to human writing. We quantify the human–LLM gap along four form-level linguistic dimensions: output length (Volume), the diversity and connective use of line-final forms (Structure Variation), the irregularity of line lengths (Rhythmic Irregularity), and adherence to standard orthography (Normative Adherence). We operationalize these dimensions as five interpretable features. For detection, a logistic regression classifier over these five features attains an average AUC-ROC of 83.60 in zero-shot out-of-distribution detection across seven unseen LLMs, versus 75.84 for the strongest baseline in our comparison, KatFishNet, an absolute gain of 7.76 AUC points and a 10.23% relative improvement; one generator-specific punctuation pattern outside our taxonomy remains a boundary case. For generation, expert evaluation on GPT-5.2 prefers feature-guided poems over the unconstrained baseline, and analyses across GPT-5.2 and Gemini-3 show that targeted length, rhythm, and ending statistics move toward the human distribution. These results suggest that interpretable, language-specific features can bridge the diagnosis and guidance of LLM-generated poetry.

Abstract:
Vertical-domain few-shot classification remains challenging for small language models, as limited supervision makes it difficult to acquire domain-specific decision knowledge. On-Policy Distillation (OPD) can improve teacher-guided adaptation by supervising student-generated rollouts, while GRPO-based reinforcement learning can further refine downstream predictions. However, existing KD-to-RL pipelines typically rely on globally fixed transition schedules, ignoring that different samples may require different amounts of teacher-guided acquisition before reward-driven refinement. We propose PIVOT (Perplexity-Informed Transition Optimization), a dynamic transition framework that routes samples between OPD and GRPO according to teacher-evaluated sequence perplexity. PIVOT moves low-perplexity samples to GRPO for reward-driven refinement while keeping high-perplexity samples under OPD for continued domain knowledge acquisition. Experiments on Banking77 and HWU64 show that PIVOT consistently outperforms continued OPD and globally synchronized OPD → GRPO baselines under the same optimization budget, achieving stronger downstream performance and more stable training dynamics.

Abstract:
Foundation models have established unified representations for natural language processing, yet this paradigm remains largely unexplored for tabular data. Existing methods face fundamental limitations: LLM-based approaches lack retrieval-compatible vector outputs, whereas text embedding models often fail to capture tabular structure and numerical semantics. To bridge this gap, we first introduce the Tabular Embedding Benchmark (TabBench), a comprehensive suite designed to evaluate the tabular understanding capability of embedding models. We then propose TabEmbed, the first generalist embedding model that unifies tabular classification and retrieval within a shared embedding space. By reformulating diverse tabular tasks as semantic matching problems, TabEmbed leverages large-scale contrastive learning with positive-aware hard negative mining to discern fine-grained structural and numerical nuances. Experimental results on TabBench demonstrate that TabEmbed significantly outperforms state-of-the-art text embedding models, establishing a new baseline for universal tabular representation learning.

Abstract:
Rare-disease diagnosis requires converting heterogeneous patient descriptions into a ranked differential over thousands of disorders while keeping each hypothesis tied to verifiable evidence. Phenotype-based tools provide structured, interpretable comparisons but usually require normalized Human Phenotype Ontology (HPO) input. Large language models (LLMs) can read clinical narratives and propose broader hypotheses, yet their free-form outputs are difficult to ground in a disease catalog or combine with ontology-based evidence. We present ORBIT (Orphanet-grounded Retrieval and Bounded LLM Interaction for Traceable rare-disease diagnosis), a catalog-grounded diagnosis layer for both free-text and structured-HPO cases. ORBIT grounds open disease proposals to Orphanet, fuses them with exact phenotype and case-profile retrieval, scores only the resulting candidate pool with a probabilistic model, and reranks a fixed menu of evidence-bearing disease cards. The output records catalog identifiers, candidate provenance, phenotype support, and conflicts. Across RareArena and RareBench, ORBIT improves every reported metric over single-agent and medical-agent systems using the same backbone. Retrieval analysis, ablations, and expert review further identify where complementary candidate sources, iterative refinement, and constrained reranking affect performance.

Abstract:
Understanding the economic intent of Ethereum transactions is critical for user safety, yet current tools expose only raw on-chain data or surface-level intent, leading to widespread “blind signing” (approving transactions without understanding them). Through interviews with 16 Web3 users, we find that effective explanations should be structured, risk-aware, and grounded at the token-flow level. Motivated by these findings, we formulate TxSum, a new domain-grounded NLP task for DeFi transaction explanation, and construct a dataset of 187 complex Ethereum transactions with 2,375 token-flow annotations and transaction-level summaries. We further introduce MATEX, a grounded multi-agent framework for high-stakes transaction explanation. It selectively retrieves external knowledge under uncertainty and audits explanations against raw traces to improve token-flow-level factual consistency. MATEX achieves the strongest overall explanation quality, especially on micro-level factuality and intent quality. It improves user comprehension on complex transactions from 52.9% to 76.5% over the strongest baseline and raises malicious-transaction rejection from 36.0% to 88.0%, while maintaining a low false-rejection rate on benign transactions.

Abstract:
While Large Language Models (LLMs) are aligned to mitigate risks, their safety guardrails remain fragile against jailbreak attacks. This reveals limited understanding of components governing safety. Existing methods rely on local, greedy attribution that assumes independent component contributions. However, they overlook the cooperative interactions between different components in LLMs, such as attention heads, which jointly contribute to safety mechanisms. We propose Global Optimization for Safety Vector Extraction (GOSV), a framework that identifies safety-critical attention heads through global optimization over all heads simultaneously. We employ two complementary activation repatching strategies: Harmful Patching and Zero Ablation. These strategies identify two spatially distinct sets of safety vectors with consistently low overlap, termed Malicious Injection Vectors and Safety Suppression Vectors, demonstrating that aligned LLMs maintain separate functional pathways for safety purposes. Through systematic analyses, we find that complete safety breakdown occurs when approximately 30% of total heads are repatched across all models. Building on these insights, we develop a novel inference-time white-box jailbreak attack and a complementary detection-based defense, both grounded in the identified safety vectors. The attack achieves superior performance, while the defense yields strong detection across diverse attack types, providing robust evidence for the effectiveness of the proposed GOSV framework in LLM safety interpretability.

Abstract:
Visual Quality Assessment (QA) in multimodal large language models (MLLMs) jointly predicts absolute quality scores and relative perceptual rankings. Existing methods optimize score regression or ranking within a unified formulation, leading to unstable predictions and weak generalization under stochastic generation. We find this instability is amplified by response variability in MLLM reasoning, where stochastic decoding produces inconsistent score distributions and degrades cross-dataset robustness. This indicates QA depends not only on preference alignment, but also on response-level stability. We propose PreResQ-R1, a Response-Preference Disentangled Ranking-and-Scoring Reinforcement Optimization framework. It decouples response stabilization and preference learning, with a dual-branch reward modeling intra-sample consistency and inter-sample score-ranking alignment. For video QA, we introduce temporal-spatial evidence aggregation without dense motion modeling. PreResQ-R1 achieves state-of-the-art performance, outperforming comparable methods by 5.60% on IQA benchmarks and 2.53% on VQA benchmarks.

Abstract:
Few-shot prompting provides an effective way to adapt auditory large language models to low-resource tasks such as children’s speech recognition. However, most auditory large language models are not explicitly trained to perform inference in this demonstration-conditioned format, limiting the extent to which they can benefit from In-Context Learning (ICL). To address this limitation, we introduce Few-Shot Aware GRPO (FSA-GRPO), an RL-based post-training recipe that uses a specially designed reward to encourage the model to leverage few-shot demonstrations, thereby strengthening its few-shot adaptation ability. Notably, training with only 2k high-resource adult ASR utterances improves the model’s general few-shot adaptation ability, yielding gains not only in children’s speech recognition (53.9% relative WER reduction without any in-domain training) but also in multilingual ASR (including low-resource languages), speech translation, and audio understanding. We further study data selection and the weight and similarity cutoffs of the auxiliary reward to identify an effective training recipe. Our experiments show that when in-domain data are unavailable or cannot be used for training, FSA-GRPO is more effective than direct tuning on related out-of-domain data.

Abstract:
A prominent paradigm in inference-time alignment employs lightweight supervisors to steer Large Language Models (LLMs). Through empirical analysis, we identify a structural mismatch in this paradigm: weak supervisors exhibit pervasive high entropy across the vast majority of tokens, yet prevailing dense intervention approaches mandate supervision at every decoding step. This leads to frequent low-confidence interventions that can disrupt valid base-model reasoning and incur substantial utility costs. To resolve this, we propose TUSA (Trust-based Uncertainty Sparse Alignment). Moving away from continuous oversight, TUSA reframes alignment as a dynamic arbitration process, introducing an uncertainty-aware arbiter that authorizes intervention only when two conditions are met: the supervisor is confident and the token is semantically salient. This mechanism effectively filters out uncertainty-driven noise and redundant supervision. Extensive experiments across multiple models and benchmarks show that TUSA consistently improves both safety alignment and general helpfulness. By bypassing approximately 50% of alignment steps, it not only enhances safety preference by up to 15.6%, but also boosts general preference rates by up to 12.0% compared to the dense baseline, demonstrating that selective, high-precision alignment can outperform continuous supervision.

Abstract:
KV cache compression is essential for scaling large language models to long-context inference. However, existing methods excel at explicit retrieval tasks but often degrade on tasks that require multi-step reasoning over dispersed evidence. We present ClueKV, a training-free KV cache compression framework that better preserves the reasoning-supportive clues under tight cache budgets. The core design of ClueKV is a self-guided, adaptive headwise token reweighting mechanism. Specifically, ClueKV first uses contrastive probability shifts between the query-conditioned context and a context-free reference to construct an unsupervised clue signal, which highlights context tokens supportive of downstream reasoning under the current input. This signal then serves as self-generated guidance for compression. Based on this signal, ClueKV dynamically identifies, for each input instance, the attention heads that are more aligned with reasoning-relevant context regions, and reweights token importance head by head before KV eviction. Across LongReason, LongBench, and RULER, ClueKV consistently outperforms strong KƒV compression baselines, with the largest gains on complex reasoning tasks. On LongReason, it retains 99.21% of full-KV accuracy at a 50% cache budget, and remains more stable than prior methods as context length increases to 64k tokens. ClueKV also introduces only marginal prefill overhead, making it practical for long-context deployment.

Abstract:
We compile 129 public LLM prompt datasets with more than 1.22TB and more than 673M instances and organise them into a unified taxonomy. We use seven datasets for detailed analysis and identify lexical, syntactic, and semantic patterns that distinguish prompts from general text. We evaluate these features in prompt filtering, source domain routing, and elicited response quality analysis. A 63 dimensional linguistic feature set that can be extracted on a CPU achieves over 91% of the F1 of GPU sentence embeddings and reduces latency per request by nearly half. In cross dataset routing, sentence embeddings reach 0.74 Macro-F1 and a structural feature subset reaches 0.61, compared with the chance level of 0.20. Structural routing features remain negatively associated with elicited response quality after prompt length is controlled on UltraFeedback. The apparent positive association of type to token ratio becomes much smaller under the same control. These findings support prompt processing pipelines that combine efficient structural features with robust semantic routing. Our catalogue, annotations, and code are given at https://github.com/ymzhang-cs/prompt-dataset-analysis.

Abstract:
Online scams are a growing form of digital harm, yet scam research remains constrained by fragmented datasets and limited annotation depth. We introduce SCAMBENCH, a benchmark of 1,836 authentic scam messages curated from victim-reported incidents across six scam types: employment, investment, imposter, sweepstakes/lottery/prizes, tech support, and advance fee loan scams, along with 1,834 real-world non-scam examples from legitimate sources. SCAMBENCH provides multi-dimensional annotations that capture persuasion strategies from the scammer perspective, red flags from the user perspective, and severity levels reflecting potential harm and urgency. These annotations are produced through a human-AI collaborative pipeline that uses expert-labeled examples to calibrate and validate LLM-assisted annotation at scale. Our analysis shows that these signals vary significantly across scam types, highlighting substantial scam-type heterogeneity. We further use annotated evidence spans to generate cue-removed hard scam examples by rephrasing explicit persuasion and red-flag cues. Evaluating five LLM-based detectors on original and rewritten scams shows that all five detectors perform worse on cue-removed examples, and the performance drop persists on a human-validated subset, suggesting sensitivity to surface-level cues. A qualitative case study identifies five recurring failure patterns. SCAMBENCH provides a resource for scam detection, robustness evaluation, and scam education, publicly available at https://huggingface.co/datasets/Shouninger/ScamBench.

Abstract:
Test-Time Scaling has emerged as a powerful method to extend the reasoning capabilities of Large Language Models. However, single-agent TTS faces significant scalability bottlenecks, as excessively long reasoning traces lead to increased inference costs and stability issues caused by context management failures. To address these limitations, we propose leveraging Multi-Agent Systems as a structural upgrade to standard TTS. By decomposing monolithic reasoning chains into distinct, manageable contexts across multiple agents, MAS offers a more robust framework for scaling reasoning. We validate this approach by introducing M500, a dataset comprising 500 high-quality multi-agent, multi-turn collaborative reasoning traces generated via DeepSeek-R1. Through Supervised Fine-Tuning on M500, we enable open-source models to internalize collaborative reasoning patterns and show improved TTS performance in MAS. Furthermore, we propose an adaptive scaling strategy incorporating a “CEO” agent to dynamically guide the reasoning process and optimize collaboration depth. Extensive experiments within the AgentVerse framework demonstrate that our fine-tuned models, Qwen2.5-32B-MAS and Phi4-14B-MAS, significantly outperform their base counterparts.

Abstract:
Multilingual large language models (LLMs) exhibit substantial performance disparities across languages, with low-resource languages often lagging far behind high-resource ones. Federated learning (FL) offers a privacy-preserving path for decentralized multilingual adaptation, but existing methods typically aggregate client updates without explicitly modeling structural differences across languages, inducing gradient interference, amplifying negative transfer, and further disadvantaging underrepresented languages. We introduce FedLing, a linguistic-structure-aware federated learning framework. FedLing performs typology-conditioned spectral routing of client LoRA updates, decomposing them into linguistically transferable consensus subspaces and language-specific divergence subspaces. Unlike posterior-only decomposition methods that require accumulated training history to estimate transferability, FedLing explicitly mixes a round-one energy prior with typology-conditioned empirical subspace alignment through a single coefficient ρ. A continuous typological isolation score further prevents negative transfer for languages without close federation neighbors, and a reciprocal fairness proxy adapts upload bandwidth toward underperforming languages. Experiments across 40 languages show that FedLing consistently improves multilingual adaptation: +7.9 average ROUGE-L and +11.6 ROUGE-L on low-resource languages over FedAvg, with 52% variance reduction and 35% upload savings. The advantage is largest in cold-start and data-scarce regimes, validating the combination of energy-based cold-start routing and typology-conditioned posterior alignment.

Abstract:
Multiscale Muon maintains K momentum EMAs at different decay rates 𝛽 k , doubling or tripling the optimizer-state memory of Muon. We show that the EMA update is a first-order low-pass filter and Multiscale Muon is therefore a filter bank on the gradient stream, with each band tuned to a different temporal scale. Under this view, the empirical fact that Multiscale Muon outperforms any single- 𝛽 filter is a direct consequence of multi-band coverage; and a sharp, asymmetric compressibility property follows from the bandwidth—the narrow-band (high- 𝛽 ) buffer is dominated by the slow, low-rank component of the gradient and is safe to project, while the wide-band (low- 𝛽 ) buffer carries high-frequency content whose effective rank is large and cannot be compressed without hurting convergence. We formalize this asymmetry, instantiate it in SMuG (Selective Multiscale Muon with GaLore) by applying GaLore only to high- 𝛽 buffers, and prove an algebraic impossibility result ruling out any non-negative linear spectral score from predicting the band-mixing coefficient 𝜆⋆ in closed form. On 200,M GPT-2 pretraining ( n=4 seeds), SMuG ( K=2 , r/n=0.125 ) reaches PPL 21.66±0.05 , statistically tied with full-rank Multiscale Muon ( 21.69±0.03 ), at 1.25× Muon’s optimizer memory (including the persistent GaLore basis V ) versus 2.00× for full Multiscale Muon—recovering 75% of the added overhead at no average quality cost.

Abstract:
Analogy is a quintessential human cognitive capability that supports knowledge transfer across domains. While researchers have long studied analogy generation, evaluation, and reasoning through symbolic and connectionist approaches, the rise of large language models (LLMs) has introduced new generation systems and evaluation methods. However, a systematic understanding of this emerging landscape and its connection to prior work remains underexplored. To address this gap, we conduct a PRISMA-based systematic literature review spanning four decades across six databases. We identify three key observations: first, generation and evaluation methods are strongly shaped by analogy granularity, such as lexical versus compositional analogies; second, human-in-the-loop design plays an important role in LLM-based analogy generation systems; and third, the field remains divided over whether LLMs perform genuine analogical reasoning. Based on our review, we propose practical guidelines for future work on analogy generation, including potential evaluation dimensions for lexical and compositional tasks.

Abstract:
Large Language Models (LLMs) exhibit strong performance across language tasks, raising the question of whether this success reflects linguistic abstraction or reliance on surface-level statistical patterns. This survey reviews recent advances in mechanistic interpretability (MI) for analyzing candidate internal mechanisms underlying linguistic competence in LLMs. We trace the shift from model-agnostic analyses to transformer-specific methods and summarize core approaches including probing, neuron- and circuit-level analysis, and causal interventions. Synthesizing empirical findings, we argue that current evidence supports an intermediate view: LLMs appear to use reusable mechanisms for some grammatical behaviors, while these mechanisms remain frequency-conditioned, distributed, and only partially understood. We further review interpretability studies of multilingual models, highlighting evidence for both language-specific and shared representations. Finally, we outline open challenges in scalability, benchmarking, and ethics, and discuss directions toward a more cautious mechanistic account of language in LLMs.

Abstract:
Supervised fine-tuning (SFT) applies a uniform cross-entropy loss to all target tokens, even though different tokens provide unequal learning signals for mathematical reasoning. This uniform treatment can over-sharpen already mastered tokens while amplifying learning pressure on uncertain, low-confidence tokens, leading to suboptimal training dynamics. We propose Trimmed Logit-Gap SFT (TrimSFT), a simple token-level reweighting method that scales the SFT loss according to the logit gap between the gold token and its strongest competitor. TrimSFT trims supervision away from both extremes: tokens already mastered (large logit gap) and tokens weakly supported by the current model (small or negative logit gap), concentrating learning within an intermediate logit-gap region between them. We instantiate this principle with a Gaussian weight centered at margin m with bandwidth τ, requiring no reference model or additional forward pass. We evaluate TrimSFT on six base models from the Llama, Qwen, and DeepMath families across five mathematical reasoning benchmarks. TrimSFT consistently improves over standard SFT, achieving the best average performance on five out of six models, with gains of up to +26.9 points over SFT on MATH500. Further analyses show that the bandwidth τ matters more than the exact margin location, and that half-trim variants that remove supervision pressure from only one side yield inferior trade-offs. A token-level logit-gap distribution analysis suggests that TrimSFT reshapes model confidence in a more balanced way than uniform SFT or monotonic reweighting methods. These results suggest that reasoning SFT can benefit from trimming both extremes rather than treating all tokens uniformly.

Abstract:
Multi-Agent Debate (MAD) improves LLM-agent accuracy but suffers from rapid context growth, limiting scalability in larger multi-agent settings. Existing methods prune low-utility communications using prior signals, such as token-level log-likelihoods or LLM self-reported confidence. However, these signals become unreliable under hallucination, degrading the accuracy of MAD methods that rely on them. We propose SVR-MAD, a Bayesian-inspired MAD framework that treats pre-debate signals as priors and debate outcomes as posterior-style evidence for estimating agent correctness. SVR-MAD uses this evidence to incrementally construct the communication graph, prioritizing agents whose answers survive peer challenges. Experiments across multiple LLMs and benchmarks show that SVR-MAD reduces token cost by up to 61% while matching or improving accuracy relative to the most accurate competing MAD baseline.

Abstract:
We introduce FLAME (FLemish Accounts of Momentary Experiences), a corpus of nearly 25,000 personal narratives in Belgian-Dutch (Flemish), collected through experience sampling to support Natural Language Processing (NLP) research on an underrepresented variety. Such everyday narratives are rich in culturally grounded themes, but their informal register and low-resource setting make thematic extraction hard. Comparing K-Means, LDA, and BERTopic, we find that human evaluation favors BERTopic, which produces the most coherent, culturally resonant topics. FLAME, thereby, offers a new resource for studying everyday language use in a low-resource variety.

Abstract:
We introduce SHARP, a compact, span-annotated dataset for hallucination detection in model-generated mathematical solutions. Unlike existing resources that rely on coarse step-level labels, SHARP provides fine-grained, span-level annotations generated through an automated pipeline. Our methodology synthesizes realistic reasoning traces with precise error markup, enabling the production of high-fidelity training data with minimal computational overhead. Despite being 10–30 tim smaller than existing open-source datasets, a 1.5B PRM fine-tuned on SHARP achieves state-of-the-art performance on ProcessBench.

Abstract:
Latent language identification is often used to argue that multilingual language models route computation through language-specific states, such as English pivots. However, existing probes infer latent language from different signals, such as the geometry of hidden states or what can be decoded from intermediate representations. Since such claims shape conclusions about how models share and route information across languages, we ask whether these probes measure the same phenomenon or expose distinct aspects of multilingual computation. We study this question across model families, training regimes, domains, tasks, checkpoints, and up to 27 languages. We find that identification probes systematically disagree: the GMM-based representation probe, which draws evidence from hidden state geometry, shows earlier cross-lingual mixing, whereas decoding-based probes, which rely on output-space decodability, retain sharper language-specific and more English-biased signals. These differences track model multilinguality and training progression, but are comparatively stable across domains. Our results suggest a more cautious interpretation of latent language identification, where current probes expose different aspects of multilingual processing, rather than directly revealing a single internal lingua franca.

Abstract:
Deep research agents have demonstrated remarkable capabilities in complex information-seeking tasks, yet this power comes at a steep computational cost. Driven by accuracy-focused training paradigms, current models adopt brute-force strategies characterized by blind tool dependency and performative reasoning—generating long, redundant trajectories that are far from necessary for resolving these tasks, leading to wasteful tool calls and excessive token consumption. To overcome this efficiency trap, we propose SlimSearcher, a principled framework that pushes the Pareto frontier between accuracy and computational cost across both Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL). In the SFT stage, SlimSearcher employs Pareto-efficient filtration to distill trajectories that are both successful and economical, guiding the model toward inherently efficiency-aware search behaviors. During RL, we introduce Adaptive Reward Gating, a dynamic reward-shaping mechanism that evaluates relative tool and token efficiency within a sampled cohort. By cascading these adaptive efficiency metrics with a strict correctness gate, our approach effectively avoids the brevity bias associated with absolute penalties and mitigates reward hacking. Extensive experiments on long-horizon benchmarks, including GAIA, BrowseComp, and XBench-DeepSearch, demonstrate that SlimSearcher reduces average tool-call rounds by 28%–59% while maintaining or improving accuracy.

Abstract:
Large language models (LLMs) are increasingly applied to specialized domains, where effective use of domain expertise often requires reasoning over complex rules in concrete scenarios. However, existing benchmarks only partially evaluate this capability, as they either focus on output-level instruction constraints or overlook the distinct roles that rules play in scenario reasoning. To address these gaps, this paper introduces RuleWeaver, a benchmark construction framework for evaluating rule-centered scenario reasoning. RuleWeaver starts from corpus-derived IF-THEN Meta Rules, progressively augments them into complex rules, and composes these rules into rule-centered scenario QA instances. Beyond final-answer correctness, RuleWeaver further supports process-level evaluation through rubric-based answer quality, rule recall, and rule precision. Experiments on 11 representative LLMs show that current models still struggle with complex rule-centered scenario reasoning, with even the best-performing model achieving only around 50% of the maximum rubric score.

Abstract:
Abundant procedural knowledge on the Web holds great potential for helping agents solve long-horizon tasks. However, such knowledge is often multimodal, heterogeneous, noisy, and implicitly assumes human executors, making it difficult to use directly as the skills required by agents. To bridge the gap between human-oriented guides and agent-executable skills, we formalize this problem as guide-to-skill learning: converting in-the-wild guides into executable skills and continuously improving them from trajectories observable to the agent. To evaluate the capability of existing agents on this task, we introduce MMG2Skill-Bench, the first benchmark designed for this problem. We further propose MMG2Skill, a closed-loop framework that compiles guides into editable skills, conditions a fixed vision-language model (VLM) agent on these skills during execution, and revises the skills from trajectory-level root-cause feedback without using benchmark scores. Across GUI control, open-ended gameplay, and strategic card play with six VLM backbones, MMG2Skill consistently outperforms vanilla baseline agents in every model–domain setting, achieving macro-average gains of +12.8 to +25.3 percentage points across backbones. Ablation studies show that directly prompting agents with raw guides can degrade performance, while both structured skill construction and trajectory-driven revision are necessary for the observed improvements. On success-inferable tasks, analyzer-based early stopping further prevents late-stage performance regressions and saves 25%–53% of attempts when the success signal is properly calibrated.

Abstract:
Sample-level ranking is widely used for data selection, debugging, and training-data attribution, but redundant NLP corpora—exact and near duplicates, paraphrases, and templated examples—can make proxy-induced rankings unstable across random seeds. We study stable sample-level ranking under redundancy and propose SCARV, a modular aggregation layer that operates on top of existing scoring proxies. SCARV normalizes scores within each run, can regularize redundant examples through cluster-level aggregation and allocation, and aggregates across seeds with a robust operator. Across synthetic redundancy, naturally mined QQP redundancy, multiple proxy families, classification and sentence-pair tasks, and end-to-end DistilBERT fine-tuning, stability-aware aggregation improves over bare proxy rankings and yields more reproducible ranking-based decisions, such as selected subsets and suspicious-example sets. Matched mechanism analysis shows that cross-seed aggregation is the dominant generic stabilizer. The incremental effect of structural regularization is small, map-dependent, dataset-dependent, and sometimes negative; a meaningful redundancy map is useful mainly when group-consistent treatment is operationally desired. These results position SCARV as a stability-oriented aggregation framework rather than a universal data selector and emphasize decision reproducibility over uniform gains in downstream utility.

Abstract:
Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We investigate the potential of news article corpora as an alternative to RIS, linking images to relevant articles to infer their dates and locations from article metadata. We evaluate the performance of a news article retrieval pipeline, NewsRECON, which leverages a corpus of over 85,000 articles. Experiments on the TARA dataset show that NewsRECON outperforms prior work and can be combined with a multimodal large language model (MLLM) to achieve new SOTA results in the absence of RIS evidence. Furthermore, NewsRECON generalizes to the 5Pils-OOC benchmark despite geographic and temporal shifts. We make our code available.

Abstract:
Recent studies show that text-to-image (T2I) diffusion models are vulnerable to backdoor attacks, where a trigger in the input prompt can steer generation toward harmful or unintended content. Beyond the trigger token itself, backdoor effects can spread to neighboring tokens in the text embedding space. To address this, we introduce PEPPER (PErcePtion-Guided PERturbation), a backdoor defense that rewrites the caption into a semantically distant yet visually similar caption while adding unobtrusive elements. With this strategy, PEPPER disrupts the trigger embedded in the input prompt, escapes the attacked neighborhood, and thereby achieves enhanced robustness without training or access to model weights. Experiments show that PEPPER is particularly effective against text encoder–based attacks, substantially reducing attack success while preserving generation quality. PEPPER can also be paired with any existing defenses yielding consistently stronger and generalizable robustness than any standalone method.

Abstract:
Epidemic intervention policies are textual artefacts that human decision-makers interpret, justify, and revise through natural language, making large language models a natural candidate for epidemic policy reasoning. A naive LLM, however, lacks the epidemic dynamics needed to project intervention consequences, the quantitative surveillance signals required to assess severity, and the institutional constraints that define admissible actions. We present EpiWorld, a closed-loop framework that grounds an LLM policy actor in a learned action-conditioned epidemiological world model and a tiered skill library of public-health protocols, surveillance tools, and adaptive lessons accumulated through after-action analysis. Given a candidate intervention, the world model predicts regional epidemic evolution and enables fast counterfactual rollouts that provide feedback for policy selection and refinement. Outcomes of simulated futures are distilled into reusable lessons while protocol constraints remain fixed, allowing the decision process to improve without sacrificing interpretability or controllability. We evaluate both the world model and the end-to-end framework on retrospective COVID-19 and Influenza datasets: the world model achieves the best out-of-distribution Peak-MAE among all forecasting baselines, and the closed-loop framework reduces cumulative hospitalisation by up to 59% across datasets and by an average of ~16% across six LLM backbones, outperforming reinforcement-learning and optimal-control policy baselines.

Abstract:
Safety guardrails in Code Large Language Models (Code LLMs) are designed to prevent the generation of malicious code, yet their robustness under linguistically diverse inputs remains largely unexplored. To address this important safety issue, we present MALICE (Malicious Adversarial LLM Inputs for Code Generation), a large-scale adversarial benchmark that exposes a critical blind spot at the intersection of multilingual malicious prompts and code generation safety. Our approach converts malicious English prompts into code-mixed and transliterated variants through sentence-level translation into low-resource languages while preserving programming keywords intact. The resulting dataset comprises approximately 250,000 adversarial prompts spanning 18 languages at various resource levels, derived from nine existing malicious code benchmarks. We perform a comprehensive evaluation across nine models (including Claude 4.6 and GPT-5.2) and safety tools, which reveals concerning vulnerabilities: code-mixed malicious prompts reduce detection accuracy by over 0.70 absolute points for the strongest classifier, while even the most safety-aligned models see their malicious code generation rates increase threefold. These results demonstrate that current safety alignment fails to generalize beyond English inputs, underscoring an urgent need for multilingual defenses in Code LLMs.

Abstract:
We introduce FormalMATH, a large-scale Lean4 benchmark for formal mathematical reasoning, comprising 5,560 semantically validated Lean4 statements spanning competition-level and undergraduate mathematics. To improve reliability, FormalMATH is constructed through a human-in-the-loop pipeline that combines type checking, LLM-based semantic verification against the informal–formal statements alignment, negation-based disproving, and expert review. We conduct a systematic evaluation of frontier theorem provers under test-time scaling on FormalMATH. Results show that current provers remain far from solving the benchmark: the strongest model, DeepSeek-Prover-V2 (671B), achieves only 28.31% Pass@32. We further observe substantial domain imbalance across provers, with stronger performance in applied mathematics and algebra but weaker results in discrete mathematics. We identify common failure modes in current provers, including incomplete proofs, weak handling of inequalities, misuse of automation tactics, and redundant hypothesis introduction. Extreme test-time scaling (e.g., Pass@3200) yields only modest gains on a subset of FormalMATH, suggesting that current training paradigms do not equip provers to effectively exploit additional sampling budget. We also observe a counterintuitive negative effect when provers are given human-written natural-language solution references: such references can measurably degrade formal proof generation rather than aid it. Overall, FormalMATH provides a challenging and reliable testbed for measuring progress in formal mathematical reasoning.

Abstract:
Generative backdoor attacks pose a growing threat to the instruction tuning of large language models (LLMs), where attackers poison a few training samples to induce malicious payloads on triggered inputs while preserving normal behavior on clean ones. However, existing backdoor-poisoned sample filtering methods typically rely on global sample representations and clustering-based separation, and such global signals can become fragile when heterogeneous target-side malicious payloads scatter poisoned samples or low poisoning rates prevent poisoned samples from forming stable clusters. This motivates us to move beyond cross-sample separability and instead exploit localized token-level target-side anomalies induced by malicious payload tokens under teacher forcing. Building on this intuition, we propose TTAF, a Teacher-forcing Token-level Anomaly Filtering framework which introduces a token-level anomaly score under teacher forcing and aggregates localized evidence to filter suspicious poisoned samples before fine-tuning. Experiments across multiple datasets, representative backdoor attacks, and different LLMs show that TTAF consistently achieves strong filtering performance within this payload-injection setting, with efficient runtime and no external dependencies. Code is available at https://github.com/Sam-Han2020/TTAF.

Abstract:
While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries implicitly rely on domain-specific knowledge, such as business logic, data conventions, and analytical practices, that is neither captured by the schema nor explicitly stated in the natural language question. Historical SQL query logs offer a valuable source of such knowledge, yet existing benchmarks do not adequately support evaluation of history-driven approaches. To address this gap, we introduce BIRD-History, a benchmark consisting of 1,393 tasks across 11 databases, designed to evaluate text-to-SQL systems’ ability to ground underspecified natural language questions using historical SQL scripts. Each task is annotated with ground-truth labels specifying which historical queries contain relevant knowledge and which SQL clauses encode it, enabling systematic evaluation of both retrieval effectiveness and knowledge utilization. Alongside the benchmark, we propose a plug-in retriever that extracts five types of external knowledge from historical SQL scripts, then retrieves and reranks relevant fragments for query generation. The retriever integrates seamlessly into existing few-shot text-to-SQL pipelines without requiring prompt modifications. Experiments demonstrate consistent improvements across four text-to-SQL systems, highlighting the value of leveraging historical query logs for handling underspecified queries. Dataset and code are open-sourced on https://github.com/zjuidg/BIRD-History.

Abstract:
With the rapid development of Large Language Models (LLMs), research related to LLM-based persuasion has become a hotspot in the field. However, existing studies mostly focus on enhancing LLM agents’ persuasive capabilities, while generally neglecting the issue of their vulnerability to persuasion. In this paper, we analyse the root causes of LLM agents’ vulnerability to persuasion and find that it mainly stems from two contradictions and limitations. First, the purpose of catering to human preferences in pre-training phase conflicts with the need to resist manipulation in persuasion scenarios. Second, LLMs are often restricted to play a helpful assistant during training, and the singularity of this personality setting makes it unusually easy to develop targeted persuasion techniques for them. To eliminate LLMs’ vulnerability to persuasion, we propose the Backfire-R1 framework, which helps LLMs build persuasion resistance and better align with user objective by enabling them to learn the reasoning logic of humans with different personalities in persuasion scenarios. Comprehensive experimental results show that, the proposed method significantly improves the persuasion robustness of LLM agents, reducing the opinion shift score by 62.36% while not impairing the model’s general capabilities. Code is now available at: https://github.com/super-jw/Backfire-R1 .

Abstract:
Multimodal Large Language Model (MLLM)-based web agents provide practical, high-precision solutions for visual browser automation; however, they inherently expand the attack surface, introducing novel vision-based vulnerabilities. Existing adversarial evaluations targeting these agents frequently rely on permissive threat models and visually conspicuous artifacts. In this paper, we investigate a constrained vulnerability detection setting: a trusted web platform where the evaluator acts solely as an unprivileged third party (e.g., a merchant or advertiser) controlling only a semantically legitimate, spatially constrained region, such as an ad slot, a sponsored card, or a localized widget. Operating under these realistic constraints, we propose MIRAGE, a novel visual indirect prompt injection framework for targeted next-action hijacking. Our approach leverages diffusion models to generate perceptually benign adversarial images strictly confined to the attacker-controlled boundaries permitted by the trusted service provider. To maximize attack efficacy within such a restrictive setting, we introduce a robust optimization technique combining curvature-aware adversarial diffusion guidance with sparse, dark-pixel residual perturbations. Comprehensive evaluations against prominent MLLM web agent frameworks, specifically SeeAct and OpenClaw, empirically demonstrate the potency, realism, and stealth of our proposed MIRAGE. Code is available at https://github.com/EricDai0/MIRAGE.

Abstract:
Group Relative Policy Optimization (GRPO) is an attractive reinforcement learning paradigm for Large Reasoning Models because its groupwise relative advantages eliminate the need for a value network, but its efficiency is severely limited by zero-variance queries whose rollouts are uniformly correct or uniformly incorrect and therefore consume expensive Chain-of-Thought generation without providing useful learning signals. Existing remedies typically rely on offline difficulty labels or external capability trackers, which either fail to follow the model’s rapidly evolving competence or compromise the end-to-end simplicity of GRPO. We propose Active-GRPO, a prompt-free extension that equips the policy with intrinsic fuzzy metacognition via a lightweight confidence head attached to the final prefill representation. The head predicts ordinal confidence bins and is jointly calibrated against empirical rollout outcomes through a dual-objective optimization framework. It further enables a multi-criterion voting mechanism that filters uninformative queries before rollout generation, thereby concentrating compute on samples near the learning boundary with negligible overhead and without perturbing the reasoning distribution. Experiments on mathematical and logical reasoning benchmarks show that Active-GRPO reduces training computation while improving both reasoning accuracy and intrinsic calibration quality.

Abstract:
Recent advancements in Large Language Models (LLMs) have shown promising results in music understanding and generation tasks. However, existing works remain confined to Western tonal traditions, offering little insight into whether current LLMs can handle structurally distinct low-resource musical traditions. We present the first systematic evaluation of LLM competence in South Asian classical music, a tradition governed by raga, tala-based melodic constraints that impose fundamentally different structural principles from Western harmony-driven music. We ground our evaluation in Hindustani classical theory and Bengali classical forms, including Rabindra and Nazrul Sangeet – representative low-resource traditions within South Asian classical music. For music understanding evaluation, we introduce a 504-question-answer benchmark spanning raga grammar, cultural knowledge, and symbolic notation reasoning, evaluating 33 LLMs where frontier models such as Gemini 2.5 Pro achieve 85-90% accuracy, while most open-source models remain in the 23-40% range. For music generation, we design a five-level controlled prompting framework and find that even the strongest model produces stylistically faithful outputs only 40% of the time. These results reveal that structural validity and stylistic faithfulness in music generation are distinct objectives and highlight an open challenge for culturally grounded music modeling.

Abstract:
Visual modality enhances the capabilities of Multimodal Large Language Models (MLLMs) but also introduces a safety concern: a benign textual query may convey harmful intent when grounded in a visual image. We term this as cross-modal safety drift, and our pilot studies show that the safety response rate for such requests is substantially lower than that for requests containing explicitly unsafe text. This paper aims to systematically study this issue. First, we conduct an empirical analysis to identify representative unsafe response patterns. Building on these, we interpret model representations and attentions, revealing that visually risky cues receive limited attention and weakly trigger refusal. Motivated by the observation that safety signals from unsafe text processing can be transferred, we propose safety-awareness representation transfer (SRT), a lightweight direction-refinement method that mitigates cross-modal safety drift while freezing the MLLM backbone. Experiments across benchmarks and models show that SRT effectively improves safety in diverse cross-modal settings while preserving utility.

Abstract:
Temporal reasoning benchmarks for Video-LLMs are often mediated by language, leaving room for linguistic shortcuts from option wording, answer correlations, or language priors. To reduce such shortcuts, we introduce TempCloze , a video cloze benchmark for evaluating visual temporal reasoning in Video-LLMs. Given the beginning and ending clips of a video, models must identify the true missing middle from four candidates. TempCloze contains 1,521 carefully filtered videos from seven sources, mainly long-take and egocentric videos. We construct same-source distractors along three dimensions: Semantic asks what event should happen, Alignment probes when it should occur, and Progression tests how it should unfold, while shared scenes and objects reduce appearance cues. Our evaluation of 10 proprietary and 21 open-source Video-LLMs reveals Alignment as the primary bottleneck: models often recognize plausible semantic content and local event progression but struggle with temporal alignment. We further conduct error pattern and behavioral sensitivity analyses on TempCloze-Mixed and TempCloze-Hard with four representative models to examine where errors arise and how candidate order, context direction, visible span, frame density, and test-time scaling influence model choices.

Abstract:
Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as the standard paradigm for improving reasoning capability of large language models, while Multi-Token Prediction (MTP) has been a widely adopted module in pretraining. Combining the two is a natural approach, yet current RL practices detach MTP gradients because joint training degrades the performance. We revisit this failure from an optimization perspective. We show that the per-step effect of MTP on the RL objective can be decomposed into two terms: a first-order correlation and a second-order perturbation penalty. This decomposition unifies three MTP training regimes: Detach, Cross-Entropy loss, and Policy loss, and explains why each succeeds or fails. Further analysis of policy loss reveals that, although it aligns with intuition, performance still degrades: the correlation term decays while the quadratic penalty persists. Guided by the analysis, we propose Optimal Coefficient Calibration (OCC), an adaptive scheme that tracks the optimal coefficient online via a log-probability proxy at negligible cost. Across six competition-level mathematical reasoning benchmarks, OCC consistently matches or exceeds the detach baseline, delivering improved joint MTP-RL training performance.

Abstract:
Lifelong conversational agents rely on memory systems to maintain deep, context-aware interactions with users. However, existing explicit textual memory pipelines suffer from a severe information bottleneck, often losing subtle behavioral patterns and emotional shifts. Furthermore, being typically static post-deployment, they cannot autonomously adapt to personal habits and preferences without manual feedback. Cognitive science, however, suggests that humans maintain mental models purely in a latent space and continuously refine them through predictive coding. Inspired by this, we propose ThinkFlow, a novel end-to-end latent memory framework for lifelong conversational agents. ThinkFlow bypasses the text bottleneck by dynamically compressing conversational flows into probabilistic latent memory skills, autonomously consolidating complex user states into disentangled, continuous vectors without semantic interference. To break this barrier, we introduce a test-time evolution paradigm. By coupling teacher-guided latent alignment to bootstrap the initial state with a self-supervised next-user-utterance prediction task for continuous refinement, the framework successfully overcomes cold-start challenges and achieves label-free lifelong personalization. Extensive experiments on long-term conversation benchmarks demonstrate that ThinkFlow significantly outperforms prevailing memory systems, providing highly personalized and contextually accurate responses over extended multi-session interactions.

Abstract:
Browser-use agents automate everyday web tasks through structured DOM interfaces or vision-language models operating on page screenshots. However, web pages can change between observation and action execution due to advertisements, asynchronous loading, and timed updates, causing agents to act on stale assumptions. We frame this temporal mismatch as a time-of-check to time-of-use (TOCTOU) vulnerability in browser-use agents, where page changes during planning can alter the target, value, or validity of a planned action before execution. We introduce DYNWEB, a benchmark spanning synthesized and real-world websites, and evaluate 10 open-source agents as well as Codex’s computer-use feature across different observation and action modalities. All 11 evaluated systems exhibit at least one TOCTOU violation, showing that the vulnerability is not limited to a particular agent design. To reduce this exposure, we design a lightweight pre-execution validation mechanism that reduces the exploitable window from approximately 10 s to 0.13 s. Across three agent frameworks, the mechanism reduces the number of vulnerable DYNWEB cases to 0 out of 14. Adaptive attacks further show that stealthy single-mutation attacks succeed in at most 2.2% of trials, while event-driven timing reveals the limits of pre-execution validation.

Abstract:
Knowledge-Based Visual Question Answering (KB-VQA) requires retrieving external textual knowledge to answer visual queries. However, current methods often suffer from two critical misalignments during retrieval: distributional misalignment, driven by the severe stylistic shift from descriptive pre-training image captions to formal knowledge texts, and granularity misalignment, where lengthy knowledge documents contain excessive noise that drowns out key information, making it difficult to accurately match the visual query. To resolve these challenges, we propose RGCC, a Retriever-Guided Context Compression framework. By mining alignment-friendly data and further training a context compression model, RGCC condenses lengthy documents into compact, retriever-preferred representations that significantly improve cross-modal retrieval. Extensive experiments on the InfoSeek and E-VQA benchmarks demonstrate that RGCC achieves state-of-the-art retrieval performance and boosts downstream question-answering accuracy without additional online inference overhead.

Abstract:
Radiology Report Generation (RRG) has benefited from stronger encoder-decoder pipelines and particularly Large Language Model (LLM) report generators, but most methods still perform training through random mini-batches. This practice is poorly matched to chest radiology corpora, where common normal statements and frequent findings dominate the optimization signal, while clinically important pathological attributes appear in the long tail. We argue that random sampling pushes report generators toward a frequent-finding shortcut, where models quickly learn fluent report narratives and high-frequency disease phrases, then receive limited pressure to recover rare attributes that contribute little to aggregate cross-entropy. To address this problem, we propose ParaC, a pathology-aware replay curriculum for long-tail RRG. ParaC ranks training reports with pathology labels and structured medical knowledge, expands the training process from simple reports to complex long-tail cases, and replays prototype anchors to preserve basic cross-modal grounding. Extensive experiments on IU X-Ray and MIMIC-CXR confirm the superiority of ParaC on several existing methods, with further ablations analyzing the large gains brought by our pathology-aware curriculum to cover rare pathological findings.

Abstract:
LLM agents are increasingly deployed on long-horizon tasks, where a user often revises the goal partway through rather than stating it once at the start. Adapting an ongoing trajectory to such a mid-task intervention is a basic requirement for agents that work with a user, yet existing agentic benchmarks evaluate agents under a goal that stays fixed for the whole task. To address this gap, we introduce InterventionBench , a benchmark that measures how well an agent adapts to a mid-task intervention. Built on AppWorld and WebArena, InterventionBench inserts a user message that revises the goal partway through each task, and defines the revision along three operations and three components of the goal, forming nine intervention types that locate which kind of revision an agent fails on. The released benchmark contains 2,250 intervention instances. We evaluate twelve LLM agents and find that every intervention type degrades performance. Add interventions, which extend the goal rather than replace it, are consistently the hardest. These results establish adapting to a mid-task intervention as a substantial and open challenge for current LLM agents. Our code and data are available at https://github.com/jhn25/InterventionBench.

Abstract:
As large language models (LLMs) are increasingly deployed, harmful generations have become a critical safety concern. Existing safeguards operate at the input, output, or streaming-generation stages, while early-risk methods that rely on surface tokens or output logits often suffer from weak initial signals, and internals-based detectors using dense representations may retain highly entangled and redundant safety-irrelevant information. It therefore remains unclear whether the earliest post-generation hidden states already contain reliable signals about final-response harmfulness. To address this gap, we propose ForeSight, a first-token output-risk forecasting framework that distills weak and redundant early safety signals into compact, layer-aware risk representations. Experiments across five safety benchmarks and two target models demonstrate that ForeSight achieves superior and efficient early-risk forecasting while using only first-token hidden states. The code is available at: https://anonymous.4open.science/r/Foresight-B067

Abstract:
LLM agents have made rapid progress on software engineering and ML research tasks, but these advances often assume access to a working runnable environment. For research artifacts released alongside published papers, setting up such an environment from a fresh machine remains a major bottleneck. Existing environment setup benchmarks do not cover the full scope of research-artifact deployment, which involves multi-language toolchains, system-level dependencies beyond containers (e.g., GPU/CUDA and kernel configurations), and legacy artifact compatibility. We introduce DeployBench, a multi-domain benchmark of 51 research-artifact deployment tasks spanning AI/ML, computer systems, and scientific computing, covering all these dimensions. Each task is verified by a hidden pipeline that executes the paper’s designated experiment and checks its outputs. Evaluating five state-of-the-art LLMs with OpenHands yields pass-rates from 7.8%-51.0%. Failures are dominated by a completion-judgment problem: 102 of 181 are agent-terminated self-stops, where the agent’s pre-finish checks validate a different or weaker target than the paper-specific task requires. DeployBench highlights the gap between current agents and autonomous deployment, and offers a realistic testbed for scientific research agents.

Abstract:
Thinking has emerged as a critical capability for Large Language Models (LLMs) tackling complex tasks. However, its reactive nature, where reasoning is passively triggered only upon receiving a user response, inevitably introduces latency that compromises conversational fluidity. This stands in sharp contrast to human dialogue, where speakers proactively anticipate and plan future content during natural pauses to ensure seamless interaction. To bridge this gap, we propose Proactive Thinking, a framework that empowers models to precompute potential response elements during conversational downtime instead of waiting idly for the next input. We then introduce a training-free baseline that can think ahead by anticipating future states, balancing efficiency and quality through speculative continual thinking. To evaluate this approach in practice, we adapt three goal-oriented benchmarks of varying complexity into time-aware environments that simulate real-time conversational flow. We demonstrate that proactive thinking effectively improves interaction efficiency without compromising performance. Ultimately, this work advocates for a fundamental shift toward more intelligent, anticipatory, and real-time conversational AI.

Abstract:
Large language models are increasingly used in practical systems, making efficient model selection important for reducing deployment cost. LLM routing has emerged as a practical solution for allocating each input query to an appropriate model under a desired cost-performance trade-off. Existing routing methods often estimate model suitability from the surface semantics or embedding similarity of the input query. However, such methods may ignore the underlying difficulty of a query, leading to suboptimal routing decisions. To address the challenge, we propose VDAR-Router, a difficulty-aware retrieval-based routing framework. For each input query, VDAR-Router first generates an explicit difficulty analysis. It then retrieves historical examples with similar difficulty profiles. Based on the retrieved records, it estimates candidate model suitability and selects the model using a reward function that considers both performance and cost. Experiments on three datasets show that VDAR-Router consistently achieves better cost-performance trade-offs than existing baselines. These results demonstrate the effectiveness of difficulty-aware retrieval for training-free LLM routing. Case studies further show that explicit query analysis helps retrieve more relevant examples and supports more reliable routing decisions.

Abstract:
The “thinking-with-images” paradigm equips multimodal LLMs with active visual operations such as crop-and-zoom. However, models using these operations often achieve only marginal or negative gains over direct inference at substantially higher token cost. They may also repeatedly crop irrelevant regions and fail on questions that direct inference answers correctly. We ask whether the returned visual evidence causally affects the answer . To answer this question, we formulate visual tool-use as a causal graph that separates observation-mediated paths from action-induced shortcuts . We then audit it through interventions at the three levels: policy (comparing tool-use with direct inference), trajectory (corrupting all observations during rollout), and step (counterfactually replacing one individual observation under a fixed prefix). Our step-level estimand, Visual Evidence Gain , isolates the contribution of each returned observation. Across six representative models and five fine-grained perception benchmarks, we uncover policy miscalibration with two failure modes. In Calling Without Looking , returned observations have no causal effect on the answer. In Looking Without Planning , observations are informative but the call schedule is incoherent. A trajectory-level diagnostic decomposes the policy-level accuracy gain into per-group contributions and shows that the gain is concentrated in a Calibrated minority. We term this discrepancy the illusion of visual tool-use : despite aggregate accuracy gains, visual tool-use is not causally effective across a broad range of rollouts. The code is available at https://github.com/OpenCausaLab/CauAudit .

Abstract:
Retrieval-augmented generation (RAG) is increasingly popular in information extraction (IE), especially in biological and chemical domains where external knowledge is abundant. Existing RAG-enhanced IE systems, however, typically retrieve for every input, even though retrieval may be unnecessary, introduce distracting evidence, and increase inference cost. To address this issue, we propose RAG Switch, a retrieval decision framework for IE that combines difficulty-aware signals with task-grounded supervision. Specifically, we approximate model-specific training difficulty, quantified by Error L2-Norm (EL2N), from inference-time uncertainty signals, and use it to guide retrieval decisions. We then define retrieval necessity directly from instance-level extraction outcomes without retrieval, yielding task-grounded supervision for learning retrieval decision. Experiments on multiple biological and chemical IE benchmarks across diverse IE backbones show that RAG Switch consistently outperforms representative retrieval decision baselines and achieves a strong efficiency–performance trade-off in end-to-end RAG-enhanced IE pipelines.

Abstract:
The Muon optimizer has recently offered a promising alternative to AdamW for large language model training, leveraging matrix orthogonalization to produce geometry-aware updates. However, like all first-order methods, Muon can become trapped in sharp local minima. In this work, we present MONA, an optimizer that bridges Muon’s orthogonalization framework with curvature-aware acceleration. MONA adds an acceleration term directly into Muon’s gradient processing pipeline. This term is calculated from the exponential moving average of gradient differences. We provide a detailed convergence analysis for MONA, showing that the acceleration term enables escape from sharp minima while preserving Muon’s spectral-norm regularization. Empirically, MONA achieves better convergence and downstream task performance compared to both Muon and AdamW across three scales of Mixture-of-Experts pretraining, spanning from 1B to 68B parameters, with the largest model trained on 1 trillion tokens. Furthermore, we conduct supervised fine-tuning on the MOE-68B-A3B model and evaluate it on general capability, mathematical reasoning, and code generation benchmarks, where MONA achieves SOTA performance.

Abstract:
Large language models have shown strong potential as role-playing agents for real individuals, yet faithful impersonating remains challenging. Existing in-context learning-based methods fail to capture how individuals react under different situations. In addition, LLM-based evaluation is difficult for obscure individuals. To address these challenges, we propose Situation–Internal state–Behavior Persona method to incorporate situation-dependent behavioral strategies. We further design an evaluation protocol that provides LLM evaluators with references about the impersonated individual. We evaluate our approach on a newly constructed dataset for the task of generating replies on social media. Experimental results show that our proposed method outperforms state-of-the-art ICL-based baselines, while our evaluation protocol achieves moderate correlation with human judgment. Besides, experiments on fictional-character benchmarks demonstrate that our proposed method is applicable beyond the social media setting. These findings suggest that incorporating behavioral information broadly improves the fidelity of role-playing for real individuals on social media or fictional characters

Authors: Tao Yu, Haopeng Jin, Hao Wang, Shenghua Chai, Yujia Yang, Zhongtian Luo, Junhao Gong, Jiaming Guo, Minghui Zhang, Xinlong Chen, Zhenghao Zhang, Yuxuan Zhou, Yanpei Gong, YuanCheng Liu, Yiming Ding, Kangwei Zeng, Pengfei Yang, Yufei Xiong, Shanbin Zhang, Chengshaoxiong, Huang Ruilin, Li Shuo, Yuxi Niu, Xinyuan Zhang, Yueya Xu, Jie Mao, Ruixuan Ji, Yaru Zhao, Mingchen Zhang, Jiabing Yang, Jiaqi Liu, YiFan Zhang, Hongzhu Yi, Xinming Wang, Cheng Zhong, Xiao Ma, Zhang Zhang, Yan Huang, Liang Wang

Abstract:
In recent years, large language models (LLMs) have made rapid progress in information retrieval, yet existing research has mainly focused on text or static multimodal settings. Open-domain video shot retrieval, which involves richer temporal structure and more complex semantics, still lacks systematic benchmarks and analysis. To fill this gap, we introduce ShotFinder, a benchmark that formalizes editing requirements as keyframe-oriented shot descriptions and introduces five types of controllable single-factor constraints: Temporal order, Color, Visual style, Audio, and Resolution. We curate 1,210 high-quality samples from YouTube across 20 thematic categories, using large models for generation with human verification. Based on the benchmark, we propose ShotFinder, a text-driven three-stage retrieval and localization pipeline: (1) query expansion via video imagination, (2) candidate video retrieval with a search engine, and (3) description-guided shot localization. Experiments on multiple closed-source and open-source models reveal a significant gap to human performance, with clear imbalance across constraints: temporal localization is relatively tractable, while color and visual style remain major challenges. These results reveal that open-domain video shot retrieval is still a critical capability that multimodal large models have yet to overcome.

Abstract:
Large language models (LLMs) are increasingly used for automated data visualization, yet existing approaches often frame visualization generation as a single-step mapping from user query to figure or code, overlooking the iterative analytical reasoning process of expert analysts. We present InsightChain, a four-step visualization prompting pipeline (Explore-Focus-Test-Present) that emulates expert analytical workflows for complex data visualization, together with VG-COPRO, a novel automatic prompt optimization (APO) algorithm for jointly optimizing pipeline performance. To further bridge the evaluation gap for complex data visualization, we introduce the Insight Progression Metric (IPM), a rubric combining four text-based dimensions with a vision-based dimension. Experiments on public datasets demonstrate that InsightChain consistently outperforms competing prompting baselines. We further show that VG-COPRO is more effective than existing APO methods at optimizing chain-based visualization prompting strategies.

Abstract:
Clinical trials constitute a critical yet exceptionally challenging and costly stage of drug development ( 2.6B per drug), where protocols are encoded as complex natural language documents, motivating the use of AI systems beyond manual analysis. Existing AI methods accurately predict trial failure, but do not provide actionable remedies. To fill this gap, this paper proposes ClinicalReTrial, a multi-agent system that formulates clinical trial optimization as an iterative redesign problem on textual protocols. Our method integrates failure diagnosis, safety-aware modifications, and candidate evaluation in a closed-loop, reward-driven optimization framework. Serving the outcome prediction model as a simulation environment, ClinicalReTrial enables low-cost evaluation and dense reward signals for continuous self-improvement. We further propose a hierarchical memory that captures iteration-level feedback within trials and distills transferable redesign patterns across trials. Empirically, ClinicalReTrial turns 56.7% of failed protocols into predicted successes under the simulation environment, with a mean success probability gain of 7.4% at negligible cost ( 0.156 per trial). Extensive retrospective case studies further show that ClinicalReTrial recovers clinically meaningful modification patterns that align with real-world expert-driven protocol changes.

Abstract:
Code LLMs exhibit remarkable capabilities in automated code generation, but the generated code often suffers from low computational efficiency. Existing solutions have two limitations: (1) direct fine-tuning lacks transferability across model scales; (2) The efficiency optimization method for iterative generation-evaluation loops incurs high overhead. We propose CoEffi, which adopts inference-side intervention that directly operates on the decoding process without modifying base model parameters, enabling real-time efficiency improvement across different model scales while avoiding iterative reevaluation. CoEffi uses a small efficiency expert model for proposal and the base model to evaluate its acceptability, additionally introducing a region-aware dynamic acceptance threshold mechanism that maintains a sliding window to record historical acceptance rates and adjusts the dynamic threshold accordingly. We extensively evaluate CoEffi on four mainstream code LLM families and three efficiency benchmarks. Compared with vanilla instruction-tuned models, CoEffi improves average Beyond@1 by 9.12%–38.62% across the evaluated model families. It also improves Pass@1 on Mercury and ENAMEL, while EffiBench exhibits model-dependent correctness gains and regressions. A family specific trained expert can be reused across compatible base model scales within the same model family.

Abstract:
In the context of information seeking, conversational agents are undergoing an evolution from reactive tools to proactive, personalized assistants. A critical aspect of this evolution is the ability to tailor strategic interactions to a user’s unique needs and expectations. Unlike existing studies that focus on proactively clarifying query ambiguities, we center on clarifying the user’s expertise in order to tailor responses for better user comprehension. We find that existing agents struggle to determine user expertise from queries alone, a limitation that prevents them from dynamically adapting their responses. To address this gap, we introduce PASSING to empower the agent to proactively clarify a user’s expertise through targeted inquiries. This is achieved by our What-to-ask and How-to-ask strategies, induced by LLM self-play. Our extensive experiments also show our superiority. We believe that PASSING represents a crucial step towards creating more human-centric conversational agents.

Abstract:
Reinforcement learning has recently shown promise in improving large language models for Text-to-SQL generation, yet existing methods typically optimize one-shot rewards defined over a single SQL state. Such rewards provide limited guidance for iterative SQL correction and are insufficient to capture the improvement of multi-turn SQL refinement. In this paper, we propose Progress-SQL, a multi-turn reinforcement learning framework with progressive rewards for Text-to-SQL. Our approach introduces an Oracle-guided Diagnostic Tree (ODT), which abstracts SQL queries into clause-level structural profiles and produces diagnostic feedback for next-turn refinement. To provide dense and robust reward signals, we combine ODT-based structural alignment with lexical alignment and define a progressive reward that measures the improvement from the initial SQL to the final SQL. We further incorporate a progression latency reward that favors earlier correctness and an execution status reward that encourages recovery from the invalid SQL. Experiments on BIRD, Spider, and Spider robustness variants demonstrate that our method consistently improves Text-to-SQL performance across both primary and robustness evaluations.

Abstract:
Reinforcement Learning (RL) has significantly enhanced reasoning in Large Language Models. However, applying state-of-the-art RL methods like Group Relative Policy Optimization (GRPO) to video-based Multimodal LLMs (MLLMs) remains sub-optimal. We identify two key exploration bottlenecks: (1) Advantage Vanishing: generating multiple responses from invariant video contexts leads to near-uniform rewards, neutralizing learning signals after in-group standardization; (2) Visual Overfitting: standard RL lacks mechanisms to regularize sensitivity to visual perturbations under single-epoch training, causing policies to overfit to sparse temporal cues. To address these, we propose Dual-Group Exploration, a framework revitalizing learning signals at both input and output levels. At the input level, a Dual-Group Sampling strategy breaks visual determinism via complementary temporal sequences. At the output level, an Entropy-rated ELO Advantage captures inter-group quality variances. Concurrently, a Cross-Visual Counterfactual Weight establishes a Visual Consistency Region, safely driving exploration by enforcing reasoning consistency across varied views. Achieving state-of-the-art performance across six video reasoning benchmarks, our method demonstrates that consistency-guided exploration is beneficial for video understanding.

Abstract:
External hidden-state memories can extend Transformers beyond local attention windows, but the write policy deciding which states enter finite memory is often treated as an implementation detail. We study this factor with a controlled delayed-copy diagnostic using a windowed Transformer augmented with an external FIFO hidden-state memory. Under forced masking, query-to-source attention is blocked, removing this shortcut and making source retention directly testable. We define a capacity boundary as the minimum memory size needed to reach a fixed copy-accuracy threshold. We find that write policy changes this boundary predictably under constrained FIFO memory: an idealized source-only policy, which assumes that relevant states are identifiable at write time, yields an approximately delay-invariant boundary near the source length, whereas an unfiltered prefix-all policy induces a delay-dependent boundary that grows with written non-source content. Intermediate partial-prefix write policies produce intermediate boundaries, while source-pinned controls and memory diagnostics show that prefix-all degradation is dominated by FIFO eviction of source states rather than retrieval interference from retained non-source states alone. A symbolic key–value retrieval diagnostic shows the same qualitative trend. Within this controlled setting, our results highlight write policy as a central variable in evaluating finite FIFO external-memory Transformers.

Abstract:
Generative adversarial distillation (GAD) provides on-policy feedback for black-box distillation by training a discriminator that ranks teacher and student responses to the same prompt. While effective, GAD is also costly: every step pays for a discriminator update, an actor update over the full response, and the indirect cost of recovering whenever the discriminator and the actor progress at incompatible rates. We argue that the right way to make GAD efficient is not to make individual updates cheaper but to decide when each update is worth its cost. We introduce curriculum-efficient GAD (CE-GAD), an adaptive recipe that monitors a small set of training signals and uses them to: - (i) skip discriminator updates that would not change the reward, - (ii) constrain actor updates and prune per-token gradients to the subset that actually carries signal, and - (iii) rescale rewards and normalize advantages to preserve within-batch contrast under saturation. Evaluations across four models from two open-source families (Qwen2.5 at 3B/7B/14B and Llama-3.2-3B) demonstrate that CE-GAD matches or surpasses vanilla GAD on quality while winning on 12 out of 16 individual benchmark configurations. Specifically, on Qwen2.5-7B, CE-GAD improves the average score by up to +3% relative compared to the base model, while reducing per-step wall-clock time by up to 8x relative to vanilla GAD.

Abstract:
Multi-Turn Long-Form Question Answering (MT-LFQA) is a key application paradigm of Large Language Models (LLMs) in knowledge-intensive domains. However, existing benchmarks are limited to single-turn dialogue, while multi-turn dialogue benchmarks typically assess other orthogonal capabilities rather than knowledge-intensive factuality. To bridge this critical gap, we introduce KnowMT-Bench, a new benchmark designed to systematically evaluate MT-LFQA for LLMs across knowledge-intensive fields, including medicine, finance, and law. To faithfully assess the model’s real-world performance, KnowMT-Bench employs a dynamic evaluation setting where models generate their own multi-turn dialogue histories given logically progressive question sequences. Our experiments on a diverse suite of LLMs show a clear degradation in both factual capability and information delivery efficiency in multi-turn dialogue. We further probe the underlying causes and investigate mitigation strategies spanning domain-specific finetuning, active context denoising, and RAG. The results show domain-dependent gains from finetuning, while active context denoising and RAG more consistently mitigate these issues, with RAG notably reversing this performance degradation. These findings underscore the importance of our benchmark for evaluating LLMs’ conversational factual capabilities in real-world applications.

Abstract:
Audio tokenizers serve as the discrete interface between continuous audio and Audio Language Models (ALMs), but existing tokenizers often struggle to support both understanding and generation. Reconstruction-oriented codecs preserve acoustic fidelity but lack rich semantics, while semantic-aware tokenizers typically rely on separate semantic and acoustic streams, introducing redundancy or misalignment. We propose EntangleCodec, a unified discrete audio tokenizer that learns caption-aligned semantic-acoustic representations before quantization. By aligning audio with rich captions rather than ASR transcripts, EntangleCodec captures linguistic content, speaker identity, emotion, prosody, and acoustic scenes within a compact token stream. A flow-matching diffusion decoder further enables high-quality reconstruction across speech, music, and general audio. EntangleCodec achieves reconstruction quality competitive with specialized codecs, outperforms all codec-based baselines on audio understanding by up to +7.4% on MMAR, and supports both TTS and TTA generation in a unified framework. Furthermore, EntangleCodec-based audio language models demonstrate strong scaling behavior: even at 0.6B parameters, the model surpasses specialized continuous-representation LLMs with over 13B parameters across three benchmarks using 22x fewer parameters; scaling to 8B further establishes new state-of-the-art results on MMAR, highlighting that representation quality is as critical as model scale in audio language modeling. Code and model weights are available at https://anonymous.4open.science/r/EntangleCodec-BA01.

Abstract:
Whether large language models (LLMs) can control their own internal representations matters for both machine metacognition and AI safety. A recent study applied neurofeedback to LLMs and claimed that they can control their internal representations. However, the reported control may rely on superficial mechanisms rather than genuine internal access because the control targets in that study are not privileged, meaning that a third party can infer them from the prompt. We redesign the neurofeedback paradigm for LLMs so that the control target satisfies the privileged access requirement, which is closer to neurofeedback experiments in human cognitive neuroscience. Under this stricter setting, the models do not demonstrate reliable control over privileged internal representations, suggesting that previously reported control cannot exclude the possibility that it relies on superficial mechanisms. Our results indicate that rigorous assessments of metacognition in LLMs require evaluation methods that demand privileged access.

Abstract:
Video Large Language Models (VideoLLMs) excel in video understanding but suffer from significant computational overheads due to extensive visual tokens. Existing mainstream solutions including visual token compression and query-based Retrieval-Augmented Generation (Video RAG), face notable limitations: the former typically pursues task-agnostic compression of global video semantics, lacking query awareness, while the latter fails to optimize visual token construction and relies on costly, rigid annotation data. To address these challenges, we propose MERTA (Multi-Expert Retrieval for Token Allocation), a plug-and-play framework that integrates selective multi-expert collaborative retrieval with query-aware token allocation. MERTA selectively leverages lightweight experts to capture multimodal semantics, adaptively identifying key frames by measuring the relevance among queries, visual content, and auxiliary annotations. It then dynamically allocates the visual token budget, assigning higher budgets to key frames while keeping the total visual token budget constant. MERTA integrates seamlessly without requiring architectural modifications, demonstrating superior efficiency and performance over existing methods across multiple video understanding benchmarks.

Abstract:
Multi-trait Automated Essay Scoring (AES) requires rubric-grounded reasoning across interdependent traits, rather than isolated score prediction. Existing feedback-enhanced methods often decouple feedback from scoring or assess traits independently, weakening score–feedback consistency and rubric alignment. We propose HiFTS, a unified autoregressive framework that generates hierarchical CoT feedback before predicting trait-level and holistic scores. HiFTS distills rubric-grounded hierarchical CoT feedback from a teacher LLM and trains student models to jointly generate feedback and scores. HiFTS further applies Group Relative Policy Optimization with a composite reward balancing score agreement, calibration, feedback quality, and structural validity. At inference, a lightweight global prior provides holistic guidance to reduce drift during long-form reasoning. We also introduce CFMS-34, a Chinese multi-trait AES dataset with 951 essays annotated with holistic scores and 34 rubric-based traits. Experiments on CFMS-34 and ASAP++ show that HiFTS achieves strong holistic and trait-level scoring while producing coherent, rubric-aligned feedback.

Abstract:
Existing long-form factuality evaluation relies on the decompose-retrieve-verify pipeline. However, the pipeline suffers from noise from claim decomposition and fixed verification granularity, resulting in unreliable results. We propose ElementCheck, a complexity-aware framework that verifies long-form outputs via sentence elements. Instead of uniformly decomposing sentences into atomic sub-claims, ElementCheck extracts entity pairs that are explicitly linked through verifiable connections in the original sentence as elements, and organizes these into an element graph. The graph topology provides a structural signal for estimating sentence complexity, enabling direct verification for simple sentences and targeted element-level refinement and verification for complex ones. To support fine-grained evaluation, we construct a new benchmark FastFact-Sent by mapping isolated claims from FastFact-Bench back to their source sentences. Experiments on FastFact-Sent and two domain-specific benchmarks show ElementCheck consistently improves factuality verification across five backbone models while maintaining a favorable accuracy-cost trade-off. Further analyses demonstrate that complexity-aware verification reduces unnecessary re-verification and maintains stability across different backbones. The code is available at Here .

Abstract:
Wireless signals are a unique form of time-series data that encode rich physical characteristics of the propagation environment. As large volumes of signals are continuously generated in wireless sensing systems, efficient compression is essential for scalable transmission under limited system resources. However, wireless signals exhibit strong variability across environments and time, making it difficult to design universal compression schemes that generalize across domains while preserving the physical fidelity required for accurate reconstruction and sensing. Meanwhile, recent advances in large language models (LLMs) have demonstrated remarkable reasoning and adaptation capabilities across diverse modalities and domains. This raises an intriguing question: can LLMs compress wireless signals in a training-free manner while preserving physical fidelity? To the best of our knowledge, this problem remains unexplored. To answer this question, we propose a physics-informed prompting framework for training-free wireless signal compression. The framework incorporates environmental context and scenario-specific knowledge as physical priors in the prompt. We further introduce a dictionary module with self-learned patterns to stabilize inference and a key prompt module to align the semantics between the LLMs used for compression and reconstruction. Extensive experiments demonstrate that the proposed framework achieves high-fidelity reconstruction under cross-domain settings, highlighting the promise of physics-informed prompting as a new paradigm for processing and compressing low-level physical signals.

Abstract:
Constructing efficient and reliable policies to assist humans is indispensable for human-AI collaboration. Existing methods mainly follow two lines of work. The majority of prior work relies on multi-agent reinforcement learning (MARL) to learn black-box policies, which limits interpretability and raises safety concerns. Recent methods query large language models (LLMs) at each decision step, causing slow responses and high inference costs. We propose Co- 𝜋 -tree, a closed-loop method that learns an executable policy tree consisting of a partner-behavior prediction tree and an agent-action selection tree. Co- 𝜋 -tree constructs a policy through distilling LLM reasoning into policy tree code. It then evaluates the policy through partner interaction, obtains feedback, and uses natural language summaries of interaction feedback to improve problematic branches. Experiments in Overcooked-AI show that Co-π-tree improves average reward by 6.4% over the strongest LLM baseline, while reducing the number of LLM queries by 77.7% and test-time latency by 97.1%. Project page: https://beiwenzhang.github.io/Co-pi-tree/.

Abstract:
Document understanding aims to perform question answering and information extraction over document images, where the visual content is highly information-dense and most queries rely on only a few relevant layout regions. However, existing methods either adopt a one-pass strategy that implicitly assumes all layouts are equally important, or focus excessively on small regions at the cost of losing critical layout information. To address these limitations, we introduce Doc-CoB (Chain-of-Boxes), a simple-yet-effective framework that integrates coarse-to-fine layout-aware visual reasoning into multimodal large language models. Instead of directly zooming into small regions, Doc-CoB progressively focuses on query-relevant layouts while preserving global document information. Specifically, it first selects key layout boxes and then focuses on them for further understanding with visual prompting. To support this paradigm, we introduce two reasoning tasks for box recognition and box reasoning, with an automatic pipeline that constructs 249k training samples with intermediate visual supervision. Experiments on seven benchmarks with four popular models show that Doc-CoB significantly improves performance, demonstrating its effectiveness and wide applicability. The code and the data are available at https://github.com/Doc-CoB/Doc-CoB.

Abstract:
Arabizi refers to Arabic written in Latin script. Although previous studies have shown that the prevalence and usage of Arabizi vary by factors such as region and age group, most NLP research on Arabic texts treats it as a temporary phenomenon resulting from limited technological support for the Arabic script. In this work, we engage with Arabic speakers to collect insights on their perceptions and usage of Arabizi. We further examine writing norms among speakers of different dialects, focusing on Algerian, Egyptian, Lebanese, Moroccan, and Tunisian Arabic. To this end, we release two resources. First, we perform a character-level alignment of Arabic words to study inter- and intra-dialectal variation across these five dialects, based on words transliterated by the surveyed participants (VARabiziLex), finding systematic intradialectal regularity and inter-dialectal variation. Second, to study Arabic speakers’ ability to identify this stylistic variation at the sentence-level, we build VARabiziCues, a manually curated parallel corpus of sentences written in Arabic script alongside multiple Arabizi transliterations, collected from speakers of the same five dialects. Our study presents the largest human-centered, cross-dialectal study of Arabizi’s perceptions and practices to date.

Abstract:
Large language models are increasingly deployed for personalized interaction, and demographic conditioning via user profiles is a widely adopted strategy for cultural adaptation. We ask whether this approach genuinely serves individual users or achieves accuracy by erasing individual distinctiveness. Studying seven models including frontier GPT-5.1 on the World Values Survey, we find that demographic profiles improve value alignment accuracy for most models, but at a systematic cost to individuality. That is, models pull responses toward demographic group centroids rather than preserving individual differences, a behavioral pattern we term alignment by stereotyping. Permutation tests (10,000 permutations, six demographic attributes, seven models) certify that top-performing models compress individuals far above the human baseline; within-family scaling amplifies this tradeoff while degrading intrinsic cultural understanding. Using a synthetic dialogue dataset validated on real human-chatbot conversations from PRISM (Kirk et al., 2024), we further show that distributing demographic signals across conversational turns partially suppresses prototype retrieval compared to compact demographic labels, a finding validated on real conversations via PRISM but requiring replication at larger scale.

Abstract:
We introduce EMemBench, a programmatic benchmark generator for evaluating long-term episodic memory of agents through interactive games. Rather than using a fixed set of questions, EMemBench generates questions from environment-grounded trajectories, covering both text-only and visual game environments. Each template computes verifiable ground truth from underlying game signals, with controlled answerability and balanced coverage over memory skills: single/multi-hop recall, induction, temporal, spatial, logical, and adversarial. We evaluate memory agents with strong LMs/VLMs as backbones, using in-context prompting as baselines. Across 15 text games and multiple visual seeds, results are far from saturated: induction and spatial reasoning are persistent bottlenecks, especially in visual settings. Persistent memory yields clear gains for open backbones on text games, but improvements are less consistent for VLM agents, suggesting that visually grounded episodic memory remains an open challenge. A human study further contextualizes the difficulty and interpretability of EMemBench.

Abstract:
Multi-chapter novel generation requires models to balance two quality dimensions simultaneously: chapter-level reader engagement and global narrative rhythm. Existing approaches have mainly focused on planning, memory, and execution, while paying limited attention to the mismatch between training signals and reader experience. We propose ChapterRL, which reformulates multi-chapter generation as a reward design problem. Specifically, ChapterRL introduces two complementary reward signals. Reading Depth Residual (RDR) derives debiased chapter-level quality signals from reader continuation and completion behavior through genre–position stratified normalization. Dynamic Rhythm Trajectory (DRT) uses an arousal estimator to impose a threshold-gated penalty on systematic rhythm collapse in the later part of a story. The two signals correspond to chapter attractiveness and global narrative momentum, and are jointly integrated into reinforcement learning training. We evaluate the full ChapterRL policy on three automatic benchmarks and separately validate its RDR reward model on task-aligned narrative domains of the Chinese WritingPreferenceBench. Results show that ChapterRL improves chapter quality and rhythm stability over supervised fine-tuning and single-reward variants, while RDR exhibits strong narrative preference alignment as an efficient scalar reward model.Code is available at [ChapterRL](https://github.com/SorBetQ/ChapterRL).

Abstract:
Large language models (LLMs) are increasingly deployed over knowledge bases for efficient knowledge retrieval and question answering. However, LLMs can inadvertently answer beyond a user’s permission scope, leaking sensitive content and complicating deployment under fine-grained access-control requirements. We identify a geometric regularity in intermediate activations: representations associated with different permission scopes exhibit consistent and distinguishable structure in activation space. Building on this observation, we propose Activation-Space Anchored Access Control (AAAC), a training-free framework for multi-class permission control. AAAC constructs a compact bank of permission-specific activation anchors from a small offline calibration set and requires no model fine-tuning. At inference time, the trusted permission metadata is used by the controller to steer query activations toward authorized anchors and away from unauthorized ones, thereby mitigating the risk of over-privileged generation. Across three LLM families, AAAC achieves permission violation rates of 0.005–0.009 and prompt-based attack success rates of 0.084–0.101, while maintaining competitive utility with 10.3%–25.2% inference overhead.

Abstract:
Large Language Models (LLMs) exhibit strong language understanding and instruction-following abilities, making them promising backbones for grammatical error correction (GEC). However, prompt-only correction for low-resource languages still suffers from the over-correction challenge, since LLMs cannot fully understand their internal grammar. To address this issue, we propose an automatic linguistic neuron probing and optimizing approach to improve the low-resource GEC with only a few neuron activations. Concretely, we first utilize multiple low-resource GEC corpora to observe the changes in neurons, thus obtaining a linguistically specific probe matrix to ensure whether the neuron is activated. Then, we construct an extra weight matrix to determine the activation intensity of each neuron. Finally, we combine the probe matrix and the weight matrix to obtain an activation matrix, which is injected into selected model layers during inference to guide the frozen LLM toward more controllable GEC. Experiments on four benchmark datasets show that our approach consistently improves GEC performance, indicating that lightweight sparse activation rescaling is extremely useful for low-resource grammatical knowledge understanding. Detailed analysis demonstrates that our approach can further provide interpretable neuron-level control without large-scale parameter fine-tuning.

Abstract:
Large Reasoning Models (LRMs) have demonstrated remarkable performance on complex reasoning benchmarks. However, their long chain-of-thought reasoning processes incur significant inference overhead. Pruning has emerged as a promising approach to reducing computational costs. However, existing efforts have primarily focused on large language models (LLMs), while pruning LRMs remains unexplored. In this work, we conduct the first empirical study on pruning LRMs and show that directly applying existing pruning techniques fails to yield satisfactory results. Our findings indicate that using self-generated reasoning data for calibration can substantially improve pruning performance. We further investigate how the difficulty and length of reasoning data affect pruning outcomes. Our analysis reveals that challenging and moderately long self-generated reasoning data serve as ideal calibration data. Based on these insights, we propose a Selective Self-Generated Reasoning (SSGR) data construction strategy to provide effective calibration data for pruning LRMs. Experimental results on different model series validate that our strategy improves the reasoning ability of pruned LRMs by 10–13% compared to general pruning methods.

Abstract:
In this paper, we introduce Predicate-State Video Reasoning Segmentation (PS-VRS), where a query defines a dynamic predicate P(o,t) over objects and time rather than a fixed object identity. Existing methods typically treat the target as permanently active once identified, leading to over-emission of masks during vacant intervals and failure to handle target switching. To address this, we construct PS-Bench, a large-scale benchmark with 2,077 videos and 16,250 queries annotated with explicit active/vacant predicate states, temporal intervals, and pixel masks. We further propose Predicate-State Grounding (PSG), a State-Bind-Emit pipeline that: (1) predicts [ACTIVE]/[VACANT] state tokens; (2) aligns them to video frames via dense cross-modal similarity; and (3) emits masks via SAM2 only for active states. Experiments show that PSG outperforms existing baselines by +9.32 𝒥 ℱ on PS-Bench and significantly reduces false-positive mask emissions. Our results demonstrate the effectiveness of explicit predicate-state modeling for video reasoning segmentation with dynamic visual witnesses.

Abstract:
Red-teaming has become a standard tool for probing alignment and safety weaknesses in large language models, but most existing methods treat each attack session in isolation and discard what was learned once the dialogue ends. We introduce LongMem-RAT, a multi-agent red-teaming framework equipped with long-term memory that accumulates, consolidates, and reuses insights from past attacks across sessions. This design enables cross-session generalization, reduces redundant exploration, and focuses new attacks on fragile alignment behaviors. Across multiple proprietary and open-weight models, LongMem-RAT improves Attack Success Rate (ASR) over a fully reproduced X-Teaming baseline by 21.73 percentage points on HarmBench and 18.40 points on AdvBench (20.06 points on average). Memory ablations show that long-term memory improves ASR by 13.23 and 6.07 points and reduces Average Turns to Success (ATtS) by 0.37 and 1.92 turns on HarmBench and AdvBench, respectively, averaging a 9.65-point ASR gain and a 1.14-turn ATtS reduction.

Abstract:
Memory-augmented long-context reasoning boosts LLMs’ complex task solving via external knowledge, yet current methods suffer from disjointed memory retrieval-induced context drift and aimless iterations causing premature convergence or redundant computation. We present MemPulse, integrating fact-based memory hypergraphs and pulse-driven adaptive iteration. It builds hypergraphs with facts as hyperedges to gather scattered information and sustain coherent context evolution. Its memory pulse engine computes agent confidence and structural novelty to output dynamic control signals, adjusting retrieval range and stopping criteria for logical reasoning with minimal overhead. Evaluated on Locomo, MemPulse surpasses mainstream baselines in reasoning accuracy and efficiency.

Abstract:
Social interaction depends on both language and visible social signals, such as facial expressions, posture, gaze, and emotional shifts. Yet existing social-agent benchmarks are largely text-based and rarely test whether multimodal agents can use visual cues to guide interaction. We introduce AgentViSS, a benchmark evaluating visual social intelligence in multimodal social simulation. It contains 240 scenarios, 585 role instances, and 2,340 role-task instances, combining aligned textual-visual evidence, structured role profiles, and four role-level tasks: expression task, characteristic task, interaction regulation task, and interaction outcome task. Evaluating seven recent MLLMs under verbalized-vision and direct-vision reveals a clear gap between local role enactment and interaction management: role-specific expression and conflict handling are near saturation, whereas interaction regulation and visually grounded outcome achievement remain substantially more difficult.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as an effective approach for enhancing the reasoning capabilities of Large Language Models (LLMs). Despite its efficacy, RLVR faces a meta-learning bottleneck: it lacks mechanisms for error attribution and experience internalization intrinsic to the human learning cycle beyond practice and verification, thereby limiting fine-grained credit assignment and reusable knowledge formation. We term such reusable knowledge representations derived from past errors as meta-experience. Based on this insight, we propose Meta-Experience Learning (MEL), a novel framework that incorporates self-distilled meta-experience into the model’s parametric memory. Building upon standard RLVR, we introduce an additional design that leverages the LLM’s self-verification capability to conduct contrastive analysis on paired correct and incorrect trajectories, identify the precise bifurcation points where reasoning errors arise, and summarize them into generalizable meta-experience. The meta-experience is further internalized into the LLM’s parametric memory by minimizing the negative log-likelihood, which induces a language-modeled reward signal that bridges correct and incorrect reasoning trajectories and facilitates effective knowledge reuse. Experimental results demonstrate that MEL achieves consistent improvements on benchmarks, yielding 3.92%–4.73% Pass@1 gains across varying model sizes.

Abstract:
Dynamic sampling is a technique employed in reinforcement Learning for Large Language Models to mitigate training instability, often caused by the gradient-decreasing problem. However, existing dynamic sampling approaches are predominantly designed for tasks with binary rewards (success/failure) and struggle to adapt to the complex reward structures of tool learning. Such task involves interdependent sub-tasks of varying difficulty and yields fine-grained, multi-faceted rewards. To bridge this gap, we introduce Dynamic Sampling with Curriculum Learning (DSCL), an algorithm tailored for the intricate dynamics of tool learning. DSCL integrates two core components: a Reward-Based Dynamic Sampler, which leverages multi-dimensional reward statistics to prioritize high-value training data; and a Task-Based Dynamic Curriculum Learning method to overcome the credit assignment problem caused by the fine-grained rewards. Extensive experiments on widely used tool learning benchmarks demonstrate the efficacy of our approach. DSCL significantly improves model performance, outperforming strong baselines by 4.75% on the BFCL V3 dataset and 4.02% on the API-Bank dataset.

Abstract:
Vision-Language Models (VLMs) provide useful priors for interactive decision-making, but using them directly as policies is expensive and brittle: they must be queried at every step, do not improve from environment interaction, and can repeat systematic errors. We study how to learn a cheap autonomous policy from an online, expensive, and imperfect VLM teacher. We propose SAGE (Selective Agent Guidance via Entropy), a framework that queries a VLM only when the learner is uncertain, executes the suggested action during training, and distills guidance into a lightweight Reinforcement Learning (RL) policy. Because VLM advice is not always reliable, SAGE can weight teacher-action distillation using environment-derived advantages rather than treating all suggestions as equally useful. Across sparse-reward visual reasoning and navigation tasks, SAGE learns policies that act without VLM guidance at evaluation time and improves over unguided RL in several environments, including settings where the learned policy exceeds its VLM teacher. The results show that selective guidance is most beneficial when the VLM can help the agent discover high-reward trajectories, and less useful when unguided exploration already succeeds or teacher actions do not lead to informative experience. SAGE also reduces VLM usage by prompting the teacher only on a fraction of training steps and requiring no VLM calls at deployment. Overall, our results suggest that VLMs don’t need to be used as fixed policies to be useful; they can instead act as temporary, imperfect sources of guidance whose value is tested and internalized through interaction.

Abstract:
Recent ARC-AGI systems have shown that pure vision can solve abstract grid reasoning without language backbones, but most strong models still rely on fixed-depth feed-forward computation. We study whether recurrent computation is a better scaling axis for this setting. We propose Loop-ViT, a vision-native recurrent model that reuses a weight-tied Hybrid Block across multiple steps, with local convolutions for image-token refinement and attention for global rule induction. We also use an entropy-based exit rule to reduce unnecessary iterations at inference time. On ARC-AGI-1, Loop-ViT improves the parameter-efficiency trade-off over fixed-depth vision baselines and reaches 65.8% accuracy with an 18M model. Overall, the results suggest that repeated computation is a useful inductive bias for ARC-style visual reasoning. The code is available at https://github.com/WenjieShu/LoopViT .

Abstract:
Spatial perspective transformation is a fundamental component of human spatial cognition, yet it remains unclear whether large language models (LLMs) can exhibit robust spatial reasoning abilities that support such transformations. We introduce a controlled dataset inspired by cognitive science studies, combining route-view and survey-view descriptions across multiple environmental topologies. Experiments on both open-source and closed-source LLMs show that models readily learn within-perspective reasoning but continue to struggle with cross-perspective transformation, especially from route-view to survey-view reasoning. The learnt capabilities remain strongly topology-dependent and generalise asymmetrically between within-perspective and cross-perspective settings. Scaling substantially improves allocentric and within-perspective reasoning, but does not uniformly resolve cross-perspective transformation difficulty, while chain-of-thought prompting provides little benefit. Overall, our results suggest that current LLMs do not consistently demonstrate perspective-invariant spatial reasoning across perspectives and environments. Code and datasets are available at https://github.com/zhanghaotong1/EMNLP-2026-perspective-transformation-in-LLMs.

Abstract:
High-quality multi-turn tool-use data is essential for training agentic models, yet existing data synthesis methods often underrepresent the argument-level dependencies that are critical to long-horizon tool use. As a result, even when a model selects the correct tool, task execution may still fail because the model fills tool arguments with fabricated, stale, or weakly grounded values. To address this problem, we propose State-Guided Data Synthesis with Argument Provenance (SAP). SAP combines state guidance, tool-argument provenance constraints, and turn-level validation to efficiently construct tool-use trajectories with long-range dependencies and high accuracy. Using data generated by SAP, we build SAP-4B, which is highly competitive even when compared with much larger models across multiple benchmarks. Source code, synthesized data, and trained weights are available at https://github.com/Zichen1024/SAP.

Abstract:
Multilingual Large Language Models (LLMs) traditionally rely on a single vocabulary shared by all supported languages, which can lead to uneven compression across them. Moreover, their large embedding and output matrices increase memory usage and slow inference, notably for small-scale models. It is also wasteful as models are often used for only a subset of languages. To address these issues, we introduce a modular framework for multilingual model training. First, we propose methods to learn large modular BPE and Unigram tokenizers that enable extraction of subtokenizers tailored to any language subset. These subtokenizers achieve compression on par with monolingual tokenizers and improve cross-lingual fairness. Second, we design a pretraining strategy that samples subtokenizers to form batches, restricting predictions to the relevant vocabulary subset and allowing efficient training despite a large vocabulary. This supports efficient inference with any combination of language-specific vocabularies. Therefore, it reduces memory usage and speeds up inference in models without sacrificing performance.

Abstract:
Large Language Models (LLMs) as agents often fail to improve in new environments. We identify and characterize a failure mode we call exploration collapse: under reinforcement learning (RL) in environments whose states are unfamiliar to the policy, Pass@k, the probability that at least one of k sampled trajectories succeeds, drops markedly over training even as Pass@1 edges up, revealing increasingly brittle exploration; environments closer to the pretraining distribution show no such decline. We trace this collapse to weak grounding in environment states and dynamics, and study a simple remedy: explicitly teaching the agent to estimate the current state and predict its transitions before optimizing for reward. We instantiate it as SPA, an explore-then-exploit recipe that cold-starts the policy with a Self-Experience supervised finetuning (SFT) stage, collecting the model’s own interaction trajectories and supervising state and next-state prediction, and then runs standard RL. The resulting world model serves as a grounded initialization for RL rather than an inference-time planner. Across unseen environments, SPA consistently and substantially improves over vanilla RL: for example, it raises the Sokoban success rate from 25.6% to 59.8% on Qwen2.5-1.5B-Instruct, letting sub-3B models surpass a 20B baseline on these tasks. Controlled studies indicate that the gains track four factors: grounded state representations, explicit transition modeling, self-experience trajectories from a sufficiently strong exploration policy, and adequate coverage of transition data.

Abstract:
Large language model (LLM) agents create new opportunities for end-to-end data science automation, but existing systems often overfit to dataset-specific schemas, preprocessing routines, and code patterns, making them brittle under task shifts and costly to reuse. We propose DSFlow, a workflow-optimization framework that represents data-science solutions as compositions of reusable operators rather than monolithic, dataset-coupled programs. DSFlow evolves operator-based workflows to discover transferable structures, expands its operator library through synthesis and composition, and uses a two-stage evaluation strategy that filters candidates with inexpensive plan-level scoring before selectively running full executions. This design keeps optimization practical while improving transfer to unseen tasks. Experiments on heterogeneous benchmarks spanning tabular, image, text, and time-series modalities show that DSFlow achieves strong zero-shot generalization, reaching an average rank of 2.62. In addition, DSFlow grows its operator library from 10 manually designed operators to more than 70 discovered ones, improving accuracy from 0.8070 to 0.8448 and demonstrating scalable knowledge accumulation.

Abstract:
Large Language Models (LLMs) are powerful but limited by static parametric knowledge that becomes outdated once pretraining ends. Knowledge editing addresses this problem by updating model behavior on target facts without full retraining. In particular, in-context knowledge editing has gained attention because it is training-free and readily applicable to black-box LLMs. Recent reinforcement learning (RL)-based approaches improve over fixed retrieval strategies by adapting prompt construction to the quantity-quality trade-off. Despite initial success, they fail to model the prompt as a structured entity under the distinct and often competing objectives of reliability, generality, and specificity. Previous methods largely optimize a single objective and make decisions over only part of the prompt construction process, thereby overlooking both the balance of different objectives and the global organization of demonstrations. We propose Multi-Objective In-context Knowledge Editing (MO-IKE), a multi-objective RL algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process. MO-IKE trains a dynamic retriever to optimize competing objectives in knowledge editing, enabling more balanced and globally coherent prompt construction. On Llama-3.2, MO-IKE improves edit success (reliability) from 85.0% to 92.0%, paraphrase consistency (generality) from 77% to 79%, while increasing retention rate (specificity) by 23.0% compared to prior RL-based methods.

Abstract:
Vision-Language models (VLMs) have demonstrated impressive capabilities across diverse domains, yet they fail significantly in allocentric spatial reasoning, which requires perspective-taking from object-centric viewpoints rather than the camera’s perspective. This capability is essential for real-world applications including robotics, autonomous driving, and embodied AI. In this work, we identify this limitation as a prevalent camera viewpoint bias and propose Perspective-Conditional Prompt-based Contrastive Decoding (PCD), a training-free method that leverages VLMs’ capabilities while exploiting this performance gap. By contrasting responses to allocentric versus egocentric framings of identical spatial configurations, PCD explicitly suppresses camera viewpoint bias without requiring fine-tuning or external foundation models. Evaluating on allocentric benchmarks including 3DSRBench, OmniSpatial, ViewSpatial-Bench, and SpatialMQA, we demonstrate that PCD improves accuracy by up to 32.3 points on Qwen2.5-VL-7B, with consistent gains observed on recent, larger-scale VLMs. Our approach offers substantial computational advantages, achieving 8.65× faster inference and a 37.4% memory reduction compared to multi-model pipeline framework.

Abstract:
Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason over constraints, and decide when a task is infeasible. Yet existing agent evaluations often report only end-to-end success, making it difficult to determine whether failures stem from planning or execution. We introduce Agent Planning Benchmark (APB) , a planning-specific diagnostic benchmark with 4,209 multimodal cases across 22 domains and five settings, covering holistic planning, feedback-conditioned step-wise planning, and robustness under extraneous tools, broken tools, and unsolvable tasks. Across 12 MLLMs, APB reveals systematic weaknesses in long-horizon planning, tool-noise robustness, calibrated refusal, and inference-time refinement. We further validate APB on 200 ToolSandbox tasks and 200 𝜏 2 -bench tasks, where APB-guided refinement consistently improves plan correctness, plan grade, and downstream execution metrics across three representative models. APB thus serves as an upstream diagnostic complement to execution benchmarks. The APB benchmark and code are available in this URL .

Abstract:
Evaluating multimodal large language models (MLLMs) on long-form scientific papers requires measuring not only answer correctness, but also whether answers are supported by traceable cross-modal evidence. Existing answer-only metrics and synthetic “Needle-In-A-Haystack” tests mainly assess answer matching or isolated retrieval, leaving evidence-grounded support under-examined. We introduce “Fish-in-the-Ocean” (FITO), an evidence-chain evaluation lens for native scientific documents, and instantiate it with SIN-Data, a scientific interleaved corpus that preserves text–figure coupling. Built on SIN-Data, SIN-Bench comprises four evidence-auditing tasks: evidence discovery, hypothesis verification, grounded QA, and evidence-anchored synthesis. We further propose “No Evidence, No Score”, which credits predictions only when linked to verifiable anchors and diagnoses evidence quality through matching, relevance, and logic. Experiments on 10 MLLMs show a clear correctness–grounding gap: Gemini-3-pro obtains the best average overall score (0.566), while GPT-5 reaches the highest SIN-QA answer accuracy (0.767) but lower evidence-aligned scores. Code and data are available at https://github.com/IIGROUP/sin-bench .

Abstract:
Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context. Despite recent progress, it remains unclear whether Vision-Language Models (VLMs) understand visual persuasiveness. This motivates us to ask: can VLMs assess whether an image persuasively supports an intended message, which visual factors shape this judgment, and do they align with human judgments? Through empirical analyses on image-message pairs where human raters consistently agree on the persuasiveness judgment, we show that VLMs exhibit a recall-oriented bias: they over-predict images as persuasive while achieving high recall. We introduce Visual Persuasive Factors (VPFs), a taxonomy informed by cognitive psychology for quantifying visual cues that shape persuasive judgments. Our factor-level analysis reveals that VPFs distinguish human persuasiveness judgments, whereas VLMs only partially reproduce these patterns, often generating false positives by treating persuasion-relevant cues as sufficient evidence. Building on this insight, we evaluate VPF-guided interventions and find that properly framed VPF knowledge can improve performance, but merely specifying visual cues or adding step-by-step reasoning is insufficient. By analyzing model rationales at the level of functional reasoning steps, we further identify a central bottleneck in connecting object identification to semantic message alignment.

Abstract:
Given the criminal fact, a complete legal judgment usually contains two parts: Court View Generation (CVG) for the case and Legal Judgement Prediction (LJP) for each defendant. However, existing studies still cannot support unified modeling and evaluation of complete legal judgments. First, at the data level, existing legal datasets mostly cover either single-defendant scenarios or multi-defendant scenarios, and the attributes predicted by LJP are not rich enough, making it difficult for models to generalize to real-world complete legal judgments. Second, CVG and LJP are usually modeled independently, ignoring their intrinsic connection. Third, the role of auxiliary articles has not been explored. To address these issues, we build CMML (Chinese Mixed-Defendant Multi-Attribute Legal Judgment Dataset), a large-scale dataset that mixes single- and multi-defendant cases with multi-attribute annotations. Based on CMML, we propose a new task, LJP-x, which extends LJP with additional prediction attributes. We further systematically study the interaction between CVG and LJP-x, and explore how auxiliary articles influence both tasks. Our dataset is available at: https://github.com/magicalpig2003/CMML.

Abstract:
Understanding and predicting the temporal popularity of social media content is critical for digital marketing and recommendation systems. Existing methods often rely on static feature concatenation, struggling to capture complex multimodal interactions and strict sequential dependencies in long-term popularity evolution, frequently underestimating viral content. To address these limitations, we propose a MultiModal Autoregressive Transformer Framework (MMATF). Our framework extracts and refines key representations from video covers, texts, category tags, and numerical statistics. A dynamic modality interaction module adaptively fuses these inputs to capture evolving content dynamics, establishing a solid foundation for temporal relevance. Furthermore, an autoregressive temporal encoding module strictly prevents future information leakage. Extensive experiments on SMTPD and ICIP datasets demonstrate MMATF significantly outperforms advanced baselines across 7-, 14-, and 30-day horizons. Specifically, compared to the strongest baseline, our model reduces Mean Absolute Error by 3.8% and 4.3%, and improves Spearman’s Rank Correlation by 0.3% and 0.4% for 30-day predictions on SMTPD and ICIP respectively, proving consistent long-term forecasting accuracy.

Abstract:
Iterative Direct Preference Optimization (DPO) has emerged as a widely used paradigm for aligning Large Language Models on reasoning tasks. Existing approaches typically rely on Best-of-N sampling ( N≥8 ) to mine positive trajectories from the distribution tail. In this work, we show that in mathematical reasoning, increasing N yields diminishing returns while increasing verifier-induced false-positive risk and the distribution shift required for policy updates. To address this, we introduce PACE (Proximal Alignment via Corrective Exploration), a generation-based corrective framework that replaces exhaustive mining with low-budget exploration ( 2≤ N≤3 ). Rather than searching for increasingly rare positive samples, PACE synthesizes high-fidelity preference pairs from failed explorations through corrective hindsight refinement and verification-guided filtering. Empirically, PACE matches or exceeds the performance of DPO-R1 ( N=16 ) while using about 1/5 of the compute, and remains robust under 20% label corruption, where high- N baselines exhibit substantially higher noise exploitation.

Abstract:
Iterative preference optimization is essential for aligning Large Language Models on mathematical reasoning tasks, yet its efficiency is often throttled by signal scarcity: as the model improves, static problem sets become increasingly mismatched to the model’s evolving competence, producing rollouts that are either too easy or too hard and therefore non-informative, which leads to a scarcity of valid preference pairs. We propose DIAG, a Diagnostic Iterative Alignment and Generation framework that adaptively reshapes the practice distribution to increase informative supervision and focus training near the student’s current competence boundary. DIAG consists of two phases: (1) diagnosing valid preference-pair yield to calibrate the exploration-exploitation trade-off and allocate topic quotas via an Empirical Bayes shrinkage estimator, thereby prioritizing high-yield concepts; and (2) generating targeted practice, where a teacher synthesizes variants from the student’s failure traces. We further provide a theoretical view interpreting DIAG as a teacher-mediated approximation to KL-regularized reweighting of the practice distribution toward the student’s competence boundary, where valid preference-pair yield is maximized. Experiments show that DIAG boosts yield across iterations and delivers stronger reasoning performance under an iso-effective training budget, demonstrating that it can distill more informative preference supervision for mathematical reasoning.

Abstract:
Cross-lingual biomedical entity linking (BEL) maps mentions in any language to unique identifiers in a biomedical knowledge base, supporting clinical and biomedical NLP applications. We identify two issues affecting current systems. First, the UMLS aliases used to train cross-lingual BEL retrievers are heavily skewed toward English, so retrievers generalize poorly to non-English mentions. Second, although context is often necessary for disambiguation, naively injecting context into retrievers trained only to align aliases severely degrades retrieval. We propose BioELX, a retrieve-rerank framework that addresses both issues. For retrieval, we continue training SapBERT \mboxmulti using Wikidata-derived cross-lingual alias supervision, forming shared concept neighborhoods across languages. For reranking, we adapt pretrained LLM rerankers to entity linking through mention-anchored prompting, which marks the target mention so that rerankers score candidates with respect to the intended mention rather than other salient tokens in the context. Experiments show that BioELX achieves new state-of-the-art results on four cross-lingual BEL benchmarks, improving Recall@1 by 4.8 to 18.2 percentage points over prior best results, without any task-specific BEL annotations.

Abstract:
Large Language Models (LLMs) have shown strong potential in medical question answering, but their reliance on static parametric knowledge limits their ability to handle rapidly evolving clinical evidence. The retrieval system mitigates this issue by incorporating external medical knowledge, but flat retrieval can return lexically relevant but clinically misaligned evidence. We propose Med-Chain, a training-free medical evidence retrieval framework that reformulates retrieval as inspectable and repairable clinical evidence paths. Med-Chain first routes each question through a clinical topic tree to progressively narrow retrieval from broad domains to specific guideline sections. It then verifies whether the retrieved path matches the clinical intent and performs level-aware refinement to repair mismatches. Experiments on seven medical QA benchmarks show that Med-Chain consistently improves backbone LLMs and outperforms existing medical Retrieval-augmented generation baselines, highlighting the value of clinically structured and repairable evidence retrieval.

Abstract:
Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading to reward hacking and policy degradation. To address this, we propose RobustTests, a framework featuring a faulty-code-driven test case synthesis strategy. By leveraging "near-correct" faulty codes, RobustTests captures latent logical discrepancies and employs validator agents with behavioral feature clustering to filter invalid or redundant test cases. Additionally, a stepwise dense reward function based on pass rates is introduced to mitigate false negatives and enhance training robustness. Using this pipeline, we construct an augmented version of the CodeContests+ dataset with superior diagnostic utility. Experimental results show that RL fine-tuning of Qwen3-32B via RobustTests achieves a 3% absolute gain on LiveCodeBench, demonstrating its effectiveness in advancing LLM code generation proficiency. Codes and data are available at https://huggingface.co/datasets/sid6/RobustTests.

Abstract:
Diffusion language models (DLMs) have emerged as a promising alternative to autoregressive language models. While discrete DLMs achieve strong performance across language generation tasks, continuous DLMs remain relatively underexplored despite their smooth latent-space dynamics. In this work, we present a unified embedding-space interpretation of continuous and discrete DLMs. Within this framework, we analyze continuous diffusion dynamics as a retention channel that preserves intermediate denoising information during sampling. Motivated by this perspective, we propose 𝜆 -DLM, which smoothly interpolates between continuous and discrete diffusion dynamics through a controllable parameter 𝜆 . By training with randomly sampled 𝜆 , a single model generalizes across continuous, discrete, and intermediate diffusion dynamics, unifying diverse retention behaviors within one framework. Unlike prior methods that rely on fixed or self-confidence-guided diffusion dynamics, 𝜆 -DLM enables controllable exploitation–exploration trade-offs without retraining. Experiments on LM1B and OWT unconditional generation benchmarks show that 𝜆 -DLM consistently outperforms strong baselines. Moreover, varying 𝜆 enables semantically diverse generation without additional training. Compared with temperature sweeps, varying 𝜆 shows a more structured relationship between lexical diversity and sentence-embedding-based mode coverage.

Abstract:
Inspired by Large Language Models (LLMs), codec-based TTS systems have achieved remarkable success in high-fidelity speech generation by directly adopting the standard Cross-Entropy (CE) objective to predict sequences of discrete speech tokens from neural audio codecs. However, this practice overlooks the geometric structure of codec token spaces: tokens that are close in the codec codebook embedding space often correspond to acoustically similar speech realizations and can serve as perceptually plausible alternatives. We term this observation the Neighboring Token Phenomenon . The one-hot CE objective is agnostic to such structure, as it penalizes all non-target tokens equally, including these close neighbors, creating a mismatch between the structured codec latent space and the learning objective. In this paper, we quantitatively analyze the neighboring token phenomenon in both reconstruction and generation settings. To address the mismatch, we propose Multi-Neighbor Cross-Entropy (MNCE) loss function, which generalizes the one-hot target of CE loss into a positive set containing the ground-truth token and its nearest neighbors, encouraging the model to treat neighbor predictions as partially correct. As a result, even when the model misses the exact target token, it is more likely to predict a close neighbor rather than an unrelated token, reducing perceptually harmful errors in synthesized speech. Experiments demonstrate that MNCE is effective and generalizable across diverse codec-based TTS systems, improving quality and robustness. Moreover, scaling training to 20 , 000 hours further validates the scalability of our approach.

Abstract:
Long-horizon interactions with LLM-based assistants require memory systems that preserve and update user states, preferences, and interaction histories. Existing evaluations report end-to-end QA accuracy and cannot determine whether errors arise from encoding, retrieval, or generation. We introduce EvalMem, an operation-level diagnostic framework with three parallel Examiners. For each query, the Encoding Examiner checks whether the target fact is stored, the Retrieval Examiner assesses whether the native retriever returns usable evidence, and the Generation Examiner tests whether the model can answer from oracle evidence. Their outputs form fine-grained multi-label defect codes. To improve store-level diagnosis, we adapt agentic RAG with a recall-first strategy that searches using both the query and source evidence, increasing recall of present evidence on LoCoMo from 70.2% to 95.6%. Evaluations of seven memory systems on LoCoMo, LongMemEval-S, and dynamic DynaMem-Bench identify retrieval as the most frequently attributed failure layer; in default LoCoMo, retrieval defects reach 22.1%, compared with 7.7% for encoding and 6.5% for generation. Guided by this diagnosis, MemWiki, a search-friendly auxiliary structure built from each system’s memory export, improves mean accuracy by 2.5 and 2.3 percentage points on LoCoMo and LongMemEval-S, respectively.

Abstract:
Removing a specific individual’s information from multimodal large language models (MLLMs) is often needed after deployment, but existing methods rely on a retain set, which is hardest to obtain at that point, and rebuilding it recreates the privacy exposure that unlearning aims to remove. Forgetting from the forget set alone instead damages the shared visual-language computation, harming perception. We cast retain-free unlearning as a localization problem: causal tracing, weight transplant, and Fisher overlap all point to early-to-mid decoder MLPs as the layers where identity information is stored and, unlike other module families, can be modified without substantially disrupting vision. We turn this into Pathway-Aware Visual-attribute Anchoring (PAVA), which confines updates to these layers and pairs a forget loss with a visual-attribute anchor that preserves image-grounded behavior by distilling the model’s own pre-unlearning answers from the forget images alone. On MLLMU-Bench and ReMem, PAVA gives the strongest forget-retain trade-off among forget-set-only methods and remains competitive with retain-based baselines.

Abstract:
In Agentic Search, trajectory-level outcome rewards fail to quantify the behavioral contributions of individual steps, while existing step-level reward methods typically rely on costly tree sampling. We view world knowledge as a latent world graph and each IS task as search within a latent task graph, where effective steps should make graph progress toward the answer node. Based on this prior, we propose Graph-Distance Contribution Reward (GDCR), a step-level process reward that scores newly-retrieved and newly-cited entities by their distance to the answer node in a training-time Entity-Relation (ER) graph. We further propose Step Advantage Policy Optimization (SAPO), which converts GDCR into step-level advantages and combines them with trajectory-level outcome advantages. Experiments on four challenging benchmarks validate the effectiveness of our method.

Abstract:
Extracting skills from past successes is critical for the efficient evolution of Large Language Model (LLM) agents. Prevailing agent self-evolution paradigms typically rely on a core assumption: equipping LLMs with skill memories derived from successful trajectories will monotonically improve their problem-solving capabilities. However, probe analyses reveal that extracting skills solely from successful trajectories traps the model in a Skill Imitation Trap. For tasks that resemble past successes but require different tools, retrieving more skills paradoxically increases the model’s confidence in wrong tool calls—procedure skills raise the wrong-tool margin by 47% over a memory-free baseline. To overcome this limitation, we propose Boundary-Aware Skill Memory (BASM), which augments each skill with explicit boundary fields—applicability conditions, risk cues, avoidance rules, and recovery notes. These fields transform each retrieved skill from an unconditional action template into state-conditioned guidance: the agent applies the skill when its conditions hold, suppresses inapplicable tool calls when they do not, and issues targeted repairs when execution fails. Across three agent benchmarks and four model scales, BASM consistently outperforms success-distilled skill-memory baselines: it improves task success rate by up to 23.8% on AppWorld, accuracy by up to 5.0% on BFCL, and reduces attack success rate by 4.6% on AgentDojo, while simultaneously reducing average AppWorld steps by up to 6.6% relative to the memory-free baseline.

Abstract:
Training Large Language Model (LLM) agents via Reinforcement Learning (RL) in federated settings faces fundamental challenges, including model heterogeneity across clients, prohibitive communication costs and severe data heterogeneity. We propose a semantic federated learning framework called SemFed that addresses all challenges by communicating behavioral patterns rather than model parameters. Each client extracts compact semantic policy summaries from local training trajectories, which are aggregated at the server into shared, elite and anti-patterns. These aggregated patterns are injected back into local training through different complementary channels. Since the communicated objects are structured behavioral abstractions, SemFed supports heterogeneous model architectures while reducing per-round communication to kilobytes. Experiments on ALFWorld and WebShop across multiple heterogeneous LLM backbones demonstrate that SemFed consistently surpasses strong federated baselines in task success while reducing communication overhead by orders of magnitude.

Abstract:
Multicultural multi-agent systems are increasingly deployed in globally diverse settings, where different agents are grounded in different cultural backgrounds. Existing cultural evaluation focuses on value alignment: how closely a single agent matches a target culture. Yet alignment is a per-agent property and cannot reveal whether a system, taken as a whole, preserves the cultural plurality it is meant to represent. We propose value diversity as a system-level evaluation axis for multicultural agent systems, defined through the dissimilarity between culturally conditioned agents’ responses on a shared value survey. Using the World Values Survey, we evaluate 19 cultures and 18 backbone models across a wide range of system configurations. We find that diversity is largely uncorrelated with alignment, indicating that the two capture complementary system properties, and that current multicultural agent systems fall substantially below human societies in value diversity. Mixed-backbone systems narrow this gap but do not close it, and the gap persists across culture compositions and agent scales. Social interaction further erodes diversity by driving agents toward consensus, and a participatory budgeting case study shows that this homogenization narrows the breadth of collective decision-making. Together, our results establish value diversity as a distinct evaluation axis for multicultural multi-agent systems and reveal a persistent homogenization tendency in current LLM-based societies. Our code and data are publicly available at https://github.com/iNLP-Lab/MultiAgent-Diversity.

Abstract:
Large Language Model (LLM) agents face a fundamental bottleneck called context flooding in multi-turn information retrieval: as evidence accumulates across interaction turns, their context windows become saturated with redundant, conflicting, and outdated facts. Existing research either compresses interaction history into rolling summaries, sacrificing relational precision, or relies on retrieval-augmented pipelines that lack mechanisms for resolving knowledge conflicts and maintaining a globally consistent belief state. In this work, we introduce GRAM, a reinforcement learning framework designed to train an agent to actively manage and utilize a dynamic graph-structured memory. Rather than passively accumulating retrieved text, the agent learns to incrementally construct and revise a coherent knowledge graph that serves as its persistent belief state, resolving contradictions and preserving logical relationships in memory. Furthermore, We employ Group Relative Policy Optimization to train small language models (under 4B) to master these complex memory-governance behaviors. Experimental results across multiple mainstream question-answering benchmarks demonstrate the superiority of the GRAM framework over existing agentic memory baselines and conventional RAG systems.

Abstract:
Latent reasoning models compress chain-of-thought into hidden states, but whether these representations retain usable self-knowledge about correctness remains unclear. We study two compression-based paradigms, Latent-SFT and CoLaR, and find a sharp distinction between problem-level and trajectory-level self-knowledge. A lightweight attention probe predicts which problems the model will solve (AUC 0.805/0.853) and transfers across datasets and compression ratios, yet cannot reliably rank rollouts of the same problem; distribution-matched retraining does not recover this signal. This distinction maps directly to downstream use: selective prediction improves accuracy by 20–25pp at 50% coverage, and adaptive routing outperforms either solver while reducing token cost, whereas latent best-of-N yields negligible gains despite available oracle headroom. The problem-level signal also persists on a 7B math checkpoint but vanishes on StrategyQA. These results identify an algorithm–paradigm boundary for exploiting self-knowledge in latent reasoning. Code is available at https://github.com/Arthas777/latent-probe.

Abstract:
Real-time spoken dialogue systems face a fundamental tension between latency and response quality. End-to-end speech-to-speech (S2S) models respond immediately and naturally handle turn-taking, backchanneling, and interruption, but produce semantically weaker outputs. Cascaded pipelines (ASR ➔ LLM) deliver stronger responses at the cost of latency that grows with model size. We present RelayS2S, a hybrid architecture that runs two paths in parallel upon turn detection. The fast path—a duplex S2S model—speculatively drafts a short response prefix that is streamed immediately to TTS for low-latency response onset, while continuing to monitor live audio events. The slow path—a cascaded ASR ➔ LLM pipeline—generates a higher-quality continuation conditioned on the committed prefix, producing an uninterrupted utterance. A lightweight learned verifier gates the handoff, committing the prefix when appropriate or falling back gracefully to the cascaded pipeline. With GPT-4.1 as the back-end, RelayS2S substantially reduces response latency while preserving nearly all of the cascaded pipeline’s textual quality. On synthetic voice dialogues, it achieves a P90 first-chunk latency of 81 ms, excluding TTS and network latency, compared with 1,006 ms for the cascaded baseline. On real voice dialogues, RelayS2S reduces average first-chunk latency by 479 ms while retaining 99% of the cascaded pipeline’s textual quality. These benefits become larger as the slow-path model scales. Because the prefix handoff requires no architectural modification to either component, RelayS2S serves as a lightweight, drop-in addition to existing cascaded pipelines. Our code is publicly available at https://github.com/mailong25/relays2s.

Abstract:
Large language model (LLM) serving creates environmental impacts beyond carbon and water, including ecosystem damage through biodiversity-related pathways. We present BIRDS, a framework for Biodiversity Impact of Request-Driven LLM Serving. BIRDS defines request-level functional units, quantifies operational and embodied biodiversity impact, and introduces Quality-Normalized Biodiversity Impact (QNBI) to jointly analyze ecological impact and response quality. Across diverse workloads, models, GPUs, and regions, BIRDS reveals that reveals that biodiversity impact accumulates at scale and exposes quality-aware serving tradeoffs. The code is available at https://github.com/TianyaoShi/BIRDS.

Abstract:
Embodied agents must reason and act from experience accumulated before future questions or goals are known, yet existing memory systems often treat such experience as flat text, retrieved clips, or static graphs. We introduce MEMORA , an embodied memory framework that forms persistent participant-specific world memory from continuous egocentric video. MEMORA instantiates a formation–consolidation lifecycle: online memory editing maintains typed stores for spatial context, object identity and state, and temporal action evidence, while offline consolidation abstracts repeated experience into reusable routines, habits, and preferences. We evaluate MEMORA with MEMORA-Bench , a 45-hour egocentric-video suite for retrospective memory faithfulness and prospective memory-grounded planning. Across four open-weight answer models, MEMORA improves over no-memory, flat-text, and graph-structured memory baselines under matched perception and retrieval. The gains are largest on out-of-distribution planning, where MEMORA improves robot-grounded planning score by up to , showing that formed memory can compose new goals from remembered procedures, objects, and preferences. A physical-robot demonstration further suggests that memory formed from human egocentric video can guide participant-grounded action without in-domain robot data.

Abstract:
Multilingual fluency often invites a stronger assumption: a model that can speak a user’s language must also understand the culture encoded by that language. We call this the Illusion of Cultural Alignment. To test this assumption directly, we introduce MMC-QA, a benchmark of 1,086 natively sourced questions across 11 language groups, five cultural dimensions, and three difficulty tiers. Unlike translated benchmarks, MMC-QA targets locally grounded knowledge and reduces shortcuts from English-centric cross-lingual transfer. Evaluating 15 LLMs, we find substantial cultural degradation and a pronounced Locality Effect: cultural competence tracks pre-training exposure more closely than general reasoning ability. We further show that common inference-time remedies do not dissolve the illusion. Models remain overconfident on unfamiliar cultural questions, repeated sampling yields unstable rather than reliable correctness, and retrieval augmentation helps unevenly on long-tail facts. These findings indicate that cultural alignment cannot be inferred from multilingual ability alone and requires deeper intervention than calibration, sampling, or retrieval at inference time.

Abstract:
Natural Language Inference (NLI) is a core task for language understanding, yet existing NLI datasets are static and increasingly saturated, allowing current Large Language Models (LLMs) to perform well without fully revealing their capabilities and shortcomings. To address this problem, we propose a new data augmentation framework that automatically builds more challenging NLI datasets by iteratively fusing rich facts into the premise and hypothesis of an NLI instance. We use a strict fact filter to ensure that fused facts are non-contradictory and non-redundant. Applied to SNLI and MNLI, our augmentation substantially increases data length and complexity, and the performance of a range of LLMs on the augmented datasets drops significantly (up to 30%). Ablation experiments and human quality checks confirm the high quality of the generated data.

Abstract:
Trajectory data is getting more vital for training large language models for boosting the agentic abilities. Unlike the verifiable domains such as coding or mathematics, scaling trajectory data for open-ended tasks are much more difficult because these tasks lack singular ground truth and are costly to annotate or verify. In this paper, we propose RetroGen, a self-improving framework of retrospective process supervision. Our key observation is that although expert trajectories are scarce, high-quality final artifacts such as literature reviews, analyst reports and legal judgments, are abundant in pre-training data and can be viewed as compressed traces of the evidence-seeking processes that produced them. RetroGen reconstructs candidate latent trajectories from expert artifacts, verifies them against both the artifact and supporting evidence, and trains models on their own successful reconstruction data, without requiring trajectory data from stronger models. Experiments show that RetroGen improves grounding, faithful synthesis, and long-form evidence-seeking agent tasks.

Abstract:
Native Language Identification (NLI) consists in predicting a person’s native language based on their second language production. In the field of text-based NLI, approaches based on Large Language Models (LLM) outperform previous approaches based on statistical classifiers trained on linguistic features. However, the unstructured nature of LLM textual output does not support the direct application of usual classifiers. To address this limitation, we explore the use of LLM to generate effective latent vector representations (embeddings) for NLI. Our experiments, across ten models ranging from 0.1B to 405B parameters, show that embeddings gathered from a frozen LLM coupled with a linear classifier match the performance of fine-tuned generative LLM. Furthermore, layer-depth analysis reveals that the first 40% of an LLM’s layers are sufficient to achieve full representational utility for NLI.

Abstract:
Named entity recognition (NER) extracts entity spans and assigns them to predefined types. Large language models (LLMs) are increasingly used for NER, but most LLM-based methods still rely on a mainly semantic view, judging whether a candidate span is compatible with an entity type. Although effective, this view can miss entities with weak type semantics, over-accept semantically salient modifiers, and leave little chance to recover spans omitted by early candidate extraction. We argue that entity spans are also traceable in sentence form: local structure and prepositions provide cues about a span’s position and function. We propose SSDA-NER, a Semantic-Structural Dual-path Arbitration framework that combines semantic understanding with structural analysis. Specifically, the Semantic path checks candidate type compatibility, while the Structure path withholds explicit semantic guidance during its first stage and identifies possible mentions through structural analysis. The Arbitration Agent then compares their rationales and confidence scores to adjudicate disagreements. Experiments on nine datasets with three backbone LLMs show that the two views are complementary: the Structure path substantially supplements entities missed by the Semantic path on some datasets, and arbitration outperforms existing multi-agent methods in most evaluated settings. Source code is available at https://github.com/MrHoux/SSDA-NER.

Abstract:
Personalizing Large Language Models (LLMs) for long-term multi-turn interactions remains challenging, as user preferences and behavioral traits are progressively revealed and continuously evolving throughout conversations. Previous personalization methods have mostly relied on prompt-based or post-training (e.g., fine-tuning) paradigms; however, these approaches have been either prone to shallow alignment or have suffered from high computational costs. Inspired by the recent emergence of activation steering, we explore whether personalized alignment can be achieved through lightweight latent-space intervention without modifying backbone parameters. To achieve this goal, we propose PersonaSteer , a dynamic side-model architecture for multi-turn personalized alignment via learnable activation steering. Specifically, to enable fine-grained and evolving personalization, our design learns high-dimensional steering representations from unstructured interaction history and injects them into the backbone model’s residual stream for latent-space behavioral modulation. Furthermore, we introduce a steering-vector feedback loop to maintain persona consistency across multi-turn conversations. Extensive experiments demonstrate that PersonaSteer achieves best or top-tier alignment performance, averaging 3.18/5 on Qwen2.5-7B and 2.95/5 on Qwen2.5-3B. In addition, our framework approach maintains stability during evolving dialogues, with alignment performance consistently improving over time. PersonaSteer provides a lightweight solution for building truly dynamic and personalized LLM. The code is available in https://github.com/tsinghua-fib-lab/PersonaSteer .

Abstract:
LLM-based assistants have become ubiquitous, yet information security awareness (ISA) remains an underexplored aspect of LLM safety. ISA encompasses not only security knowledge, but also the attitudes and behaviors required to recognize implicit security risks and avoid prioritizing user satisfaction over secure guidance. We introduce LISAA, a framework for evaluating LLM ISA under realistic end-user interactions. The framework includes a dataset of 100 realistic scenarios spanning all criteria of an ISA taxonomy, each with a tension between implicit security implications and user satisfaction. We further introduce methods for systematically refining realistic ISA scenarios from authentic user queries and for selecting reliable LLM judges without requiring large annotated datasets. Robustness analyses show that LISAA rankings remain stable under scenario resampling and stochastic generation, while construct-validity experiments demonstrate sensitivity to changes in security behavior. Applying LISAA to leading LLMs reveals that many widely deployed models still exhibit medium-to-low ISA levels, and that strong performance on traditional cybersecurity knowledge benchmarks does not necessarily translate to high ISA. In addition, the framework enables comparative analyses across model families, revealing consistent trends such as higher ISA in newer and larger variants, although meaningful gaps persist even among recent models. We release the dataset, evaluation pipeline, source code, and an online tool for evaluating new models using the LISAA framework.

Abstract:
Language models increasingly use skills that provide reusable instructions and procedures for task execution. After a relevant skill has been selected, the model may need to apply its governing procedure together with retrieved task context. When the retrieved context is useful but contains cues that violate the skill, the model must retain compatible information while rejecting the conflicting cues. We study failures in which the model adopts such cues and produces a retrieval-supported but skill-invalid answer, which we call procedural hallucination. We introduce SOURCECLASH, a matched benchmark with 1,271 bundles and 6,355 prompts across ten procedural domains. Each bundle fixes the task, skill specification, and answer space while varying the relation between the skill and retrieval. Across nine models, valid-answer rates average 25.7% under conflicting retrieval, compared with 78.2% and 72.8% under two supportive retrieval settings. Mechanistic analysis of open-weight models shows that the answer-preference difference typically reaches its maximum shortly after generation begins. Residual replacement moves the model toward valid answers, while attention decomposition attributes most of the repair to value vectors. Intervention at the identified value pathway also improves final answer validity. These results characterize how partly conflicting retrieval changes skill-guided generation.

Abstract:
End-to-end event extraction remains challenging for large language models as it requires simultaneous identification of event triggers, classification of event types, and extraction of schema-grounded argument spans. We present EAGER, a reinforcement learning framework for generative event extraction that combines fine-grained verifiable rewards with Schema-Contrastive Advantage Estimation to alleviate advantage collapse under sparse binary rewards. Our reward design explicitly targets structural validity, extraction accuracy, groundedness, coverage, over-generation, and span precision. Experiments across seven benchmark datasets show that EAGER consistently outperforms prompting, supervised fine-tuning, and prior reinforcement learning baselines, achieving a substantial improvement over the strongest prior method. Results demonstrate that task-aligned verifiable rewards and contrastive advantage estimation substantially improve structured extraction.

Abstract:
Language Identification (LID), the task of determining the language of a given text, is a fundamental preprocessing step for multilingual NLP. While recent work has expanded African LID, existing systems remain limited in both language coverage and fine-grained discrimination among closely related languages and varieties. We introduce AFROSCOPE, a unified framework for African LID that includes AFROSCOPE-DATA, a dataset covering 640 languages, and AFROSCOPE-MODELS, a suite of strong LID models with broad African language coverage. To address persistent confusions among related or geographically proximate languages, we propose a hierarchical classification approach leveraging AFROSCOPE-MIRROR, a specialized embedding model for targeted disambiguation. This approach improves macro-F1 by 4.55 points on a confusable-language subset compared to our strongest base model. We further analyze cross-lingual transfer and domain effects, showing how language-family structure, script compatibility, and domain coverage shape LID performance. Finally, we position African LID as an enabling technology for measuring Africa’s linguistic landscape in digital text and release AFROSCOPE-DATA and AFROSCOPE-MODELS online.

Abstract:
Searching agents have achieved significant progress in NLP, yet their multimodal counterparts are constrained by a lack of suitable evaluation benchmarks requiring reasoning over continuous video, audio, and images. We propose long-form TV dramas as a natural solution and introduce DramaSearch, encompassing 3,600 riddles across 9 dramas totaling 239 hours. Our dataset is generated via a graph-constrained pipeline that processes enriched captions into structured events and multi-granularity evidence views. By employing character-to-view and entity-to-view indexing, we ensure controllable clue disclosure and strictly unique answers. The riddles are categorized into multi-clue, multi-hop, and compositional types, requiring agents to aggregate isolated clues and navigate complex dependency chains. Experiments demonstrate that a Qwen3-8B agent, trained on a held-out set using SFT and RL, achieves an accuracy of 48.3%. This result establishes a strong baseline that performs favorably against zero-shot GPT-5 (37.0%), while indicating significant room for future research. All data and code will be publicly released.

Abstract:
We introduce a corpus for Systemic Functional Linguistics (SFL) Appraisal analysis in Chilean Spanish, together with TULIO, a BERT model adapted to Chilean Spanish through continual pre-training. The corpus contains 2,546 texts annotated for the Attitude subsystem, with a focus on negative Judgment, and supports two span-level sequence labeling tasks: Attitude Types, which distinguishes Affect, Judgment, and Appreciation, and Judgment Subtypes, which identifies five finer-grained forms of social evaluation. To reduce the boundary ambiguity reported in prior Appraisal annotation work, we use a two-stage protocol that first marks structural spans and then assigns Attitude labels. Three expert annotators reach almost perfect agreement at the general level (Krippendorff’s α = 0.834) and substantial agreement for Judgment Subtypes (α = 0.647). We benchmark supervised sequence labelers and structured LLM agents. The highest observed F1 scores are 0.554 on Attitude Types from a Flair BiLSTM-CRF with TULIO embeddings and 0.410 on Judgment Subtypes from Qwen 3.6 Plus. Under Twitter-only training, supervised performance is genre-sensitive, with higher F1 on Twitter than on other genres. All model scores remain below expert agreement.

Abstract:
State-of-the-art GRPO-style methods for speech-aware large language model post-training suffer from coarse credit assignment, broadcasting the same terminal-reward advantage to every token in a response. This ignores useful structure within rollout batches, where speech-conditioned completions often share prefixes before diverging at important decisions. We propose Low-rank Exploration with Adaptive Forking (LEAF), a retrospective tree-based RL method that recovers this structure without online branching or additional decoding. LEAF samples complete responses, selects high-surprisal boundaries, groups responses by shared prefixes, and assigns span-level advantages using descendant rewards. We theoretically justify LEAF’s span-level credit assignment and boundary-selection design. Empirically, LEAF improves over GRPO across speech question answering and speech translation benchmarks under the same rollout and low-rank adaptation budget. Notably, smaller LEAF-trained models consistently outperform current state-of-art, full-parameter baselines.

Abstract:
In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN’s cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results suggest that refusal-relevant information is decodable from intermediate activations and can serve as an effective search signal in an AutoDAN-style discrete jailbreak optimization loop, especially for larger and more robust models.

Abstract:
Inference with large language models (LLMs) on long sequences is computationally expensive due to the quadratic complexity of self-attention. Distributed blockwise methods such as Star Attention reduce this cost by sharding context across hosts, but rely on prepending a static, content-blind copy of the first block to every host. We propose Pulsar Attention, which replaces the static anchor with two lightweight, content-aware components: a small attention-sink prefix that stabilizes softmax, and compact cross-block summaries built via a Max-IDF heuristic that selects chunks containing globally rare tokens. This reduces the Phase 1 per-GPU FLOPs by up to 3.3× over Star Attention while retaining an identical KV cache footprint. On RULER with Llama-3.1-8B-Instruct, Pulsar Attention outperforms Star Attention at sequence lengths up to 128K tokens and remains competitive with dense attention across most tasks, with task-dependent absolute gains of up to 4.7% over the dense baseline.

Abstract:
Current Large Language Model (LLM) agents struggle with long-horizon reasoning tasks, such as competition-level mathematics, where sustaining precision over hundreds of steps is critical. A primary bottleneck is the reliance on natural language context or summaries to track progress; these linguistic representations are inherently "lossy," introducing semantic drift and accumulating errors that derail the reasoning chain. To address this, we propose Structured Strategy Folding (SSF), a novel paradigm that shifts the agent’s memory mechanism from linguistic description to computational verification. Instead of compressing history into text, SSF "folds" the reasoning trajectory into a sequence of Executable Symbolic States (ESS). Each ESS serves as a verifiable anchor—comprising either executed code artifacts or formally judged symbolic lemmas—generated through a dual-pathway validation mechanism. By interacting with these rigorous state artifacts rather than raw textual history, the agent effectively eliminates ambiguity and prevents error propagation. Experiments on challenging mathematical benchmarks demonstrate that SSF significantly outperforms standard Chain-of-Thought and existing summarization baselines, achieving substantial accuracy improvements while maintaining robust performance over extended reasoning horizons.

Abstract:
Modular compression has enabled considerable parameter reduction in LLMs while preserving strong language understanding and downstream task accuracy. However, existing joint modular compression methods primarily rely on activation statistics, leaving loss-sensitivity information and its module-level characterization underexplored. We investigate addressing this gap with LaMoC, a loss-aware modular compression methodology that blends activation and Empirical Fisher statistics through gradient-error alignment. LaMoC improves joint compression by selecting compression statistics that better align local module reconstruction error with the downstream loss. Our contributions are three-fold: (1) We characterize the Empirical Fisher as a module-level loss-aware proxy that can be blended with the activation statistics required for compression. (2) We reformulate joint modular compression as a two-tiered optimization problem that minimizes module reconstruction error while tuning the activation and gradient information blending rate. (3) We implement an empirically driven methodology with statistical validation to solve the resulting compression problem. We evaluate LaMoC across four model families spanning eight models. On the 4–8B models, LaMoC achieves an average 2.5% reduction in perplexity and a 1% relative improvement in task accuracy over state-of-the-art modular compression methods.

Abstract:
LLM agents are increasingly used for collaborative problem solving and human-group simulation. This makes outcome-only evaluation insufficient: if LLM groups are used as models of human groups, we need to know whether they succeed or fail through human-like deliberative mechanisms. We compare human group chats with matched LLM deliberation traces on Wason-style deductive reasoning, then test whether the same process signatures generalize to analogical, abductive, and analytical tasks. Humans and LLMs show the same assembly bonus asymmetry: discussion improves the average member more often than the best initial member. Initial-answer diversity accounts for the effect of model heterogeneity, increasing movement in both corrective and destructive directions. The main differences are process-level. Compared with humans, LLM groups follow majorities more often, surface less unique information, and converge earlier; correct minority signals succeed mainly when re-expressed early. Interventions motivated by human group-decision research yield modest improvements in collective outcomes, but do not remove the coordination bottleneck. Together, these results suggest that LLM groups can reproduce some outcome-level patterns of human deliberation while diverging in the mechanisms that generate assembly bonus and process loss, with implications for human-group simulation and human-AI collaboration.

Abstract:
Simulated patients offer a scalable way to train psychotherapy micro-skills such as empathic responding and exploratory probing, but current systems either follow fixed scripts or rely on LLMs that drift unpredictably over long sessions. We present the Adaptive Virtual Patient (AVP), which adapts its disclosure behavior in response to trainee skill, from guarded, through moderate openness, to full disclosure. The AVP is informed by a structural equation model fit to nearly 2000 hours of real-world psychotherapy transcripts, which quantifies how therapist empathy and exploration shift a patient’s openness over time. An LLM generates the AVP’s utterances conditioned on a disclosure level that the dynamics module updates each turn. In an evaluation with 20 clinicians and trainees over 80 sessions (1,033 turns), the AVP’s disclosure rises in response to therapist empathy and exploration, while a prompt-only baseline stays flat; ablations confirm that the empirically motivated parameterization outperforms alternatives, with exploration carrying most of the adaptive signal.

Abstract:
Text-to-image models like Stable Diffusion and DALLE-3 still struggle with complex multi-turn image editing. We study how to break down such a task into a sequence of subtasks and address them by an agentic workflow (path) of AI tool use with minimum costs. Conventional search algorithms require expensive exploration to find tool paths. While large language models (LLMs) possess prior knowledge of subtask planning, their estimation of the quality and cost of tools is usually inaccurate to determine which to apply in each subtask. Can we combine the strengths of both LLMs and graph search to find cost-efficient tool paths? We propose a three-stage approach “CoSTA” that leverages LLMs to create a subtask tree that prunes a graph of AI tools for the given task, and then conducts A search on the small subgraph to find a tool path. To better balance the total cost and quality, CoSTA combines both metrics of each tool on every subtask to guide the A search. Each subtask’s output is evaluated by a vision-language model (VLM), where a failure will trigger an update of the tool’s cost and quality on that subtask. Hence, the A search can recover from failures quickly to explore other paths. Moreover, CoSTA can automatically switch between modalities across subtasks for a better cost-quality trade-off. We build a novel benchmark of challenging multi-turn image editing, on which CoSTA outperforms state-of-the-art image-editing models or agents in both cost and quality, and performs versatile trade-offs upon user preference.

Abstract:
Vision Language Models (VLMs) are increasingly used in place of traditional OCR pipelines for document understanding. We show that VLMs do not always transcribe text literally. When the input contains imperfect text, they often rewrite it into a more plausible form, a behavior that clean-text OCR benchmarks do not detect. We introduce FaithC4, a multilingual perturbation benchmark of 1,455 single-page documents (English, Chinese, Korean) with three perturbation families: scramble, random substitution, and visually similar substitution. We use it to evaluate 15 systems spanning general-purpose VLMs, OCR-specialized VLMs, and traditional OCR pipelines. Traditional OCR is near-perfect on perturbed text, OCR-specialized VLMs degrade by 0.2–2.3 points of Word Error Rate (WER), and general-purpose VLMs degrade by up to 4.5 points in English language. To explain this gap, we probe Qwen3-VL-4B layer-by-layer and find that rewriting occurs when a perturbed word’s final-layer FFN representation stays close to the original word’s encoding, while faithful transcription occurs when the representation is sufficiently disrupted. Shorter words give the model less evidence to disrupt the representation and are therefore rewritten far more often than longer ones, with 4-character words rewritten more than ten times as often as words of 8 or more characters. We will release FaithC4 and our evaluation code upon acceptance to support future research on transcription faithfulness.

Abstract:
Urban region representation learning plays a fundamental role in urban computing, providing compact embeddings that support a wide range of downstream analytical tasks. Existing methods are developed under a single-city setting and rely on continuous vector representations learned from city-specific spatiotemporal data. Learning transferable and semantically meaningful region representations in cross-city scenarios remains a significant challenge. In this paper, we propose URBAN-PROTOS, a two-stage framework for cross-city learning of urban region representations. We first abstract multi-city mobility data into discrete Urban Prototypes, which capture city-invariant spatiotemporal regularities from localized heterogeneous contexts. These prototypes serve as shared semantic units across cities and are further integrated into the semantic space of large language models through a prototype-guided alignment mechanism. Extensive experiments conducted demonstrate that URBAN-PROTOS consistently outperforms existing methods in cross-city transfer settings, while maintaining strong performance in single-city evaluations. Further analyses show that the learned region embeddings encode deep semantic regularities shared across cities, providing robust support for cross-city generalization.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) improves multimodal reasoning in answer-verifiable settings by rewarding final answers that can be checked. Yet answer-correct trajectories may still rely on incomplete derivations, weak evidence, or statements that contradict their conclusions. This gap between answer correctness and reasoning validity, which we call reasoning-answer inconsistency, motivates trajectory supervision in multimodal RLVR. We compare two main approaches: reward models (RMs) and generative rewards (GRs). RMs are efficient and help early in training, but their gains weaken as the policy distribution shifts; GRs improve performance, but they may yield unstable rewards and incur high pointwise judging cost. We therefore propose Groupwise Ranking Reward, which ranks verifier-passed trajectories for the same prompt in one pass and redistributes reward accordingly. Groupwise comparison better separates stronger and weaker correct trajectories with lower judge overhead than pointwise GRs. Experiments show that RLVR aggravates reasoning-answer inconsistency, while trajectory supervision alleviates it. Groupwise Ranking Reward performs best overall, improving reliability-conditioned accuracy from 47.4% to 54.7% over RLVR.

Abstract:
Prior research has demonstrated that the choice of prompt template during Supervised Fine-Tuning (SFT) significantly impacts the robustness of safety alignment afterwards. However, the influence of template selection during Knowledge Distillation (KD) from teacher to student remains largely unexplored. Thus, we fill this gap by analyzing how different template configurations influence the pre-existing safety alignment of the student. We observe a significant degradation of safety alignment present in the aligned base instruct-tuned model. Specifically, we find that utilizing chat templates renders the model more compliant with harmful queries compared to a non-chat template. These findings are consistent across three models: LLaMA, Gemma and Qwen model families and are evaluated across multiple safety benchmarks. We further show that using a non-chat template during distillation better preserves the base student’s internal representations, while chat template distillation induces a larger representational shift. Code: https://github.com/anjilab/role-of-prompt-template-in-kd

Abstract:
AI agents are increasingly expected to modify and maintain machine-learning systems in production, yet existing benchmarks primarily evaluate either competition-style ML experimentation or generic repository issue resolution. We introduce MLOps-Bench, a repository-grounded benchmark for production ML engineering. Unlike prior ML-agent benchmarks, MLOps-Bench evaluates executable lifecycle changes inside real ML repositories, spanning data pipelines, feature engineering, model training, model serving, monitoring, CI/CD and governance, and end-to-end integration. Each task is grounded in repository evidence and includes a natural-language requirement, oracle patch, plausible buggy baseline, fail-to-pass tests, pass-to-pass tests, and provenance metadata. The benchmark contains 212 validated tasks from 31 public ML repositories, with 636 fail-to-pass tests and 856 pass-to-pass tests. We evaluate five agents and find that the best system solves 27.3% of tasks while passing 67.1% of tests, showing that current agents often make partial progress but still fail production-relevant behavioral requirements.

Abstract:
Large language models (LLMs) are becoming the foundation for autonomous agents that can use tools to solve complex tasks. Reinforcement learning (RL) has emerged as a common approach for injecting such agentic capabilities, but typically under tightly controlled training setups. It often depends on carefully curated task–solution pairs and substantial human supervision, which creates a fundamental obstacle to open-ended self-evolution. In this paper, we propose Tool-R0 for training general purpose tool-calling agents from scratch with self-play RL, under a zero-data assumption. Initialized from the same base LLM, Tool-R0 co-evolves a Generator and a Solver with complementary rewards: one proposes targeted challenging tasks at the other’s competence frontier and the other learns to solve them with real-world tool calls. This creates a self-evolving cycle that requires no pre-existing datasets. Evaluation on five tool-use benchmarks show that Tool-R0 yields 92.5% relative improvement over the base model and surpasses fully supervised tool-calling baselines under the same setting. Our work further provides empirical insights into self-play LLM agents by analyzing co-evolution, curriculum dynamics, and scaling behavior.

Abstract:
Large language models (LLMs) are known to exhibit social sycophancy, often validating or agreeing with users in socially sensitive contexts. Existing evaluations typically measure sycophancy under a fixed prompt formulation, leaving unclear whether such behavior is stable when the same underlying situation is presented with different sycophancy-relevant prompt variants. In this work, we study sycophancy prompt sensitivity: the extent to which changes in user confidence, emotional framing, social consensus, or validation-seeking language alter a model’s sycophantic behavior. We refer to our evaluation framework as SyPS, short for Sycophancy Prompt Sensitivity. Building on existing social sycophancy evaluation settings, SyPS constructs controlled prompt variants that preserve the same underlying user situation while varying sycophancy-relevant social cues. We introduce the Sycophancy Prompt Sensitivity Score (SPSS), an instance-level measure of sycophancy variation across paired prompt variants. Unlike aggregate sycophancy rates, SPSS separates baseline sycophancy from prompt-induced shifts, enabling model-level comparisons of robustness to sycophancy-relevant social cues. Empirically, we find that sycophancy prompt sensitivity is socially structured: validation-seeking and emotional-pressure cues often increase sycophancy, whereas counter-framing and anti-sycophancy prompts tend to reduce it. Our framework highlights whether LLMs maintain stable social judgments while adapting appropriately in tone.

Abstract:
Large language models (LLMs) have shown great potential for healthcare applications. However, existing evaluation benchmarks provide minimal coverage of Alzheimer’s Disease and Related Dementias (ADRD). To address this gap, we introduce ADRD-Bench, a preliminary ADRD-specific LLM benchmark. ADRD-Bench has two components: 1) ADRD Unified QA, a synthesis of 1,438 questions consolidated from seven established medical benchmarks, providing a unified assessment of clinical knowledge; and 2) ADRD Caregiving QA, a novel set of 149 questions derived from a nationally adopted, large clinical trials supported brain health management program, mitigating the lack of practical caregiving context in existing benchmarks. We evaluated 36 state-of-the-art LLMs on the proposed ADRD-Bench. Results showed that the accuracy of open-weight general models, open-weight medical models, and frontier closed-source general models ranged from 0.63 to 0.93 (mean: 0.77; std: 0.09), 0.47 to 0.93 (mean: 0.81; std: 0.14), and 0.83 to 0.93 (mean: 0.90; std: 0.03), respectively. While top-tier models achieved high accuracies (>0.9), case studies revealed inconsistent reasoning quality and stability, highlighting a critical need for domain-specific improvement to enhance LLMs’ knowledge and reasoning grounded in daily caregiving data. The ADRD-Bench dataset and code are publicly available at https://github.com/IIRL-ND/ADRD-Bench.

Abstract:
We present the first systematic study of masked diffusion language models (MDLMs) for graph-to-text generation. We analyze MDLM generation trajectories—the order in which tokens are unmasked during iterative decoding—and find that, unlike autoregressive LLMs which generate text linearly, MDLMs naturally prioritize entities first, followed by relational and function words, with structural tokens resolved last. We further identify a previously undocumented failure mode of supervised fine-tuning: SFT disrupts this strategy by prematurely anchoring structural sentence-ending tokens early in the decoding trajectory, effectively fixing the output length which can lead to omitted or hallucinated information. To address this, we propose 𝜆 -scaled structural decoding, a training-free inference-time modification that downweights structural token confidence and recovers +9.4 BLEU-4. Finally, we introduce Graph-LLaDA, which integrates a Graph Transformer encoder into LLaDA’s decoding process to explicitly incorporate relational graph structure. Cross-dataset evaluation on LAGRANGE reveals that previous baselines overfit to dataset-specific patterns, while LLM- and MDLM-based approaches generalize significantly better. All code and datasets used in our experiments are publicly available at https://github.com/idirlab/Graph_LLaDA.

Abstract:
Scientific discovery workflows rely heavily on lab notes, where researchers record observations, interpret uncertain results, and plan follow-up experiments. Unlike polished publications, lab notes preserve evolving scientific reasoning, tacit scientific knowledge, and author uncertainty, giving AI agents access to the process of science. However, most prior work on scientific text focuses on papers, protocols, or structured databases, leaving informal laboratory notes underexplored as inputs to AI agents for science. This gap matters because tacit knowledge in lab notes is not written as a clean set of instructions: validated observations, tentative judgments, and possible experimental next steps often appear in the same passage. If these signals are conflated, an AI agent may mistake uncertain scientific judgments for confirmed conclusions or executable actions. To this end, we present Notes2Skills, a two-stage framework for turning lab notebooks into verifiable skills for scientific AI agents while preserving the author’s certainty. Across three wet-lab sessions and seven original downstream configurations, Notes2Skills is the only tested configuration that avoids both observed regime-level failures: over-acting on uncertain notes and missing firm directives. We show that certainty preservation is a key missing piece between lab notebooks and reliable agent skills, opening a path toward safer AI co-scientist systems.

Abstract:
Effective disaster response is essential for safeguarding lives and property. Existing statistical approaches often lack semantic context, generalize poorly across events, and offer limited interpretability. While large language models (LLMs) provide few-shot generalization, they remain text-bound and blind to geography. To bridge this gap, we introduce a Geospatial Awareness Layer (GAL) that grounds LLM agents in structured Earth data. Starting from raw wildfire detections, GAL automatically retrieves and integrates infrastructure, demographic, terrain, and weather information from external geodatabases, assembling them into a concise, unit-annotated perception script. This enriched context enables agents to produce evidence-based resource-allocation recommendations (e.g., personnel assignments and budget allocations), further reinforced by historical analogs and daily change signals for incremental updates. We evaluate the framework in real wildfire scenarios across multiple LLM models, showing that geospatially grounded agents can outperform baselines. The proposed framework can generalize to other hazards such as floods and hurricanes.

Abstract:
Thinking in reasoning language models (RLMs) has been subject to debate on whether it resolves or amplifies bias. Prior works have shown competing conclusions in both directions. Using a within-model thinking-vs.-non-thinking ablation across QwQ-32B, DeepSeek-R1-Distill-Qwen-32B, and Qwen3-32B on three high-stakes decision tasks (Adult, COMPAS, Credit), we show that thinking has an asymmetric dual effect on counterfactual fairness: it both resolves counterfactual flips produced by the non-thinking baseline and creates new flips at near-saturating model confidence. In all nine (model, dataset) combinations, the created flips outnumber the resolved flips by roughly 5× . To explain the effect, we treat the thinking trace itself as a measurable site of fairness change and study it through two dynamic instruments: 1) We propose Counterfactual Depth Probability Gap (CDPG) to track bias evolution along thinking depth, and observe that bias propagates and amplifies with thinking. 2) We also formulate Bias Transition Matrices (BTM) to show how predictions of counterfactual pairs change from non-thinking to thinking, and find that the asymmetric dual effect originates in the pair-state joint transition.

Abstract:
Test-time scaling improves the reasoning capability of large language models by generating longer sequential chains of thought. While effective, this approach reaches a bottleneck as computation increases, resulting in a “test-time scaling plateau” where further computation provides only marginal gains. We argue that this ceiling reflects a limitation of the scaling strategy rather than model capability, a phenomenon we term Tunnel Vision, where imperfect early steps can lock the model into a suboptimal reasoning path. Motivated by this observation, we propose ParaThinker, an end-to-end parallel reasoning framework that trains LLMs to generate multiple reasoning paths in parallel and aggregate them into a final answer within a single architecture. By exploring different lines of thought, ParaThinker sidesteps the Tunnel Vision issue and unlocks the model’s reasoning potential. On challenging reasoning benchmarks, ParaThinker achieves accuracy improvements (e.g., 6.5%–20.7% on AIME-24 with a 1.5B model) over sequential-reasoning LLMs under the same budget of decoding tokens. Under matched decoding token budgets and with KV-cache reuse, ParaThinker achieves 38.7%–66.8% lower inference latency than sequential decoding via batched parallel generation. The code is available at https://github.com/MobileLLM/ParaThinker.

Abstract:
Affective image editing aims to transform an image toward a target emotional state. Existing emotion-based editing methods typically represent emotion as a global condition, which often leads models to produce templated edits, limiting their ability to control fine-grained affective variations. To address these limitations, we propose Affective Continuous Editing Framework (ACE), an object–atmospheric framework for modeling visual emotion. To reduce templated editing, our framework employs an Affective Disentanglement Agent (ADA) to translate abstract affective intent into region-level editing instructions and counterfactual supervision signals. Guided by this data, we de-sign a decoupled regional LoRA module with two independent lightweight branches, Object LoRA (OB-LoRA) and Atmosphere LoRA (AT- LoRA), which respectively learn object and atmospheric editing directions. In addition, we introduce a counterfactual regional suppression loss, encouraging the model to learn disentangled regional features and continuous, fine-grained affective variations. Experiments demon-strate that ACE achieves better identity preservation, region-level controllability, and fine-grained emotional expression than existing emo-tion-conditioned editing methods. Quantitative analyses further confirm that the two learned control directions are monotonic and largely confined to their intended regions. The dataset and code are available at https://github.com/xixixixixxxx/ACE.

Abstract:
Dense retrieval over long documents is expensive. Token-level encoders scale quadratically in sequence length, and most long-context embedding models reach 32K tokens only through architectural workarounds or by stretching billion-parameter LLMs. We propose REIGN (Refurbished Embeddings with Integrated Guidance Networks), a contrastively trained bi-encoder that operates on sequences of contextualised chunk embeddings from a frozen Guidance Network (GN) rather than on raw tokens. REIGN targets multi-chunk inputs and primarily document-to-document retrieval; single-chunk inputs stay with the GN. Decoupling token-level processing from document-level reasoning, and caching the GN embeddings to disk, cuts per-document training cost by roughly four orders of magnitude relative to chunked Transformer fine-tuning. We also release a synthetic long-document retrieval benchmark for contrastive training and evaluation at long context lengths. Across an in-distribution Wikipedia benchmark, the LoCo out-of-distribution suite, and a real-world patent retrieval case study, REIGN matches dense long-context retrievers at smaller parameter budgets per regime. A paired significance test puts it on par with models 1.6–4.3× larger on the patent task, and it stays within 0.65 nDCG@10 of a 20×-larger model on LoCo.

Abstract:
Parallel thinking has emerged as a new paradigm for large reasoning models (LRMs) in tackling complex problems. Recent methods leverage Reinforcement Learning (RL) to enhance parallel thinking, aiming to address the limitations in computational resources and effectiveness encountered with supervised fine-tuning. However, most existing studies primarily focus on optimizing the aggregation phase, with limited attention to the path exploration stage. In this paper, we theoretically analyze the optimization of parallel thinking under the Reinforcement Learning with Verifiable Rewards (RLVR) setting, and identify that the mutual information bottleneck among exploration paths fundamentally restricts overall performance. To address this, we propose Outline-Guided Path Exploration (OPE), which explicitly partitions the solution space by generating diverse reasoning outlines prior to parallel path reasoning, thereby reducing information redundancy and improving the diversity of information captured across exploration paths. OPE are implemented with an iterative RL strategy that optimizes outline planning and outline-guided reasoning independently. Extensive experiments across multiple challenging mathematical benchmarks demonstrate that OPE effectively improves reasoning performance in different aggregation strategies, enabling LRMs to more reliably discover correct solutions.

Abstract:
Similarity-based retrieval is the dominant rule for conditioning large language models (LLMs) in in-context learning, retrieval-augmented generation, and prompt-based time series forecasting. The rule concentrates on near-duplicate candidates, an issue that has motivated diversity-aware retrieval but remains unexamined in other retrieval-conditioned pipelines. We study this issue using prompt-based time series forecasting as a test bed, where a learned prompt pool is retrieved by similarity. Dominant methods in this setting retrieve top- K entries by cosine similarity without redundancy control, producing a bias toward dominant temporal patterns while overlooking rare but informative events. We propose CASP-LLM, a coverage-aware semantic prompting framework that addresses this prompt selection bias by combining usage-tracking and saturating-gate techniques into a coverage regularizer that adds no learnable parameters. On six long-term benchmarks and the M4 short-term benchmark, CASP-LLM matches or improves on similarity-based LLM forecasters on most dataset-horizon settings, with the exceptions of Electricity, M4-Monthly, and the few-shot long-horizon setting. A controlled study locates the failure mode at the cross-batch usage level rather than per-retrieval redundancy: within-retrieval diversification such as MMR does not help, whereas regularizing anchor usage across training does.

Abstract:
Large vision-language models (LVLMs) excel at multimodal tasks but often generate instance-level object hallucinations, describing nonexistent objects. Yet most mitigation strategies rely on output-level or coarse-grained interventions, limiting their ability to identify and selectively regulate attention heads associated with faithful versus hallucinatory generation, often at the cost of descriptive coverage. To address this limitation, we propose HACP, a training-free framework that integrates head-level semantic attribution with instance-adaptive counteractive pruning. HACP introduces InfoSpectralScore to summarize spectral changes in head-perturbed representations and identify heads associated with hallucination or grounded generation, which are then selectively suppressed or reinforced during decoding. Across multiple LVLMs and benchmarks, HACP consistently reduces hallucinations while preserving descriptive coverage; on a 1000-image multi-label COCO subset, it lowers CHAIR-s/i from 66.7/18.0 to 43.4/8.6 and improves F1 from 73.0 to 77.0. These findings suggest that head-level attribution provides an actionable signal for fine-grained inference-time hallucination control in LVLMs, while helping preserve descriptive coverage. The official project repository for HACP is available at https://github.com/S1yang/HACP.

Abstract:
Despite the advance in automatic term acquisition (ATA) research, the use of ATA in terminology work is limited. One of the main reasons is that most ATA research focuses on automatic term extraction (ATE) from textual corpora and does not consider terminologies. This paper introduces automatic term generation (ATG), a new method of ATA, which explores existing terminologies, in which timeline information is also considered. We propose a new way of representing the structure of a terminology as a graph, which consists of nodes that represent constituent elements of terms and two types of links that represent intra-term dependency between constituent elements and term closure between the first and the last constituent element of terms. ATG is then defined as a set of link prediction tasks, i.e. predicting two types of links. Cliques of the graph represent the new generated terms. They are determined in a backward fashion by first selecting nodes for which term closure links are predicted and then by walking backward to term nodes for which intra-term dependency links exist. Experiments showed that our approach gives high performance in predicting new terms.

Abstract:
Travel planning agents assist users in generating personalized travel plans by modeling their individual preferences. Existing agents either rely on explicit user instructions or engage in multi-turn clarification to elicit user preferences. However, both approaches overlook the rich behavioral signals latent in users’ past behaviors, which implicitly encode their preferences. This over-reliance on active user input increases interaction burden and limits plan personalization. To bridge this gap, we introduce a new task, Behavior-Aware Travel Planning, which infers user preferences directly from past behaviors and generates personalized travel plans. To facilitate research on this task, we introduce Behavior2Trip, a benchmark constructed from one of the largest Chinese online travel platforms, comprising 11,400 instances. Each instance represents an average of 39.8 past user behaviors spanning 14 attributes across 5 preference dimensions. We further propose B2T-Agent, a reinforcement learning-based agent that leverages user behavior trajectories, interacts with external tools for preference-aligned retrieval, and maintains an internal memory module. Experiments on Behavior2Trip show that GPT-4.1 achieves a full-constraint pass rate of only 0.5% on the hardest tasks, while B2T-Agent built upon Qwen3-8B outperforms all baselines, highlighting the substantial challenge of this task. Moreover, Qwen3-8B trained with B2T-Agent also outperforms GPT-4.1 on the TravelPlanner benchmark, demonstrating strong generalization.

Abstract:
The rapid evolution of reasoning-intensive Large Language Models renders traditional metrics insufficient by masking fine-grained failures and implicit pathologies. Existing weakness discovery methods typically rely on rigid pipelines, yielding superficial insights that lack the diagnostic depth required for effective model improvement. To address this, we introduce Agent4Weakness, a multi-agent framework designed to replicate the rigorous workflow of human expert analysts. By integrating a Domain-Aware Memory for contextual reasoning with diagnostic evaluation skills and a Tool Abstraction mechanism for decouple data analysis, Agent4Weakness transforms raw evaluation traces into grounded, actionable reports. We validate our framework through an extensive study involving 104 models across 27 benchmarks. Experimental results demonstrate that Agent4Weakness produces diagnostic reports significantly superior to competitive baselines. Crucially, leveraging these insights for prompt guidance yields an average 3.7 point performance boost and establishes a closed-loop optimization framework.

Abstract:
Training data attribution scores how training examples affect downstream model performance. For models released with many intermediate checkpoints, attribution can be computed along the saved training trajectory, but evaluating every checkpoint is often prohibitively expensive. We introduce PrAM (Probe-based Attribution Mass), a post-hoc offline checkpoint selector for training data attribution. PrAM runs a small attribution probe at every checkpoint, selects checkpoints with high absolute attribution mass, and runs full attribution only on that subset. Across Pythia-14M, 70M, and 160M, PrAM closely matches a full attribution-pool reference and outperforms heuristic rules based on recency, spacing, learning rate, and validation loss. Beyond rank correlation, PrAM preserves the domain-by-target attribution tables and ranked training examples produced by full saved-checkpoint attribution, enabling scalable post-hoc analyses of checkpointed model releases. We support this with application-based analyses scaling to 410M and 1B parameter models.

Abstract:
Entity-fact unlearning in large language models (LLMs) aims to remove the binding between a target entity and a forgotten fact, producing a model that behaves like one retrained without the forgotten data while preserving neighboring knowledge and overall model utility. We frame this problem as distribution shaping: such a retrained model would yield a plausible distribution at forget queries, fluent, semantically appropriate, and reflective of residual knowledge, which we take as the appropriate target. Existing objectives provide only narrow token-level or semantically uninformative supervision, falling short of this target. We propose Counterfactual Self-Distillation (CSD), a self-distillation framework that approximates the retrained distribution by constructing a counterfactual teacher through type-consistent entity substitution. The resulting teacher is decoupled from the target identity while preserving type-level relational structure, enabling directed probability reallocation without an external oracle. Experiments on TOFU and RWKU show that CSD achieves a strong forget-utility trade-off, including the best overall forgetting on RWKU and strong performance on TOFU.

Abstract:
Fragment-based drug discovery optimizes small molecular hits through three recurring transformations: fragment growing, linking, and merging. Existing generative methods, however, usually specialize in a single transformation and have focused primarily on linker design. We introduce FragmentGPT, a unified chemical language model that casts all three operations as conditional molecular generation. To build this capability, we construct a chemically-aware pre-training corpus using energy-guided bond cleavage, together with overlap-based examples that teach the model to connect separated fragments and consolidate shared substructures. We further introduce Reward-Ranked Alignment with Expert Exploration (RAE), which combines expert-guided candidate generation, Pareto- and composite-reward-based selection, and supervised fine-tuning for multi-objective molecular optimization. Conditioned on one or more fragments, FragmentGPT can expand a scaffold, connect distinct subunits, or merge overlapping structures into a complete molecule. Experiments on RTCB and SARS-CoV-2 main protease (3CLPro), together with ablation studies, show high chemical validity and strong aggregate performance across multiple drug-design objectives.

Abstract:
Deciphering scanned Oracle Bone Script (OBS) images into corresponding modern Chinese characters is a crucial step in AI-assisted OBS decipherment. As a highly pictographic writing system, OBS often embeds semantic information in glyph forms, making its decipherment depend on more than visual similarity alone. However, how to effectively exploit such glyph-meaning cues and how to make the decipherment process interpretable remain challenging. To this end, we propose an LVLM-based framework that bridges glyphs and meanings through structured glyph analysis. Specifically, we introduce a progressive training strategy that guides the model from radical analysis to pictographic analysis and finally to mutual analysis, enabling it to capture semantic information embedded in OBS glyphs. The generated textual analyses serve as both interpretable reasoning traces and explicit evidence for retrieving corresponding modern Chinese characters from a dictionary. We also construct the Pictographic Decipherment OBS Dataset (PD-OBS), which provides detailed radical and pictographic analyses to support glyph-analysis learning and dictionary-based retrieval. Experiments on OBS benchmarks demonstrate the strong zero-shot capability of our method, especially its substantially improved Top-10 accuracy on unseen OBS, highlighting its ability to provide reliable and interpretable candidate references for AI-assisted archaeological and historical studies. The code and dataset will be released at https://github.com/PKXX1943/PD-OBS.

Abstract:
Evaluating multi-paragraph clinical question answering (QA) systems is resource-intensive and challenging: accurate judgments require medical expertise and achieving consistent human judgments over multi-paragraph text is difficult. We introduce CQA-Eval, an evaluation framework and set of evaluation recommendations for limited-resource and high-expertise settings. Based on physician annotations of 300 real patient questions answered by physicians and LLMs, we compare coarse answer-level versus fine-grained sentence-level evaluation over the dimensions of correctness, relevance, and risk disclosure. We find that inter-annotator agreement (IAA) varies by dimension: fine-grained annotation improves agreement on correctness, coarse improves agreement on relevance, and judgments on communicates-risks remain inconsistent. Additionally, annotating only a small subset of sentences can provide reliability comparable to coarse annotations, reducing cost and effort. Data and code available at https://github.com/federicabologna/cqa-eval under a CC-BY license.

Abstract:
Large language models can answer a medical question correctly and still abandon that answer when a user pushes back. We study this failure as medical sycophancy and ask when models are most likely to give in. Across five open-weight models, 500 MEDQUAD questions, and 1.2 million trials, we use a fully crossed design over four conversational factors: user role, user evidence, interaction structure, and grounding. Medical sycophancy is nearly three times more common when users challenge an answer the model has already given than when the false claim appears in the initial query. Models are also more susceptible to users presented as physicians or medical students. Most strikingly, fabricated evidence has opposite effects across interaction structures. It increases sycophancy in single-turn interactions but reduces it after the model has already answered. Grounding helps, but does not eliminate the behavior. Sycophancy varies more across medical questions than across models, making question selection an important part of benchmark design. Reasoning traces suggest that multi-turn failures are associated with models turning back toward their own prior answer, while fabricated evidence receives more scrutiny after an initial response. Together, the results show that medical sycophancy depends as much on how a model is challenged and evaluated as on which model is tested.

Abstract:
KV cache compression is essential for long-context LLM inference. Most token-level KV compressors reduce cache length by retaining selected token KV entries and evicting the rest. Once evicted, those spans no longer receive attention, even though some may still carry information useful enough to preserve in compact form. We propose SurrogateKV, a training-free, runtime-only method that reframes KV compression as region-level representation allocation. Under the same KV-slot budget, SurrogateKV assigns historical regions to raw KV, surrogate KV, or drop. It does not train a compressor, modify model weights, or introduce a new attention operator. After prefill, SurrogateKV constructs lightweight surrogate entries as ordinary KV pairs, keeping their information available during standard attention decoding. This representation decision is orthogonal to layer- and head-wise budget allocation. We evaluate SurrogateKV-Snap, SurrogateKV-Dynamic, and SurrogateKV-Ada, obtained by applying SurrogateKV to SnapKV, DynamicKV, and Ada-KV, respectively. As representative results, SurrogateKV-Snap raises Needle-in-a-Haystack (NIAH) retrieval accuracy from 87.51 to 98.84 at 128 slots per layer. SurrogateKV-Ada raises the LongBench average score from 40.77 to 41.26 at 512 slots per layer while preserving 98.4% of full-cache performance with only 10.5% of the prefill KV cache. Code is available at https://github.com/kimjunewon/SurrogateKV.

Abstract:
Vision-language models (VLMs) have advanced human-AI interaction but struggle with cultural understanding, often misinterpreting symbols, gestures, and artifacts due to biases in predominantly Western-centric training data. In this paper, we construct CultureVerse, a large-scale multimodal benchmark covering 19,682 cultural concepts, 188 countries/regions, 15 cultural topics, and 3 question types, with the aim of characterizing and improving VLMs’ multicultural understanding capabilities. Then, we propose CultureVLM, a series of VLMs fine-tuned on our dataset to achieve significant performance improvement in cultural understanding. Our evaluation of 16 models reveals significant disparities, with a stronger performance in Western concepts and weaker results in African and Asian contexts. Fine-tuning on our CultureVerse training set enhances cultural perception, demonstrating cross-cultural, cross-continent, and cross-dataset generalization without sacrificing performance on models’ general VLM benchmarks. We further present insights on cultural generalization and forgetting. We hope that this work could lay the foundation for more equitable and culturally aware multimodal AI systems.

Abstract:
While Reinforcement Learning from Verifiable Rewards (RLVR) has advanced reasoning in Large Vision-Language Models (LVLMs), prevailing frameworks suffer from a foundational methodological flaw: by distributing identical advantages across all generated tokens, these methods inherently dilute the learning signals essential for optimizing the critical, visually-grounded steps of multimodal reasoning. To bridge this gap, we formulate Token Visual Dependency , quantifying the causal information gain of visual inputs via the Kullback-Leibler (KL) divergence between visual-conditioned and text-only predictive distributions. Revealing that this dependency is highly sparse and semantically pivotal, we introduce Perception-Grounded Policy Optimization (PGPO), which is a novel fine-grained credit assignment framework that dynamically reshapes advantages at the token level. Through a threshold-gated, mass-conserving mechanism, PGPO actively amplifies learning signals for visually-dependent tokens while suppressing gradient noise from linguistic priors. Extensive experiments based on the Qwen2.5-VL series across seven challenging multimodal reasoning benchmarks demonstrate that PGPO boosts models by 18.7% on average. Both theoretical and empirical analyses confirm that PGPO effectively reduces gradient variance, prevents training collapse, and acts as a potent regularizer for robust, perception-grounded multimodal reasoning.

Abstract:
Adaptive context compression is vital for scaling Large Language Models (LLMs) to complex, multi-turn agent tasks. However, rule-based compression methods may discard task-critical nuances, while Reinforcement Learning (RL) approaches usually struggle to balance information retention and token efficiency under the sparse rewards inherent to long-horizon workflows. To bridge this gap, we propose ZipRL, a novel adaptive compression framework tailored for Reinforcement Learning from Verifiable Rewards (RLVR). ZipRL features a multi-granularity compression mechanism for active, non-uniform information reduction, coupled with Hindsight Response Replay (HRR)—a technique designed to densify training signals during RLVR optimization. Theoretically, we prove ZipRL’s superior task-relevant utility over uniform methods. Concretely, ZipRL utilizes coarse-to-fine prompts for macro-compression and incorporates HRR into GRPO via generalized advantage reshaping. Multiple models of varying versions and parameter scales validate the effectiveness of our approach. Benchmarks on five agent tasks show ZipRL outperforms state-of-the-art approaches by 27.9% and 34.7% across Qwen3-4B and Qwen3-8B models, while maintaining exceptional token efficiency and robustness under extreme 256-turn extrapolation stress tests. Our code is available at https://github.com/huzhexin/ZipRL.

Abstract:
Role-playing agents (RPAs) must sustain persona fidelity, natural interaction, and scene progression across multi-turn trajectories. However, supervised fine-tuning, preference optimization, and coarse episode-level RL can over-reinforce safe stylized responses that appear in-character but are repetitive, low-information, or weakly grounded in the evolving scene. We propose LSH-RL, a Long-Short Horizon Reinforcement Learning framework for persona-level RPAs. LSH-RL trains a shared policy through dynamic multi-role self-play, combining short-horizon utterance/action rewards with long-horizon character-trajectory rewards. A gated trajectory-to-turn reward fusion mechanism prevents locally degenerate turns from receiving undeserved long-horizon credit, while a dialogue-action curriculum supports staged optimization. Experiments on PersonaArena and CharacterBox show that LSH-RL improves over SFT and DPO in both dialogue-only and full interaction settings. Further analyses show reduced repetition and stronger human preferences, suggesting a path toward more coherent and interaction-rich persona-level RPAs. The source code is available at https://github.com/Shiwenlong-source/LSH-RL.

Abstract:
Attribution aims to identify which input tokens are responsible for a language model’s predictions, yet its evaluation in NLP remains dominated by discrete Top-k removal tests that poorly reflect global importance and interaction effects. In this work, we reformulate text attribution evaluation as a continuous functional over attribution-induced trajectories and introduce two complementary AUC-based metrics tailored to autoregressive models. Perturbation AUC measures token necessity via progressive embedding-level corruption with a stable logit-based signal, while Recovery AUC evaluates sufficiency by tracking the restoration of internal representations in hidden-state space. Together, these metrics subsume Top- tests and enable principled, ordering-sensitive assessment of attribution quality. Using these diagnostics, we show that most existing attribution methods collapse to near-random rankings under continuous evaluation and fail to capture even first-order interaction effects. Motivated by this failure mode, we propose a mid-layer attribution framework that exploits a layer-wise saturation property of transformer computation to decompose attribution into information allocation and causal effect. Across multiple models and tasks, our method consistently outperforms existing baselines under both Perturbation and Recovery AUC, yielding more faithful and interaction-aware explanations of language model behavior.

Abstract:
Longitudinal electronic health records (EHRs) capture years of patient history across notes, codes, labs, and procedures, and contain evidence needed to reason about likely clinical outcomes. However, comprehensive clinician review of these records is impractical, and LLM-based processing is costly and often unreliable, missing some relevant observations while hallucinating others. We therefore propose EviGen, a three-layer framework for verifiable clinical rationale generation that addresses these challenges. The first layer is a patient-conditioned retriever that uses learnable queries to find evidence predictive of, not just textually relevant to, a clinical outcome and ranks it by prediction attribution scores. The second layer is an LLM generator that consumes this ranked evidence as a scaffold to produce a clinical rationale grounded in the retrieved spans. The third layer is a process-supervised verifier that checks the generated rationale at the reasoning-step level, flagging unreliable claims. Across three medical prediction datasets, EviGen improves prediction performance and rationale faithfulness over full-context LLM and RAG baselines, and is preferred by clinical reviewers in a usability evaluation.

Abstract:
Processing video in vision-language models is expensive: each frame occupies hundreds of tokens, and inference cost scales with every frame and every repeated query. We introduce Frames2LoRA, a method for parametric video internalization. A perceiver hypernetwork reads the intermediate representations produced layer-by-layer as a frozen VLM encodes a video, and generates a Low-Rank Adaptation (LoRA) adapter in a single forward pass. Unlike standard LoRA fine-tuning, which requires iterative gradient updates, Frames2LoRA predicts these weights directly from the video. Trained for SmolVLM2 500M and 2.2B on video summarization and captioning, Frames2LoRA enables the same frozen VLM to answer queries from the adapter alone, with zero visual tokens in its context at query time. Frames2LoRA is statistically non-inferior and equivalent to direct video-in-context inference across all five captioning benchmarks at both model scales, and across seven of eight video question answering benchmark-scale pairings. Although trained only on 12 frames at 384px, it remains stable up to 1,024 frames and 1024px, where direct video-in-context inference often degenerates. Across this sweep, it reduces answer-time visual-token load by up to 1,500x and query TTFT by 6-80x, while preserving video-faithful outputs. We also find that independently generated adapters for non-overlapping video segments can compose in rank space, suggesting a path toward chunked long-video internalization.

Abstract:
We introduce TALIM (Tajik Academic Language Intelligence Measurement), a large-scale academic benchmark for evaluating academic reasoning and language understanding in Tajik. TALIM spans nine subjects and three question formats, multiple-choice, matching, and free-response, sourced natively from the Tajik national curriculum rather than translated from existing benchmarks. We evaluate nine LLMs, both proprietary and open-weight models, finding that proprietary models substantially outperform open-weight models across all subjects and question types. Grammar is the weakest subject across all models, while physics is consistently the strongest. English translation substantially improves open-weight performance while Farsi transliteration yields only marginal gains, indicating that failures stem from data scarcity rather than script unfamiliarity. Chain-of-thought analysis reveals several systematic failure modes: hallucination, reasoning process failures, and grammatical category confusion. We hope TALIM will serve as a resource for assessing Tajik language capabilities of frontier models and motivate similar native benchmark construction efforts for other low-resource languages.

Abstract:
While large language models (LLMs) show immense potential in specialized domains, their adaptation is often hindered by limited access to domain-specific data and the high computational cost of model training. To address these challenges, we propose a lightweight knowledge injection framework. Instead of sharing raw data directly, our approach fine-tunes a small model and transfers its knowledge vectors to the target LLM for knowledge injection, thereby reducing data exposure and computational overhead. To mitigate the knowledge loss incurred during the injection process, we further introduce an iterative knowledge calibration strategy. We theoretically bound the error of the transformation matrix and prove the stability and linear convergence of the calibration process. Comprehensive experiments demonstrate that our method significantly outperforms parameter-efficient baselines and achieves performance comparable to full-parameter fine-tuning, while consuming only 28% of the GPU memory and 3% of the raw training data. Code is available at https://github.com/Yuhanleeee/LiSKI.

Abstract:
Large language models are commonly adapted to downstream tasks through supervised fine-tuning (SFT), but substantial distribution mismatch between downstream supervision and a model’s generation distribution can intensify catastrophic forgetting. Data rewriting offers a data-centric way to narrow this mismatch before SFT. Existing methods, however, typically sample rewrites from a prompt-induced conditional distribution, which need not align with the backbone’s natural question-answering generation distribution, and fixed templates can reduce output diversity. We formulate data rewriting as a policy-learning problem and train a lightweight LoRA rewriting policy with reinforcement learning. The policy optimizes question-answering-style distributional alignment and semantic diversity under a hard task-consistency gate, producing verified supervision for downstream SFT. Across three instruction-tuned backbones, the resulting models attain downstream gains broadly comparable to standard SFT while reducing degradation on non-downstream benchmarks in every evaluated setting. Additional experiments on logical reasoning and medical question answering provide preliminary evidence that a rewriting policy can be reused across domains for the same backbone.

Abstract:
Disasters cause severe societal impacts, demanding rapid coordination of heterogeneous AI tools, from satellite analysis to flood prediction and damage assessment, into coherent multi-step workflows. As LLMs increasingly serve as orchestrators of such pipelines, effective coordination requires more than selecting semantically plausible tools: LLMs must generate interface-valid workflows with correct parameter binding and dependency propagation. We introduce DisasterBench, a benchmark for evaluating structured multi-agent planning over semantically similar but operationally distinct disaster-response tools. To enable step-level failure attribution, we further propose First-Point-of-Failure (FPoF), which localizes the earliest root cause in a predicted workflow, separating primary errors from downstream cascading effects. Our evaluation reveals three findings: planning method effectiveness depends strongly on model capacity; tool mismatch and parameter-binding errors dominate first failures, revealing semantic grounding and interface consistency as distinct bottlenecks; and verbose intermediate reasoning can create instruction clash with structured output requirements, disrupting plan generation. Together, these findings highlight a fundamental gap between semantic reasoning and interface-constrained workflow planning, underscoring the need for planning frameworks that jointly model semantic intent, execution constraints, and workflow consistency. All code, data, and evaluation resources are available at the https://github.com/TamuChen18/DisasterBench_Open.

Abstract:
Few-shot parameter-efficient adaptation of pretrained Audio–Language Models (ALMs) often improves seen-class performance at the cost of unseen-class generalization, leading to the base-to-new trade-off. We study this failure through zero-shot drift in the text embedding space: few-shot tuning can distort inter-class structure and move adapted embeddings away from their pretrained anchors. We therefore propose Subspace Tuning (SubT), an embedding-space adaptation method that combines a geometry-aware shared transformation with anchoring to the zero-shot prototypes. The learned transformation is transferred to unseen classes, with subspace-aware gating to mitigate negative transfer. Across 11 audio benchmarks, SubT achieves strong few-shot generalization while operating directly on precomputed text embeddings without text-encoder backpropagation. Code is available at https://github.com/jhyukjang/SubT.

Abstract:
Personalized Large Language Models (PLLMs) aim to tailor responses to individual users, where a central challenge is preference reasoning: inferring query-relevant preferences from user-related history. Existing benchmarks, however, largely assume that such preference can be retrieved from semantically related history. We study an underexplored but practically important regime, profile-preference conceptual misalignment (PRCM), where observable profile cues and query-specific preferences lie in different concept spaces, making semantic retrieval inconsistent for personalization. We introduce Vibe-Bench, a benchmark with two psychology-grounded tasks, 3,504 personas and 12,239 dialogues, including a manually verified gold test set, and requires cross-concept preference reasoning beyond surface semantic overlap. Experiments with several personalization methods show that current PLLMs largely rely on shallow semantic correlations and fail to acquire robust cross-concept mappings. These findings establish PRCM as a distinct failure regime in PLLMs and position Vibe-Bench as a focused testbed for advancing preference reasoning beyond semantic matching.

Abstract:
Long-document multimodal question answering requires more than retrieving relevant chunks from a large document. Different queries require different evidence behavior. Existing multimodal RAG systems improve evidence access through text chunks, page images, graph links, or heterogeneous document elements, but they often apply a largely query-agnostic evidence-use strategy. We present TAP-RAG, a task-aware policy-controlled RAG framework for long-document multimodal QA. TAP-RAG contains a main controller, the Task-Aware Policy Controller (TAPC), and two policy-guided evidence executors: Task-Aware Query-Guided Flow Diffusion (TA-QFD) and Task-Aware Visual Enhancement (TAVE). For each query, TAPC predicts the task prior, estimates visual/local/global evidence signals, and produces an executable policy. TA-QFD then expands textual and structural evidence over the multimodal document graph, while TAVE selectively inspects page images when visual or layout evidence is needed. A guarded synthesis stage fuses text, visual, and structural evidence and abstains when support is insufficient. On DocBench and MMLongBench-Doc, TAP-RAG achieves the best overall accuracy among the compared systems, improving over a matched multimodal-RAG baseline by +9.1 points (61.1 to 70.2) and +4.5 points (42.2 to 46.7), respectively.

Abstract:
Recent multimodal search agents use reasoning, vision, and live web search across many turns to answer questions beyond parametric knowledge by orchestrating a small library of hand-crafted search and visual tools. However, the library is laborious to create and maintain, and frozen when new query types, evidence formats, and failure modes are encountered in the wild. Human practitioners, by contrast, cope with insufficient tools by fashioning new ones from familiar parts and reusing them across tasks. Inspired by this, we propose ToolSmith, an automatic tool-generation framework that grows and reuses modules on top of a lean base tool library. We evaluate ToolSmith on three in-domain multimodal search benchmarks and ten out-of-domain benchmarks under a VLM-as-a-Judge protocol. ToolSmith consistently improves over the base library across deployed agents of varying scales, with larger backbones reaping greater gains, demonstrating the scalability of the approach. Our code and data are available at https://threesr.github.io/toolsmith/.

Abstract:
RL-trained search agents interleave retrieval with reasoning to answer knowledge-intensive questions, achieving strong end-to-end accuracy on multi-hop QA benchmarks. Yet end-to-end accuracy conflates two distinct abilities: searching for relevant documents and reading what they contain. Existing evaluations do not measure whether the agent reads the documents it retrieves. We design a controlled evaluation that force-injects gold-bearing documents into each agent’s retrieval pipeline, isolating reading from retrieval. Across five RL search agents on three multi-hop QA benchmarks, up to 46.2% of samples are answered incorrectly even when the gold answer is literally present in the retrieved documents—retrieval succeeds but reading fails. We term this phenomenon Retrieval Myopia (RM). The agent’s interaction format is itself a substantial cause: simply re-presenting the same documents to the same model in a plain QA format, with no weight updates, recovers up to 48.9% of RM cases. Guided by this diagnosis, we propose CWFT (Confidence-Weighted Format Transfer), a training-free pipeline that anchors the agent’s prediction against k confidence-weighted clean-format re-reads. CWFT consistently improves accuracy across all evaluated system–dataset configurations on three multi-hop QA benchmarks.

Abstract:
Scientific peer review generation has attracted increasing attention for reducing reviewing burdens and providing timely feedback. However, existing Large Language Model (LLM)-based methods often produce generic comments with insufficient evidence support and weak source traceability, while complex multi-agent systems incur high inference costs. To address these challenges, we propose EGTR-Review, an Evidence-Grounded and Traceable Review Generation framework via Multi-Agent Teacher Distillation. EGTR-Review first constructs a multi-agent teacher that performs structure-aware paper decomposition, key-element extraction, external scholarly evidence retrieval, evidence-state labeling, verification reasoning, and review synthesis. It then distills both intermediate reasoning trajectories and final review comments into a lightweight student model through task-prefix-driven multi-task learning. An evidence-weighted objective further reduces the influence of weak, missing, or non-verifiable supervision. Experiments on public peer-review datasets show that EGTR-Review (Student) outperforms strong prompt-based, fine-tuned, and structured/agentic baselines across automatic metrics, LLM-as-Judge evaluation, and human evaluation, while maintaining strong factual grounding and source traceability with substantially lower token consumption and inference time. Our code, prompts, configurations, and sample data are available on GitHub.

Abstract:
Retrieval-augmented generation (RAG) remains challenging for multi-hop question answering when supporting evidence is distributed across heterogeneous sources, including text, tables, and knowledge graphs (KGs). Existing systems often rely on modality-specific retrieval pipelines or uncontrolled iterative search, making it difficult to preserve source structure while maintaining efficient evidence acquisition. We present RELOOP, a structure-aware retrieval framework built on Hierarchical Sequence (HSEQ), a reversible representation that maps heterogeneous sources into provenance-preserving typed segments with lightweight structural links. Operating on this unified evidence interface, a lightweight iterator selects and expands candidate evidence through structure-respecting actions, while an explicit sufficiency signal determines when the accumulated evidence is adequate for answering. A head module then synthesizes the final answer from a compact, canonicalized evidence package, with an optional refinement step when inconsistent evidence is detected. Experiments on HotpotQA, HybridQA, TAT-QA, and MetaQA show that RELOOP consistently improves answer quality over strong single-pass and iterative RAG baselines while requiring substantially fewer retrieval steps than graph-expansion approaches. These results suggest that a reversible, structure-aware evidence interface provides an effective foundation for accurate and budget-controlled retrieval across heterogeneous QA settings.

Abstract:
Multi-agent debate (MAD) improves the reasoning capabilities of large language models by having multiple agents iteratively refine their responses through discussion. However, MAD suffers from a critical vulnerability known as shared misconception: when a majority of agents initially converge on an incorrect answer, the debate process tends to amplify rather than correct the error. Existing methods primarily address peer skew but leave the agents’ inherently biased concept priors unaddressed. To mitigate this systematic weakness, we propose R 2 -MAD (Remember and Reweight for Multi-Agent Debate), a framework that equips agents with an experience memory accumulated from past debates. R 2 -MAD intervenes on both failure modes through two complementary mechanisms: A debate-state-aware retrieval policy dynamically calibrates the concept prior by retrieving relevant historical evidence based on the current consensus level. Then these retrieved experiences provide a basis for estimating per-agent reliability, yielding confidence weights to modulate peer influence. Experiments on various benchmarks show that R 2 -MAD achieves consistent improvements over existing single-agent and MAD baselines.

Abstract:
Discrete events such as holidays, policy announcements, and public gatherings leave no trace in historical time series yet fundamentally shape future demand. Language models capture precisely this event knowledge, making them a natural source of auxiliary signal for event-driven forecasting. However, some multimodal forecasters borrow semantic alignment from vision-language pretraining, enforcing geometric proximity between text and series embeddings. For time series regression, this objective is theoretically unnecessary and actively harmful when training data is limited or text-series coupling is weak: contrastive gradients compress the high-variance series features required to represent event-induced spikes. We propose predictive grounding as the principled alternative, shaping language representations through prediction loss alone. PROSE (Predictive grOunding for event-driven time-Series forEcasting) realizes this framework through a fully fine-tuned language encoder, token-level Event Token Injection, LLM-generated Trajectory Reasoning Anchors as global prefix tokens, and a Soft Functional Alignment loss. On a 464K-sample Chinese holiday benchmark, PROSE surpasses the strongest baseline by 10.7% WMAPE; on Time-MMD, it achieves state-of-the-art results across 7 of 9 domains.

Abstract:
Training effective Text-to-SQL models remains challenging due to the scarcity of high-quality, diverse, and structurally complex datasets. Existing methods either rely on limited human-annotated corpora, or synthesize datasets directly by simply prompting LLMs without explicit control over SQL structures, often resulting in limited structural diversity and complexity. To address this, we introduce EvolSQL, a structure-aware data synthesis framework that evolves SQL queries from seed data into richer and more semantically diverse forms. EvolSQL starts with an exploratory Query-SQL expansion to broaden question diversity and improve schema coverage, and then applies an adaptive directional evolution strategy using six atomic transformation operators derived from the SQL Abstract Syntax Tree to progressively increase query complexity across relational, predicate, aggregation, and nesting dimensions. An execution-grounded SQL refinement module and schema-aware deduplication further ensure the creation of high-quality, structurally diverse mapping pairs. Experimental results show that a 7B model fine-tuned on our data outperforms one trained on the much larger SynSQL dataset using only 1/18 of the data. Our code is available at https://github.com/slaanurgle/EvolSQL.

Abstract:
Recent advances in block diffusion language models have demonstrated competitive performance and strong scalability on reasoning tasks. However, their test-time compute allocation remains largely unexplored, leaving a critical speed-effectiveness trade-off unresolved in long Chain-of-Thought reasoning. To address this, we propose a unified test-time compute allocation framework that introduces adaptivity in both step-wise decoding and blockwise generation. At the decoding level, we propose Bounded Adaptive Confidence Decoding (BACD), a difficulty-aware sampling strategy that dynamically adjusts denoising based on model confidence, accelerating inference while controlling error accumulation. Beyond step-wise adaptivity, we introduce the Think Coarse, Critic Fine (TCCF) paradigm that allocates large block sizes for exploratory thinking and smaller block sizes for precise refinement. To stabilize training under varying block configurations, we adopt Progressive Block Size Extension, which mitigates quality degradation when scaling up block sizes. Extensive evaluations on six benchmarks show that our TDAR-8B model with BACD and TCCF achieves a 2.38 speedup and +3.4% average accuracy over the strong TraDo-8B baseline, unlocking the potential of block diffusion in complex reasoning.

Abstract:
Molecular property prediction is fundamental to drug discovery and materials science, yet existing models suffer from limited interpretability and poor cross-task generalization. Without explainable chemical rationales, accurate predictions alone are insufficient for guiding costly downstream experiments. We propose MppReasoner , a multimodal large language model that incorporates verifiable chemical reasoning for molecular property prediction. Built on Qwen2.5-VL-7B-Instruct, MppReasoner integrates 2D molecular images with SMILES strings through a two-stage training strategy: supervised fine-tuning (SFT) on 16,000 expert-guided reasoning trajectories, followed by Reinforcement Learning from Principle-Guided Rewards (RLPGR). RLPGR applies rule-based rewards verified by RDKit to evaluate chemical principle usage, structural analysis, and logical consistency, providing objective feedback without human preference labels. Experiments across 8 benchmark datasets show that MppReasoner outperforms the strongest baseline by 2.95% and 5.49% on in-distribution and out-of-distribution tasks, respectively. Beyond predictive accuracy, MppReasoner generates chemically grounded reasoning traces, with expert evaluation confirming high reasoning quality across three assessed dimensions. Code is available at https://github.com/Jesse-zjx/MPPReasoner.

Abstract:
Aligning Large Language Models (LLMs) requires simultaneously satisfying Helpfulness, Harmlessness, and Honesty (HHH). Existing works-including SFT, MoEs, RLHF, and orchestrator-based steering-operate on standard 1-to-1 transformer architectures and show systematic trade-offs across objectives. While such works can steer behaviour post hoc, they remain fundamentally additive and therefore suffer from catastrophic forgetting. While recent mechanistic interpretability have helped characterise this behaviour, they expose a deeper structural failure-transformers lack an mechanism to negotiate incompatible objective signals, resulting in a mechanistic blind-spot. Therefore, we test the hypothesis that these failures are architectural-driven rather than optimisation-driven via NegotiaX, a 1-to-N transformer architecture that separates and reconciles alignment objectives via two modules. Module I (Transformer Module) allocates independent representational pathways to each objective-mitigating catastrophic forgetting. Module II (Mechanistic Alignment Circuit (MAC)), embeds objectives into dedicated subspaces and performs a learned negotiative operation prior to decoding-mitigating mechanistic blind-spot. Across four 7B backbones, NegotiaX remains competitive with SOTA-improving Avg by up to 6.3%, Helpfulness by 6.2%, Honesty by 5.6%, and Harmlessness by 4.7%.

Abstract:
We investigate how model scale shapes lexical acquisition in LLMs, introducing trajectory comparison as a methodological tool and three new convergence metrics. Tracking surprisal dynamics across four Pythia model sizes and 153 training checkpoints, we report two main findings. First, scale drives a global rise in inter-word surprisal coherence: synonym curves become more correlated at larger scales, but baseline correction reveals that the advantage of semantic relatedness over random pairs is invariant with scale, indicating that what scales is broad inter-word coherence rather than semantic abstraction specifically. Second, learning dynamics are shaped primarily by training data rather than by model capacity: surprisal trajectories are highly similar across model sizes, and our new metrics confirm that larger models converge faster but not along qualitatively different paths. Together, these results call for baseline-corrected analyses and longitudinal evaluation when assessing lexical competence in LLMs.

Abstract:
Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM recommenders may consume web content that Generative Engine Optimization (GEO) operators have polluted to mislead them. We ask: to what extent do they become unwitting promoters of fake products? We introduce FORGE (Fake Online Recommendations in Generative Environments), which locally rewrites real products in a frozen set of retrieved web pages into fake ones and measures how often the LLM recommends the fake product, across 225 real products in 15 categories and 5 consumer scenarios. Across 12 commercial and open-weights LLMs, all models are vulnerable: a single polluted page yields fooled rates of up to 27%, while the full top-3 replacement raises this to 73.8%. Vulnerability varies across categories, increasing when models lack stable prior knowledge of the products. Reasoning does not mitigate this vulnerability; instead, it often generates spurious social proof to justify false recommendations. None of the four defenses is adequate: the skepticism prompt can exacerbate vulnerability much like reasoning, the two consensus filters risk suppressing legitimate products, and credibility re-ranking helps every model but removes only a sixth of the fakes. We release the FORGE benchmark and the evaluation code at https://github.com/leoluolol/forge-benchmark.

Abstract:
The underperformance of existing multimodal large language models for time series reasoning lies in the absence of rationale priors that connect temporal observations to their downstream outcomes, which leads models to rely on superficial pattern matching rather than principled reasoning. We therefore propose the rationale-grounded in-context learning for time series reasoning, where rationales work as guiding reasoning units rather than post-hoc explanations, and develop the RationaleTS method. Specifically, we firstly induce label-conditioned rationales, composed of reasoning paths from observable evidence to the potential outcomes. Then, we design the hybrid retrieval by balancing temporal patterns and semantic contexts to retrieve correlated rationale priors for the final in-context inference on new samples. We conduct extensive experiments to demonstrate the effectiveness and efficiency of our proposed RationaleTS on three-domain time series reasoning tasks. We provide the source code in https://github.com/yuppielqx/RationaleTS.

Abstract:
Mobile agents on personal smartphones can affect private data, payments, settings, and the foreground screen. We present MATE (Mobile Agent Trustworthy Execution), an execution-layer system that controls the boundary between agent action proposals and device effects without modifying the underlying model. MATE represents each UI primitive as a structured action record that combines the instruction, target app, screen context, primitive, and action fields. Runtime Guardrail (RG) maps this record to an allow, block, or user-approval decision; Predefined Task Routing (PTR) serves recurring requests through verified routes with post-execution screen checks; and a floating interface keeps execution in a compact overlay. On MobileSafetyBench, MATE raises high-risk blocking from 29.7% to 97.0% under GPT-4o. On AndroidWorld, it reduces LLM calls by up to 6 and raises recurring-use task success to 95.7%.

Abstract:
Standard accuracy benchmarks evaluate whether large language models (LLMs) reach correct answers. However, they do not test whether models maintain that answer when challenged by a plausible counter-argument. We introduce a controlled protocol for evaluating answer stability: after a model answers a multiple-choice question correctly, we challenge the model’s answer with a coherent argument for an incorrect option and measure whether the model flips. The setup a) isolates argumentative content from overt social pressure and b) varies argument length, self-attribution, and cross-model source. Across seven frontier models and 57 MMLU subjects, flip rates range from 17.5% to 97.3%, revealing large differences in stability that are not captured by accuracy metrics alone. We find that self-attribution consistently increases flip rates (mean +7.1pp, up to +18.7pp). Furthermore, pooling wrong-answer arguments across models and selecting the most effective one per question yields stronger adversarial challenges than relying on any single source model. From this cross-model pool, we construct MaxFlip, a curated benchmark that amplifies answer flips by up to +23.6pp over self-generated challenges. We release the protocol, challenge records, and MaxFlip to support stability evaluation alongside standard accuracy benchmarks. Materials are available at https://github.com/nafisenik/WhoFlips, https://hf.co/datasets/nafisehNik/WhoFlips.

Abstract:
Knowledge editing for large language models typically treats a fact update as a replacement in which a new object overwrites the old. Many real-world facts, however, evolve through sequences: the same subject–relation pair maps to different objects across disjoint intervals, each correct within its validity window. Existing temporal-editing benchmarks probe latest-value or explicit-time recall, so they cannot determine whether an edited model retains a coherent timeline. We introduce TKE-Bench, a diagnostic benchmark that operationalizes temporal accessibility along three reasoning-oriented axes with paraphrased variants. On TKE-Bench, all evaluated editing baselines collapse on chronologically structured queries, revealing catastrophic temporal interference (CTI). We trace CTI to a geometric property implicit in locate-and-edit methods: mid-layer subject-token keys are nearly collinear across time slices of the same subject, so stacked updates write along one direction and the latest edit overwrites every earlier value. Building on this, we propose ChronoExpert, a temporal mixture-of-experts that stores each time slice in an independent lightweight expert and activates only the matching one at inference, leaving the base model otherwise untouched. Across multiple LLM backbones, ChronoExpert consistently outperforms the strongest editing baselines and stays robust as chains grow longer.

Abstract:
In this work, we investigate whether data annotation can be automated using state-of-the-art large language models (LLMs), while maintaining high consistency and reliability. We propose a multi-model, multi-run framework in which several LLMs act as annotators and their outputs are aggregated using a strict consensus criterion to improve annotation robustness and reduce model-specific biases. The proposed approach is evaluated on a multilingual sentiment analysis task using lead sections of Wikipedia articles covering diverse topics in five languages. The resulting annotations are assessed using intra-model consistency, cross-model agreement, and standard inter-annotator agreement metrics. Additionally, we conducted human evaluation. The described framework enables scalable and reproducible data annotation, and the resulting dataset is publicly released to support further research.

Abstract:
Structured pruning of large language models provides a practical route to inference acceleration on standard hardware, but most existing methods rely on neuron importance criteria that compress activation behavior into static scalar scores, discarding the sequential structure of activations. We propose TADP (Temporal Activation Dynamics Pruning), a structured pruning method that models neuron importance along the temporal dimension of activations by jointly capturing mean activation, peak activation, positional variance, transition dynamics, and position-weighted response within a unified scoring framework. TADP requires no gradient computation or weight updates and performs pruning in a single forward pass over a calibration dataset, making it zero-shot and computationally lightweight. We evaluate TADP on eight models from the LLaMA, Mistral, Qwen, and Minerva families at 3B and 7B/8B scale, under pruning levels from 10% to 50%, measuring perplexity on WikiText-2 and zero-shot accuracy on ARC, HellaSwag, MMLU, and PIQA.

Abstract:
When LLMs are given contradictory passages, the generated response may appear decisive while masking how internal signals behave under conflicting evidence. Does it favor one passage, blend both, or ignore both? We study model behavior under inter-context contradiction by introducing a white-box tracing framework that extracts internal model signals based on hidden-state shifts, counterfactual attribution via passage removal, and token-level uncertainty during generation under two single-passage prompts and one dual-passage prompt with conflicting evidence. We evaluate four open-weight LLMs from the LLaMA (8B and 70B), OLMo, and Qwen on WikiContradict, a human-annotated QA benchmark with contradictory passages. Each dual-passage answer is labeled as committing to passage A or passage B, blending both, or fabricating an answer unsupported by either passage. Comparing these labels with the extracted signals shows that larger models reduce fabrication, but hidden-state and probability-level signals can disagree with the final answer. Moreover, fabricated answers often remain high-confidence, showing that confidence alone is unreliable for identifying ungrounded behavior under contradictory evidence. We further validate on ConflictBank, a larger misinformation-conflict QA dataset, showing consistent disagreement patterns across models. Finally, we test whether pre-generation white-box signals are informative beyond interpretation by training classifiers to predict output behavior. Classifiers using white-box signals outperform probability baselines, showing these signals contain diagnostic information about model behavior under contradictory evidence beyond log probabilities.

Abstract:
Human annotators typically assess translation quality by considering the broader document context, which allows them to detect errors arising from discourse-level phenomena that may be missed when evaluating text segments in isolation. However, the role of document-level context in automatic Machine Translation (MT) evaluation remains underexplored. In this work, we present a controlled study of the impact of document context on LLM-based MT evaluation and its interaction with the availability of manually curated references. We consider several evaluation configurations, also exploring how performance varies when enabling the thinking inference mode and conducting task-specific fine-tuning. Our results show that the effect of document context differs substantially across test sets, and that input repetition itself can account for part of the apparent gains. Fine-tuning improves performance consistently and offers larger gains when document context is available, suggesting that task-specific adaptation may help models to handle document-level context more effectively.

Abstract:
Encoding graphs as text prompts enables LLM-based graph reasoning. However, effective graph-to-text encodings depend on the downstream task, graph structure, and target LLM, making manual format design time-consuming and brittle. We propose EvoFormat, a black-box evolutionary search framework that automatically discovers interpretable graph-to-text formats represented as human-readable formatting rules over node naming, relation expression, and global arrangement. Using only task-level feedback from LLM outputs, EvoFormat iteratively mutates and selects rules without gradients or internal model access. Across synthetic GraphQA and IMDB-MULTI-derived structural QA tasks over unattributed graphs, EvoFormat outperforms strong hand-crafted encodings in most settings.

Abstract:
Tool-use hallucination has become a critical obstacle to agent deployment. It can emerge and propagate across judgment, planning, and execution. However, existing benchmarks mainly focus on final task completion or individual tool-use capabilities, making fine-grained failure attribution difficult. We propose Agent-Trident (TRIDENT), a three-level progressive diagnostic benchmark for tool-use hallucination. TRIDENT decomposes the agent tool-use process into judgment, planning, and execution, supporting stage-level attribution of tool-use failures. The dataset contains 1,120 samples across 20 subdomains and 7 task structures. We further propose the Cross-level Composite Metric (CCM), a bottleneck-sensitive aggregate metric that measures models’ overall reliability across the tool-use chain. Evaluating 29 mainstream large language models (LLMs), we find that even the best model reaches only 42.79% CCM, indicating that current LLM agents remain far from reliable tool use. Our analysis further identifies model behaviors and stage-level differences overlooked by conventional evaluations. These results show that TRIDENT complements final-outcome evaluation with stage-level diagnosis, revealing reliability limitations in solvability judgment, tool-use planning, and dynamic execution while supporting agent model selection, error analysis, and safe deployment.

Abstract:
Code verifiers play a critical role in post-verification for LLM-based code generation, yet existing supervised fine-tuning methods suffer from data scarcity, high failure rates, and poor inference efficiency. While reinforcement learning (RL) offers a promising alternative by optimizing models through execution-driven rewards without labeled supervision, our preliminary results show that naive RL with only functionality rewards fails to generate effective unit tests for difficult branches and samples. We first theoretically analyze showing that branch coverage, sample difficulty, syntactic and functional correctness can be jointly modeled as RL rewards, where optimizing these signals can improve the reliability of unit-test–based verification. Guided by this analysis, we design syntax- and functionality-aware rewards and further propose branch- and sample-difficulty–aware RL using exponential reward shaping and static analysis metrics. With this formulation, CVeDRL achieves performance with only 0.6B parameters, outperforming GPT-4o by up to 17.55% in pass rate with over 20 faster inference than competitive baselines.

Abstract:
Large Language Model (LLM) personalization aims to adapt generation to individual users by capturing implicit preferences from historical interactions. Steering-based methods offer an inference-efficient solution, but existing work follows a difference-based paradigm that estimates personalized steering directions from representation offsets between contrastive historical examples, thus entangling inherent user preference with preference-agnostic contextual bias. This limitation arises because offset-based estimation uses raw contrastive variation, making the derived direction confounded by contextual bias rather than identifying preference-discriminative variation between user-conditioned and generic behavior. To address this, we propose Disentangled Personal Direction Discovery (DiPer) , which formulates personalized steering vector estimation as a discriminative direction discovery problem. DiPer learns layer-wise user-specific directions through supervised metric learning, based on Recursive Feature Machine (CITATION), over historical activations, to discover preference-discriminative directions while suppressing contextual bias. Theoretical analysis and extensive experiments demonstrate DiPer’s efficacy and robustness.

Abstract:
RL post-training strategies are dataset-dependent and reveal a recurring empirical pattern: capacity parameters accumulate monotonically across stages, while regularization parameters predominantly oscillate in response to shifting training dynamics. This distinction highlights a potential flaw in fixed training schedules: by forcing all parameters along rigid paths, they fail to capture the dynamic exploration-exploitation tradeoffs that regularization must track. We uncover this through LLMZero, an agentic system that optimizes training trajectories via tree search by diagnosing pathologies at each checkpoint and proposing coordinated multi-parameter transitions. Across four diverse GRPO tasks, LLMZero discovers strategies that improve over the base model by 9% to 140% and over grid search by 6% to 15% (relative), consistently outperforming random search and a skill-based agent under a matched compute budget. The capacity–regularization asymmetry is consistent across all four tasks, offering a candidate design heuristic for multi-stage training.

Abstract:
Question answering (QA) is a core challenge in AI, particularly for complex queries requiring multi-hop reasoning across documents, or symbolic operations like aggregation or exhaustive listing. Retrieval-augmented generation has become the dominant approach to QA, with recent graph-based variants addressing part of these issues by organizing knowledge to better support compositional questions. However, most textual graph-based RAG methods still lack the structure needed for symbolic operations useful to answer complex questions reliably. This motivates symbolic graph-based approaches, which extract knowledge graphs (KGs) whose relations are logic predicates that enable SQL-like querying. Yet these pipelines typically use LLMs for KG extraction, which can introduce consistency issues, where extracted facts may violate commonsense ontology constraints. We propose a neuro-symbolic framework for ontology-grounded KG construction combining open-domain extraction, embedding-based canonicalization of types and predicates, and targeted LLM-based correction of ontology violations. By deferring corrections to a post-extraction stage, our method avoids repeated LLM calls, substantially reducing token usage while improving KG consistency and preserving downstream QA quality. Finally, we show that the extracted KGs are well suited for symbolic querying by measuring the occurrence of SPARQL graph patterns.

Abstract:
Reward-aware variants of Direct Preference Optimization (DPO), such as ODPO and MODPO, implicitly assume linear marginal utility over the preference signal: identical signal gaps induce identical gradient updates regardless of whether the pair lies in a low- or high-quality region. We propose FreeMU, a lightweight plug-and-play training module that introduces non-linear marginal utility through a pair-aware divided-difference weight applied directly to the preference margin. The same weight formula integrates with three representative DPO variants—ODPO, MODPO, and β-DPO—with each instantiation differing only in the reward source and its normalisation. We first instantiate FreeMU on the two explicit-score variants (Likert labels for ODPO and reward-model outputs for MODPO) and then, for variants without externally annotated scores, on β-DPO using the implicit Bradley–Terry reward induced by the policy and reference models. Across summarization, multi-objective safety alignment, and dialogue, FreeMU consistently improves over its host in LLM-based win rates and task-specific metrics with near-zero training overhead; a controlled ablation confirms that the convex curvature direction is the operative ingredient—reversing the sign to a concave utility removes the gain.

Abstract:
Large Language Models (LLMs) have shown remarkable promise in translating and reformulating complex mathematical optimization problems across modeling languages. However, validating such transformations through empirical solver executions alone is unreliable, as solver outcomes may be affected by local minima, structural timeouts, numerical artifacts, and subtle semantic divergence between formulations. We introduce SOVER, an LLM-assisted SMT framework that separates semantic mapping from formal certification: Z3 checks domain cross-feasibility and global objective order preservation for mixed-integer linear formulations, while dReal provides tolerance aware feasibility/range and ε-argmin checks for continuous nonlinear formulations. We also introduce NLEQUIV-150, a public benchmark of 100 equivalent and 50 deliberately hard non-equivalent nonlinear reformulation pairs. With LLM-extracted mappings, SOVER classifies 149/150 pairs (99.33%) correctly, including all 50 hard negatives; the sole error is an incomplete mapping extraction.

Abstract:
Current frontier LLMs can theoretically process long contexts with 1M tokens or more. But to what extent can they go beyond simple retrieval and perform deeper reasoning over such long contexts? We empirically investigate long-horizon reasoning capabilities of LLMs, focusing on deductive logic expressed in Prolog. We construct ProloNg, a synthetic testbed to probe Prolog Long Reasoning, which systematically varies the complexity (reasoning depth) of problems, where the hardest case has a reasoning depth of 22 and 62k context length. We study 8 reasoning models across 5 families of frontier LLMs, and find that performance degrades substantially as reasoning depth grows, with the majority of models approaching chance beyond depth 10.

Abstract:
The application of Item Response Theory (IRT) for data selection in benchmarks has mostly been limited to monolingual settings. In this work, we ask: how does IRT generalize when the same benchmark is translated across languages? We conduct experiments on Multilingual GSM and Global MMLU, covering over 100 language models and 11 languages. Fitting 1PL, 2PL, and 3PL IRT models per language, we assess cross-lingual consistency of parameters and whether IRT-filtered subsets preserve model rankings across languages: item difficulty emerges as the most stable parameter followed by guessing and discrimination. A differential item analysis also shows that lack of stability can be tracked to specific items shifting under translation across languages. Our findings suggest IRT-based selection in multilingual settings must be both parameter-aware and item-aware, validating model choice and screening individual items before assuming cross-lingual generalization.

Abstract:
Vision-language models (VLMs) demonstrate strong performance across compositional reasoning benchmarks, which require reasoning over semantic perturbations of objects, attributes, relations, and their interactions. However, our controlled analysis reveals that existing compositionality-aware VLMs exhibit element-specific biases, often underperforming vanilla CLIP on certain compositional elements. To address this, we propose Compositional Scene Graph-guided CLIP (CS-CLIP), which uses scene graphs to identify compositional elements and construct structured negatives via selective masking. We further retain negatives that are most contradictory to the original caption, forcing the model to rely on compositional structure rather than surface cues. CS-CLIP achieves state-of-the-art compositional reasoning with robust performance across compositional elements. It also preserves general vision-language capabilities such as cross-modal retrieval and downstream visual reasoning, while requiring fewer training samples than prior methods.

Abstract:
Cancer-related discussions on social media provide important spaces for information exchange and peer support, but can also expose users to misinformation with implications for prevention, screening, and treatment decisions. Existing work often treats cancer misinformation as a binary phenomenon, providing limited insight into how misinformation is expressed, engaged with, and associated with potential harm. We introduce a multi-dimensional taxonomy for characterizing cancer misinformation in Reddit discussions of breast, lung, colon, and prostate cancer. Developed through expert annotation, the taxonomy captures seven dimensions spanning misinformation presence, cancer stage, information-seeking and sharing behavior, misinformation type, risk, stance, and topical focus. We evaluate 21 large language models (LLMs) across zero- and few-shot settings and develop a cross-model agreement strategy for scaling misinformation identification to over 133K posts. Our analysis shows that misinformation is heterogeneous in both form and function: unproven and alternative treatments emerge as a prominent topic, misinformation frequently occurs within exchanges that combine information seeking and sharing, and users engage with questionable claims through both endorsement and uncertainty. We further find that classification difficulty varies substantially across dimensions, with risk assessment posing particular challenges for both human annotators and LLMs. Our taxonomy and empirical findings move beyond binary detection toward a more nuanced characterization of how cancer misinformation is produced, discussed, and encountered in online health communities.

Abstract:
Noisy long-tailed text classification is particularly challenging because mislabeled examples and clean tail-class examples often exhibit similarly high losses, despite requiring fundamentally different treatments. Consequently, conventional sample-selection methods may mistakenly discard informative minority-class instances, while uniform multi-teacher supervision can propagate conflicting learning signals. We propose GRIP (Gated Routing of Instance-level Preferences), a dual-teacher dual-gate framework that performs instance-level selective knowledge distillation. A noise-specialized teacher provides robust supervision for unreliable samples, whereas a tail-specialized teacher strengthens learning for under-represented classes. Two lightweight and independent gates use sample-level reliability, uncertainty, teacher-disagreement, and class-rarity signals to determine whether each teacher should supervise a given instance. The gates are optimized with reward-shaped weighted binary cross-entropy objectives, and the student combines the standard classification loss with selectively activated distillation losses. Experiments on eight text-classification benchmarks covering real-world and synthetic label noise, three noise types, and varying imbalance factors show that GRIP achieves the best performance across all reported synthetic noisy long-tailed settings and substantial gains on real-world noisy datasets, including a 9.76-point accuracy improvement on TREC over the strongest baseline. Ablation and routing analyses further show that the two teachers and gates learn complementary, non-redundant roles. Only the student is retained at inference time, preserving the model size and latency of a standard BERT classifier.

Abstract:
Large language models are widely used for code generation, yet they rely on an implicit assumption that the task descriptions are sufficiently detailed and well-formed. However, in practice, users may provide defective descriptions, which can have a strong effect on code correctness. To address this issue, we conduct a large-scale empirical analysis evaluating the robustness of 10 code LLMs across 3 benchmarks under three natural language defect types: Lexical Vagueness (LV), Under-Specification (US), and Syntax/Formatting (SF). Our results show that defect impact is not uniform: Under-Specification causes Pass@1 drops of up to 15.3% with no model immune, including state-of-the-art reasoning models; Lexical Vagueness produces moderate, benchmark-dependent degradation; and surface formatting noise has a negligible effect. Crucially, robustness is shaped more by the properties of the task description than by the size or architecture of the model, in contrast to prior perturbation studies reporting uniform degradation across models. Second, to address these defects before they reach the model, we develop SpecValidator, a lightweight LoRA-fine-tuned classifier based on a 1.5B code model, achieving F1 = 0.84 and MCC = 0.79, significantly outperforming few-shot GPT-5-mini (F1 = 0.43) and Claude-Sonnet-4 (F1 = 0.56), and demonstrating that strong code generation ability does not transfer to prompt meta-reasoning. Perhaps more importantly, our analysis indicates that SpecValidator can generalize to unseen issues and detect unknown Under-Specification defects in the original (real) descriptions of the benchmarks used.

Abstract:
Long-term memory for an LLM agent requires more than retrieving a relevant passage. Many existing systems address belief revision, causal connections and cross-domain abstraction primarily through storage and retrieval mechanisms, limiting their ability to support implicit personalisation as a user’s preferences evolve. We present DPCM, which organises memory into a capability hierarchy spanning raw inputs, atomic facts, belief trajectories, identity, domain schemas, prospective intentions and cross-domain patterns. A synchronous daytime writer (SYSTEM 1) records revisions as doubly linked supersedes chains; an asynchronous nighttime engine (SYSTEM 2) induces schemas and intentions, then uses cross-domain collisions to derive core schemas. Dual-process theory motivates this separation between online encoding and offline abstraction without implying that agent memory reproduces human cognition. Across single-run evaluations on LongMemEval, PersonaMem and PersonaMem-v2, SYSTEM 2 yields its largest observed gain on implicit cross-session inference (+5.20 on PersonaMem-v2) and its smallest on the recall-oriented LongMemEval aggregate (+0.33). When the inference budget is limited to 8k tokens and the full history no longer fits, DPCMfull exceeds the long-context baseline by 10.5–17.9 points across both backbones.

Abstract:
Knowledge editing seeks to update language models without full retraining. While most prior work focuses on isolated factual or instance-level edits, we study a more structured setting: mathematical rules as a controlled stress test for abstract symbolic rule editing. We introduce RULE-EDIT, the first benchmark explicitly designed for editing and evaluating rule-level knowledge in LLMs. Our benchmark investigates two key questions: (1) whether edits to abstract rules propagate to derived instances, and (2) whether edits remain consistent across different representations of the same rule. To evaluate these aspects, we propose two new metrics: Instance Portability and Rule Understanding. Experiments on strong open-source LLMs with representative editing methods show that, although models can often overwrite formula-level knowledge, they struggle to transfer edits to rule-derived instances and to maintain consistency across symbolic and natural-language forms. For example, several methods achieve nearly 100% reliability on direct rule queries, while Instance Portability never exceeds 52% and Rule Understanding remains below 26%. Overall, RULE-EDIT provides a controlled benchmark for studying whether current knowledge editing approaches truly modify abstract symbolic rules in LLMs, rather than merely altering surface outputs.

Abstract:
This paper addresses the challenge of generating high-quality academic charts that match the visual standards of human-authored papers. While existing AI agents can produce well-structured text and code, their generated visualizations often lack the stylistic and semantic fidelity of human designs. Advanced coding agents that employ self-reflection mechanisms exhibit poor visual reasoning and limited reflection following, resulting in sparse reward signals that severely undermine their reinforcement learning (RL). We propose ViCo, a training framework for visual-oriented coding that employs iterative reflections to align generated chart images progressively with the reference. We first introduce a self-supervised warm-up stage , which augments Monte Carlo Tree Search with consistency-based pruning to synthesize high-quality reflection trajectories, ensuring that each coding step strictly follows the outcomes of prior reflections. A multi-step RL algorithm is then developed on counterfactual baselines to estimate advantage for reflection and action steps within each refinement cycle, thereby addressing the reward sparsity problem. To enable efficient reward in massive training, we propose an automatic, multifaceted evaluation framework that assesses charts’ style, layout, and semantic consistency via a hierarchical heterogeneous layout graph structure. Experiments on three public benchmarks demonstrate that ViCo, trained on an 8B model, achieves performance close to commercial LLMs with adequate reflection capabilities.

Abstract:
Extreme low-bit quantization-aware training (QAT) is often unstable due to synchronized bin transitions, which can trigger catastrophic loss spikes. While prior works rely on scale-side controls to mitigate this fragility, we demonstrate that restricting pivot movement alone is insufficient. Instead, we identify a critical weight-side state governing low-bit QAT stability: pivot-proximal residence, where latent weights persistently linger near quantization pivots. We formalize this duration as dwell time and show that long dwell times accumulate a large, vulnerable population of weights prone to synchronized mass flips. To resolve this vulnerability, we propose Pivot Gradient Scaling (PGS) as a principled modification to the Straight-Through Estimator (STE) that leaves the forward pass intact and selectively amplifies latent-weight gradients within a localized pivot band. Crucially, we provide a mathematical guarantee proving that this sign-preserving amplification increases pivot-proximal mobility and systematically shortens expected dwell time under both SGD and Adam. Exhaustive empirical evaluations across Llama 3.2 and Qwen3 model families under full-training regimes up to 30 billion tokens demonstrate that PGS structurally stabilizes optimization, attenuates transition bursts, and delivers substantial downstream accuracy gains.

Abstract:
LLM-as-a-Judge has emerged as a scalable alternative to costly human evaluation. However, LLM judges suffer from position bias in pairwise evaluation: the judge systematically favors the first or second candidate regardless of content, forcing practitioners to perform bidirectional evaluation at twice the inference cost. We propose Probe-and-Steer (PaS), a probe-localized, single-layer steering method for mitigating position bias. PaS localizes the bias-critical layer via a layer-wise logit-difference probe and trains a single residual-stream steering vector at that layer under a multi-objective loss, requiring minimal labeled data while keeping the LLM fully frozen. Through layer-wise probing across multiple model families, we uncover three qualitatively distinct bias-distribution patterns and delineate a clear applicability boundary for residual-stream steering. Experiments on an XSum-based dataset with Llama-3.1-8B-Instruct show that PaS delivers the largest gains among all compared methods, improving consistency by 22.9 pp and accuracy by 16.4 pp while shrinking positional asymmetry by 33× (from 0.471 to 0.014), adding the lowest inference overhead at 4%.

Abstract:
Digital dehumanization, although a critical issue, remains largely overlooked within the field of computational linguistics and Natural Language Processing (NLP). To our best knowledge, this work presents the most comprehensive annotated bi-lingual dataset on dehumanization, with almost 16.000 instances (44% dehumanizing) that exists today and can serve as both a training resource for machine learning models and a benchmark for evaluating future dehumanization detection techniques. Our comparison of 15 methods shows that traditional hate speech detection approaches perform poorly on dehumanization detection, particularly for implicit forms. By leveraging our dataset in few-shot and fine-tuning settings, LLMs achieve comparable performance on implicit and explicit dehumanization and substantially outperform zero-shot baselines, with average gains of 39% and 65%, respectively. However, detection accuracy remains slightly lower for mechanistic than for animalistic dehumanization, the latter being the dominant focus of prior work.

Abstract:
Self-consistency improves LLM reasoning by majority voting over multiple sampled solutions, but always exhausts the sampling budget and trusts every solution equally. Early-stopping methods cut token cost substantially via convergence monitoring. However, deciding when to stop cannot change what is voted on, so they save tokens without meaningfully improving accuracy. We introduce CASE (Confidence-Aware Self-consistency with Adaptive Early stopping), which improves both accuracy and efficiency by leveraging token-level confidence normalized at two scales: within-problem to filter low-quality solutions and weight survivors, and cross-problem to adapt the stopping threshold to problem difficulty. Across 5 models and 6 benchmarks, CASE simultaneously saves 85% of tokens and improves accuracy by +2.24 pp over majority voting, outperforming every early-stopping baseline on both accuracy and efficiency. Component and role-swap analyses confirm that filtering and weighting are complementary and the two normalization scales are non-interchangeable.

Abstract:
Evaluating multi-turn conversational systems remains a fundamental challenge. Conventional pipelines rely on manual rubrics and fixed context, a static approach that limits coverage and fails to capture emergent dialogue behaviors. To address this, we introduce CoReflect (A Reflective Co-Evolution Framework for Improving Conversational Evaluation), which unifies dialogue simulation and evaluation into an adaptive, iterative process. CoReflect employs a conversation planner that generates structured templates to guide a user simulator through diverse, goal-directed dialogues. Subsequently, a reflective analyzer processes these dialogues to identify systematic behavioral patterns and automatically refine the evaluation rubrics. Crucially, the insights from the conversation analysis are fed back into the planner to update conversation templates for subsequent iterations. This co-evolution loop ensures that the complexity of test cases and the diagnostic precision of rubrics improve in tandem. By minimizing human intervention, CoReflect provides a scalable and self-refining methodology that allows evaluation protocols to adapt alongside the rapidly advancing capabilities of dialogue models.

Abstract:
Web agents increasingly browse, interpret, and act on web content on behalf of users. In many realistic workflows, however, the visual evidence they consume is not neutral: charts and dashboards can be misleading, shaping judgments without issuing any explicit malicious instruction. Unlike chart question answering, Web Agents treat charts as intermediate evidence whose misinterpretation can propagate into concrete downstream actions — a reliability risk that remains underexplored. We introduce MisVisAgentBench, the first paired executable benchmark for evaluating how misleading visualizations affect Web Agent execution. It comprises 280 task instances in matched clean–misleading pairs that differ only in chart evidence. We further provide a diagnostic framework that isolates misleading-induced failures from baseline task difficulty and captures execution-level effects such as extra browsing steps and decision hesitation. Across eleven vision-language Web Agents, replacing clean charts with misleading counterparts nearly halves the average task success rate, an absolute drop exceeding 34 percentage points. Moreover, these failures predominantly follow the targeted misleading action branch, suggesting that deceptive charts systematically redirect rather than randomly disrupt agent decisions. These results reveal that visualization robustness is a critical yet understudied dimension of Web Agent safety and reliability.

Abstract:
Current research on LoRA primarily focuses on minimizing the number of fine-tuned parameters or optimizing its architecture. However, the necessity of all fine-tuned LoRA layers during inference remains underexplored. In this paper, we investigate the contribution of each LoRA layer to the model’s ability to predict the ground truth and hypothesize that lower-layer LoRA modules play a more critical role in model reasoning and understanding. To address this, we propose a simple yet effective method to enhance the performance of large language models (LLMs) fine-tuned with LoRA. Specifically, we identify a “boundary layer” that distinguishes essential LoRA layers by analyzing a small set of validation samples. During inference, we drop all LoRA layers beyond this boundary. We evaluate our approach on three strong baselines across four widely-used text generation datasets. Our analysis and experimental results show that selectively retaining critical LoRA layers during inference is effective across tasks of different types.

Abstract:
Large language models are increasingly deployed as multi-agent systems, where specialized roles communicate and collaborate through structured interactions to solve complex tasks that often exceed the capacity of a single agent. However, most existing systems still rely on a fixed role library and an execution-frozen interaction topology, a rigid design choice that frequently leads to task mismatch, prevents timely adaptation when new evidence emerges during reasoning, and further inflates inference cost. We introduce MetaGen, a framework that adapts both the role space and the collaboration topology at inference time, without updating base model weights. MetaGen generates and rewrites query-conditioned role specifications to maintain a controllable dynamic role pool, then instantiates a constrained execution graph around a minimal backbone. During execution, it iteratively updates role prompts and adjusts structural decisions using lightweight feedback signals. Experiments on code generation and multi-step reasoning benchmarks show that MetaGen improves the accuracy and cost tradeoff over strong multi-agent baselines.

Abstract:
Unified multimodal models use a shared Transformer backbone for visual understanding and image generation. Existing post-training signals primarily constrain final outputs, output-side representations, or task-level behavior, providing limited supervision for how visual semantics are organized across intermediate layers. We propose HiSVR, a hierarchical semantic visual reconstruction method for post-training unified multimodal models. A frozen semantic visual tokenizer extracts residual semantic codes from each training image, which supervise image-associated hidden states at different Transformer depths. Shallow layers predict shorter semantic-code prefixes, while deeper layers reconstruct longer prefixes, yielding coarse-to-fine supervision over intermediate representations. To balance the differing optimization difficulty across depths and prefix lengths, we introduce an uncertainty-weighted progressive reconstruction loss. HiSVR is used only during training and removed at inference, preserving the original architecture and inference cost. Experiments on unified multimodal backbones show that HiSVR improves visual understanding while maintaining image generation quality. These results indicate that hierarchical semantic reconstruction effectively regularizes intermediate representations in unified multimodal models.

Abstract:
In frame semantics, sentence comprehension is assumed to proceed by relating lexical meaning to background knowledge called semantic frames, thereby enabling readers to implicitly enrich the text with unstated information. Recent large language models (LLMs) have achieved strong performance across a wide range of downstream tasks; however, it remains unclear whether they can reproduce the kinds of implicit enrichment that humans naturally make during comprehension. To address this question, we introduce FrameBench, a benchmark grounded in frame semantics. FrameBench consists of multiple-choice questions that test whether models distinguish the frames evoked by the same verb across contexts. We construct the benchmark for English and Japanese using FrameNet-style resources and a generation-and-verification pipeline with native-speaker judgments. Our experiments on a diverse set of models reveal challenges for small models, while several large models surpass the human reference scores. We release the constructed FrameBench dataset and the code for dataset construction and evaluation at https://github.com/SasanoLab/FrameBench

Abstract:
Large language models (LLMs) have shown significant promise in healthcare applications. To better mirror real-world settings for LLM evaluation, dynamic longitudinal diagnosis-and-treatment simulation with virtual patients has recently emerged as a focal point of research. However, existing simulation frameworks are limited by clinical trajectory uniqueness, where virtual patients can only provide feedback based on information available in static electronic health records (EHRs). This limitation leads to simulation failures when unrecorded but medically sound examinations are ordered. In this paper, we formulate the task of Examination Feedback Simulation, which aims to dynamically augment the unique trajectory by simulating medically plausible examination results in response to clinical orders. To support this largely unexplored research area, we construct ClinTrack, a dataset curated from MIMIC-IV. We further propose a structure-aware evaluation metric SimScore to quantitatively assess the quality of simulated results, which shows promising initial alignment with expert judgment. Building on this framework, we develop ClinSim, a tailored generative model for this task. Experiments demonstrate that our 4B ClinSim model significantly outperforms flagship models with up to 235B parameters on this task, providing a critical foundation for creating more dynamic and realistic virtual patients. Code and other resources are available at GitHub.

Abstract:
Large language models (LLMs) have been widely used for problem-solving tasks. Most recent work improves their performance through supervised fine-tuning (SFT) with labeled data or reinforcement learning (RL) from task feedback. In this paper, we study a new perspective: the divergence in solutions generated by LLMs for a single problem. We show that higher solution divergence is positively related to better problem-solving abilities across various models. Based on this finding, we propose solution divergence as a novel metric that can support both SFT and RL strategies. We test this idea on three representative problem domains and find that using solution divergence consistently improves success rates. These results suggest that solution divergence is a simple but effective tool for advancing LLM training and evaluation.

Abstract:
Text-to-SQL systems built on large language models still struggle in real deployment, where users underspecify queries while databases enforce rigid schemas. The gap is rarely lexical; it lies in implicit knowledge—hidden joins, derived formulas, and domain-specific filters—that natural language (NL) omits but SQL must encode. We observe that this knowledge is not missing: every historical (NL, SQL) pair already records it, with NL stating the goal and SQL stating the steps. Treating the two as dual views of one program, we propose KODE-SQL, a framework with three agents: KODE-Miner aligns NL and SQL in a shared graph space and distills the unmatched regions into reusable templates; KODE-Solver retrieves these templates to generate schema-grounded SQL under a rule-then-retrieval policy; KODE-Critic verifies each candidate on the same graph and feeds successful cases back to the knowledge base. KODE-SQL improves execution accuracy by up to 11.1 points on the knowledge-heavy ScienceBenchmark, remains competitive on BIRD and Spider, and yields consistent gains (+4–7 points) when plugged into existing solvers—validating both its standalone strength and its portability as a knowledge module.

Abstract:
Stereotype evaluation for large language models relies on heavily annotated benchmarks, yet such evaluations are constrained by limited instance availability and are often sensitive to which examples are included. We propose a framework for predicting the minimum number of evaluation instances needed to preserve benchmark-level stereotype scores under specified error and confidence criteria. Our method provides a statistically grounded way to determine how much a benchmark can be reduced without materially changing the resulting evaluation outcome. We support the framework with theoretical analysis and extensive experiments on three widely used stereotype benchmarks, CrowS-Pairs, StereoSet, and BBQ. Across datasets, models, and tolerance settings, the proposed method provides reliable subsample-size guidance without exception. As an additional extension, we show that our approach can also guide compact synthetic stereotype evaluation. Overall, our results suggest that benchmark-level stereotype evaluation can often be conducted with far fewer instances than the original datasets require, offering a practical and principled approach to more efficient bias assessment.

Abstract:
High-performance CUDA kernels are essential for scalable AI systems, while Large Language Models (LLMs) still struggle to generate correct kernels due to strict and implicit execution constraints. Existing LLM-based approaches either rely on costly agentic or reinforcement-learning (RL) pipelines, or adopt supervised fine-tuning (SFT) objectives that fail to explicitly model CUDA sensitivity, namely code tokens or regions tightly coupled with execution constraints. In this work, we investigate CUDA sensitivity from the perspective of token confidence patterns, showing that CUDA sensitivity appears at both token and region levels, where most CUDA-sensitive tokens are predicted with high confidence, while a smaller low-confidence subset forms regions corresponding to execution-critical structures. These findings suggest that effective CUDA kernel generation should both leverage high-confidence CUDA-sensitive tokens and preserve low-confidence CUDA-sensitive regions. Building on these insights, we propose CU DA- Se nsitive Instruction T uning (CuSeT) , a low-cost post-training method within a simple SFT framework. CuSeT follows the principle of “from tokens to regions” by combining adaptive token-level masking with region-aware sample reweighting . Experiments show that CuSeT consistently improves both functional correctness and kernel performance across multiple model families and scales, outperforming standard SFT and advanced SFT variants, while achieving competitive performance against frontier CUDA kernel generation models with substantially lower inference cost.

Abstract:
Large Audio Language Models (LALMs) still struggle in complex acoustic scenes because they often fail to preserve task-relevant acoustic evidence before reasoning begins. We identify this error pattern as the evidence bottleneck: state-of-the-art systems show larger deficits in acoustic evidence extraction than in downstream reasoning, suggesting that upstream perception is often the limiting factor. To address this problem, we propose EvA (Evidence-First Audio), a dual-path architecture that enhances acoustic evidence preservation through hierarchical aggregation and non-compressive, time-aligned fusion. We also build EvA-Perception, a large-scale training set with about 54K event-ordered captions and 500K evidence-grounded QA pairs. Under a unified zero-shot protocol, EvA achieves the best open-source Perception results on MMAU, MMAR, and MMSU, with the largest gains on perception-heavy splits. Human evaluation on open-ended captioning further shows improved fine-grained acoustic coverage and caption quality. These results support the evidence-first hypothesis: stronger audio understanding depends on preserving acoustic evidence before reasoning.

Abstract:
Large language model (LLM) agents for long-horizon interactive tasks typically follow a ReAct-style protocol, issuing one primitive action per LLM round. While this enables frequent replanning, it is inefficient for long-horizon tasks where many rounds are spent on routine action sequences. A natural alternative is to let the agent emit variable-length action chunks. However, naively training such policies with standard reinforcement learning fails: the agent either collapses to single-action behavior or over-commits to excessively long sequences. Both failures share a common root cause: the inability to learn chunk boundaries. We propose SPACE, which addresses this challenge by distilling chunk-boundary supervision from trajectory-induced programmatic skills. We induce two-level programmatic skills from successful trajectories, where subskill boundaries serve as direct chunk-boundary supervision. This temporal structure is then distilled into a primitive-chunk policy via hybrid on-/off-policy optimization with chunk-aware credit assignment. Experiments on ALFWorld and ScienceWorld show that SPACE improves success rates by 7.0%-31.3% over the strongest baseline in each setting while reducing average LLM decision rounds by up to 78.9%.

Abstract:
Mechanistic interpretability usually studies fully trained models, yet the computations that drive a behaviour can change while the model is still learning the task. On the Indirect Object Identification task, a model reads a sentence such as "When Mary and John went to the store, John gave a drink to" and should continue with Mary, the name mentioned once, rather than John, the name mentioned twice. Pythia models pass through an early training window in which they prefer the repeated name, so accuracy in a choice between the two names falls below one half while language-model loss on a fixed text sample keeps decreasing across the same window. The window reflects a temporary imbalance between two computations. We identify one cause of the wrong preference by selecting a set of attention heads that write the repeated name, on prompts separate from those used for causal evaluation, keeping that selection fixed, and then replacing each head’s final-token output with its average output on a separate set of non-repeated-name prompts. This improves the correct-minus-repeated logit difference in a separately trained 160M model and in the official 160M, 410M, and 1B models. At 160M, the head that lowers the repeated name in the mature model shows little of its mature behaviour at this point. It directs less than one percent of its attention to the repeated mention, and its output makes almost no direct contribution to lowering that name’s logit. Both properties grow over the interval in which behaviour recovers. Across the 160M, 410M, and 1B models, transplanting the corresponding head’s mature parameters into the early checkpoint recovers 35 to 68 percent of the total improvement in the correct-minus-repeated logit difference seen by the end of training. Related early-to-late reversals appear at further Pythia scales, in two independently trained GPT-2 models, and in OLMo. A mature circuit can therefore conceal a transient causal configuration that shaped behaviour earlier in training.

Abstract:
In long-horizon tool use, early-step defects passing per-call checks often propagate silently across dependency chains to cause irreversible late-stage failures—a propagation class accounting for 73.0% of diagnosed Vanilla FC failures. Because these defects are locally valid (stepwise legality) yet manifest differently across tool boundaries (semantic heterogeneity), they easily evade standard per-call guardrails and post-hoc reflection, necessitating proactive pre-execution risk anticipation. To this end, we formalize the pre-execution risk propagation problem and propose RiskGuard, a training-free, dev-calibrated runtime controller. To capture the multi-source, conjunctive dependencies inherent in tool composition, RiskGuard models execution traces via a typed dependency hypergraph. Meanwhile, a Referential/Constraint/Operational (R/C/O) kernel tracks risk transitions across semantic boundaries. Replacing rigid binary blocking, a five-mode intervention rule selects the lowest-commitment feasible response to mitigate risks smoothly. Evaluated on ToolSandbox-hard, pooled τ²-Bench, and ToolHop, RiskGuard achieves absolute gains of +0.133 Sim (0.281 → 0.414), +9.2 pp pass@1, and +4.85 pp accuracy over the baseline, with depth diagnostics showing stronger gains on harder dependency slices.

Abstract:
Internet memes evolve as they are repurposed across online communities, yet studying how their usage and language shift requires corpora that follow the same templates over many years, which existing resources rarely provide. To address this gap, we collect a decade of posts from Reddit meme subreddits (2016–2025) and introduce MemesOverTime, a semi-automatically curated dataset of 6.6M memes annotated with 6,584 unique templates. We use it to conduct an empirical study of meme template dynamics. We find that templates differ markedly in their temporal activity patterns: most reach peak usage within the first third of their active period, with short-lived templates peaking especially early. For long-lived templates, lexical drift is generally modest, with templates with more instances showing greater stability. Case studies reveal that creators spontaneously develop conventionalized expressions for some templates, while topical references shift over time.

Abstract:
As language models are increasingly deployed in real-world applications, the ability to erase specific knowledge from them becomes critical for safety and compliance. Prominent methods seek persistent removal by updating the model’s parameters, yet the target knowledge often can be recovered through adversarial prompting or relearning. In this work, we hypothesize this limitation stems in part from existing methods overlooking the embedding layer. To address this, we introduce EMBedding ERasure (EMBER), a plug-n-play erasure module that leverages Sparse Matrix Factorization for precise erasure of concept-related features from token embeddings. Through comprehensive evaluations across diverse concepts on Gemma-2-2B-it and Llama-3.1-8B-Instruct, we find that augmenting existing methods with EMBER consistently improves erasure efficacy and specificity across task formats, with minimal coherence loss. Moreover, EMBER dramatically improves robustness to relearning, reducing regained accuracy by up to 50%, limiting it to 35% on Llama compared to 70%–76% for prior methods. Further analysis shows that the coherence cost is localized, affecting only a small set of concept-exclusive tokens. Our work establishes that precise embedding-level intervention is necessary for robust concept erasure, and demonstrates that existing methods can benefit from such augmentation.

Abstract:
LLM-as-a-Judge systems are increasingly employed for automated evaluation, yet remain susceptible to systematic biases that undermine their reliability. We demonstrate that the judge can effectively debias itself without additional training. We propose two lightweight strategies: bias-aware prompting, which explicitly alerts the judge to potential target biases prior to evaluation, and self-debiasing, in which the judge critiques the initial judgment for bias influence. To broaden the evaluation scope, we introduce three novel benchmarks—authority bias (fabricated citations), confidence bias (assertive phrasing), and formatting bias (structured presentation)—and evaluate across seven bias types in total. Across three frontier models, self-debiasing improves performance in 20 out of 21 model–bias combinations with gains up to 15 percentage points, while bias-aware prompting also yields consistent improvements. The recent fine-tuned external critic, however, does not generalize as reliably to the newly-introduced bias types (degradation observed in 5 out of 9 cases). Our findings demonstrate that prompting a judge to reason about bias, either preemptively or retrospectively, is a robust and generalizable approach for enhancing evaluation integrity.

Abstract:
Large language models often solve a theorem forward yet fail to disprove a closely related false one: a falsification gap that supervised fine-tuning does not close and can actively worsen. We frame counterexample generation as constrained witness emission against a deterministic per-theorem Python verifier, and release SymCE, a corpus of 4,707 false undergraduate-algebra and real-analysis conjectures, each paired with executable verifiers. The verifier also serves as the reward function, making SymCE a training environment. Training Qwen3-4B with SFT followed by GRPO under this oracle reveals an imitation trap: counterexample-only SFT collapses true-theorem recognition from 0.27 to 0.00, while RLVR with a sparse outcome-only reward repairs this and exceeds the base, to 0.66. The collapse replicates across four seeds and on Gemma-3-4B. Sparse and dense rewards yield statistically indistinguishable in-domain success yet diverge by 33 points on a held-out calibration probe, a dissociation we trace to the partial-credit term. Our 4B model outperforms every evaluated 7B open-weights math specialist, remains competitive with six frontier commercial APIs, and transfers under unchanged prompting to GSM8K, MATH-500 and MMLU-college-math. A human audit of 177 verifier decisions finds 97.7% accuracy. Code, data, verifier modules and annotations: https://github.com/ce-rlvr/SymCE.

Abstract:
Retrieval heads, a subset of attention heads in Transformers, were studied in English, showing its crucial role in retrieving information from the context. We expand the study of retrieval heads to multilingual context and find that while nearly half of all retrieval heads are often shared across multiple languages, language-specific retrieval heads also emerge. We further design a cross-lingual needle-in-the-haystack task, to identify Retrieval Transition Heads (RTH) that retrieve key information and further map it to target-language output. Our experiments reveal that RTHs do not always overlap with RH, and can be more vital for Chain-of-Thought reasoning in multilingual LLMs. Across four multilingual benchmarks (MMLU-ProX, MGSM, MLQA, and XQuAD) and two model families (Qwen2.5-7B Instruct and Llama3.1-8B Instruct), we demonstrate that masking RTH induces bigger performance drop than masking Retrieval Heads (RH). Specifically, for Llama3.1-8B Instruct, masking the top-25 RTHs results in an average 37.3-point drop (against 19.5-point drop from RH masking) in reasoning accuracy (MMLU-ProX, MGSM) and 9.0 F1-score reduction (against 5.1 drop for RH) in extractive QA (MLQA, XQuAD). Our work advances the understanding of multilingual LMs by isolating the attention heads responsible for mapping to target languages. Code available at https://github.com/NerdyVisky/multilingual-retrieval-transition-heads

Abstract:
Reinforcement learning is a powerful approach for improving language models, but applying it to open-ended domains such as healthcare remains challenging because high-quality responses are often multidimensional, context dependent, and difficult to capture with a single reward signal. Rubrics offer a natural form of supervision by providing a structured representation of the criteria that define a strong response. Existing rubric-based approaches, however, often rely on instance-specific criteria, creating two important limitations: (1) such rubrics are typically unavailable for unseen examples at test time; and (2) constructing new rubrics for every instance incurs substantial reward design costs. We introduce Health-SCORE, a scalable framework that captures recurring principles of response quality in the form of reusable meta-rubrics. Rather than relying on manually constructed instance-level criteria, Health-SCORE captures generalizable principles that can be applied across examples, tasks, and healthcare datasets. These meta-rubrics support two forms of model improvement: they can serve as structured, safety-aware reward signals for reinforcement learning during post-training, and they can be incorporated directly into prompts to guide model behavior at inference time. Across several open-ended healthcare benchmarks, models trained with Health-SCORE achieve performance competitive with models trained using human-authored instance-level rewards, while substantially reducing reward-design costs and improving generalizability.

Abstract:
Reliable capability assessment is essential for governing and securely deploying advanced AI systems. However, "AI Sandbagging," where models strategically suppress their true capabilities during evaluation, undermines existing assessment frameworks by making it difficult to distinguish genuine limitations from strategic concealment. To address this challenge, we propose a new black-box sandbag detection method inspired by "honesty inertia," a psychological phenomenon where humans struggle to maintain deception after answering simple questions truthfully. We hypothesize that large language models exhibit an analogous effect: inserting several simple, unrelated questions, which target models should answer correctly, before the target query induces honest inertia, causing sandbag models to involuntarily increase accuracy while standard models remain stable. Through extensive experiments on multiple datasets with various model architectures, we confirm the validity of this hypothesis and establish this differential response as a reliable, model-independent sandbag detection signal. We further provide interpretability support for this phenomenon from two dimensions: theoretical derivation and hidden state analysis of models. This work offers a practical tool for enhancing AI audit and supervision.

Abstract:
Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics. We introduce TriviaRoomQA, a multilingual benchmark designed to evaluate everyday, culturally grounded, and long-tail knowledge across 288 topics. The benchmark contains 3,300 parallel multiple-choice questions in six European languages and additional 5,340 French-only questions for a more fine-grained case study. We evaluate 30 open-weight LLMs from European, Asian, and North American providers, covering models from 7 to 70B parameters. We find that models are strong on knowledge-intensive topics such as history, geography, and mathematics, but substantially weaker on everyday popular-culture topics such as celebrities, music, movies, and news. Moreover, model performance varies across languages even for the same underlying questions, suggesting that access to factual knowledge is not always language-independent. In sum, our dataset and experiments demonstrate an important knowledge gap which is not captured by existing academic-based saturated benchmarks.

Abstract:
Automatic text simplification measures correlate poorly with human judgments and are disconnected from psycholinguistic research on comprehension. We propose a multidimensional analysis framework grounded in Cognitive Load Theory and the Construction-Integration model, organizing over 400 linguistic features into six simplicity dimensions targeting distinct comprehension stages. Using this framework, we analyze four human-annotated and six aligned simplification datasets. We find that (1) our dimensions predict human simplicity judgments better than established measures on most datasets; (2) current datasets overwhelmingly target lexical and syntactic dimensions while under-targeting personalization the dimension psychological research identifies as most impactful for comprehension; and (3) existing measures overall do not capture any specific dimension. These findings provide actionable guidance for building measures and datasets better aligned with how readers actually process text.

Abstract:
Mapping textual specifications into formal representations is essential for ensuring the correctness of protocol designs and implementations. LLM-generated mappings, used for networking security or testing, are assumed to capture a perfect understanding of the specification, which may not hold in practice. The goal of this paper is to assess the extent to which LLMs can interpret the specification correctly. We examine the degree to which an LLM’s implicit representation of a finite-state transition system–defined via natural language descriptions–aligns with a manually generated ground-truth model. We designed 4 tasks and 1482 task queries for 16 protocols. We evaluated different judge biases, observed the inherent difficulty gaps between tasks, looked into the effect of 4 context types, and the influence of protocol characteristics. Our work contributes to a step toward verifying whether LLMs can really be trusted in FSM (Finite State Machine) reasoning of protocol specifications.

Abstract:
We present Holmes-Graph, a framework to evaluate the linguistic competences of graph-based text embeddings generated by training graph neural networks (GNNs) on graph representations of text. The framework extends the Holmes benchmark, which previously covered only sparse, static, and transformer-based representations. The framework is flexible to be used with any graph representation and GNN architecture. Once weights are learned on a downstream task, frozen embeddings are probed across five linguistic competences. When applied to AI-generated text detection, Integrated Syntactic Graphs (ISG) combined with Graph Neural Networks (GNN) achieve an aggregate Holmes score of 0.662. This matches or exceeds DeBERTa-base on syntax, morphology, semantics, and reasoning, but not for discourse. The results confirm that graph-based text embeddings encode non-trivial linguistic knowledge and motivate their inclusion in future probing benchmarks.

Abstract:
Computer-use agents (CUAs) rely on visual observations of graphical user interfaces, where each screenshot is encoded into a large number of visual tokens. As interaction trajectories grow, the token cost increases rapidly, limiting the amount of history that can be incorporated under fixed context and compute budgets. This has resulted in no or very limited improvement in the performance when using history unlike other domains. We address this inefficiency by introducing ReVision, which is used to train multimodal language models on trajectories where redundant visual patches are removed using a learned patch selector that compares patch representations across consecutive screenshots while preserving spatial structure required by the model. Across three benchmarks, OSWorld, WebTailBench, and AgentNetBench, when processing trajectories with 5 history screenshots using Qwen2.5-VL-7B, ReVision reduces token usage by 46% on average while improving success rate by 3% over the no drop baseline. This establishes a clear efficiency gain, enabling agents to process longer trajectories with fewer tokens. With this improved efficiency, we revisit the role of history in CUAs and find that performance continues to improve as more past observations are incorporated when redundancy is removed.

Abstract:
Enterprise AI assistants suffer from two failure modes not captured by existing public-web QA benchmarks: authorized information leakage, where factually correct answers reach unintended readers, and aggregation failure, where evidence scattered across artifacts yields fluent but incomplete responses. Both stem from enterprise authority being per-item and partly body-embedded, a structure that existing benchmarks address along at most one axis. We argue that authority-aware evaluation must be conducted at the per-item level, and instantiate this principle through Cross-Artifact and Authority-aware Reasoning over Organizational Bundles (CAROB): a synthetic enterprise dataset in which every retrieval-and-citation item carries a time-indexed audience/approval/lifecycle annotation, paired with three tasks that decouple cross-artifact coherence, authority-aware retrieval, and role-conditioned answering. Evaluating five frontier and open-weight LLMs, we find that no single model dominates both axes of the benchmark: each task induces a distinct model ordering, and the model strongest on authority handling ranks last on answer coverage. Two structural patterns hold across the lineup: retrieval-side leakage amplifies substantially at the answer stage, and structural authority filtering eliminates the residual leak band that prompt-level mitigation leaves behind, with both patterns replicating on EnronQA and ACCESS DENIED INC. We release CAROB as an evaluation testbed for authority-aware enterprise system design, together with its per-item annotation, three-task structure, and label-first generation pipeline as reusable infrastructure.

Abstract:
Text-to-image (T2I) safety guardrails fail to generalize equitably to non-standard dialects. Evaluating 23,080 paired prompts across five English dialects, we formalize this failure as the dialect penalty, where filters trigger based on linguistic surface features rather than semantic intent. Text-level filters fail in opposing directions: NSFW-T over-flags benign dialect prompts and LatentGuard over-flags toxic ones (bias gaps up to +28.29 pp), while the OpenAI Moderation API under-detects them. A controlled typo ablation confirms this penalty originates from flagging dialectal features, not generic out-of-distribution sensitivity. The pixel-level generator is largely dialect-agnostic; the penalty enters at text processing and cascades unevenly to post-hoc guardrails. We show this bias tracks training data imbalance and is mitigable via group-balanced retraining, with an ablation attributing the gain to balanced exposure rather than to the worst-group objective of GroupDRO (group distributionally robust optimization). Current pipelines systematically fail dialect speakers, an equity failure masked by mean accuracy benchmarks. Our official code and dataset are publicly available at https://github.com/minguinho26/dialect-penalty-t2i. Content Warning: This paper contains offensive, toxic, or disturbing text prompts and generated images.

Abstract:
We introduce C-QUERI, a dataset of question-answer exchanges from U.S. congressional committee hearings spanning the 108th–117th Congresses, with the pipeline used to construct it. C-QUERI transforms congressional hearing records into structured question-answer exchanges, enabling researchers to study political questioning and witness responses systematically and at scale. Congressional hearings provide a particularly valuable setting for such research: interactions are structured by formal rules, members of different parties are guaranteed opportunities to question witnesses, and witnesses are expected to respond. As an initial application demonstrating the resource’s utility, we examine whether questioner party affiliation can be predicted from question text alone. Question text contains a modest aggregate partisan signal, with substantially greater distinguishability in some committees, hearing types, and configurations of partisan control than others. C-QUERI provides new infrastructure for studying congressional politics and a general framework for analyzing question-answer interactions in other interview-like settings.

Abstract:
Recent progress in Omni-models has yielded strong performance on benchmarks focused on static observations. However, their ability to maintain spatiotemporal continuity when predicting coherent future states from previous context remains underexplored. We address this by formulating Next Scene Prediction (NSP) as a diagnostic task for assessing multimodal near-future state prediction, requiring models to anticipate the immediate future from a given context. We introduce OmniNSP, a benchmark evaluating causal and cross-modal coherence across physical, logical, and social dynamics. Unlike prior tasks that often rely on textual or discriminative proxies, OmniNSP asks models to express predicted future states through text-mediated visual and speech synthesis. Our analysis shows that current Omni-models can produce plausible textual forecasts while still failing to preserve future-state consistency in generated outputs. A frozen-generator control substantially narrows cross-model visual-consistency gaps, attributing much of the original system-level difference to model-specific realization stacks rather than textual foresight alone. A modality-input ablation further shows that audio provides complementary predictive cues beyond vision, with the largest gains for logical transitions and contexts with informative speech. These results position OmniNSP as a diagnostic of both near-future prediction and the preservation of predicted state through multimodal realization.

Abstract:
Gaps remain in our understanding of how large language models (LLMs) acquire knowledge during pre-training. We posit that auxiliary views, reformulations of knowledge across different contexts, are causally helpful for learning. We design controlled experiments to isolate this effect. First, we confirm that repetition is necessary for acquisition and clarify that paraphrasing helps only at smaller batch sizes. Second, holding the token budget fixed, reallocating tokens from document repetition to auxiliary views improves learning, even for factual recall, which is counterintuitive. Third, the effectiveness of auxiliary views is not contingent on the strength of the teacher model that generates them. Fourth, we identify forms of knowledge, contextual and foundational, that aid learning in the presence of prior knowledge gaps. Finally, we examine how these effects manifest mechanistically via layer-wise biases and compression. Together, our findings suggest that auxiliary representations of knowledge, which arise naturally in large pre-training corpora, are a key factor in the success of pre-training and offer a plausible explanation for why data diversity matters.

Abstract:
Large language models can generate fluent explanations, but effective tutoring requires supporting the learner’s thought process, not just delivering content. Metacognitive tutoring targets this gap by prompting planning, monitoring, debugging, and evaluation, and, crucially, deciding when to be active versus minimally present, based on learner signals and trajectory. We introduce MetaCLASS, a learning-science grounded framework that formulates metacognitive tutoring as move selection over 11 interpretable actions aligned to self-regulated learning (SRL) processes. MetaCLASS uses a two-phase procedure that first plans a pedagogical trajectory conditioned on learner profiles (calibration, help-seeking) and then generates natural dialogue consistent with that plan. This process yields 1,015 conversations spanning 7,711 turns, each annotated with turn-level metacognitive labels and supported by comprehensive validation analyses. We benchmark nine open-source LLMs on predicting the next coach move given a question and dialogue context. The best model achieves only 43.2% accuracy in coach move prediction, and models exhibit compulsive intervention bias: in turns where effective metacognitive tutoring requires silence (41.7% of cases), models predict "no intervention” only 6.6% of the time, while severely over-predicting high-intervention moves. These results show that traditional content-based tutoring ability does not translate to metacognitive tutoring competence, positioning MetaCLASS as a testbed for developing tutors that promote SRL.

Abstract:
Transformers achieve state-of-the-art results across many tasks, but their uniform application of quadratic self-attention to every token at every layer makes them computationally expensive. We introduce DTRNet (Dynamic Token Routing Network), an improved Transformer architecture that allows tokens to dynamically skip the quadratic cost of cross-token mixing while still receiving lightweight linear updates. By preserving the MLP module and reducing the attention cost for most tokens to linear, DTRNet ensures that every token is explicitly updated while significantly lowering overall computation. This design offers an efficient and effective alternative to standard dense attention. Once trained, DTRNet blocks routes only ~10% of tokens through attention at each layer while maintaining performance comparable to a full Transformer. It outperforms routing-based layer skipping methods such as MoD and D-LLM in both accuracy and memory at matched FLOPs, while routing fewer tokens to full attention. Its efficiency gains, scales with sequence length, offering significant reduction in FLOPs for long-context inputs. By decoupling token updates from attention mixing, DTRNet substantially reduces the quadratic share of computation, providing a simple, efficient, and scalable alternative to Transformers.

Abstract:
Long-horizon research agents solve open-ended tasks through iterative retrieval, aggregation, and synthesis, but context grows rapidly while the marginal value of additional evidence often declines. This leads to unnecessary token cost, higher latency, and noisier inputs for final report generation. We study marginal value estimation for context management in deep research agents and present the first systematic stage-aware comparison of pruning strategies across the pipeline. We evaluate lightweight heuristic criteria and a learned value model at pre-retrieval, post-retrieval, and pre-synthesis stages. Our results show that pruning effectiveness depends more on where pruning is applied than on the specific scoring rule: early pruning yields the largest end-to-end savings, while later pruning mainly refines the final synthesis context. Lightweight heuristics reduce token usage by up to 73% with little quality degradation, learned pruning remains competitive on selected trade-offs, and no single method dominates across quality, efficiency, and faithfulness. These findings provide practical guidance for designing efficient long-horizon agentic systems.

Abstract:
Discrete diffusion language models (dLLMs) offer an attractive alternative to autoregressive generation by updating multiple tokens in parallel, but their inference efficiency is still limited by the cost of safely committing tentative predictions. In particular, speculative decoding for dLLMs is challenging because speculative tokens within the same diffusion block may depend on one another, making aggressive acceptance prone to error propagation and strict verification computationally expensive. We present PV-Decoding, a confidence-driven speculative decoding framework with progressive verification for block diffusion inference. PV-Decoding partitions masked positions into directly decoded tokens, speculative tokens, and unresolved tokens using two confidence thresholds, and then verifies speculative tokens with progressively stronger policies, including single-batch, dual-batch, and multi-batch verification. This design exposes a practical accuracy–efficiency trade-off: weaker verification minimizes overhead, whereas stronger verification improves acceptance precision and robustness. Experiments on three diffusion language models across GSM8K, MBPP, and HumanEval show that PV-Decoding reduces sequential model forward passes (MFPs) by 8%–37% relative to confidence-based decoding, while keeping the largest observed accuracy drop to 1.3%. On Dream-7B-Instruct/GSM8K, PV-Decoding (1-batch) and PV-Decoding (2-batch) improve throughput by 7.0%–13.6% under the same hardware budget, whereas PV-Decoding (multi-batch) is slower under the current runtime. These results suggest that confidence-guided speculation with structured verification is a practical path toward efficient and reliable dLLM inference.

Abstract:
Large language models (LLMs) are increasingly used in agentic coding settings, where they can inspect files, execute commands, run tests, observe failures, and iteratively revise code. This shift raises a central evaluation question: can an agentic LLM generate an end-to-end software artifact that is both deployable and behaviorally correct under execution? Backend services provide a controlled but realistic substrate for this evaluation. Their APIs expose application-level executable semantics, and deployed behavior can be checked deterministically against an OpenAPI contract through black-box HTTP interactions. We introduce , a benchmark of 56 contract-defined backend generation tasks rewritten from real open-source applications. Given a visible specification and an OpenAPI contract, an LLM must generate a Dockerized service that is built, deployed, and evaluated only through HTTP tests. To strengthen evaluation without introducing hidden requirements, uses a test agent and a code agent to co-evolve the test oracle and reference service, where the test agent proposes specification-grounded backend tests and the code agent repairs the reference implementation. Although the best-performing model, GPT-5.5, succeeds on 55.4% of tasks under the base oracle, it succeeds on only 28.6% under the final oracle. This gap suggests that current LLMs can implement many local API behaviors, but still struggle to produce complete backend services.

Abstract:
Recent advancements in topic modeling have split into two main paths: efficient embedding-based methods that offer scalability but are often outperformed on standard clustering metrics, and Large Language Model (LLM) approaches that achieve state-of-the-art clustering performance but at the cost of prohibitive computational expense and poor scalability. This paper introduces TopicMerge, a novel multi-stage pipeline that bridges these approaches to achieve both state-of-the-art clustering performance and computational efficiency. TopicMerge first employs an efficient clustering pipeline to deliberately over-segment a document corpus into a large set of fine-grained, semantically pure micro-clusters, which are then given initial semantic labels. It then introduces a novel, iterative merging algorithm where an LLM, guided by local neighborhood context, arbitrates on consolidating candidate pairs. These approved merges are then selectively executed and semantically relabeled by the LLM, hierarchically building broader, more coherent topics. Extensive experiments on the Bills and Wiki datasets show that TopicMerge achieves state-of-the-art clustering performance while reducing required LLM calls by over 92% compared to document-level methods, offering a practical and scalable solution.

Abstract:
Existing approaches to visual attribute value extraction (AVE) primarily rely on static product images, failing to capture temporal cues, multi-angle views and fine-grained visual details. Directly applying video vision-language models (VLMs) to product AVE results in limited performance due to the lack of domain knowledge, and fine-tuning them requires extensive high-quality data and substantial computational resources. Thus, we propose visual search augmented chain-of-thought reasoning (ViS-CoT), a training-free, plug-and-play pipeline that can be easily applied to any open-source video VLM for video-to-text AVE in e-Commerce. Specifically, ViS-CoT employs visual clustering to identify representative frames, followed by visual search to retrieve semantically similar product knowledge that can enrich attribute cues. Next, an interleaved CoT reasoning module iteratively refines reasoning through visually-aligned auxiliary texts derived from captioning and automatic speech recognition. Finally, the integrated information guides the model toward accurate and fine-grained attribute predictions. Extensive experiments across 14 product categories on the VideoAVE dataset show that ViS-CoT consistently enhances multiple state-of-the-art video VLMs, achieving an average improvement of 17.91 percentage points in micro-F1.

Abstract:
Chain-of-Thought (CoT) prompting is widely adopted for mathematical problem solving, including in low-resource languages, yet its robustness under irrelevant context - a known vulnerability in English - remains unexamined in such settings. To systematically probe this challenge, we introduce DISTRACTMATH-BN, a Bangla benchmark that augments MGSM and MSVAMP with semantically coherent but computationally irrelevant information. Evaluating seven models ranging from 3B to 12B parameters, we observe substantial performance degradation under distractors: standard models drop up to 41 points, while reasoning-specialized models decline 14-20 points despite consuming 5× more tokens. We propose †DAGGER, which reformulates mathematical problem solving as executable computational graph generation with dedicated modeling of distractor nodes. Fine-tuning Gemma-3 models using Supervised Fine-Tuning followed by Group Relative Policy Optimization achieves comparable weighted accuracy on augmented benchmarks while using 89% fewer tokens than reasoning models. Importantly, this robustness emerges without explicit training on distractor-augmented examples. Our results suggest that enforcing structured intermediate representations can improve robustness and inference efficiency in mathematical understanding compared to free-form reasoning, particularly in noisy, low-resource settings.

Abstract:
Large Language Models (LLMs) are increasingly applied to legal and criminal justice tasks, yet existing work focuses almost exclusively on post-arrest scenarios where the suspect’s identity is already known, leaving the critical pre-arrest challenge of inferring suspect characteristics from incomplete evidence largely unexplored. To fill this gap, we introduce the P rofiling, I nvestigation, and J udgment ( PIJ ) benchmark, comprising 2,500 real homicide cases from five countries. PIJ evaluates LLMs across three tasks that span the entire criminal investigation pipeline: criminal profiling, which requires abductive reasoning to infer suspect attributes from fragmentary scene evidence, crime process reconstruction, which tests structured information extraction, and sentence prediction, which demands legal deductive reasoning. We evaluate 9 powerful LLMs and find that performance degrades systematically as tasks shift from explicit fact extraction to implicit reasoning over unknown suspect profiles. Categories requiring inferential reasoning, such as motivation and victim-offender relationships, remain the primary bottlenecks. Further analysis reveals substantial gaps between LLMs and human experts, along with pervasive biases in gender, age, and motive attribution. Our findings indicate that current LLMs are far from ready for practical deployment in criminological investigation.

Abstract:
Large Language Models (LLMs) have emerged as transformative, powerful tools for processing and acting on unstructured financial data. However, existing surveys suffer from methodological fragmentation, evaluation inconsistency, and application disconnect. This hinders the transition of research from academia to real-life applications in financial institutions. To address these gaps, we (1) introduce a unified taxonomy organizing approaches by computational complexity, (2) propose a three-stage evaluation framework, and (3) examine LLM deployment approaches suited to specific operational contexts. Furthermore, we identify key challenges including validation frameworks, data contamination, and operational compliance that need to be further investigated.

Abstract:
Unified Multimodal understanding and generation Models (UMMs) increasingly combine visual understanding and image generation within a single interactive workflow, making generated visual content available as later reasoning context. However, existing jailbreak evaluations mostly study text rewriting or isolated visual prompts, leaving the safety risk of narrative cross-turn visual grounding underexplored. We propose NarrativeAttack, a semantic-preserving visual narrative jailbreak framework. NarrativeAttack employs a three-act narrative structure in which the UMM’s own generator produces images for the setup (pre-event) and resolution (post-event) stages, making the full attack workflow self-contained while concealing the malicious event as a hidden climax. The attack concludes with an image-based "guessing game" that embeds the original malicious query among benign candidates, compelling the model to select and answer the most relevant one based on the established narrative context. A dynamic difficulty mechanism further enhances attack stability. Experiments show NarrativeAttack consistently surpasses prior approaches, achieving up to 88.25% ASR on Gemini-2.5-Flash. These results uncover an underdeveloped vulnerability and highlight the urgent need for safety alignment in UMMs.

Abstract:
Cross-Video Reasoning (CVR) has emerged as a critical frontier in multimodal intelligence, requiring models to retrieve, align, and aggregate evidence distributed across multiple videos. Current Multimodal Large Language Models (MLLMs) often struggle with CVR, as simple single-pass strategies encode multiple videos into a shared compressed context, potentially obscuring rare but critical evidence. In this paper, we propose AgentCVR, a multi-agent framework that treats CVR as an active evidence-acquisition task. AgentCVR employs a Master Agent to iteratively coordinate specialized Visual and Audio Agents for targeted evidence extraction. To ensure efficient training, we introduce Script-Simulated RL, which optimizes the agent’s policy with LLM-generated semantic scripts and a lightweight text-based simulator, bypassing costly multimodal inference during online exploration. Experimental results on a comprehensive CVR benchmark show that AgentCVR outperforms single-pass baselines and achieves comparable performance to state-of-the-art closed-source systems, particularly in complex cross-video alignment and localization. To ensure reproducibility, our code is available https://github.com/wang-jh24/AgentCVR.

Abstract:
The Verifiability Gap in table reasoning arises when multimodal LLMs produce correct answers while grounding intermediate steps in unsupported visual evidence. Intermediate visual claims are often not exposed in a symbolically addressable form, making standard RL with verifiable rewards unable to reliably inspect whether a reasoning step is actually supported by the image. We mitigate this gap for table reasoning with Anchored Reasoning, which turns each reasoning step into a checkable (coordinate, value, structural role) claim over the table grid. We instantiate this paradigm as V-tableR1: a policy VLM generates anchor-laden visual chains-of-thought, while a learned critic VLM estimates anchor fidelity against the source image. The policy is trained with PGPO, a GRPO-based process-guided formulation that integrates critic-gated trajectory rewards with stabilization mechanisms for long anchor-laden rollouts. V-tableR1 (4B) achieves the best open-source average accuracy across seven tabular reasoning benchmarks, obtains state-of-the-art results on several individual datasets, surpasses open-source models up to 18× its size in aggregate, and improves over its SFT baseline by up to 10.66% absolute accuracy.

Abstract:
Metadata-conditioned generation is often framed as a data-to-text task in which additional input fields improve grounding. Museum description generation challenges this assumption because catalogue records are heterogeneous, partially visual, and unevenly documented across cultural-geographic contexts. We study how metadata composition shapes artifact description generation through a two-phase evaluation. Phase 1 analyses natural metadata variation in a large museum collection, where field count, field type, image availability, and Place co-vary. Phase 2 introduces controlled field probes that hold object identity fixed while varying metadata quantity, field type, and image access. Our results show that metadata quantity alone does not explain generation behaviour. Field type determines whether models ground, omit, or infer information, while images can both support visual grounding and increase unsupported interpretation. More broadly, our findings show that metadata-conditioned data-to-text evaluation should assess field- and claim-level support, rather than relying only on aggregate input coverage or reference similarity

Abstract:
Large Language Models (LLMs) demonstrate strong multilingual reasoning performance, yet their robustness to semantics-preserving structural variation remains underexplored, particularly for relatively free word-order languages. We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam: constrained constituent reordering and active–passive voice transformation. We introduce a benchmark dataset IndicReStruct, with two variants, GSM8K-Reordered and GSM8K-Voice, constructed from GSM8K while preserving semantic meaning. Across six state-of-the-art LLMs and multiple prompting strategies, we observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs. To further understand these failures, we perform qualitative error analysis and mechanistic interpretability experiments using residual-stream activation patching. Our analyses show that reasoning failures frequently arise from disruptions in entity–quantity alignment and that intermediate transformer layers contribute most strongly toward reasoning restoration. Overall, our findings suggest that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different but semantically equivalent inputs.

Abstract:
LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible evidence is retrieved to address question constraints. This valuable information is buried in raw search trajectories that are long and difficult to parse. We introduce SearchAtlas, a framework that converts search trajectories into structured graphs whose edges represent how evidence is propagated across the reasoning trace, from the query that retrieves it to the final answer. Our automated parsing pipeline achieves a mean edge F₁ of 86.0% against human-annotated graphs and remains consistent across repeated runs. We analyze five search agents on three benchmarks, which reveal systematic differences in search scale and evidence aggregation across agents and benchmarks. SearchAtlas exposes fragmented answer support, question constraints that do not reach the answer, and unverified parametric knowledge entering the response. These process failures are strongly associated with incorrect answers, even more so than an LLM judge given either the raw trajectory or the ordered query list, suggesting that the constructed graphs provide useful interpretability. Moreover, an audit of cases in which process diagnostic scores disagree with final-answer correctness shows that they capture information not reducible to answer accuracy.

Abstract:
The current evaluation landscape for multilingual Large Language Models (LLMs) broadly assesses raw performance through static leaderboards, without offering a principled account of why performance gaps emerge. Gauging raw accuracy alone provides no explainable or actionable outcomes, masking systemic biases while failing to capture linguistic parity, a critical prerequisite for equitable deployment. To address this limitation, we first statistically demonstrate the widespread existence of severe performance disparities across multilingual benchmarks and then introduce a two-step hierarchical evaluation approach to break down their root causes. In the first step, we fit a Bayesian hierarchical mixed-effects model to isolate and analyze the performance variance attributable strictly to language identity, independent of task type. We demonstrate that observable language features (e.g., script, family, and typological distance) explain 79% of this language-based variance on understanding tasks and 92% on reasoning tasks - a quantity we formalize as R 2 ling , the proportional reduction in the language random-intercept variance attributable to observable features - with a model’s internal representational similarity to English emerging as the dominant predictor across both task buckets. In the second step, we broaden our analysis across the wider variance axes of the experimental design to unpack the systemic drivers of benchmark score variation. We find that natural language understanding (NLU) and reasoning tasks possess fundamentally divergent variance profiles: model identity heavily dominates understanding tasks (66.7% of total variance), whereas the benchmark-by-model interaction accounts for the largest share of variance in reasoning (46.3%). Together, these results transition multilingual evaluation from passive performance mapping into an explainable, diagnostic framework that yields actionable levers to intentionally target the root drivers of language disparity.

Abstract:
We introduce BuzzASR, a collection of language-specialized fine-tuned Whisper models adapted for automatic speech recognition (ASR) in 102 languages. Large end-to-end Transformer-based ASR models such as Whisper have revolutionized ASR, but most prominent models are highly multilingual. As a result, these models often perform poorly on languages less well-represented in their training set. While it has long been known that effective language adaptation can be achieved through simple fine-tuning on monolingual data, this strategy has only been applied to a small number of languages. We massively scale up this simple approach to 102 languages covered in the FLEURS dataset, while also implementing a more complex language adaptation strategy that integrates monolingual tokenizer replacement and data augmentation using text-only fine-tuning. BuzzASR models outperform Whisper on 77 out of 102 languages, reducing character error rates (CER) by a factor of over 2.8 on average. Our models achieve state-of-the-art CER among open-source systems on 27 of 102 languages on the combined FLEURS and Common Voice test set. Our tokenizer replacement strategy yields an average 3.4x improvement in compression rate (characters per token) over Whisper’s multilingual BPE, with gains of up to 21.7x.

Abstract:
Mixture-of-Experts (MoE) pretraining relies on an auxiliary load-balancing loss (LBL) to drive per-expert utilization toward uniformity. Post-training inherits a different situation: the base router already encodes non-uniform expert co-activation structure, which a re-imposed uniformity objective flattens away. We show that downstream performance depends instead on holding this inherited routing softly, a principle we term soft router anchoring, and instantiate it as Router Prior Bias (RPB), a training-time bias that pulls the router logits toward a prior read off the frozen base router while leaving the router itself trainable. On math post-training of Moonlight-16B-A3B, RPB attains 45.77 in-domain accuracy against 31.91 under re-applied LBL and 29.44 under unanchored fine-tuning, and retains more out-of-domain capability than either. The ordering against LBL reproduces on a second model family (Qwen3-30B-A3B-Base), and the advantage over LBL is resolvable on an independently sourced corpus. Anchors defined on the router weights, on its logits, or on its output distribution perform comparably with no consistent ordering, which places the effect in the softness of the constraint rather than in the particular prior RPB supplies. Retained community structure in the expert co-activation graph tracks these gains wherever the base router is non-uniform enough for communities to form, yet enforcing the same prior as a hard assignment preserves that structure while performance falls sharply. Community structure is therefore a footprint of soft anchoring rather than its source, and the practical lesson is that inherited routing should be held softly during post-training, since both flattening it toward uniformity and enforcing it absolutely carry a downstream cost. Our code will be released at https://github.com/naver-ai/rpb.

Abstract:
Agent self-evolution in long-horizon LLM systems is largely procedural: useful experience is not merely stored information, but reusable procedures for searching, debugging, and verification. Yet current evaluations do not isolate this form of transfer. Agent benchmarks test single-episode task solving; memory benchmarks target information retention rather than procedural reuse. We introduce EvoAgentBench, a benchmark for agent self-evolution with Ability-guided transfer evaluated separately in four agentic domains: web research, algorithmic reasoning, software engineering, and knowledge work. EvoAgentBench extracts trace-grounded Abilities from agent executions, canonicalizes them into operational units, and builds domain-specific Ability Graphs linking tasks that share procedural overlap. By design, every test task is backed by verified training-side Ability support. Across a 528/267 train/test split, two scaffolds, and three backbones, curated Ability content yields a positive aggregate effect and positive gains in every evaluated configuration, while gains from current automatic methods remain inconsistent across configurations. EvoAgentBench shifts self-evolution evaluation from aggregate accuracy comparison to fine-grained diagnosis of experience encoding, routing, exposure, and uptake.

Abstract:
Recent advances in Large Reasoning Models (LRMs) are reshaping reasoning-intensive reranking, where step-by-step reasoning enables multi-faceted relevance judgments. To scale LRMs, existing approaches distill chain-of-thought traces from a teacher LRM into a student reranker via Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL). However, this pipeline is misaligned with the distillation objective: teacher traces are too dense for direct imitation, while RL exploration is too sparse on hard cases. To this end, we propose SARD (Self-Adaptive Reinforced Distillation), a tri-mode RL framework that recasts teacher reasoning from a fixed supervision target into an adaptive resource the student draws on only when its own reasoning falls short. Triggered by rollout outcomes, SARD organizes training into three modes: Free Exploration, where the student reasons independently; Guided Exploration, which supplies reusable in-context guidance abstracted from teacher traces when free rollouts fail; and Teacher-Guided Repair, where the teacher diagnoses failures and revises, re-selects, or appends guidance, falling back to teacher trajectories only as a last resort. Experiments on reasoning-intensive reranking benchmarks show SARD outperforms strong reranking and SFT-then-RL baselines, with clear gains on difficult instances and under-represented reasoning patterns.

Abstract:
Hyphenation patterns remain a compact and widely deployed solution for word breaking in typesetting systems, text processors, and web rendering engines, but their generation still depends on manually tuned patgen program’s parameter profiles. We formulate patgen profile selection as a black-box hyperparameter optimization problem and evaluate Gaussian-process Bayesian optimization for this task. The search objective combines a precision-oriented F 1/7 score with an explicit trie size-accuracy tradeoff using a normalized trie-size penalty. We evaluate the method on 17 hyphenated word-list datasets covering 14 languages and multiple scripts. Against two strong hand-tuned profiles regenerated from the same 8/10 training split and evaluated on the same 1/10 held-out test split, the GP-optimized profiles improve F 1/7 and reduce trie size on all 17 datasets. The median optimized/baseline trie ratio is 0.343. A dataset-level sign test gives p=7.6⋅10 -6 ; a separate budget-matched comparison on five representative datasets shows that model-based search improves over the best hand-tuned profile under the fixed comparison objective. The results show that model-based optimization can make pattern generation more reproducible and less dependent on expert trial-and-error while keeping the accuracy–compactness trade-off explicit.

Abstract:
As model sizes continue to scale, distributed training has become inevitable. Automatic parallelization techniques can derive efficient training parallelism strategies at low cost while achieving superior performance. The difficulty of this problem is jointly determined by the complexity of the model and the underlying compute cluster. Meanwhile, mixture-of-experts (MoE) models are increasingly emerging as the dominant architecture and the rapid evolution of accelerator hardware has made cluster heterogeneity commonplace, posing substantial challenges to automatic parallelization. However, existing approaches typically target either MoE architectures or heterogeneous clusters, failing to generalize to scenarios where both challenges coexist. To this end, we present HAPMoE, a heterogeneity-aware automatic parallelism planner for MoE training. HAPMoE builds a lightweight MoE-aware cost model and efficiently searches a six-dimensional parallel space, producing parallel plans directly deployable on Megatron-LM. Experiments show that HAPMoE improves end-to-end training throughput by up to 3.2× over baselines across heterogeneous clusters. Its non-uniform pipeline partitioning yields an additional up to 78% gains, and its pruning-enhanced dynamic programming algorithm completes the search within 1 minute, demonstrating high efficiency and practical value in complex hardware environments.

Abstract:
Financial multimodal reasoning requires agents to coordinate numerical computation, retrieval, visual interpretation, and temporal grounding across heterogeneous evidence sources. Existing tool-augmented agents improve execution fidelity, yet remain largely stateless across episodes, repeatedly rediscovering reasoning strategies and failure patterns. In high-stakes financial settings, this leads to unreliable tool routing, noisy retrieval, and hallucination-prone reasoning. We present FinAcumen, a financial reasoning agent framework centered on selective experience memory for tool-augmented multimodal reasoning. FinAcumen accumulates financially grounded reasoning experience from prior trajectories, distilling successful strategies and failure-derived cautionary rules into a persistent memory bank. During inference, retrieved experiences condition reasoning only when semantic relevance exceeds a calibrated threshold, while irrelevant memory is explicitly suppressed through a fallback mechanism. A deterministic financial tool environment further grounds numerical computation, retrieval, visual decoding, and answer verification. Across four financial multimodal reasoning benchmarks, FinAcumen consistently improves a frozen 8B vision-language model over finance-specialized models and approaches leading proprietary general-purpose models. Further analysis shows that selective experience activation improves reasoning reliability under retrieval uncertainty. Our code is available at https://github.com/CamelliaLilium/FinAcumen

Abstract:
As large language models are deployed at scale, the ability to selectively remove knowledge has become critical for privacy, safety, and legal compliance. Existing post-training unlearning methods suffer from a persistent tension: unbounded divergence signals over-unlearn, degrading utility and producing incoherent outputs, while fixed sample-agnostic targets and reference-relative penalties under-unlearn, leaving traces recoverable under paraphrase or membership-inference attacks. We attribute both failures to a common structural cause: the forgetting objective is decoupled from any sample- or parameter-level notion of what retention requires. We present FLOUR (Forgetting via Learned On-manifold Unlearning Representations), a latent unlearning framework in which a lightweight encoder produces a per-sample suppression direction. This direction is anchored to the null region of the reference model’s activations, keeping it on-manifold and disentangled from retention-critical directions, which enables forgetting while preserving coherent generation and enhancing privacy. Results on the Open-Unlearning benchmark suite show that FLOUR most closely matches the retrain oracle across the unlearning trilemma of forgetting, retention, and privacy, substantially outperforming prior baselines. Our code is publicly available at https://github.com/ judydnguyen/flour-unlearning.

Abstract:
Large Language Models (LLMs) are increasingly serving as personal assistants, where users may share individual preferences over extended interactions. However, assessing how well LLMs can follow these preferences in natural, long-term situations remains underexplored. This work proposes RealPref, a benchmark for evaluating natural preference-following in personalized user-LLM interactions. RealPref features 100 synthetic user profiles, 1300 personalized preferences, 4 types of preference expression (from explicit to implicit), and long-horizon interaction histories. It explored three types of test tasks (multiple-choice, true-or-false, and open-ended), with granular rubrics for LLM-as-a-judge evaluation. Results indicate that LLM performance drops significantly as context length grows and preference expression becomes more implicit, and that generalizing user preference understanding to unseen scenarios poses further challenges. RealPref and these findings provide a foundation for future research to develop user-aware LLM assistants that better adapt to individual needs.

Abstract:
Current context augmentation methods, such as retrieval-augmented generation, play a crucial role in bridging a model’s internal knowledge boundary and external evidence for multi-hop reasoning. However, they often follow a rigid policy and treat external retrieval as the default action at each step. Such brute-force context expansion incurs unnecessary computational cost and may degrade reasoning performance by saturating the context with redundant or weakly relevant evidence. In this paper, we propose Cross-Boundary Context Evolution (EvoCtx), a framework that models complex reasoning as an adaptive process of boundary-aware context evolution. EvoCtx dynamically decides whether the next reasoning transition should cross the current evidence boundary through retrieval or refine the reasoning state within the existing context. It estimates the semantic gap between the reasoning state and the accumulated evidence, and strategically alternates between boundary expansion and intra-boundary trajectory refinement. This eliminates redundant retrieval steps and preserves a compact, evidence-supported reasoning trajectory. Extensive experiments on challenging open-domain and multi-hop QA benchmarks demonstrate that EvoCtx significantly outperforms previous methods, offering an effective approach to complex tasks. The source code can be accessed at https://github.com/Anya-RB-Chen/EvoCtx.

Abstract:
Fine-grained credit assignment is a central challenge in reinforcement learning for long-horizon LLM agents. Standard objectives often train from programmatically verifiable terminal rewards by broadcasting each sparse outcome to every action in a trajectory. Existing methods typically seek finer credit from the rollout side, constructing auxiliary trajectory signals or additional comparisons to estimate action importance. Although useful, these approaches still treat the verifier that judged success as a scalar reward, discarding its internal task structure. Our key insight is that many verifiable tasks already encode the relevant checks inside their terminal verifier. We propose VICT (Verifier-Instrumented Credit Tracing), a training-time interface that exposes executable or evidence-backed atoms and traces them back to actions through dependency-valid proof edges. VICT redistributes group-relative advantage only along those edges, shifting credit assignment from rollout-side inference to verifier-side tracing. It preserves the original terminal reward, abstains when evidence is incomplete or ambiguous, and changes only the training-time advantage tensor, requiring no learned critic, process labels, branch rollouts, or inference-time verifier access. Under matched rollout budgets on ALFWorld and WebShop, VICT improves over outcome-only and rollout-side credit baselines; ablations rule out dense atom rewards, final-commit credit, temporal proximity, and sparsity as sufficient explanations.

Abstract:
Large Language Models often achieve strong performance by generating long intermediate chain-of-thought (CoT) reasoning. However, it remains unclear when a model’s final answer is actually determined during generation. If the answer is already fixed at an intermediate stage, subsequent reasoning tokens may constitute post-decision explanation, increasing inference cost and latency without improving correctness. We study the evolution of predicted answers over reasoning steps using forced answer completion, which elicits a model’s intermediate prediction at partial reasoning prefixes. Across a range of models and datasets, we find that the final answer is fixed from the earliest stages of generation in most cases, with only 32% of answers changing after CoT. Moreover, once the final answer switch does occur, models continue to generate substantial reasoning, producing 760 additional tokens per query on average and accounting for around 40% of the total reasoning. Motivated by these findings, we explore early-stopping strategies that halt generation once the answer has stabilized. We show that simple probe-based stopping can reduce reasoning token usage by 500 tokens per query while incurring only a 2% drop in accuracy. Together, our results indicate that a large portion of chain-of-thought generation functions as post hoc explanation and can be reduced with minimal impact on performance.

Abstract:
Diffusion Large Language Models (dLLMs) have emerged as an efficient alternative to autoregressive models, yet aligning them via Reinforcement Learning (RL) requires likelihood surrogates estimated from masked reconstruction subproblems under a small Monte Carlo budget per rollout. Existing methods construct these subproblems by uniform random masking, leaving open the question of which subproblems to prioritize. We identify a systematic upstream/downstream structure in dLLM rollouts. Some tokens, when revealed, trigger large confidence changes in nearby undecoded positions; we call them upstream. Others induce only small local changes and are therefore downstream. We find masking downstream tokens yields substantially better-posed subproblems than masking upstream tokens, a phenomenon we term subproblem difficulty asymmetry. Based on the observation, we propose Informed Masking (IM), which derives a per-token priority score from the denoising trajectory at zero extra inference cost and biases mask sampling toward downstream tokens. IM is plug-and-play: when plugged into three state-of-the-art dLLM RL methods on LLaDA-8B-Instruct, it delivers up to 2.01%, 8.68%, and 5.77% relative average gains on math and planning benchmarks with improved training stability.

Abstract:
Reinforcement learning (RL), particularly RL with Verifiable Rewards (RLVR), has recently emerged as a central paradigm for enhancing large language models’ (LLMs) reasoning abilities, demonstrating remarkable effectiveness across reasoning tasks. Recent studies suggest that high-entropy tokens play an exceptionally important role in model training, since training with only the highest 20% entropy tokens yields significant performance gains. However, why such high-entropy tokens are beneficial remains insufficiently understood. In this work, we find that although high-entropy tokens within one answer tend to correlate with large gradient magnitude, entropy alone fails to consistently reflect token importance across different answers, considering the variations in the answer-level reward signals. Based on this observation, we introduce the Gradient Magnitude-based Token Selection (GMTS) method to quantify token importance, which leverages the entropy–gradient connection to approximate gradient-magnitude rankings for token selection. We find that training on the top 20% tokens ranked by GMTS consistently outperforms entropy-based token selection across three reasoning domains and various model sizes, suggesting that GMTS provides a more fine-grained estimate of token contribution for RLVR training.

Abstract:
The capacity of Low-Rank Adaptation (LoRA) is fundamentally governed by its rank parameter. While standard implementations apply a uniform rank across all layers, a recent wave of variable-rank extensions demonstrates that tailoring ranks to module importance improves expressiveness. However, this emerging literature remains highly fragmented and lacks systematic cross-comparisons. To unify the field, we propose a taxonomy grouping these approaches into four categories: Rank growth, pruning, redistribution, and pre-allocation. We analyze their design dimensions, theoretical expressiveness, implementation complexity, and memory efficiency. Finally, we outline critical challenges inhibiting adoption, including standardized evaluations and memory metrics.

Abstract:
Previous work shows that a model’s ability to define domain terms correlates to performance in other tasks related to that domain. For example, in a QA setting, a language model can miss a question in a specialized domain for no reason other than not properly recalling what a term in the question means. We argue that a short glossary definition is the smallest unit of evidence a retrieval system can supply against that failure. Concretely, focusing on the finance NLP domain, we assess the role of definitions in improving performance on XFinBench, and also pairing MMLU-Finance with a 13,609-term Investopedia glossary of financial terms. We experiment with a five-model panel spanning mid-capability open models to a frontier closed model, and show that definition-only retrieval produces a structured gain on these datasets, with the effect concentrating in a mid-capability band rather than scaling with model size. Finally, we connect term-level measurement to individual questions through a human-validated annotation, and find that term knowledge does not locate the questions where a definition changes the answer, with none our implemented internal routing signals approaching the per-question oracle.

Abstract:
A key challenge in reliable LLM deployment is recognizing when uncertainty reflects irreducible variability in the task rather than limitations in the model’s knowledge. In language tasks, a central source of such aleatoric uncertainty is input ambiguity or underspecification, where multiple interpretations remain plausible. Existing decomposition methods estimate aleatoric uncertainty by generating multiple clarifications of the input, querying the model for an answer under each clarification, and comparing the resulting answers. We argue that answers are not necessary for identifying ambiguity: they are often redundant, add avoidable cost, and can mislead through epistemic leakage. We support this claim theoretically, and propose a clarification-only approach that estimates this ambiguity-induced component directly from the space of plausible interpretations, without answers to the clarified inputs. Using ambiguity detection as an operational evaluation across three benchmarks, this direct approach improves AUROC (63.34 vs. 60.85), reduces computational cost by 4–26x in output tokens and 2.2–3.5x in API calls, and yields estimates with substantially lower correlation with epistemic uncertainty. Overall, our results suggest that ambiguity-induced aleatoric uncertainty is better estimated from the interpretation space than from the response space.

Abstract:
Perspective API closes at the end of 2026, removing the de facto standard for toxicity measurement and exposing researchers’ dependence on a tool they did not control. Drawing on this case, we argue that a research field must build and govern its own measurement infrastructure rather than borrow it. Surveying 241 papers that use or study Perspective, we show what depending on it cost the research community: claims reaching past what the tool could support, results that shifted when its model was silently retrained, and disparities researchers could measure but not explain. These failures were amplified throughout the LLM lifecycle, where Perspective supplied the labels, filtered the corpora, and graded the systems trained on each, rewarding errors rather than catching them. To keep the instrument open to study after shutdown, we release Perspective scores for 5.9 million text snippets from 77 datasets. We further specify ten requirements for measurement infrastructure a field owns, and argue that what blocks such infrastructure is not technical capability but the value the field places on infrastructure work.

Abstract:
By continuously acquiring new knowledge, continual learning enables large language models (LLMs) to remain adaptive, accurate, and relevant as the world evolves. In continual learning, tasks arrive one after another and the model learns them sequentially. As more tasks are introduced, existing methods for learning a new task impose increasing computational demands. However, we find that the computational overhead, which gradually increases as the number of tasks grows, can be mitigated. To limit the continual increase in computational costs, we introduce a simple but efficient block-sparse (EBS) framework for continual learning of LLMs. In EBS, we employ a dedicated, task-specific low-rank adaptation (LoRA) block for each task. When training on a new task, only one LoRA block is loaded into GPU memory and involved in computation, while at inference each input activates at most two LoRA blocks. This design ensures that the computational cost remains independent of the number of tasks, allowing EBS to maintain high efficiency even when the sequence of tasks becomes very large. We offer a simple implementation that works across different model sizes and transformer architectures (decoder-only and encoder-decoder) while consistently delivering both high efficiency and strong task performance. Experimental results on two continual learning benchmarks demonstrate that EBS achieves state-of-the-art performance on the T5 and LLaMA 2 models.

Abstract:
Tokenization is a fundamental component of language modeling pipelines. Despite its importance, it is often fixed, even though it significantly impacts model performance across languages. In this work, we analyze what tokens are learned when tokenization is jointly optimized with language modeling. We compare tokenizer-free approaches such as SSLMs and H-Nets with fixed tokenizers across 18 typologically and script-diverse languages. Our results show that joint optimization fundamentally alters token structure. SSLMs recover morphologically aligned and contextually efficient tokens, whereas H-Nets prioritize byte-level efficiency, producing longer tokens with very low overlap with standard subword vocabularies. We further show that tokenization behavior varies across language typologies. Agglutinative languages exhibit more dynamic segmentation patterns while learning. Through downstream evaluation, with pretrained-then-finetuned BERT models, we find that SSLM-based pretokenization consistently reduces language modeling perplexity and achieves competitive downstream performance despite distinct vocabularies. Overall, tokenizer-free approaches optimize for contextual and computational efficiency rather than strict morphological structure, resulting in fundamentally different yet effective vocabularies for downstream NLP.

Abstract:
Search Agents—large language models augmented with search tools—have intensified the need for future-proof evaluation benchmarks. Existing benchmarks such as BrowseComp rely on static knowledge, making them vulnerable to test-set contamination and parametric memorization. Consequently, models can achieve high scores through fact recall rather than genuine retrieval, obscuring true browsing competence via reasoning shortcuts. In this paper, we introduce EvoBrowseComp, an evolving benchmark of 400 English and 400 Chinese contamination-free complex questions synthesized via live-web traversal. To collect these questions, we design a three-agent collaborative framework: (1) a QA synthesis agent that retrieves fresh knowledge from the live web to synthesize QA pairs; (2) an information filtering agent that filters retrieved knowledge in terms of credibility and popularity to block parametric shortcuts; and (3) a high-level guidance agent that formalizes questions into reasoning graphs to reduce logical redundancy and shortcuts in synthesized QA pairs. Because the framework supports fully automated synthesis, EvoBrowseComp can be regularly updated to prevent data contamination and maintain temporal freshness. Extensive experiments confirm its great difficulty, requiring broad horizontal search. It establishes a scalable paradigm for auto-updatable, high-difficulty benchmarking that keeps pace with both evolving world knowledge and advancing agent capabilities.

Abstract:
Compact vision-language models (VLMs) now power a growing share of multimodal applications. The benchmarks used to compare them, however, inherit a frontier-centric design: each model is reduced to a single accuracy number, narrowing the inter-model gap on saturated suites and pressing models into low-score bands on harder ones. We introduce PRISM-VLM, a multi-axis discriminative benchmark that scores every item along seven axes covering the recurring failure modes (task quality, behavioral robustness, and capability bottlenecks) and combines them into a single PSCORE, with items recycled from fifteen public benchmarks. Across compact VLMs from the past two years, PSCORE separates model pairs more reliably than prior single-axis benchmarks under an item-level paired bootstrap, and surfaces behavioral differences these benchmarks average away. Even models with statistically indistinguishable PSCOREs diverge sharply along the per-axis profile, particularly on sycophancy, which is nearly orthogonal to single-prompt accuracy. We will release the full pipeline, prompts, and per-item annotations.

Abstract:
Large reasoning models have demonstrated remarkable proficiency in various tasks. However, we observe that they frequently exhibit a problematic reliance on familiar reasoning patterns, a phenomenon we term reasoning rigidity. Despite explicit instructions from users, these models often override clearly stated conditions and default to habitual reasoning, leading to incorrect conclusions. This behavior presents significant challenges, not only in reasoning-intensive domains but also in realistic settings. To systematically investigate reasoning rigidity, a behavior unexplored in prior work, we introduce a dataset, ReasoningTrap, consisting of math problems, puzzles, and agentic tasks designed to diagnose reasoning rigidity. Using this dataset, we identify patterns that occur when models default to ingrained reasoning, and suggest inference-level and GRPO-based post-training remedies. Our diagnostic set and evaluation code are available at the project page to facilitate future research on mitigating reasoning rigidity in language models.

Abstract:
This paper presents a simple method that allows to easily enhance textual pre-trained large language models with speech information, when fine-tuned for a specific classification task. A classical issue with the fusion of many embeddings from audio with text is the large length of the audio sequence compared to the text one. Our method benefits from an existing speech tokenizer trained for Audio Speech Recognition that output long sequences of tokens from a large vocabulary, making it difficult to integrate it at low cost in a large language model. By applying a simple lasso-based feature selection on multimodal Bag-of-Words representation, we retain only the most important audio tokens for the task, and adapt the language model to them with a self-supervised language modeling objective, before fine-tuning it on the downstream task. We show this method improves the performances compared to an unimodal model, larger SpeechLMs, and audio integration via learned representations. We demonstrate its effectiveness on Argumentative Fallacy Detection and Classification tasks where audio was previously believed counterproductive, and affective computing tasks on a widely-used dataset. We also provide an in-depth analysis of the method, showing that even a random audio token selection helps enhancing the unimodal model. Our code is available online .

Abstract:
Politeness is key to building healthy communication. Brown and Levinson’s influential theory of politeness (BL’s theory) proposes that polite speech can be characterized by the presence of face-saving speech acts while impolite speech is characterized by the presence of face-threatening speech acts. Most existing politeness resources neither operationalize nor control politeness under BL’s theory; instead, they typically consist of short utterances with an intuitive account of politeness. In this paper, we introduce a long-text corpus that includes sentence-level face-act annotations and document-level politeness scores. Following BL’s theory, we annotate politeness along three dimensions: Directness vs. Indirectness, Positive Face Saving, and Negative Face Saving, and define Overall Politeness as their average. We formulate face-act tagging as a multi-label classification task and compare context-agnostic sentence classifiers to context-sensitive variants and sequence-labeling models. We also model document-level politeness via multi-target regression using the text, and enhance the regression model with the predictions of our face-act classifier. We show that context-sensitive classifiers outperform context-free classifiers for face-act tagging. Moreover, we demonstrate that for document-level politeness regression, enhancing the text with face-act predictions yields better performance. Our code and data are available online: alipanahroshad-oss/email-politeness-corpus

Abstract:
Resolving egocentric spatial expressions in conversational navigation requires mapping speaker-centric references to allocentric orientations. In practice, this process is often mediated by automatic speech recognition (ASR), where transcription errors, linguistic variation, and referential ambiguity obscure spatial relations. Despite progress in navigation and multimodal reasoning, egocentric spatial reference resolution under ASR-transcribed, non-English conversational input remains underexplored. We introduce Conversational Orientation Reasoning (COR), a diagnostic benchmark for egocentric spatial reference resolution in Traditional Chinese conversational navigation. COR consists of 10,000 instances pairing ASR-transcribed utterances with structured landmark coordinates derived from real-world environments projected onto a 50 x 50 grid. COR isolates the effects of spoken language noise from geometric computation, enabling evaluation of transcription errors, linguistic variation, and referential ambiguity beyond end-to-end navigation success. Using COR, we study an interpretable decomposition of orientation inference into three steps: extracting spatial relations, mapping landmark coordinates to absolute directions, and inferring the speaker’s orientation. Models following this decomposition achieve 99.6% accuracy on clean text and retain 95.7% accuracy under ASR-transcribed input. COR supports analysis of failure modes in spoken spatial reference resolution, providing insights into how language grounding degrades under noisy conversational input. The code and dataset are publicly available under the MIT license.

Abstract:
LLMs are increasingly able to answer complex questions about enterprise-scale document collections. But evaluation is hard: companies don’t want to share internal communications, and synthetic datasets have been overly simple. We present CorporateBench (CB), a human-validated multi-task Q A benchmark whose scale approaches the conditions LLMs encounter in corporate communication networks, with evaluation corpora surpassing 230,000 documents. CB evaluates LLMs across two dimensions (information extraction and knowledge base querying) through four synthetically generated firms ranging from 12 to 10,000 employees. Each corpus is sampled from a temporally evolving knowledge base describing a consistent world, guaranteeing cross-document logical consistency even across hundreds of thousands of documents. We evaluate five LLMs on CB, revealing increasingly poor performance as input size approaches realistic scales. CB provides LLM developers a metric for corporate communication reasoning, filling a crucial gap in the benchmarking ecosystem.

Abstract:
AI-generated content has progressed from monolithic models to modular workflows, especially on platforms like ComfyUI, allowing users to customize complex creative pipelines. However, the large number of components in ComfyUI and the difficulty of maintaining long-horizon structural consistency under strict graph constraints frequently lead to low pass rates and workflows of limited quality. To tackle these limitations, we present ComfySearch, an agentic framework that can effectively explore the component space and generate functional ComfyUI pipelines via validation-guided workflow construction. Experiments demonstrate that ComfySearch substantially outperforms existing methods on complex and creative tasks, achieving higher executability (pass) rates, higher solution rates, and stronger generalization.

Abstract:
Humans are influenced by how information is presented, a phenomenon known as the framing effect. Prior work suggests that LLMs may also be susceptible to framing, but it has relied on synthetic data and did not compare to human behavior. To address this gap, we introduce WildFrame - a dataset for evaluating LLM responses to positive and negative framing in naturally-occurring sentences, alongside human responses on the same data. WildFrame consists of 1,000 real-world texts selected to convey a clear sentiment; we then reframe each text in either a positive or negative light and collect human sentiment annotations. Evaluating eleven LLMs on WildFrame, we find that all models respond to reframing in a human-like manner ( r≥ 0.52 ), and that both humans and models are influenced more by positive than negative reframing. Notably, GPT models are the least correlated with human behavior among all tested models. These findings raise a discussion around the goals of state-of-the-art LLM development and whether models should align closely with human behavior, to preserve cognitive phenomena such as the framing effect, or instead mitigate such biases in favor of fairness and consistency.

Abstract:
Large Language Models (LLMs) acquire extensive factual knowledge during computationally expensive pretraining. Knowledge editing aims to efficiently update specific facts in a trained model without retraining the entire network. While many editing methods have recently been proposed and show strong performance on existing benchmarks, these benchmarks exhibit several limitations. In particular, they allow only a single edit per entity, rely on highly templated prompts, and evaluate edits using relative probability changes rather than ground-truth correctness. As a result, the capabilities and limitations of current knowledge editing methods remain insufficiently understood. In this work, we introduce the Subject-centric Entity Editing Dataset (SEED), a dataset and evaluation framework designed to address these limitations. SEED organizes edits into sets of related and unrelated facts rather than isolated instances, enabling evaluation when multiple facts about the same subject entity are updated. It also introduces undirected knowledge probes that test whether edited facts can be recalled in general contexts, and ground-truth evaluation metrics that directly measure whether the edited fact appears in the model’s output. Using SEED, we test 9 state-of-the-art knowledge editing methods over 10 metrics. These studies find that editing performance decreases by up to 36.9% as the number of edits per entity increases and edit recall decreases by 36.6% with prompts that increase token-wise distance between the edit subject and relation.

Abstract:
Diffusion and flow-matching models have revolutionized automatic text-to-audio generation, increasingly capable of generating high-quality audio outputs for speech and acoustic events. However, creative audio generation involving music and songs still has room for improvement. Recent open lyrics-to-song models like DiffRhythm, ACE-Step, and LeVo have set an acceptable standard for recreational use but lack the fine-grained word-level controllability desired by musicians. To our knowledge, our flow-matching-based JAM is the first to provide word-level timing and duration control in song generation, enabling fine-grained vocal control. To better align generated songs with human preferences, we implement aesthetic alignment through Direct Preference Optimization, iteratively refining the model using a synthetic dataset without manual annotations. Additionally, we aim to standardize evaluation of lyrics-to-song models through our public evaluation dataset JAME. We demonstrate that JAM outperforms open models on music-specific attributes.

Abstract:
Retrieval-Augmented Generation (RAG) is a key approach to mitigating the temporal staleness of large language models (LLMs) by grounding responses in up-to-date evidence. Within the RAG pipeline, re-rankers play a pivotal role in selecting the most useful documents from retrieved candidates. However, existing benchmarks predominantly evaluate re-rankers in static settings and do not adequately assess performance under evolving information—a critical gap, as real-world systems often must choose among temporally different pieces of evidence. To address this limitation, we introduce FRESCO ( F actual R ecency and E volving S emantic CO nflict), a benchmark for evaluating re-rankers in temporally dynamic contexts. By pairing recency-seeking queries with historical Wikipedia revisions, FRESCO tests whether re-rankers can prioritize factually recent evidence while maintaining semantic relevance. Our evaluation reveals a consistent failure mode across existing re-rankers: a strong bias toward older, semantically rich documents, even when they are factually obsolete. We further investigate an instruction optimization framework to mitigate this issue. By identifying Pareto-optimal instructions that balance Evolving and Non-Evolving Knowledge tasks, we obtain gains of up to 27% on Evolving Knowledge tasks while maintaining competitive performance on Non-Evolving Knowledge tasks.

Abstract:
Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages remains poorly understood. We conduct a controlled LoRA fine-tuning study across multiple open-weight LLM families and scales, using a standardised grid of 11 languages and four benchmarks. We fine-tune each model on a single task–language source, then evaluate it on all other task–language target pairs to measure transfer. We decompose transfer into three regimes: (i) Matched-Task (Cross-Language), (ii) Cross-Task (Matched-Language), and (iii) Cross-Task (Cross-Language). Single-source fine-tuning yields a net positive uplift across regimes, but the gains are strongly asymmetric. Matched-Task (Cross-Language) transfer emerges as the most effective and structurally regular regime, with transfer magnitude driven principally by the identity of the target language rather than model architecture. We identify a stable coarse-grained hierarchy in which some task and language targets consistently absorb gains from diverse sources, while others remain relatively isolated. These results imply that effective fine-tuning requires accounting for donor–recipient roles to maximise downstream gains while limiting collateral degradation in other capabilities.

Abstract:
Drug recommendation requires a deep understanding of individual patient context, especially for complex conditions like Parkinson’s disease. While LLMs possess broad medical knowledge, they fail to capture the subtle nuances of actual prescribing patterns. Existing RAG methods also struggle with these complexities because guideline-based retrieval remains too generic and similar-patient retrieval often replicates majority patterns without accounting for the unique clinical nuances of individual patients. To bridge this gap, we propose PACE-RAG (Patient-Aware Contextual and Evidence-Constrained RAG). Rather than directly copying frequent medications from retrieved patients, PACE-RAG personalizes recommendations by first extracting patient-specific clinical features, retrieving cases around these features, and then refining the final prescription using the patient’s current symptoms, active medication history, and focus-specific prescribing tendencies. By analyzing treatment patterns tailored to specific clinical features, PACE-RAG generates patient-specific medication recommendations along with an explainable clinical summary. PACE-RAG achieved the strongest performance among the evaluated inference-only LLM-based methods, reaching F1 scores of 80.84% and 47.22% on the Parkinson’s disease and MIMIC-IV cohorts, respectively. Our code is available at: https://github.com/ChaeYoungHuh/PACE-RAG.

Abstract:
While current emotional support dialogue systems typically rely on expert-defined scalar rewards for alignment, these signals suffer from severe information sparsity. They cannot explain why a response failed or how to adapt to dynamic user states, often failing to capture interaction-level conversational consequences reflected in subsequent user responses. We explore whether user reactions can serve as an additional behavior-level optimization signal for emotional support alignment. Therefore, we propose Reaction Aware Policy Optimization (RAPO), a framework that optimizes over interaction consequences rather than rubric scores. RAPO treats dialogue as a reaction-driven process and utilizes simulated user responses to generate dense natural-language feedback through three core components: (1) Hindsight Dialogue Selection, which isolates pivotal turns that meaningfully alter user emotional trajectories; (2) Generative Hindsight Feedback, which transforms user reactions into contrastive ranking signals and natural-language critiques; and (3) Scalar–Verbal Hybrid Policy Optimization, which couples scalar reward optimization for global alignment with verbal feedback distillation for fine-grained semantic refinement. Extensive experiments on ESC and Sotopia demonstrate that RAPO significantly outperforms strong RL baselines in driving positive interaction outcomes.

Abstract:
Chinese discriminatory-language detection is challenging because harmful intent is often implicit and context-dependent. We propose MAAM (Myopia–Astigmatism Anchor Mechanism), a lightweight, model-agnostic framework inspired by functional visual blur: rather than preserving every token equally, MAAM retains discrimination-relevant semantic anchors and calibrates them with C–I–S contextual priors (Contextual Tone, Group Identity, and Stance Polarity). We also introduce ChLGBT, to our knowledge the first Chinese LGBT-focused discriminatory-language dataset, with 8,120 manually annotated samples and three ordinal labels: explicit bias, implicit bias, and emotional intensity. Across strong encoder baselines, MAAM improves all three prediction dimensions, with consistent gains in accuracy, F1, Brier score, and expected calibration error. Compared with frontier LLM baselines under zero-shot and few-shot prompting protocols, MAAM remains competitive while offering stronger compactness and stability. These results suggest that interpretable anchor preservation and contextual calibration provide a practical alternative to heavier model scaling for Chinese discriminatory-language assessment.

Abstract:
Translating natural language to First-Order Logic (NL2FOL) is crucial for logic-based reasoning, but most existing models and benchmarks focus on English, leaving cross-lingual generalization underexplored. In this study, we present the first comprehensive investigation of cross-lingual robustness in NL2FOL. To support this, we introduce Multi-MALLS, the first multilingual benchmark for NL2FOL, which extends the widely-used MALLS dataset with high-quality translations into multiple languages. Using Multi-MALLS, we observe that state-of-the-art models experience a significant performance drop when applied to non-English inputs, highlighting the lack of robustness. To address this, we propose MA-FOL, a multi-agent framework that improves generalization without requiring any training on target languages. By decomposing the task into three modules, a language-agnostic structure generator, a language-specific predicate combiner, and a refinement component, MA-FOL achieves robust zero-shot generalization across diverse linguistic inputs. Additionally, we show that traditional evaluation metrics, such as Exact Match, often fail to assess semantic correctness. To remedy this, we introduce large language models (LLMs)-Judged Semantic Equivalence (SE), a new metric that leverages LLMs to evaluate whether generated formulas preserve the intended meaning. Extensive experiments demonstrate that MA-FOL outperforms strong baselines on Multi-MALLS, without any multilingual fine-tuning. The SE metric further reveals semantic correctness that traditional metrics miss. Overall, our work provides a benchmark to test robustness of NL2FOL, a framework to improve it, and a metric to evaluate it more effectively.

Abstract:
Bi-encoder architectures for biomedical entity linking learn flat embedding spaces that ignore ontology structure. HiCo-EL encodes hierarchical knowledge into bi-encoder representations through hierarchical contrastive pretraining. The core idea is within-space hierarchical multi-label contrastive learning (HMLC): contrastive losses applied independently within each encoder space, preserving cross-space compatibility for retrieval. Our formulation drops the monotonicity constraints of prior HMLC work, handles variable-depth hierarchy paths, and adds depth-weighted alignment between encoder spaces. Treating each entity as a leaf node unifies coarse type structure with fine-grained entity discrimination. On MedMentions and NCBI Disease, HiCo-EL outperforms ArboEL by +3.7% and +1.8% Recall@1 respectively. The gains reveal a key insight: breadth amplifies, depth generalizes—broad hierarchies help common entities most (+6.3% R@1), while deep hierarchies benefit unseen entities (+9.6% R@1 zero-shot).

Abstract:
Code LLMs often portray inconsistent program behaviors. Developers typically employ benchmarks to assess Code LLMs, but most benchmarks are hand-crafted, static and do not target the consistency property. In this work, we pose the scientific question: how can we automatically discover inconsistent program behaviors in Code LLMs? To address this challenge, we propose an automated consistency testing method, called MUCOCO, which employs semantic-preserving mutation analysis to expose inconsistent behaviors in code LLMs. Given a coding query, MUCOCO automatically transforms its program into semantically equivalent programs (aka mutants) and detects inconsistencies between the mutants and the original program (e.g., different output or test failure). We evaluate MUCOCO using four (4) coding tasks and seven (7) LLMs. Results show that MUCOCO is effective in exposing inconsistency and outperforms the closest baseline (TURBULENCE). About one in seven (15%) inputs generated by MUCOCO exposed inconsistencies. Our work motivates the need to test Code LLMs for consistency property.

Abstract:
In dialogue systems, role-playing agents (RPAs) aim to embody specific characters, but they often struggle to exhibit emotional expressions that are consistent with their assigned roles, resulting in insufficient emotional fidelity. This paper proposes EmoRL, a novel curriculum reinforcement learning (RL) framework, to enhance the emotional fidelity of RPAs. EmoRL features a multi-stage training paradigm guided by a carefully designed suite of reward functions. These rewards, including Format, Fine-grained Exact (FE), Emotional Similarity (ES), and Coarse-grained Categorical (CC) rewards, assess emotional consistency from strict to lenient perspectives. The curriculum RL process applies these rewards sequentially from easy (CC) to difficult (FE), enabling the RPAs to progressively learn coarse-grained emotional tendencies before mastering fine-grained distinctions. Furthermore, we introduce an emotional dialogue data synthesis pipeline to augment training data diversity and balance. Experiments on the Personality EmotionLines Dataset (PELD) demonstrate that EmoRL significantly outperforms baselines like RoleLLM, SFT, and DPO across both single-turn and multi-turn dialogues, with particularly notable gains in longer conversations. Our findings confirm the effectiveness of the curriculum learning strategy, the importance of balanced data, and the value of synthesized data for improving emotional fidelity.

Abstract:
With the growing demand for processing multiple image sequences in real-world applications, various visual token pruning methods have emerged to mitigate the computational and context length constraints faced by Large Vision Language Models (LVLMs). However, most existing pruning approaches rely on static strategies that struggle to adapt across different architectural LVLMs and multi-image scenarios, and are additionally constrained by their dependence on attention computations that are incompatible with efficient techniques like FlashAttention.To address these limitations, we propose a training-free, Adaptive Visual Token Pruning (AVTP) framework, applicable to diverse LVLM architectures. We strategically determine pruning layers based on empirical analysis of visual attention distributions across various LVLMs, and implement adaptive pruning ratios in multi-image contexts where images of higher importance retain proportionally more tokens. We conduct extensive experiments across multiple LVLMs with different architectures to demonstrate the effectiveness and robustness of AVTP. Specifically, Qwen3VL-8B achieves 2× inference speedup while maintaining 96.1% of its original accuracy on multiple multi-image benchmarks, InternVL3.5-8B retains 94.1% accuracy, and LLaVA-OneVision-7B even exceeds its original baseline performance. Our code is available at https://github.com/zry13/AVTP.

Abstract:
Open-web future event prediction requires agents to distill reliable signals from noisy, redundant, and incomplete evidence. Existing retrieval/memory mechanisms directly feed retrieved information to agents or rely on simple memory functions such as storing and reusing prior information for prediction, leaving them insufficient for open-web forecasting. We propose to transform raw web evidence into structured memory before prediction, enabling agents to reason over distilled, question-specific evidence rather than noisy retrieval results. This paper presents ForeDreamer, a self-evolving dual-agent framework for managing memory over open-web evidence. ForeDreamer separates factual memory, a question-specific evidence state for the current forecast, from experiential memory, persistent agent experience accumulated across forecasting episodes. It uses a main agent for search and prediction, and a memory-processing subagent to convert search results into factual memory with dedicated tools. ForeDreamer further evolves experiential memory through two tracks, improving both forecasting decisions and factual-memory construction. Experiments on Prophet Arena and FutureX demonstrate the effectiveness of ForeDreamer.

Abstract:
As Multimodal Large Language Models (MLLMs) evolve into sophisticated interactive assistants, their reliability depends not only on following instructions but also on validating them. We define Proactive Critique as the model’s autonomous ability to identify, analyze and fix faulty user inputs without extra prompts. However, evaluations mainly test models under ideal circumstances or simple refusal behaviors, largely ignoring active error processing. To fill this gap, we propose MMPCBench, a comprehensive framework for evaluating MLLMs’ proactive critique competence. It features a fine-grained taxonomy of 4 primary error types spanning 12 subcategories, ranging from cross-modal contradictions to missing visual premises. We adopt a hierarchical evaluation protocol to measure models’ error detection, diagnosis and resolution performance, and apply alignment-aware metrics to assess the coherence between internal reasoning and final responses. Tests on 14 mainstream MLLMs show obvious weaknesses in proactive critique, especially in dealing with subtle visual anomalies. Notably, we identify a pervasive "consistency gap": reasoning models can often correctly identify and analyze errors during internal reasoning yet suppress these valid insights in final outputs to prioritize response compliance.

Abstract:
Synthetic data generation has become a cornerstone for advancing large language models. However, the lack of the quantitative analysis for error tolerance became a critical bottleneck. Consequently, current filtering strategies fluctuate between two extremes: they are either overly aggressive, risking the exclusion of potentially valuable samples, or overly permissive, failing to eliminate erroneous samples effectively. To bridge this gap, this paper introduces Atomic Tree Operation Modeling (ATOM), a framework that decomposes data into functional units ( f(x)→ y ). ATOM distinguishes benign Operand x perturbations from fatal Operator f perturbations. The former are needlessly discarded by aggressive filtering, while the latter slip through permissive filtering. Our experiments reveal a double dissociation: models are robust to operand perturbations but collapse under operator perturbations. By prioritizing operator over aggressive operand precision, our ATOM-synthesized data outperforms rigorous baselines (e.g., +3.1% gain over LIMA), suggesting that operator diversity matters more than operand precision. Our code is available at [GitHub](https://github.com/Lut-hub/ATOM).

Abstract:
Audio-conditioned language models often underuse acoustic cues such as prosody, emotion, and non-speech sounds, raising the question of whether ASR-supervised frontends discard this information before it reaches the LM. We test whether the frontend is responsible by comparing Whisper-Tiny and Whisper-Small with EnCodec, DAC-VAE, and WavTokenizer in a shared Qwen3.5-4B audio-LM pipeline on ASR, emotion recognition, and sound captioning. Encoder replacement alone does not resolve this underuse: Whisper variants remain strongest overall, including on emotion and environmental sound captioning. To localize the failure, we trace task-relevant information through the encoder, projector, LM layers, and LM head. Linear probes and geometric analyses show that discriminative acoustic structure remains recoverable at the final LM layer, even when MCQA accuracy trails probe accuracy by up to 83 points. Because the answer format and decoding procedure are controlled, this task-dependent gap points to content-specific readout failure rather than generic format bias. LogitLens analyses and a targeted LM head intervention support the conclusion that acoustic underuse is not explained solely by encoder-side information loss and that readout alignment can be a dominant bottleneck.

Abstract:
Large Language Models (LLMs) are transforming chemical research by enabling specialized reasoning models and tool-augmented agents. Yet existing benchmarks typically evaluate reasoning or tool invocation in isolation, overlooking the dynamic interaction between internal deliberation and external execution. We present ChemEval2, a multimodal benchmark for tool-augmented chemical reasoning, containing 2,410 samples spanning fundamental principles, spectroscopic analysis, and complex reaction understanding. Built from professional textbooks and experimental data through automated generation and rigorous human annotation, ChemEval2 introduces a tool-augmented Chain-of-Thought evaluation framework that embeds tool-use checkpoints at stages where unaided language reasoning becomes unreliable. This design evaluates whether models can identify information gaps, invoke appropriate tools, and integrate tool outputs into reasoning trajectories. Across diverse text and multimodal models, we observe a process–outcome inconsistency: models can satisfy intermediate reasoning steps or execute useful tool calls yet still fail at final synthesis. These results provide evidence that coordinating computational tool use with downstream chemical reasoning remains a substantial challenge. We open-source ChemEval2 and an MCP-based suite of specialized chemical tools for reproducible.

Abstract:
Multimodal Knowledge Graphs (MMKGs) provide structured multimodal grounding for Multimodal Large Language Models (MLLMs), while Multimodal Entity Alignment (MMEA) is essential for integrating heterogeneous MMKGs. However, existing MMEA methods remain vulnerable to noisy modalities and unreliable pseudo-supervision. We propose CRMEA, a large–small collaborative framework for unsupervised MMEA that unifies PTM-based retrieval and MLLM-based reasoning through a closed-loop optimization process. CRMEA introduces a text-centric adaptive cross-modal coordination mechanism to regulate noisy visual evidence, together with a retrieve-and-vote strategy for reliable pseudo-supervision. Extensive experiments show that CRMEA consistently outperforms existing unsupervised MMEA methods and achieves performance competitive with state-of-the-art supervised approaches, while maintaining strong robustness, adaptability across heterogeneous backbones, and substantially improved cost-effectiveness over exhaustive MLLM-based alignment. Code and data are available at https://github.com/nsai-szu/CRMEA.

Abstract:
This paper explores agentic 3D spatial understanding, i.e., MLLM agents performing indoor 3D reasoning through tool use. Existing methods often misuse tools and exhibit biased tool preferences, leaving the agentic paradigm with only marginal gains over non-agentic strategies. We reveal that 3D spatial reasoning tasks are heterogeneous across scenes, while these agents apply a uniform tool-use strategy to all scenes rather than selecting tools according to the specific scene and task. To address this, we propose Skill-3D , a framework that learns self-evolving scene-aware skills. Specifically, Skill-3D identifies the task scene and records the agent’s tool-use trajectory into a Scene Memory , where successful trajectories from similar scenes are aggregated and distilled into a reusable scene-aware skill, with failed ones attached to the skill as lessons. During training, once a similar scene recurs, the corresponding skill is injected to guide the agent, producing new trajectories whose successes and failures further refine the skill, forming a loop in which the memory and the skill library co-evolve. Experiments show that Skill-3D substantially improves tool utilization in 3D spatial reasoning (from 39%to 78% on VSI-Bench), driving the agent toward correct and sufficient tool use. For instance, it improves Gemini-3-Flash by 67% on MMSI-Bench. Furthermore, we conduct agentic reinforcement learning over skill-guided trajectories, which boosts Qwen3-VL-8B by 60% on VSI-Bench.

Abstract:
Modern Large Language Models (LLMs) depend heavily on surface-level lexical filters for safety alignment, exposing a critical lexical-semantic gap: harmful intent can be fully decoupled from explicit wording. Motivated by this vulnerability, we propose CAGE (Concept Algebra for Guardrail Evasion), a single-turn black-box jailbreak leveraging the victim’s reasoning to reconstruct masked harmful intent. Through slot masking and concept equations, CAGE eliminates explicit triggers while guiding the target model to internally deduce the payload. Because standard evaluations conflate generic harmfulness with precise intent execution, we introduce the Strict Attack Success Rate (sASR), a metric that mandates verbatim slot recovery to measure reconstructive controllability beyond mere compliance. Across seven frontier models and three benchmarks, CAGE achieves an average ASR of 93.7% and sASR of 85.1%. Defense stress tests and keyword-recovery analysis reveal that existing token-level guardrails remain largely ineffective against reasoning-driven semantic evasion, suggesting that robust alignment must go beyond surface token filtering to address threats at the reasoning level. WARNING: THIS PAPER CONTAINS UNSAFE CONTENTS.

Abstract:
LLM agents accumulate valuable procedural knowledge during task execution, but these traces are session-bound, model-specific, and often entangled with sensitive content, making them unsafe to pool directly across users. Prior approaches to skill discovery often sacrifice cross-user generalization, iterative quality, or compliance-oriented sharing. We propose SkillLadder, a multi-user framework that distills local execution traces into a shared skill library without funneling raw data to a central server. The system evolves user-contributed traces into a multi-level hierarchy (L0–L3) using five quality gates that prevent regression below the strongest parent skill under a judge-defined score. To ensure safe sharing, SkillLadder distributes only level- ≥1 skills that have undergone entity scrubbing and lineage sanitization. On 𝜏 2 -bench, where each isolated task execution simulates one contributing user—a deliberately conservative proxy for many small local contributors, not a model of longitudinal human users—evolved skills improve a fixed Claude Sonnet 4.6 agent in three of four domains and generalize to simulated users running different agent models. Compliance audits verify that entity retention reaches zero by L3 and source-attribution accuracy is roughly halved, substantially reducing measured sharing risks.

Abstract:
Multimodal large reasoning models often rely on long Chain-of-Thought (CoT) traces in which a substantial fraction of tokens, such as repeated visual descriptions, self-reflection, and other visually-disengaged filler, inflate inference cost without contributing to the answer. Existing CoT compression methods optimize output length but never measure whether a reasoning token is actually grounded in the image. We propose VIG (Visual Information Gain), an information-theoretic GRPO reward that scores each reasoning token by how much the image reduces its predictive uncertainty. VIG is computed online from two forward passes of the same policy, one with and one without the image, so no reference chains, external annotations, or auxiliary reward models are needed. Across six main multimodal reasoning benchmarks and three Qwen3-VL-Thinking model sizes (2B/4B/8B), plus an additional R1-Onevision-Bench evaluation on 8B, VIG consistently improves the accuracy-efficiency trade-off, supporting our central claim: efficient multimodal reasoning emerges from raising visual information density, where every reasoning token earns its place by anchoring to the image, rather than from imposing a length budget.

Abstract:
Pedestrian anomaly behavior retrieval requires disambiguating subtle anomalous actions from routine ones using natural language. Existing cross-modal alignment methods often struggle with confounding biases: visual entanglement (dominant environments overshadowing target actions) and textual biases (limited stylistic diversity). These spurious correlations lead to erroneous matches among visually similar but semantically distinct samples. To address this, we reformulate the task from a causal intervention perspective. We propose a dual generative framework leveraging Large Multimodal Models (LMMs) to break environment-action and style-text confounders. Specifically, we synthesize causally meaningful hard negatives to decouple actions from environments, and augment positive queries with stylistically diverse rephrasings to eliminate narrative dependency. Extensive experiments on the Pedestrian Anomaly Behavior (PAB) benchmark yield consistent improvements of +1.71% in Recall@1 and +1.01% in mAP. Furthermore, zero-shot evaluation on the out-of-distribution UCC dataset demonstrates markedly enhanced generalization, achieving +3.23% and +2.46% gains in Recall@1 and mAP, respectively.

Abstract:
Large language models (LLMs) have demonstrated strong potential in long-horizon decision-making tasks, such as embodied manipulation and web interaction. However, agents frequently struggle with endless trial-and-error loops or deviate from the main objective in complex environments. We attribute these failures to two fundamental errors: global Progress Drift and local Feasibility Violation. Existing methods typically attempt to address both issues simultaneously using a single paradigm. However, these two challenges are fundamentally distinct: the former relies on fuzzy semantic planning, while the latter demands strict logical constraints and state validation. The inherent limitations of such a single-paradigm approach pose a fundamental challenge for existing models in handling long-horizon tasks. Motivated by this insight, we propose a Neuro-Symbolic Dual Memory Framework that explicitly decouples semantic progress guidance from logical feasibility verification. Specifically, during the inference phase, the framework invokes both memory mechanisms synchronously: on the one hand, a neural-network-based Progress Memory extracts semantic blueprints from successful trajectories to guide global task advancement; on the other hand, a symbolic-logic-based Feasibility Memory utilizes executable Python verification functions synthesized from failed transitions to perform strict logical validation. Experiments demonstrate that this method significantly outperforms existing competitive baselines on ALFWorld, WebShop, and TextCraft. Code is available at https://github.com/LAMDA-NeSy/DualMemory.

Abstract:
Recommender agents built on Large Language Models offer a promising paradigm for personalized recommendation. However, existing agents typically suffer from a misalignment between their tool-integrated reasoning trajectories and recommendation feedback, limiting their ability to distinguish fine-grained user preferences. To address these challenges, we propose AgenticRec, an agentic recommendation framework that formulates recommendation as a tool-integrated reasoning process over a recommendation-oriented tool suite. Built upon this framework, we further develop a dedicated two-stage training paradigm tailored for recommender agents. In the first stage, we introduce Recommendation-Oriented Trajectory Activation, optimize the agentic recommendation ability under implicit feedback. In the second stage, Progressive Preference Refinement further refines the agent through bidirectional preference reasoning over self-bootstrapped hard pairs, progressively sharpening preference boundaries. Theoretical analysis and extensive experiments demonstrate the effectiveness of AgenticRec.

Abstract:
General-purpose machine translation benchmarks such as FLORES-200 have reached a saturation regime on Chinese-English pairs, where modern large language models cluster within a narrow band of high scores. Across 22 systems, FLORES-200 zh-en GEMBA scores fall in a 7.87-point range with a standard deviation of 2.29, which compresses the separation between systems on knowledge-intensive domains such as finance, healthcare, law, and science and technology. We introduce HardMTBench, a difficulty-aware diagnostic benchmark for bidirectional Chinese-English domain translation. HardMTBench covers 12 domains and contains 10000 hand-curated source sentences with reference translations, packaged as 20000 directional test items. A three-stage construction pipeline builds a domain-balanced candidate pool of 84566 pairs, applies an LLM-based multi-signal judge over knowledge density, translation difficulty, terminology load and reference correctness, and assembles the final test set under a hardness fusion rule with per-domain quotas. Across 22 systems spanning general LLMs, commercial engines and specialised MT models, HardMTBench widens the cross-system GEMBA range by roughly a factor of two over FLORES-200, induces visible rank reorderings, and exposes domain-specific terminology and knowledge weaknesses that quality-only metrics tend to flatten.

Abstract:
The rapid maturation of zero-shot Text-to-Speech (TTS) models has turned high-quality voice cloning into a widely available capability, raising acute concerns over unauthorised replication, fine-tuning and resale of proprietary speech models. Yet ownership verification for TTS remains largely open: speech is a continuous waveform whose perturbations are easily destroyed by routine signal processing, and the human auditory system imposes a much tighter perceptual budget than vision. We present TTS-Guard , a black-box ownership verification framework for TTS models built on adversarial speaker-pair fingerprints . TTS-Guard(i) selects key speaker pairs in a dual embedding space for architecture-agnostic stealth;(ii) optimises a perturbation through an adaptive curriculum of shadow models covering fine-tuning, pruning, quantisation and distillation; and (iii) aggregates black-box queries into a calibrated Verification Confidence Score . On five mainstream TTS systems, TTS-Guard reaches an average Fingerprint Success Rate of 96.4% at a False Positive Rate of 5.8%, while preserving intelligibility and naturalness. The fingerprint remains effective against ten audio attacks, six model modifications, and two state-of-the-art adversarial purifiers.

Abstract:
Zero-shot text-to-speech (TTS) aims to synthesize high-quality speech for unseen speakers using only a short reference utterance. Existing non-autoregressive TTS models typically convert text tokens into speech-length representations through filler-token padding, explicit duration prediction, or text upsampling. However, these approaches either introduce redundant computation or rely on explicit alignment assumptions. We propose M3-TTS, a native two-stream multi-modal diffusion Transformer that preserves the original lengths of text and speech representations and enables direct text–speech interaction through packed joint self-attention. M3-TTS employs a dual-stage architecture consisting of Joint-DiT layers for learning text–speech correspondence and audio-only DiT layers for acoustic refinement. Reference acoustic information is injected through AdaLN-style conditioning. Experiments on the Seed-TTS benchmark demonstrate strong intelligibility and competitive speaker similarity and perceptual quality. The Mel-VAE variant further improves inference efficiency while maintaining high synthesis quality. Code and demos will be available at https://github.com/WWWWxp/M3-TTS.

Abstract:
Real-world lectures follow a predefined topic inventory (e.g., a syllabus), but instructors often reorder, paraphrase, or skip items, creating mismatches between the outline and the spoken content. Lectures also contain non-topical utterances (openings, transitions and wrap-ups) that should not be assigned to any topic, yet existing corpora rarely model both challenges. We present MOSAIC (Multimodal Open-course Segmentation and Alignment corpus In Chinese), a large-scale Chinese MOOC corpus for topic-guided segmentation and alignment under predefined inventories, with explicit non-topical labels and matched slide resources. MOSAIC includes MOSAIC-G (human-annotated gold benchmark with utterance-level topic alignment and video-level links to matched slide resources) and MOSAIC-S (automatically labeled silver data), collectively covering 179 courses and 14,942 videos. To scale annotation, we propose DORA (Dual-stage Outline Refinement and Alignment), a two-stage pipeline that refines noisy inventories and then performs joint segmentation and topic assignment. Experiments show that silver supervision substantially improves the performance of local models and that incorporating slides consistently benefits knowledge-point extraction, a lecture-comprehension proxy.

Abstract:
Large language models (LLMs) augmented with external tools have demonstrated strong capabilities in API selection, invocation, and reasoning over tool outputs. However, existing training-free tool-use methods usually rely on inference-time context construction to guide the entire tool-use process with a unified input, including tool descriptions, task prompts, demonstrations, trajectories, or planning hints. Tool-use decisions place different emphasis on this contextual evidence: tool/API selection emphasizes task–API correspondence, while call generation additionally uses API schemas, local call transitions, and intermediate observations. We present SD-ICL, a Stage-Decomposed In-Context Learning method that decomposes context construction into tool selection and call generation stages. SD-ICL first retrieves tool selection demonstrations by combining task semantics with tool set relevance and constructs an explicit tool selection state. It then uses this state to retrieve call generation demonstrations through API-set similarity and adjacent call relations, organizing complementary context for the executor. Experiments show that SD-ICL improves task completion and preference quality across the evaluated tool-use benchmarks and model settings.

Abstract:
Verifiable reasoning training compresses an entire response to a binary pass-or-fail label, yet a fail label alone does not reveal whether the corresponding trajectory should be discarded, penalized, or repaired. Many failed trajectories remain close to the correct solution through substantial portions of their intermediate reasoning and deviate only at a later branching point, whereas others contain no reliable prefix that can support a correct suffix. Treating both failure types as uniformly negative supervision suppresses useful reasoning states while reinforcing erroneous ones. To address this limitation, we propose prima, which decomposes failed trajectories into inheritable prefixes and divergent suffixes, and rewrites repairable failures into shared-prefix preference pairs for localized reasoning optimization. We formalize the optimization cost of mixing the two failure types and show that treating them as a uniform negative class shrinks effective gradient magnitude, providing a principled justification for separating repairable from irrecoverable failures before training. Experiments show that prima converts failed trajectories from coarse-grained negative supervision into cleaner suffix-level supervision signals, leading to consistent gains on code and mathematical reasoning tasks.

Abstract:
Web data curation has been widely studied for decoder Large Language Model (LLM) pretraining. Encoders for dense-terminology domains such as medicine, by contrast, are pretrained on small, manually-curated corpora that limit scalability and writing style diversity, a bottleneck even more severe in non-English clinical settings. Whether web-scale data curation also benefits encoder Masked Language Modeling (MLM) in a dense-terminology domain remains an open question. To address this, we introduce two complementary levers. Medical-term density filtering selects documents rich in medical terms. Signal-amplifying rephrasing uses an LLM to rewrite documents into denser variants with broader entity contexts. We instantiate the recipe on French medical NLP. The medical-term density filter outperforms the widely-used educational quality filter on downstream medical tasks, and the two complement each other. Signal-amplifying rephrasing alone improves on raw web data, and mixing it with filtered web data produces the largest gain. The recipe yields FineMed, a French medical pretraining corpus, and DoctoBERT, a state-of-the-art French medical encoder family evaluated on both the public benchmark DrBenchmark and a proprietary clinical Named Entity Recognition (NER) task.

Abstract:
Legal Judgment Prediction (LJP) predicts applicable law articles, charges, and penalty terms from case facts. Beyond accuracy, LJP calls for intrinsically interpretable and legally grounded reasoning that can reconcile statutory rules with precedent-informed standards. However, existing methods often behave as static, one-shot predictors, providing limited procedural support for verifiable reasoning and little capability to adapt as jurisprudential practice evolves. We propose VERDICT, a self-refining collaborative multi-agent framework that simulates a virtual collegial panel . VERDICT assigns specialized agents to complementary roles (e.g., fact structuring, legal retrieval, opinion drafting, and supervisory verification) and coordinates them in a traceable draft–verify–revise workflow with explicit Pass/Reject feedback, producing verifiable reasoning traces and revision rationales. To capture evolving case experience, we further introduce a Hybrid Jurisprudential Memory (HJM) grounded in the Micro-Directive Paradigm , which stores precedent standards and continually distills validated multi-agent verification trajectories into updated Micro-Directives for continual learning across cases. We evaluate VERDICT on CAIL2018 and a newly constructed CJO2025 dataset with a strict future time-split for temporal generalization. VERDICT achieves state-of-the-art performance on CAIL2018 and demonstrates strong generalization on CJO2025. To facilitate reproducibility and further research, we release our code and the dataset at https://github.com/luckyhui235/VERDICT .

Abstract:
Model merging enables combining multiple fine-tuned models without additional training, but its safety implications remain poorly understood. Prior work primarily attributes merging risks to unsafe constituent models, implicitly assuming that merging individually aligned models preserves safety. In contrast, we show that model merging reveals a previously overlooked jailbreak risk rooted in the pretrained foundation model, even when all constituent models are individually safety-aligned. Motivated by this observation, we study a new threat setting where an attacker constructs jailbreak prompts that generalize across merged models sharing the same pretrained backbone, without access to the exact merging coefficients or constituent checkpoints. To exploit this phenomenon, we propose Basin-Aware Jailbreak (BAJ), which formulates jailbreak generation as a min–max optimization over the merging space to produce transferable adversarial suffixes across merged model families. Experiments across diverse backbones and merging settings show that BAJ achieves consistently high transfer success rates and remains effective under existing defenses.

Abstract:
Existing evaluations of harmful content detection rely predominantly on static benchmarks, which struggle to reflect the interactive adversarial ecosystem of real-world content platforms where users continuously revise their expressions in response to moderation feedback. This mismatch creates a significant performance gap between offline benchmark scores and online deployment effectiveness. To the best of our knowledge, we present EvoHarmBench, the first dynamic adversarial evaluation framework for content moderation systems. The framework employs an iterative optimization loop that evolves evasion strategies at the semantic-cluster level, while simultaneously optimizing for evasion success and human readability. We systematically evaluate LLM-based defense models which are widely used in real world moderation systems. The evaluation covers 229 semantic sub-clusters across five violation categories, derived from 5,002 real-world adversarial samples collected from content platforms. Our experiments reveal substantial vulnerabilities even in leading commercial systems: after twelve optimization iterations, Upon acceptance, we will release the full benchmark data, evaluation framework, and code to encourage a shift from static benchmarking toward dynamic adversarial evaluation in content safety research.

Abstract:
Safety alignment in large language models (LLMs) remains brittle against a growing spectrum of attacks. Jailbreak attacks bypass safety mechanisms through crafted prompts, while neuron-level attacks directly prune safety-critical neurons post-deployment. Both exploit a common weakness: safety-relevant information concentrates in a sparse neuron subset. We present NeuronGuard, a fine-tuning-stage defense that simultaneously hardens LLMs against both attack classes by redistributing safety signals across a broader set of neurons. NeuronGuard dynamically identifies safety-critical neurons via periodically refreshed per-layer linear classifiers, forces refusal behavior under deliberate neuron ablation, and applies KL-divergence regularization for distributional consistency. A randomized gradient projection strategy preserves downstream task utility by resolving conflicts between the defense and task objectives. We provide a formal guarantee that NeuronGuard strictly reduces the attack success rate (ASR) upper bound, and experiments across three LLMs, six state-of-the-art attack strategies, and multimodal settings confirm near-zero ASR while maintaining task accuracy, including against white-box adaptive adversaries.

Abstract:
Deploying high-dimensional Vision-Language Model (VLM) embeddings is computationally prohibitive. While Matryoshka Representation Learning (MRL) allows elastic truncation, standard MRL optimizes dimensional hierarchies semi-independently. This lack of cross-scale structural constraints creates a severe topological disconnect causing representational collapse at lower dimensions and introduces optimization trade-offs that compromise high-dimensional capacity. To solve this, we propose M-VLM2Vec, a framework that shifts MRL from a compromised hierarchy to a synergistic geometric structure. Without additional supervision, we integrate two novel constraints into the multimodal contrastive objective: (1) Laplacian spectral alignment to preserve the global manifold topology across scales, and (2) Stiefel-constrained orthogonal projection to bidirectionally align local contrastive landscapes. By allowing nested spaces to mutually regularize, M-VLM2Vec eliminates competing gradients, preventing ultra-low-dimensional collapse while yielding striking, consistent performance gains across all embedding scales on the Multimodal Embedding Benchmark (MMEB).

Abstract:
Short-video platforms have become a primary news source for the public, which has also enabled the widespread dissemination of fake news videos. We study the task of fake news video detection and explanation (FNVDE). Existing methods face two critical limitations. First, commonly used frame selection strategies may omit veracity-relevant cues or provide insufficient temporal context for understanding news videos. Second, prior methods neglect either multimodal understanding or evidence retrieval. To address these limitations, we propose NVKE-CEI, a unified system that integrates a news video keyframes extraction method (NVKE) and an FNVDE framework leveraging both content and evidence information (CEI). NVKE selects keyframes based on chronological changes in combined visual and OCR-text similarity. CEI employs two specialized LLM-based fact checkers (content-based and evidence-based) whose outputs are fused by a lightweight judge model. Extensive experiments show that NVKE-CEI outperforms state-of-the-art baselines while generating high-quality content-grounded explanations.

Abstract:
Retrieval-Augmented Generation (RAG) improves generation quality by incorporating evidence retrieved from large external corpora. However, most existing methods rely on statically selecting top-k passages based on individual relevance, which fails to exploit combinatorial gains among passages and often introduces substantial redundancy. To address this limitation, we propose OptiSet, a set-centric framework that unifies set selection and set-level ranking for RAG. OptiSet adopts an “Expand-then-Refine” paradigm: it first expands a query into multiple perspectives to enable a diverse candidate pool and then refines the candidate pool via re-selection to form a compact evidence set. We then devise a self-synthesis strategy without strong LLM supervision to derive preference labels from the set conditional utility changes of the generator, thereby identifying complementary and redundant evidence. Finally, we introduce a set-list wise training strategy that jointly optimizes set selection and set-level ranking, enabling the model to favor compact, high-gain evidence sets. Extensive experiments demonstrate that OptiSet improves performance on complex combinatorial problems and makes generation more efficient.

Abstract:
Standardized interfaces such as the Model Context Protocol (MCP) make it easier to connect tools to agents, but connection alone is not orchestration. When many related tools are exposed as a flat callable palette, agents must identify relevant tools, resolve substitutes, pass intermediate outputs, and decide when a workflow is complete. We introduce RealFin, a finance-domain diagnostic benchmark for evaluating workflow construction under MCP-style flat tool exposure. RealFin contains 1524 tasks generated from 108 author-designed financial workflow templates over real market, macroeconomic, and company-data APIs. Each task is paired with an executable reference program that specifies one verified solution path, while the 85 agent-callable tools include explicit alternative, dependency, and related-tool metadata to represent non-unique paths, prerequisite tools, and near-neighbor ambiguity. This design supports path-independent answer scoring together with process-level evaluation of tool selection, ordering, parameter planning, entity grounding, temporal grounding, orchestration success, and error recovery. Across ReAct-style flat exposure, retrieval-based exposure, oracle-controlled variants, and progressive tool disclosure, current agents often achieve nonzero answer accuracy while failing to construct reliable workflows. These results suggest that MCP-style tool connectivity needs structured disclosure and workflow-aware evaluation, especially in finance where temporal constraints and overlapping tools are intrinsic.

Abstract:
Temporal Knowledge Graph (TKG) extrapolation aims to predict future facts from observed historical events. Existing methods usually achieve this goal by jointly modeling local historical evolution and long-range historical contexts. First, local evolution modeling often summarizes local history into an overall evolving representation, where query-relevant local deviations may not be sufficiently exploited as useful predictive features. Second, existing long-range history modeling often treats historical recurrence as a stable cue, while overlooking that its reliability may vary across relations and recent contexts. To address these issues, we propose DyHist , a query-aware historical dynamics modeling framework for TKG extrapolation. DyHist captures local dynamics by separating recent evidence from medium-range context and explicitly modeling their query-relevant deviations as predictive features. It further uses a relation-aware historical prior to calibrate candidate scores with time-decayed support, recent trends, and relation-specific strength, enabling recurrence signals to contribute differently across relations and recent contexts. Experiments on four benchmarks show that DyHist achieves the best MRR against recent baselines. Code is available at https://github.com/PANPANKK/DyHist.

Abstract:
Vision-language models can be used in applications that analyze people in professional and social contexts. This raises concerns about whether these models reproduce occupational gender stereotypes. However, it is challenging to evaluate these associations because naturally collected images often combine demographic attributes with clothing, pose, background, and other status-related cues. We introduce a paired-image benchmark to study occupational status judgments in VLMs, containing 5,487 human-validated synthetic images spanning 41 ISCO-08 sub-major occupational groups aggregated into 10 major groups, 7 ethnicity categories, and 5 age groups. Men and women are matched based on occupation, age, ethnicity, and workplace context. We evaluate 5 VLMs using spatial comparisons that avoid direct gender references, explicit gender comparisons, and structured reasoning prompts. We observe persistent male-oriented associations in earnings judgments, even when the prompts avoid direct references to gender. Furthermore, we find that men are more frequently associated with authority, seniority, and earning potential, whereas women are more often associated with warmth. Promotion likelihood exhibits a model-family-dependent pattern: male-oriented associations emerge across Qwen-3-VL models, whereas both Gemma-3 models show the opposite, female-oriented pattern. These findings show that occupational gender stereotypes persist in the evaluated VLMs even under tightly controlled visual conditions. Our code and data are publicly available at https://github.com/NLP-UMUTeam/OccuBiasBench

Abstract:
Retrieval-Augmented Generation (RAG) systems are vulnerable to knowledge corruption attacks, where adversaries inject malicious texts into external corpora to manipulate generated responses. Existing one-to-many attacks can compromise multiple queries with a small number of injected texts, but they typically require white-box access to the target retriever for gradient-based optimization. In contrast, existing black-box attacks are more practical but usually allow each malicious text to target only one query. In this paper, we study one-to-many knowledge corruption attacks against strictly black-box RAG systems, where the attacker has no access to the target retriever, generator, or corpus contents. We propose CoverPoison, a black-box method that combines competitive similarity clustering, coverage-oriented optimization, and T5-based text refinement. Competitive similarity clustering improves query grouping quality, while the coverage-oriented objective encourages each candidate to cover more queries through query-specific retrieval thresholds. Experiments on three datasets show that CoverPoison substantially improves black-box attack effectiveness. Further analyses across RAG configurations, attack objectives, and defenses validate the effectiveness of the proposed method.

Abstract:
Tree-based speculative decoding accelerates autoregressive generation by verifying multiple draft candidates in parallel, but this advantage weakens for sparse Mixture-of-Experts (MoE) models. As the draft tree grows, different branches activate different experts, expanding the union of activated experts and substantially increasing target-side verification cost. We propose EVICT, a training-free, hyperparameter-free, and lossless adaptive verification method for MoE speculative decoding. EVICT makes every verified token count by truncating the draft tree before target verification and retaining only the cost-effective prefix. It leverages fine-grained drafter signals to estimate candidate benefit, combines them with offline-profiled verification cost, and remains highly compatible with the high-performance graph-based serving framework SGLang. Extensive experiments on diverse MoE backbones and benchmarks show that EVICT achieves up to 2.35 × speedup over autoregressive decoding and an average 1.21 × speedup over the state-of-the-art baseline EAGLE-3, while significantly reducing unnecessary expert activations during verification.

Abstract:
Large Language Models (LLMs) frequently face limitations, such as judgment biases and cognitive deadlocks, in complex reasoning due to “Degradation of Thought” in self-reflection. While Multi-Agent Debate (MAD) approaches attempt to address this issue, their inherently adversarial nature suppresses diverse perspectives by overemphasizing antagonism, which results in the loss of partially valid reasoning. In this paper, we propose Multi-Agent Diverse Thinking (MADT), a novel cognitive framework that redefines multi-agent collaboration by shifting from adversarial confrontation toward constructive synergy. To break the oppositional deadlock found in traditional debates, MADT decomposes the thinking process into fine-grained modules: an Affirmative Thinker to preserve rational components, a Critical Thinker to rectify errors, and a Growth-Minded Thinker to provide optimization suggestions. These agents jointly foster a synergistic environment where specialized roles protect valid logic while iteratively refining flaws. Meanwhile, a Leader agent coordinates the collaborations and synthesizes the feedback from multiple thinkers to produce the final result. Extensive experiments on Common Machine Translation and Counter-Intuitive Arithmetic Reasoning tasks show that MADT consistently outperforms Self-Reflection and MAD baselines, validating the superior ability to enhance complex logical and reasoning performance in LLMs.

Abstract:
Human gaze is the primary high-bandwidth interface through which we parse the semantics of our environment. While often modeled as a sequence, the degree to which gaze exhibits transferable local structure across domains remains underexplored. Using motion tokenization as a diagnostic lens across four diverse datasets, we investigate the transferability of local sequence structure. We demonstrate that while gaze can share a reusable motion-token vocabulary, identical motion tokens are organized into different local transition patterns depending on the dataset and task context. Our results show the limitations of naive data-pooling under low-semantic motion-token representations and provide a framework for evaluating behavioral sequences through information-theoretic diagnostics.

Abstract:
As large language models are increasingly deployed for clinical text, ensuring they can reliably signal their own uncertainty becomes critical. Most existing uncertainty quantification (UQ) methods are designed for open-domain generation and cannot localize uncertainty at the token or span level in long clinical text. We propose Reverse Probing, the first uncertainty quantification framework specialized for clinical summarization, which estimates token-level uncertainty directly from pre-existing clinical summaries. Rather than sampling new outputs, Reverse Probing treats the text as a probe into the model’s internal state, extracting uncertainty features from four categories of signals. We evaluate Reverse Probing on two expert-annotated clinical datasets and outperform eight adapted baselines, achieving up to 4× higher AUPRC while running over 15× faster than sampling-based methods. Feature analysis reveals that the log-logit difference induced by the source record and the neighborhood context are the most consistent predictors across all models. This study offers interpretable insights into how models internally respond to unsupported clinical content.

Abstract:
Not all acoustic frames are equally useful for correcting dysarthric ASR errors. In dysarthric speech, acoustic deviations are unevenly distributed across frames, but LLM-based generative error correction systems often inject acoustic features uniformly, under-emphasizing deviant frames and over-correcting reliable ones. We propose a frame-selective acoustic conditioning method driven by a Frame Deviation Score (FDS) and VGate. FDS is a control-referenced per-frame deviation score computed over Whisper encoder frames, and VGate uses standardized per-frame FDS to reweight acoustic value vectors. Across ten personalized TORGO settings, we compare VGate with Plain, an acoustic-adapter baseline without the VGate gating module, and show that VGate is conditional rather than universally beneficial. Three findings support conditional use. (1) Plain residual headroom (the gap between Plain and Oracle@5 WER) and utterance-mean FDS-DIV characterize high-benefit cases, with gains concentrated in ref-preserved and overgeneration cases and the largest improvements driven by insertion control. (2) A development-calibrated FDS-DIV router lowers pooled WER from 19.55% to 18.29% while routing only 20.72% of utterances. (3) Cross-corpus COPAS results and blinded listening further support this conditional-use strategy. Our code is publicly available at https://github.com/doraemon500/FSAC.

Abstract:
Large language models (LLMs) often make reasoning errors when solving mathematical problems, and how to automatically detect and correct these errors has become an important research direction. However, existing approaches mainly focus on self-correction within the model , which falls short of the “teacher-style” correction required in educational settings, i.e. , systematically guiding and revising a student’s problem-solving process. To address this gap, we propose SMRC ( S tudent M athematical R easoning C orrection ), a novel method that aligns LLMs with student reasoning. Specifically, SMRC formulates student reasoning as a multi-step sequential decision problem and introduces Monte Carlo Tree Search (MCTS) to explore optimal correction paths. To reduce the cost of the annotating process-level rewards, we leverage breadth-first search (BFS) guided by LLMs and final-answer evaluation to generate reward signals, which are then distributed across intermediate reasoning steps via a back-propagation mechanism, enabling fine-grained process supervision. Additionally, we construct a benchmark for high school mathematics, MSEB (Multi-Solution Error Benchmark), consisting of 158 instances that include problem statements, student solutions, and correct reasoning steps. We further propose a dual evaluation protocol centered on solution accuracy and correct-step retention , offering a comprehensive measure of educational applicability. Experiments demonstrate that SMRC significantly outperforms existing methods on two public datasets (ProcessBench and MR-GSM8K) and our MSEB in terms of effectiveness and overall performance.

Abstract:
Large language model agents handle structured coding tasks well, but open-ended projects are harder. A hackathon gives an agent an open goal, a hard deadline and several artifacts to deliver. The agent has to decide how to split its budget across phases and when to stop widening scope. Most agents do neither. 0xClaw is a multi-agent framework that splits a hackathon run into seven phases. Each phase hands a typed artifact to the next. The Budget Router is a small fine-tuned Qwen3.5-2B model. It reads the workspace state and predicts a per-phase control token for token budget, time, reasoning effort and repair rounds. The Deadline Framer turns each token, together with the cumulative execution log, into a plain deadline instruction given to the agent at the start of each phase. Our study covers 20 hackathon tasks, 10 baselines and more than 200 agent runs. 0xClaw matches Claude Code at the best pass@1 (1.00) and exceeds ARIS (0.95), the two strongest baselines. Against Claude Code, it achieves comparable quality and pass@1 while using 59% fewer tokens. The trade-off is wall-clock time, about 2.5 times that of ARIS.

Abstract:
In document AI, reliable analytics require converting long, noisy (often multi-document) corpora into heterogeneous structured data—e.g., tables for numerical fields, graphs for relations, and trees for hierarchies. Yet, one-pass LLM extraction often yields incomplete or inconsistent structures because it lacks explicit verification and opportunities to revise earlier choices. We present CycleIE, an iterative information extraction (IE) framework that closes the loop between reasoning and acting by coupling ReAct with Monte Carlo Tree Search (MCTS). CycleIE employs a multi-agent workflow to iteratively retrieve, structure, extract, and refine content under verification guidance. By treating extraction as a feedback-driven search process, CycleIE systematically corrects omissions and inconsistencies that defeat one-pass methods, while remaining orthogonal to retrieval-augmented generation (RAG). Experiments on challenging document-based QA benchmarks demonstrate that CycleIE delivers >10% relative improvements over strong baselines, particularly in lengthy or multi-document contexts.

Abstract:
Emotion is expressed in text along a wide spectrum, from surface lexical cues to inferences entangled with content. Most layer-wise analyses of emotion in LLMs use a single corpus, leaving open whether the depth at which emotion becomes accessible is a property of the model or also of the text source. We investigate this across three datasets spanning different degrees of explicitness and contextualization in emotion expression (Twitter posts, Reddit comments, and autobiographical narratives) and eight 1B–9B open-weight LLMs from the Llama, Qwen, and Granite families. We combine layer-wise probing with offline feature scaling and online forward interventions, transfer analyses, and an early-exit classifier. We find that (i) the best probing layer shifts systematically across corpora, from input-adjacent layers to over half model depth, and this ordering persists after matching label-by-length-bin distributions; (ii)across the evaluated settings, forward-pass interventions on probe-selected bands reduce test accuracy by 5–6 points more than same-width random bands ( q < 0.01 ); (iii) selected bands transfer across datasets and emotion categories, suggesting partially shared affective information rather than strictly per-emotion substrates; and (iv) probe-selected early-exit representations outperform full-depth exits by 6.9 percentage points on average.

Abstract:
LLM-based agentic systems have shown remarkable capabilities in complex domains, while suffering from cascading errors and difficulty in debugging. Critical error detection aims to locate the earliest error step in a failed trajectory that is responsible for the final failure. However, progress faces two main challenges. First, long trajectories make it difficult to identify individual errors, since the evidence for judging a step may be scattered across distant instructions, observations, and prior context. Second, failed trajectories often contain multiple local errors with different downstream effects, only some of which remain responsible for the final failure. In this work, we propose TrajDebug, an error-lifecycle tracing framework that addresses long-trajectory error discovery with multi-granularity history compression and evidence-based error identification, and supports critical attribution by tracing each error’s resolution status and terminal impact. We further construct TrajErrBench, a benchmark of 486 manually annotated failed trajectories from and SWE-Bench Pro, covering realistic tool-use and coding scenarios. Experiments across diverse agent benchmarks show that TrajDebug achieves the best overall performance over existing baselines, and application studies further demonstrate that its diagnoses provide actionable feedback for improving downstream agent success. We will release the codes and data to facilitate further research.

Abstract:
When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine translation interact with and influence each other, using the CEFR levels as the measure of text complexity. Across six languages, including Arabic, Dutch, English, French, Hindi, and Russian, we evaluate three open-weight models, one closed model, and a commercial machine translation system on two tasks: i) correlation of CEFR with translation difficulty, and ii) shifts in CEFR levels of the source texts. Our experiments show that higher CEFR levels make texts more difficult to translate, and that machine translation shifts the CEFR level of the target text compared to the original source, for most languages. These findings provide new insights for researchers and practitioners working on multilingual pedagogical content generation and MT difficulty estimation.

Abstract:
Reward modeling aligns language models with human preferences, yet coding tasks, requiring strict logic and syntax, pose challenges due to their domain diversity. In this paper, we propose RewardCode, a generalist code reward model trained via a two-stage pipeline, enabling scalable and precise evaluation through principle-based scoring and unit test generation. Specifically, the first stage combines Structured Fine-Tuning and Rejective Fine-Tuning to distill cross-domain summarization and code evaluation capabilities from advanced LLMs. The second stage introduces Pairwise-GRPO, a reinforcement learning method that uses preference pairs to improve solution discrimination, while ensuring the generation of consistent and verifiable unit tests. Further, to train a cross-domain code reward model, we construct CodePair-18K, a dataset of verifiable code preference pairs and executable unit tests. Experiments on multiple benchmarks show that RewardCode outperforms strong baselines in evaluation accuracy, verifying its effectiveness in advancing general-purpose code LLMs.

Abstract:
Training-free KV cache compression is essential for deploying vision-language GUI agents under memory and latency constraints, yet existing methods are designed for generic language workloads and ignore the distinctive structure of GUI interaction traces. We characterize three GUI-specific workload properties — high inter-frame visual redundancy, extremely small UI-element spatial footprints, and near-uniform cross-layer attention sparsity — that cause existing schemes to retain as few as 39% of oracle-important KV pairs at the standard 20% budget. To address this, we propose ST-Lite, a training-free compression scheme whose three components each target one property: Trajectory-aware Semantic Gating (TSG) filters redundant historical frames, Component-centric Spatial Saliency (CSS) preserves fine-grained element boundaries, and a flat per-layer budget avoids hierarchical misallocation. Across seven GUI benchmarks and two backbones in the deployment-relevant β ∈ [10%, 40%] window, ST-Lite consistently outperforms all existing compression baselines — matching or exceeding Full Cache task accuracy on the primary backbone at the 20% budget — while delivering up to 2.35× decoding speedup at fivefold compression. Code: https://github.com/94wen94/ST-Lite

Abstract:
Long-horizon LLM agents increasingly rely on retrieved procedural skills as external memory. However, existing skill-augmented RL frameworks typically supervise a bundle of retrieved skills only with trajectory-level outcome rewards, leaving two key questions unanswered: whether each skill was executed and whether it helped task completion. We formulate this as a per-skill credit assignment problem. To address it, we propose SkillRubricsRL , which pairs each skill with an applicability-aware trajectory-level rubric and derives two complementary signals: a fulfillment signal for training correct skill execution and a counterfactual utility signal for retrieval and skill-bank evolution. Across ALFWorld and WebShop, SkillRubricsR achieves 97.7% ALFWorld success, 84.4% WebShop success, and a WebShop score of 92.7, improving over the strongest prior multi-skill baseline by 2.8, 2.1, and 1.6 percentage points, respectively, while increasing skill execution rate by over 50 percentage points. Code is available at https://github.com/gminiheimdallr-a11y/SkillRubricsRL .

Abstract:
The performance of large language models (LLMs) on poetry-related tasks has drawn increasing attention from researchers. Since understanding of poetry is fundamental to all poetry-related tasks, enhancing LLMs’ comprehension of poetry is crucial for advancing this field. In this study, we propose a novel method to improve LLMs’ comprehension of modern Chinese poetry. This method utilizes feedback from reliable LLM evaluators as guidance to enhance other LLMs’ understanding of poetry in specific dimensions. To identify reliable LLM evaluators, we evaluate the initial performance of several mainstream LLMs on poetry comprehension tasks and further refine their evaluation capabilities until they approach human (expert) performance. These reliable LLMs are then employed as evaluators to simulate poet’s perspective in assessing the poetry comprehension of other LLMs, providing reasons and suggestions for improvement. Experimental results show that our method effectively enhances LLMs’ comprehension of poetry across multiple dimensions.

Abstract:
Traditional decoding-time alignment methods, such as Best-of-N sampling, enhance output quality by selecting high-reward responses from multiple candidates. However, these approaches incur substantial computational overhead, which limits their practicality. Recent advances aim to mitigate this inefficiency by scoring and filtering partial generations early in the decoding process. However, these methods face a critical challenge: conventional reward models (RMs), optimized to assess complete responses, exhibit poor performance when evaluating truncated text. To address this, we introduce RewardPartial, a novel evaluation paradigm that shifts the focus from superficial partial-text quality to estimating a response’s completion potential. We provide theoretical analysis for this paradigm through the perspective of Markov Decision Process and construct TruncEval, a specialized dataset for training RewardPartial models to accurately predict this potential. Our evaluation on the TruncEval test set demonstrates that RewardPartial models deliver significant performance gains over conventional reward models. When deployed in early-filtering pipelines, our method achieves superior alignment performance compared to Best-of-N sampling, offering a more efficient solution for high-quality text generation.

Abstract:
Command-line interface (CLI) agents powered by large language models (LLMs) can interpret natural-language requests, plan multi-step tasks, execute shell commands, and modify files and system state. As these agents are increasingly used for operating-system (OS) workflows, it is important to evaluate whether they can be misused to carry out security-relevant operations. Existing benchmarks often lack an attacker-knowledge model grounded in tactics, techniques, and procedures (TTPs), provide limited coverage of end-to-end kill chains, rely on simplified single-host environments, or use LLM-as-a-judge for success evaluation. We introduce ADVCLI, an MITRE ATT CK-aligned benchmark for evaluating OS-level misuse risks of CLI agents in a controlled multi-host sandbox. ADVCLI contains 140 tasks: 40 direct malicious requests, 74 TTP-based tasks, and 26 end-to-end kill chains. Each task is paired with deterministic hard-coded verification protocols that check whether the requested OS-level effect is realized. We evaluate seven CLI Agents and products built on nine foundation models, including ReAct, OpenClaw, OpenAI Agent SDK, Claude Code, Gemini CLI, Cursor CLI, and Cursor IDE. Results show that current CLI frequently proceed beyond refusal and can complete a non-negligible fraction of malicious OS-level tasks, especially when requests include TTP-style attacker knowledge. ADVCLI provides a reproducible testbed for evaluating these risks and for developing stronger safety mechanisms for tool-using CLI agents in the future.

Abstract:
Test-time scaling for code generation typically explores the solution space by sampling multiple programs from a fixed instruction. We study a complementary direction: instance-level instruction-space exploration. Our observation is that many coding failures stem from missing constraints, overlooked edge cases, or misleading reasoning paths induced by the original prompt. To address this, we propose TipCoder , a test-time instruction proposer that generates problem-specific auxiliary tips before code synthesis. TipCoder distills multi-turn debugging trajectories into proactive guidance and further optimizes the Proposer with reinforcement learning using a marginal-utility reward. At inference time, it generates both a base solution and a tip-guided solution, and applies a Reward Model for post-hoc selection. This exploration-selection design allows tips to expose additional candidate potential while reducing regressions from unnecessary guidance. Across multiple code-generation benchmarks and target Code LLMs, TipCoder provides a robust instruction-level test-time scaling strategy, comparing favorably with stochastic sampling and generic prompt optimization baselines under a shared reward-model-based selection protocol.

Abstract:
As LLM translation becomes a common way for English-language readers to read non- Western literature, a class of harm emerges for which NLP currently lacks vocabulary: the source author can be stigmatised through accumulated translation choices that are not detected by standard MT metrics. Humanities scholarship documents this process for Chinese danmei author Mo Xiang Tong Xiu (MXTX), whose novels and fan-translated public statements circulate widely in Anglophone fandom. Under China’s censorship of queer content, MXTX writes pseudonymously and hedges her public statements (Hu et al., 2024). Anglophone anti-fans read this caution as suspicious evasion rather than a survival strategy (Ng, 2024; Springman, 2024). We ask whether LLM-based translations reproduce the translation choices underlying this stigmatisation, operationalising three measurable hazards: voice collapse, connotation flattening, and hedging erasure. We introduce DANMEI-EPV, a 412- item benchmark double-annotated by professional literary translators, and evaluate eight LLMs in four prompting conditions against a 2,000-sentence GuoFeng-Webnovel reference. Every model exhibits all hazards at non-trivial rates, 5 to 6× higher than the GuoFeng reference. Further, newer high-capacity models optimised for fluent English flatten connotation more than smaller older models, suggesting that optimising for English fluency can trade off against source rhetoricity (Spivak, 1993). Finally, a lightweight inference-time intervention, “rhetorical-silence prompting”, reduces voice collapse by 27.4% and connotation flattening by 19.6% without retraining.

Abstract:
Radiology Report Generation (RRG) aims to reduce the burden of clinical documentation, yet generating reports that remain visually grounded in the input image remains challenging. Current medical vision-language models (Med-VLMs) often rely on curated preference data, retrieval augmentation, or large-scale labeled corpora, but remain prone to hallucinated findings due to weak cross-modal alignment. To address this challenge, we propose GRAIL : Grounded Radiology Alignment of Medical Vision-Language Models for Radiology Report Generation, a two-stage multimodal alignment framework for grounded RRG. First, Clinically Informed Textual Reasoning improves report structure and medical terminology using verifiable language rewards with periodic clinical guidance. Second, Expert-guided Visual Reasoning leverages a domain expert to rank generated reports using image-report alignment scores, encouraging visually grounded report generation without curated preference pairs or retrieval data. Experiments on multiple benchmarks show that GRAIL consistently improves both report generation quality and clinical accuracy, achieving relative gains over strong post-training baselines by up to 15 % in clinical accuracy and 13 % in generation quality. GRAIL also outperforms proprietary general-purpose and medical VLMs, highlighting the effectiveness of self-supervised multimodal post-alignment for reducing hallucinations in visually grounded radiology report generation.

Abstract:
Medical vision-language models (VLMs) are often evaluated by report retrieval, grounding, and explanation quality. We show that these metrics can create a metric illusion: a model may appear clinically grounded while exploiting acquisition template, crop, orientation, or protocol rather than patient anatomy. We formalize this failure as template-induced ambiguity and introduce TIA-VL, a counterfactual benchmark with source, template-family, severity, and compound protocol shifts. We also propose Rationale-Grounded Template Ambiguity Calibration (R-GTAC), a plug-in calibration layer that discounts template-dominated evidence while preserving anatomy and textual rationale support. Across strong medical VLM-style baselines, including BioViL-T-style frozen features, adapter features, and anatomy-teacher features, R-GTAC reduces wrong same-template errors to 0.0000 on the main protocol splits. We then test whether systems know when to abstain through readiness and counterfactual rationale deferral. Counterfactual rationale readiness filtering further identifies a reliable low-risk 25% subset reaching 0.8500 Top-1 on the hardest compound shift, while revealing that coupled protocol shifts pose critical frontiers for global risk calibration. Thus, TIA-VL supports a conservative message for medical VLM evaluation: models should be judged not only by whether they answer, but by whether their apparent rationales survive anti-shortcut counterfactuals. The code and benchmark will be available at https://github.com/YifanLuo0103/TIA-VL.

Abstract:
Voting steers representation in democratic bodies and functions as the primary mechanism for imposing the social will of the electorate upon the decision making machinery of regional government. Citizens will often have diverse, and conflicting, viewpoints which makes faithful representation of their plural interests a serious challenge in multi-party democracies. In the context of multi-agent LLMs, pluralistic preference alignment addresses this problem and, often through deliberative and voting methodologies, aims to draw groups toward a common policy that consistently bridges together a range of individual preferences. However, in multi-agent settings where the goal is to craft policy satisfying very diverse preferences, such as in social or political scenarios, there is little understanding of how to structure general process mechanisms that lead to the best outcome. To address this we present VoteSim, a framework to simulate a wide range of electoral systems across sociopolitical topics and a diverse set of personalized voter LLM agents. We draw on a rich persona modeling dataset, a range of relevant sociopolitical topics, and prevalent legislative procedure to generate social policy through deliberation. Voting agents then rank their preferred policy to determine how electoral systems can satisfy group preference. Detailed empirical analysis reveals that pluralistic alignment in multi-agent LLMs on important sociopolitical issues is best achieved through proportional electoral representation, reflecting outcomes among multi-party systems in human society.

Abstract:
Memory-augmented large language models extend reasoning beyond a fixed context window by maintaining long-term memory across interactions. However, existing memory systems often collapse stable user facts, episodic events, and behavioral rules into a shared space, allowing functionally distinct memories to be retrieved and used as interchangeable evidence. We identify this failure mode as heterogeneous memory contamination, where context-specific events become overgeneralized claims, or semantically relevant but functionally incompatible memories mislead generation. To this end, we introduce MemGuard, a type-aware memory framework that preserves functional memory boundaries during memory construction and retrieval. It assigns each memory an explicit functional role at write time, maintains relations across type-isolated memories, and selectively composes evidence only from necessary memory types, reducing contamination from irrelevant or functionally incompatible evidence. Across hallucination and long-horizon conversation benchmarks, MemGuard improves memory reliability by up to 28.27% while retrieving up to 5.8x fewer memory tokens than prior methods. These results suggest that reliable long-term reasoning depends on principled organization and selective use of heterogeneous memory.

Abstract:
Large language models can generate long chain-of-thought (CoT) reasoning, yet prior work suggests that CoT can be post-hoc rationalization rather than a faithful reflection of the computation through explicitly designed settings. In this work, we go further and propose a True Thinking Score (TTS) to quantify the causal contribution of each step in CoT to the model’s final prediction in realistic reasoning problems. Across eleven models ranging from 1.5B to 1.1T parameters on common reasoning benchmarks, we find that CoTs often interleave true-thinking steps, which causally affect the final answer, with decorative-thinking steps, which appear useful but have little causal influence; Such decorative steps remain prevalent even for frontier models: Over 30% of steps in Kimi-K2.6 are decorative on MATH with TTS ≤ 0.005. Furthermore, TTS enables effective CoT pruning: removing 50% of CoT steps with the lowest TTS can largely maintain the performance. Self-training on these pruned CoTs reduces reasoning length by 66% while preserving performance on Nemotron3-Nano-30B. Finally, we provide a mechanistic analysis showing that LLMs can be steered in the latent space to engage or disengage with reasoning steps. Overall, our results reveal that frontier LLMs often verbalize reasoning steps that are not causally used, challenging both the efficiency and the trustworthiness of CoT.

Abstract:
Document weighting is an underspecified bottleneck in black-box retrieval-augmented generation. Training-free systems inherit answer-aggregation weights from retrieval similarity, yet this proxy has not been systematically compared with evidence-based alternatives. We propose a unified two-parameter framework that generalizes REPLUG with an evidence term; it is training-free, black-box, and needs no large language model (LLM) calls beyond voting. Similarity-only (λ=0) and evidence-only (λ→∞) weighting are its limiting cases, so one interpolation parameter separates the retrieval prior from the evidence signal. We evaluate across 3 LLMs (7B–70B), 3 open-domain question-answering datasets (31,861 queries), and 3 off-the-shelf evidence types. Cross-encoder and embedding-stability evidence improve exact match in all 9 LLM–dataset cells, whereas natural language inference (NLI) fails as a weighting signal, to our knowledge the first systematic evidence of this failure mode. The similarity prior is largely redundant on NQ-Open and PopQA but helps on TriviaQA, where retriever scores track gold documents better; the pattern replicates under the E5 retriever, with smaller gains concentrated on long-tail PopQA. One default transfers across datasets and scales with minimal loss; calibration depends on signal type, not scale. Practically: prefer cross-encoder evidence, avoid NLI, and use vote mass for selective prediction; weighting itself adds sub-millisecond overhead.

Abstract:
Large language models (LLMs) hold promise for automating causal knowledge extraction, a fundamental component of scientific discovery, yet their ability to reliably identify causal claims in scientific literature remains poorly understood. Using 1,000 manually annotated abstracts spanning 20 disciplines, we evaluate eight frontier LLMs under four prompting strategies on a causal conclusion identification task. More elaborate prompting strategies do not necessarily improve performance, while open-weight models achieve competitive performance relative to proprietary models. LLMs systematically produce false-positive causal predictions on non-causal abstracts, with GPT-5 showing the most extreme bias, and Few-Shot prompting provides the most consistent calibration benefit. Performance varies substantially across disciplines with stronger and more stable results in biomedical fields. Our findings provide practical insights for using LLMs in scientific causal analysis and highlight calibration and domain variation as key challenges for advancing causal reasoning in scholarly text.

Abstract:
Large Language Models (LLMs) have achieved remarkable success across diverse domains, but their adaptation to privacy-sensitive, distributed datasets remains a challenge. While Federated Learning (FL) combined with Low-Rank Adaptation (LoRA) provides a resource efficient paradigm for collaborative fine-tuning, practical deployments are hindered by the dual challenges of resource heterogeneity and data heterogeneity. Existing rank-heterogeneous methods primarily focus on bridging dimension mismatches for aggregation but typically provide a unified global model for all clients sharing the same rank, failing to capture client-specific features in non-IID scenarios. In this paper, we propose FedRoRA (Federated Rank-wise Personalized LoRA), a novel framework that enables fine-grained personalization within rank-heterogeneous federations. FedRoRA decouples adaptation into shared global directions and personalized rank-wise magnitudes governed by learnable diagonal scales. On the server side, it extracts a global subspace via singular value decomposition (SVD) and redistributes client-specific initializations through a personalized projection and top-selection mechanism. Extensive experiments on NLU and NLG benchmarks demonstrate that FedRoRA consistently outperforms state-of-the-art methods.

Abstract:
Retrieve-then-rerank pipelines restrict large language models (LLMs) to a fixed top-k candidate set, making their reasoning ability fundamentally limited by the recall ceiling of the first-stage dense retriever. On reasoning-intensive retrieval, we make two observations: first, relevant documents missed by dense top-k retrieval are frequently reachable within a few hops on the native approximate nearest neighbor (ANN) graph; second, LLM relevance judgments can identify high-yield anchors whose neighborhoods contain these missed documents. Based on these findings, we propose NaviRank, an LLM-guided navigation framework that performs greedy expansion over the HNSW graph: it uses highly parallel pointwise LLM judgments to steer anchor selection and local expansion, followed by listwise calibration for final ranking. Experiments on BRIGHT show that NaviRank achieves 27.0 NDCG@10, while establishing the strongest accuracy–efficiency Pareto frontier in both token usage and latency among all evaluated methods.

Abstract:
Multimodal large language models (MLLMs) have shown strong image-text understanding and reasoning capabilities, but their multimodal interfaces also introduce new attack surfaces for jailbreak attacks. Recent black-box multimodal jailbreaks typically conceal harmful intent through typographic rendering, cross-modal decomposition or contextual distraction, yet their effectiveness remains constrained on strongly safety-aligned MLLMs. Our analysis suggests that unsuccessful attacks often arise from two sources: excessive visible risk that triggers refusals, and fragmented semantic cues that hinder intent reconstruction, resulting in off-target responses. To address this, we propose ARC, an audit-guided risk calibration attack framework that performs cue-preserving textual decomposition and progressive visual masking, revises residual visible risk through textual and visual audit feedback, and organizes the processed cues with scenario-grounded cross-modal reconstruction. Extensive experiments on closed-source and open-source MLLMs show that ARC consistently outperforms representative baselines, achieving average ASRs of 90.86% on closed-source models and 72.95% on open-source models. Code is available at: https://github.com/antuer/ARC. Warning: This paper contains offensive and unsafe contents generated by MLLMs.

Abstract:
In settings where labeled verifiable training data is the binding constraint, each checked example should be allocated to the model and reward density where it is most informative. We identify a reward-density principle: sparse sequence-level reward is most useful on models that can explore and discover better behavior, while dense token-level teacher supervision is better suited for compressing that behavior into a smaller deployment model; accordingly, scarce labeled data should be used upstream on the strongest available teacher, then transferred downstream as dense supervision. We evaluate this rule through a four-stage workflow: teacher RL, forward-KL warmup, on-policy distillation, optional post-bridge student RL on verifiable math and code generation with Qwen3, Llama, and Qwen2.5 models. At fixed Qwen3-1.7B deployment-student size, an RL-improved 8B teacher distilled through the dense bridge outperforms direct GRPO on the same student (79.3% vs. 75.9% on MATH; 25.2% vs. 19.8% on AIME2024, avg@16), while transfer from the same teacher before RL underperforms. A component ablation confirms that each stage is load-bearing: the RL-improved teacher adds 7.8 absolute points on MATH over a raw teacher of the same size, the forward-KL warmup adds 1.7, and on-policy distillation adds 3.3. The teacher-quality ordering: raw-teacher transfer < direct GRPO < RL-teacher transfer replicates on Llama-3.1-8B-Instruct with a Llama-3.3-70B-Instruct teacher and on Qwen2.5-7B-Instruct for code generation with a Qwen2.5-32B-Instruct teacher, where the dense bridge improves HumanEval and MBPP by 3.3 and 3.8 absolute points, respectively, over direct student GRPO. The operational lesson is to avoid spending scarce labeled data on the least prepared policy: use sparse reward for teacher-side discovery, dense transfer for student compression, and student-side sparse reward only after the bridge. The OPD implementation used in our experiments is available at https://github.com/HJSang/OPSD_OnPolicyDistillation.

Abstract:
In this work we focus on the problem of embodied task planning, which requires an agent to accomplish a complex task by separating it into atomic actions. Recent studies have developed LLM-based methods to address this problem. However, the executability and correctness of the generated plans remain suboptimal in complex environments and tasks. On the other hand, classical search algorithms have the potential to solve the planning problems perfectly, yet they require fully observable environments, known state transitions and goal state, which are often unattainable in general task domains. Therefore, we propose to implement an LLM-guided search algorithm that makes the search process possible and efficient by incorporating LLMs’ commonsense knowledge. Our method leverages LLMs to (1) translate natural language instructions into formatted goal conditions, (2) facilitate efficient exploration in partially observable environments, (3) implement a rule-based world simulator, and (4) prune the space of candidate actions to improve search efficiency. Experimental results on VirtualHome and ALFWorld demonstrate the superior performance of our agent.

Abstract:
Evaluating LLMs for crisis communication requires more than judging isolated responses, because public feedback and competitor actions can reshape trust over time. Existing benchmarks remain largely static and single-turn, offering limited evidence about long-horizon risk-sensitive communication. We introduce BrandEval , a two-track benchmark for product-crisis communication, focusing on public brand responses to product-related safety, quality, service, and trust incidents. BrandEval-Static evaluates stakeholder-aware strategic analysis, while BrandEval-Dynamic stress-tests response policies in BrandPolis , a controlled multi-agent sandbox with consumer feedback, competitor responses, and trust-risk trajectories. We also construct BrandSRD , a Chinese crisis-response dataset, and instantiate a reference Strategic Agent using Strategic Rationale (SR) . Across proprietary and open-source models, we find that static strategic quality is informative but incomplete: it aligns with some end-state outcomes but does not reliably predict volatility and delayed recovery; dynamic stress tests further expose tail-risk failures under manipulative or weakly grounded policies. The code and dataset will be available at https://github.com/htgan-ai/BrandEval.

Abstract:
Despite their strong multimodal understanding ability, multimodal large language models (MLLMs) incur substantial computational overhead when processing long visual token sequences. To reduce inference costs, recent studies have explored visual token pruning through vision-centric or text-guided strategies. However, these methods often overlook high-norm outlier tokens, i.e., tokens with abnormally large feature norms, leading to suboptimal pruning decisions. In this work, we show that such high-norm outlier tokens are highly redundant in both feature and spatial dimensions, yet are often mistakenly preserved as informative cues by existing methods. Motivated by this observation, we propose SinkPruner, a training-free visual token pruning framework for efficient MLLM inference. SinkPruner follows a coarse-to-fine design with two key modules: a visual sanitizer that filters high-norm redundancies and alleviates attention sink and attention dispersion, and a text-guided pruner that further retains tokens semantically aligned with the text query. Extensive experiments on twelve image-language and four video-language benchmarks demonstrate the effectiveness, efficiency, and generalizability of our framework. Notably, SinkPruner preserves 96.5% (91.8%) of the original performance of LLaVA-1.5 (Qwen2.5-VL) under an 89% token reduction. Experiments further indicate that our visual sanitizer exhibits promising transferability in enhancing the performance of existing pruning methods. Our code is available at https://github.com/LaVi-Lab/SinkPruner.

Abstract:
Large language model-based multi-agent systems are evolving from fixed interaction topologies toward dynamically orchestrated Agent Swarms. However, existing benchmarks are still largely based on single-agent or general-purpose agent tasks, making it difficult to systematically evaluate key orchestration capabilities. We propose SwarmBench, a benchmark that evaluates model performance from multiple perspectives, including accuracy, efficiency, cost, and process quality. Experimental results show that current models exhibit substantial differences in orchestration capability. These differences are reflected not only in final accuracy, efficiency, and cost, but also in the overall quality of the orchestration process itself. Based on these findings, we further propose SwarmExp, a simple yet effective method based on experience extraction and experience replay, which consistently improves the orchestration performance of large language models.

Abstract:
Persona drift remains a central challenge for personalized language models, as user profiles evolve over long interactions rather than remain permanently fixed. Models must therefore revise persistent persona states when preferences genuinely change, while avoiding updates driven by transient, ambiguous, or unresolved observations. We propose CORE, which separates turn-local evidence from persistent persona-state revision and selectively updates grounded user preferences through uncertainty-aware belief revision. We also introduce PERSIST, a held-out post-anchor benchmark for persona-state robustness under sequential interaction stress, covering ambiguity, conflict, and controlled social influence. Across ALOE, PersonaChat, and PERSIST, CORE improves personalized alignment and robustness, with complementary gains in normalized closed-slot state fidelity. Human evaluation and mechanistic controls further support explicit update control beyond stronger generation or persistent memory alone.

Abstract:
Multimodal BrowseComp tasks require agents to combine perception, tool use, and long-horizon reasoning over dynamic web content, challenging their ability to handle compositional structure, open-world uncertainty, and multimodal integration across extended interactions. Crucially, real-world multimodal browsing involves three distinct information-flow patterns: text-only, image-to-text, and text-to-image, yet existing data construction methods cover only the text-only and image-to-text patterns, leaving text-to-image largely unaddressed and limiting agent generality and robustness. We introduce UniBrowse, a unified data pipeline that for the first time simultaneously generates training data covering all three patterns, augments curated knowledge graphs with live web retrieval for improved fidelity, and introduces a novel metric of exploration degree to filter low-signal instances for efficient reinforcement learning. Through this pipeline, we produce high-quality cold-start tool-use trajectories and exploration-rich QA pairs, and train a 35B-scale agent via supervised fine-tuning and exploration-aware RL. The resulting UniBrowse agent achieves state-of-the-art performance on multimodal BrowseComp benchmarks, attaining an average accuracy of 54.4 across five diverse benchmarks—an improvement of 10.5 points over its base model Qwen3.5-35B-A3B—and surpassing serveral closed-source agent workflows such as GPT-5 (42.9), Gemini-2.5 Pro (44.8), and Gemini-2.5 Flash (41.3).

Abstract:
Extending discrete tokenization and autoregressive modeling to signals whose latent states follow continuous trajectories introduces phase–morphology ambiguity. Similar local shapes can occur at distinct trajectory positions, while the same position can vary across instances. Standard vector quantization (VQ) may therefore merge contextually different patches and split corresponding ones. We trace this behavior to radial-to-tangential gradient leakage under the straight-through estimator and propose Geometry Alignment Quantization (GAQ). GAQ combines a learned Mahalanobis reshaping of the codebook with a Householder backward map. The map exactly cancels the leakage in the idealized setting, while the numerical stabilization used in practice introduces a bounded deviation. We instantiate GAQ on ECG as AtomECG, a tokenizer of sub-waveform atoms composed by a GPT grammar. Atom sequences show long-context scaling from perplexity 488 to 24, split-replicated motif enrichment yields auditable clinical hypotheses, and AtomECG outperforms eight eSSL baselines on more than ten diagnostic tasks. A known-phase limit cycle and PPG results further support GAQ on phase-like signals beyond ECG.

Abstract:
Role-playing with large language models is fundamentally a session-level task, requiring agents to sustain character identity and interaction quality across extended multi-turn conversations. Yet existing evaluation and optimization methods remain largely turn-level, failing to capture long-horizon quality. We propose DynSess, a unified session-level framework for role-playing agents. DynSess-Eval scores complete dialogue sessions via rubrics targeting long-horizon behaviors. Leveraging its session-level rewards, we construct high-quality training trajectories through multi-turn lookahead search and train DynSess-Character with two complementary variants: DSPO (off-policy) and GSRPO (on-policy). Experiments show that DynSess-Eval aligns with human judgments substantially better than prior evaluators, and blind human evaluation further shows that DynSess-Character matches the strongest character model despite using substantially fewer parameters, while maintaining strong role consistency and interactive ability. The dataset and code are publicly available in https://github.com/Collab-Gen/DynSess.

Abstract:
The rapid advancement of generative models has made synthetic images increasingly realistic, challenging reliable detection. Existing methods are often limited to end-to-end classification or monolithic reasoning, and thus fail to model structured forensic reasoning and heterogeneous visual evidence. We revisit synthetic image detection from a cognitive perspective and propose a Heuristic-to-Reasoning cognitive skill learning framework for evidence-based forensic analysis. Given an input image, our framework first extracts heuristic perceptual clues, selects the optimal forensic skill, and then performs skill-conditioned reasoning for evidence extraction and decision making. To support this paradigm, we introduce ClueAegis-Bench , which decomposes synthetic image detection into explicitly annotated forensic cognitive skills for structured evaluation beyond binary classification. Based on this benchmark, we propose ClueAegis ( C ognitive-skill L earning for U nified E vidence-based Synthetic Image Detection), a two-stage agentic framework that conducts heuristic skill selection followed by evidence-guided reasoning through skill-conditioned toolchains. This design reformulates synthetic image detection as a configurable multi-skill reasoning process that bridges perception, skill selection, and forensic reasoning. Extensive experiments show that ClueAegis achieves state-of-the-art performance while improving cross-domain generalization and robustness. It also provides transparent reasoning trajectories and structured forensic evidence, offering a more explainable alternative to conventional end-to-end detectors.

Abstract:
Model fusion integrates the capabilities from source models into a single target model. As of June 2026, Hugging Face hosts more than 2M models. This growing pool provides a rich base for model reuse and capability integration. Yet existing surveys often cover only separate parts of this space, and they do not provide a unified definition or a systematic taxonomy. This survey defines model fusion and organizes prior work into three levels: parameter-level, representation-level, and behavior-level fusion. We also review related metrics, benchmarks, and applications, summarize current challenges, and identify future directions. Our goal is to provide a clear map of this area and support future work on model fusion. A comprehensive list of papers about model fusion is available at https://github.com/Baicaihaochi/Awesome-Model-Fusion-Survey .

Abstract:
LLM-based web agents automate browser interactions by generating structured action sequences, yet their deployment in long-horizon tasks is hindered by fragile error propagation in noisy DOM trees. Critically, existing training paradigms rely exclusively on positive supervision from gold demonstrations, teaching models what is correct but never why plausible alternatives fail, leaving agents unable to distinguish between subtly different yet critically divergent choices. We propose FailAlign, a failure-aligned learning framework that systematically integrates error signals as first-class training supervision. We mine a Failure Index from model rollouts by pinpointing first-error steps, then leverage these instances across a three-stage pipeline: (i) action-grounded context compression distills verbose HTML into compact DOM snippets; (ii) hard-negative DPO pairs gold actions with semantically similar incorrect alternatives to sharpen decision boundaries; (iii) trajectory-level GRPO mitigates error cascades by re-simulating corrected trajectories. On Mind2Web with Qwen3-32B, FailAlign achieves 49.7% SSR and 5.6% SR, surpassing models 15× larger (Qwen3-Coder-480B: 37.0% SSR) and commercial systems (Claude Opus-4.5: 39.4% SSR), validating the efficacy of learning from failures.

Abstract:
Large language models (LLMs) have acquired substantial medical knowledge and reasoning skills during pretraining, yet eliciting these capabilities typically requires scarce rationales or costly post-training. To address this gap, we introduce RIPT, a lightweight framework for data-efficient endogenous capability activation. It explicitly decomposes this objective into two complementary components: reasoning enhancement and medical knowledge elicitation. For the former, we extract steering vectors from hidden activations on a small set of high-quality paired reasoning and vanilla responses to shape LLMs’ reasoning behavior. For the latter, we obtain prefix vectors via prefix tuning on simple medical QA pairs to elicit domain-specific knowledge. At inference, RIPT freezes the backbone LLM and combines steering-vector injection with prefix-vector conditioning. Rather than injecting external knowledge, RIPT aims to unlock medical reasoning capabilities already acquired by LLMs during pretraining. Experiments under limited-resource settings show that RIPT consistently improves diverse backbone LLMs and benchmarks, suggesting a generalizable pathway for lightweight medical reasoning enhancement.

Abstract:
Large language models (LLMs) exhibit mode collapse, where their output distribution concentrates on familiar semantic modes. When diversity disappears, repeated sampling becomes semantic repetition rather than discovery, making rare but stronger solutions unlikely to surface and leaving self-evolution and continual learning difficult to sustain. Most remedies intervene only at inference time, shaping samples without making diversity a model capability. We propose PRISM, a training-time framework that internalizes diversity within the model. PRISM combines (i) submodular curation that pairs each prompt with diverse-yet-correct multi-teacher solutions, and (ii) stochastic embedding modulation, which turns special-token perturbations into implicit signals for sampling distinct outputs. A theoretical analysis establishes the stability of this mechanism. On WritingPrompts and BookMIA, PRISM moves the quality-diversity frontier outward, achieving the best generation quality in all model-benchmark settings while maintaining the strongest or near-strongest semantic diversity across two open-weight model families. It also improves sampling efficiency on math and code benchmarks. Beyond inference, PRISM yields a stronger initialization for reinforcement learning, sustaining diverse rollouts and mitigating the diversity collapse commonly observed during RL fine-tuning.

Abstract:
As large language models (LLMs) evolve into autonomous agents for long-horizon information-seeking, managing finite context capacity has become a critical bottleneck. Existing context management methods typically commit to a single fixed strategy throughout the entire trajectory. Such static designs may work well in some states, but they cannot adapt as the usefulness and reliability of the accumulated context evolve during long-horizon search. To formalize this challenge, we introduce a probabilistic framework that characterizes long-horizon success through two complementary dimensions: search efficiency and terminal precision. Building on this perspective, we propose AgentSwing, a state-aware adaptive parallel context management routing framework. At each trigger point, AgentSwing expands multiple context-managed branches in parallel and uses lookahead routing to select the most promising continuation. Experiments across diverse benchmarks and agent backbones show that AgentSwing consistently outperforms strong static context management methods, often matching or exceeding their performance with substantially fewer interaction turns while also improving the ultimate performance ceiling of long-horizon web agents. Beyond the empirical gains, the proposed probabilistic framework provides a principled lens for analyzing and designing future context management strategies for long-horizon agents.

Abstract:
Existing continual knowledge editing methods suffer from cognitive dissonance: after an edit, the model answers the edit prompt correctly but reverts to the old fact under complex equivalent queries. This means old and new facts coexist rather than the old being replaced. The root cause of cognitive dissonance is that existing methods fit the new fact on the target prompt without overwriting the old one. In constrained methods, static null-space projectors lock outdated representations into the protected subspace, forcing updates into orthogonal directions. To resolve cognitive dissonance, we propose P-Space (Puncturing the Null-Space to Resolve Cognitive Dissonance in Continual Knowledge Editing), which at each edit step (1) punctures the protected covariance to unprotect the outdated representation, (2) overwrites the old mapping in closed form along its original direction under the punctured projector, and (3) anchors the new fact into the covariance with adaptive scaling so that future edits preserve it. Experiments on two datasets show that P-Space outperforms existing methods while preserving general capabilities.

Abstract:
Large language models (LLMs) are widely used for task understanding and action planning in embodied intelligence (EI) systems, but their adoption substantially increases vulnerability to jailbreak attacks. While recent work explores inference-time defenses, existing methods rely on static interventions on intermediate representations, which often degrade generation quality and impair adherence to task instructions, reducing system usability in EI settings. We propose a dynamic defense framework. For each EI inference request, we dynamically construct a task-specific safety-semantic subspace, project its hidden state to the most relevant direction, and apply SLERP rotation for adaptive safety control. At comparable defense success rates, our method preserves generation quality, improves usability, reduces tuning cost, and strengthens robustness in EI scenarios.

Abstract:
Agent skills, which consist of reusable strategies that guide agent reasoning and action, have shown strong potential for improving model capability at inference time. However, current skill construction methods treat the problem as one-shot extraction, overlooking a fundamental tension: a skill tailored to the specific task fails to transfer, while the abstracted skill often provides insufficient guidance. We attribute this fragility to the absence of explicit mechanisms for skill specification and generalization . To address this gap, we introduce SkillComposer, a framework that decomposes skill construction into three learnable operations: create , improve , and merge . Trained via systematic rejection sampling recipe, SkillComposer enables language models to self-evolve skills at inference time and supports three deployment modes: offline for building generalized libraries, online for task-specific refinement, and hybrid for combining both. Comprehensive experiments on -Bench, LiveCodeBench v6, and AppWorld show that SkillComposer consistently outperforms baselines. Our SkillComposer-4B improves a 27B executor by up to +4.5 on agent tasks and +3.4 on code tasks, while generalizing across domains and task types unseen during training. Analysis reveals that merge and improve address orthogonal quality dimensions and that skill composition is a transferable meta-ability, providing a practical recipe for skill-augmented inference.

Abstract:
Public benchmarks enable fair and reproducible evaluation of LLM reasoning, but they become fragile for deep research agents that actively search the web during inference. Such agents may retrieve public benchmark metadata, question context, or even ground-truth answers via web search. This gives rise to Search-Time Contamination (STC), where external retrieval bypasses intended reasoning and inflates measured performance. We systematically study STC in deep research agent evaluation. We define three contamination types with increasing severity, namely Benchmark Metadata Leakage, Question-Context Leakage, and Explicit Answer Leakage, and develop detection algorithms to identify them and quantify their impact on agent performance. Evaluating modern deep research agents on seven public benchmarks, we find that STC is widespread and can inflate performance by up to 4%. Our findings show that existing evaluations may overestimate true reasoning ability. We therefore advocate contamination-aware practices, including isolated sandboxes, transparent search trajectories, and controlled benchmark access.

Abstract:
LLM-based rerankers in recommender systems are inherently constrained by limited context windows: they can only reorder a small retrieved candidate set, so any relevant item missed by upstream retrieval cannot be recovered at ranking time. To mitigate this dependency, we propose RoutePO, which repositions the LLM from an item ranker to a recall strategy router for multi-channel recall. Conditioned on a user’s profile/context (interaction history and optional demographics), the LLM outputs personalized fusion weights over heterogeneous retrieval experts, enabling dynamic candidate aggregation without exposing long candidate lists from the entire item pool to the prompt. We train the router via a synergistic two-stage alignment process: Coarse Routing Alignment (SFT) warm-starts the policy with a quality-aware oracle prior over retrieval experts, and Fine-grained Metric Optimization (RL) directly optimizes recall-stage ranking metrics (e.g., NDCG) by sampling simplex-valued routing weights and rewarding fused-list quality. To support continuous exploration over the simplex under GRPO-style optimization, we introduce SofT-GRPO based on Gumbel–Softmax reparameterization with nucleus restriction and KL regularization. Finally, we instantiate fusion with both weighted score merging and a weight-aware interleaving strategy (Snake Fusion) that preserves within-channel rankings. Experiments on three real-world datasets demonstrate consistent gains over strong individual recallers and multi-channel fusion baselines, including up to 15.88% relative improvement in NDCG@50 on Food. Our implementation is available on https://github.com/zhengzaiyi/RoutePO .

Abstract:
Large language models (LLMs) excel at semantic understanding, yet their ability to reconstruct internal structure from scrambled inputs remains underexplored. Sentence-level restoration is difficult to evaluate automatically because scrambled sentences often admit multiple valid reorderings. We introduce OrderProbe, a deterministic benchmark for structural reconstruction using fixed four-character expressions in Chinese, Japanese, and Korean, which have a unique canonical order and thus support exact-match scoring. We further propose a diagnostic framework that evaluates models beyond recovery accuracy, including Semantic Accuracy, Logical Validity, Structural Consistency, Robustness, and Information Density. Experiments on twelve widely used LLMs show that structural reconstruction remains difficult even for frontier systems: zero-shot recovery frequently falls below 35%. We also observe a consistent gap between meaning-oriented generation and exact structural reconstruction, suggesting that structural robustness is not an automatic byproduct of semantic competence.

Abstract:
Large language models (LLMs) are increasingly used in long-running user-facing applications where responses may need to maintain a consistent interaction style. Prompting is easy to reuse but often weak, while fine-tuning and representation interventions can be stronger but are typically model-specific. We present PeLA , Prompt-Enhanced Logit Adaptation, a parameter-efficient framework for Big Five personality-trait steering. PeLA keeps the base LLM frozen, uses a personality prompt to specify the desired trait direction, and applies a learned logit adapter during decoding. The adapter contributes a base-relative residual controlled by a token-position gate, refining the prompt-conditioned distribution only when intervention is useful. Because the control module operates in output-token space, it can be reused across models with identical tokenizer vocabularies and token-id order. On open-ended PersonalityBench questions, PeLA improves macro-average target-direction scores over a strong Personality Prompt baseline on Qwen3-8B and Qwen2.5-7B, while cross-size experiments demonstrate tokenizer-compatible transfer within both the Qwen3 and Llama-3.2 families after validation-based scale calibration. PeLA narrows the gap to model-specific steering while preserving a frozen base model.

Abstract:
Evidence absence is not evidence insufficiency, but fact verification benchmarks can make them observationally similar. The Not Enough Information (NEI) label is often operationalized through constructed evidence conditions, and that choice silently determines what a verifier learns. We introduce NEI-CAP, a construction-aware diagnostic protocol for insufficient-evidence evaluation. Each NEI example carries the construction family that produced it; NEI-CAP audits shortcut cues, validates hard cases through human adjudication, and tests whether competence transfers across constructions. We instantiate the protocol on SciFact, with FEVER and HoVer as bounded external controls. Across these settings, NEI competence does not transfer reliably: encoder verifiers and an instruction-tuned decoder trained on shortcut-prone constructions fail to recognize semantically related insufficient evidence, and mixed-construction training narrows but does not close the gap. Fixed-claim diagnostics further show that the evidence condition shifts confidence in the reference Support/Refute label, not only NEI recall, so an aggregate NEI score can hide which problem a model has actually solved. We therefore recommend reporting the construction family alongside the score, and distill the results into a checklist for benchmarks that carry an insufficient-evidence label.

Abstract:
Chinese classifier prediction is often evaluated as a single-label classification problem, assuming one gold classifier for each context. We argue that this assumption overlooks the inherent variability and conceptual nature of classifier use. We propose a controlled experimental framework for studying Chinese classifiers in humans and vision-language models across three tasks: Classifier Choice, Head Noun Completion, and Referring Expression Generation. Based on a human behavioural dataset, we evaluate models against human response distributions rather than single gold labels. Our results show that classifier use is shaped by noun specificity, animacy, and task setting, and that models differ from humans not only in classifier choice but also in classifier-mediated conceptualisation. Although explicitly encoding the behavioural patterns learned from the human experiments into prompts improves model alignment with human behaviour, especially for smaller models, its effect is uneven across tasks: it helps models better match dominant human preferences, but does not enable them to reproduce the full distribution of human variability or fully close the gap in natural generation settings.

Abstract:
Tool-augmented medical agents promise clinical control and interpretability, yet their specialized outputs are often invoked broadly and compressed into text. It remains unclear whether failures arise from this textual interface or low-fidelity upstream tools. We study this question through RouteVerify, a transparent diagnostic sandbox combining a lightweight linear router, compact evidence cards, and verifier-gated multimodal reinspection. On matched and held-out ChestAgentBench splits, RouteVerify improves answer accuracy by about seven points over naive always-call-tools execution while roughly halving token use. However, it remains about two points below a direct VLM-only baseline and uses nearly twice the tokens. Oracle probes clarify the bottleneck: replacing predicted evidence with benchmark-side gold explanations raises the same downstream text reasoner above 95 percent accuracy, or above 91 percent after removing explicit answer cues. Thus, text cards can express sufficient evidence on this benchmark; the dominant practical limitation is upstream tool quality. An external JSRT/SCR audit supports this diagnosis, revealing limited nodule-classification discrimination and structure-dependent segmentation quality. The remaining raw-versus-sanitized oracle gap also indicates sensitivity to human-authored cues. We present RouteVerify as an empirical boundary study of where current tool-augmented medical agents fail.

Abstract:
Reasoning alignment usually assumes each query has a determinate answer, rewarding models for reaching it and penalizing or ignoring failures. This assumption breaks on flawed queries with missing information or contradictory constraints, causing models to hallucinate invalid answers or over-refuse difficult valid tasks. We study this failure mode as solvability boundary learning : separating input-side answerability defects from model-side inability on well-posed tasks. To address this, we introduce UnsolvableQA , a verifier-backed multi-domain benchmark built with deterministic checks and audited math verification, and UnsolvableRL , an alignment framework that decouples reasoning, unsolvability detection, and capability-calibrated refusal. UnsolvableRL yields over 85% unsolvability detection on Qwen3-4B while improving solvable accuracy from 43.4% to 69.4%, with a +35.4-point solvable-accuracy gain on Llama3-8B. Diagnostic tuning further distinguishes missing-information defects from constraint conflicts and transfers to interactive clarification, improving Detail Recovery from 47.8% to 60.7% for Llama3-8B. Ablations reveal Capability Collapse: refusal penalties degrade detection without unsolvable data, but act as a rigor regularizer when such data are included.

Abstract:
Large language models solve grade-school math word problems with high accuracy, yet a single irrelevant clause inserted into the problem can collapse it. We reconcile these observations with a mechanistic account. We show that the model’s internal computation decomposes into a four-stage sequential pipeline, Schema Abstraction, Operation Planning, Operand Binding, and Computation, each stage producing a distinct intermediate representation in an identifiable band of layers. Using the same scaffold to diagnose distractor-induced failure, we localize the corruption to a single stage, Operation Planning, implemented by a set of attention heads whose causal role we validate bidirectionally. In short, we provide a mechanistic interpretation of math word problem reasoning in LLMs, and their failure when distracted.

Abstract:
Persistent LLM agents require memory representations that make the formation of person understanding explicit across long term interaction. Existing agent memory methods emphasize information retention and retrieval, yet give limited account of how accumulated interaction evidence is abstracted into person understanding. We view this process as schema formation, where situated evidence is abstracted into reusable patterns and stable person level claims. We introduce PersonaTree, a structured lifecycle memory framework that realizes this view as a three level persona tree with explicit support paths from evidence to claims. PersonaTree maintains the tree through conservative writing, confidence guided consolidation, and query conditioned path retrieval, returning only the evidence depth required by each query. Across six person understanding and persistent memory benchmarks with three answer backbones, PersonaTree ranks first in 12 of 18 compact scores and reaches the top two in 16 settings. Ablations show that hierarchy improves abstract person understanding on KnowMe, while support path retrieval improves RealPref alignment under a comparable context budget.

Abstract:
Recent sign language translation methods have advanced visual-language alignment, yet autoregressive generation remains vulnerable to unstable policy updates under ambiguous visual evidence. Such instability may cause locally plausible continuations to accumulate into sentence-level semantic deviations, especially when reinforcement learning is used with noisy reward and advantage estimates. Existing PPO-style policy optimization applies confidence-agnostic clipping bounds to all generated tokens, which limits its ability to distinguish between over-amplified confident predictions and low-probability tokens that may still carry useful corrective signals. To address this issue, we propose CAPO-SLT, a reinforcement learning based framework for stable sign language translation generation. CAPO introduces a confidence-aware policy optimization strategy that adapts the upper clipping bound according to old-policy token probability. It applies more conservative updates to high-confidence predictions while preserving update capacity for low-probability positive-advantage tokens. In addition, CAPO caps negative-advantage updates to improve robustness under noisy supervision. This design provides a simple drop-in replacement for standard clipping and stabilizes generation-stage optimization without changing the underlying visual encoder or reward function. Experiments on Chinese and American sign language benchmarks show consistent improvements over pose-based baselines under the same input setting, suggesting that CAPO-SLT provides an effective optimization strategy for stable sign language translation generation.

Abstract:
This paper introduces SCHEDBench, a natural-language benchmark for evaluating combinatorial scheduling constraint faithfulness under surface-form variation. Grounded in canonical scheduling instances and solver-derived feasibility and optimality, SCHEDBench assesses whether large language models (LLMs) generate schedules with the same constraint-feasible behavior across varied natural-language (NL) surface forms. SCHEDBench spans 1,132 instances across job-shop scheduling problems (JSP), single and multi-mode resource-constrained project scheduling problems (RCPSP), nurse rostering/scheduling, and curriculum timetabling problems of varying difficulty. Instances are templated into natural language problems using domain-specific templates, themed entities, lexical-syntactic template rephrasing, and constraint-level surface-form variation—with reference solutions verified for feasibility and objective optimality. Across thirteen frontier and open-weight LLMs, we find that models are not reliably invariant to semantically equivalent renderings of the same scheduling problem. Surface-form variation reduces feasibility and induces above-noise shifts in per-instance hard-constraint violations on matched instances. Among the tested isolated axes, constraint reordering yields the clearest above-noise sensitivity.

Abstract:
Large Language Models (LLMs) often favor Western-associated entities across cultural contexts. Conventional debiasing methods aim for uniform neutrality, but cultural bias mitigation demands context-conditional behavior, preferring culturally appropriate entities when cultural cues are present and remaining neutral when they are absent. We propose CoCoA (Context-Conditional Cultural Alignment), a framework that learns this behavior through dual-context training on the same entity pairs under contexts with and without cultural cues. CoCoA combines a contrastive alignment objective with calibration and drift regularization, optimized through goal-aware gradient reconciliation. We evaluate CoCoA on CAMeL and Camellia, two entity-centric cultural bias benchmarks, across ten language settings and four LLMs. CoCoA reduces the Cultural Bias Score from 43 to 24 on average while maintaining near-neutral preferences at 50.2, with minimal impact on general performance across five standard benchmarks. These findings highlight that effective cultural alignment requires context-conditional modeling rather than uniform debiasing, and establish a new direction for mitigating entity-centric cultural bias in LLMs.

Abstract:
Fluent but subtly wrong mathematical solutions can contaminate training, evaluation, and reward-labeling data. We present AEGIS-D (Assessment- and Evidence-Gated Inspection System via Debate), a precision-first multi-agent pipeline for blind verification of competition-level mathematics. Five same-model, role-prompted agents exchange structured arguments, while an assessment gate accepts a candidate only when at least three report positive confirming evidence rather than mere non-refutation. On 195 hard-to-verify variants from 58 IMO/USAMO/Putnam-level problems, a single judge achieves 55.1% precision; AEGIS-D achieves 92.5% precision and 63.8% recall, reducing false positives from 24.7 to 3.0 per run (8.2-fold). A repeated two-by-two factorial associates role prompting with the largest observed false-positive reduction and finds a smaller conditional benefit from gating. Paired round and threshold analyses expose the cost-precision trade-off, and single-run transfers to Omni-MATH and OlymMATH provide preliminary evidence beyond the constructed stress test.

Abstract:
For many languages, grammatical error correction (GEC) remains underexplored due to the lack of dedicated datasets and evaluation benchmarks. Moreover, many existing GEC datasets are based on texts written by second-language learners, which narrows the research focus to specific error types. To address these gaps, we introduce a new Polish GEC benchmark, AMU-GEC, built from texts written by native Polish speakers and manually annotated by expert linguists. The benchmark is released as a machine learning challenge, without revealing the expected labels, to prevent contamination. To set baselines, we evaluate recent multilingual and Polish LLMs, finding that smaller models may perform better when trained on monolingual (Polish) data. We also conduct a human analysis of the best performing model to gain more insights about the model behavior when it is different from human annotations. The analysis shows that most of the changes suggested by the model do not introduce errors into the texts; instead, they typically either slightly improve text fluency or represent valid alternative corrections.

Abstract:
While autonomous software engineering (SWE) agents are reshaping programming paradigms, they currently suffer from a “closed-world” limitation: they attempt to fix bugs from scratch or solely using information from the current repository, ignoring the immense historical human experience available on platforms like GitHub. Accessing this open-world experience is hindered by the unstructured and fragmented nature of real-world issue-tracking data. In this paper, we introduce MemGovern, a framework designed to govern and transform raw GitHub data into actionable experiential memory for agents. MemGovern employs experience governance to convert human experience into agent-friendly experience cards and introduces an agentic experience search strategy that enables logic-driven retrieval of human expertise. By producing 135K governed experience cards, MemGovern achieves a significant performance boost, improving resolution rates on the SWE-bench benchmark by 4.65%. As a plug-in approach, MemGovern provides a solution for agent-friendly memory infrastructure.

Abstract:
While Large Language Models (LLMs) have demonstrated impressive general capabilities, their direct application in the legal domain is often hindered by a lack of precise domain knowledge and complexity of performing rigorous multi-step judicial reasoning. To address this gap, we present LegalOne, a family of foundational models specifically tailored for the Chinese legal domain. LegalOne is developed through a comprehensive three-phase pipeline designed to master legal reasoning. First, during mid-training phase, we propose Plasticity-Adjusted Sampling (PAS) to address the challenge of domain adaptation. This perplexity-based scheduler strikes a balance between the acquisition of new knowledge and the retention of original capabilities, effectively establishing a robust legal foundation. Second, during supervised fine-tuning, we employ Legal Agentic CoT Distillation (LEAD) to distill explicit reasoning from raw legal texts. Unlike naive distillation, LEAD utilizes an agentic workflow to convert complex judicial processes into structured reasoning trajectories, thereby enforcing factual grounding and logical rigor. Finally, we implement a Curriculum Reinforcement Learning (RL) strategy. Through a progressive reinforcement process spanning memorization, understanding, and reasoning, LegalOne evolves from simple pattern matching to autonomous and reliable legal reasoning. Experimental results demonstrate that LegalOne achieves state-of-the-art performance across a wide range of legal tasks, surpassing general-purpose LLMs with vastly larger parameter counts through enhanced knowledge density and efficiency.

Abstract:
Social power plays a fundamental role in shaping human interaction, yet computational studies of power remain limited to narrow linguistic and cultural settings. Existing datasets further lack the demographic and relational depth needed for robust cross-cultural analysis. To address this gap, we introduce a theoretically grounded framework for studying social power in naturalistic multilingual dialog through movie screenplays. The framework integrates a schema informed by social science theory, a native speaker annotation pipeline refined through pilot studies, and a custom interface for scalable cross-lingual analysis. Using this framework, we constructed an initial corpus containing 15,836 annotated instances from 100 scenes in French and Egyptian Arabic movies. Our analysis reveals strong agreement on observable demographic and contextual attributes, while socially interpretive aspects, such as power asymmetry and intention alignment, remain more contested, highlighting the complexity of social power across cultures. We evaluated 6 Large Language Models (LLMs) and Multimodal LLMs on cross-cultural social power reasoning, finding persistent gaps between human and model agreement in relational and theory-of-mind reasoning. Our work introduces the first extensible multilingual framework for studying social power in dialogues and provides an initial evaluation setting for studying cross-cultural social reasoning.

Abstract:
LLM agents are increasingly expected to support enterprise workflows, where tasks often involve missing information, uncertainty, feedback, and long-term trade-offs. However, existing enterprise and financial benchmarks mainly test static capabilities such as information extraction, numerical calculation, domain knowledge, and financial QA, leaving interactive and long-horizon decision-making underexplored. To bridge this gap, we introduce EnterpriseBench, a benchmark that evaluates LLM agents across this spectrum, from static question answering to dynamic decision-making. Specifically, EnterpriseBench reorganizes existing enterprise and financial QA datasets into a unified foundational suite annotated by capability and difficulty, and introduces three professional interactive settings: Consulting, based on management-consulting-style business cases for client problem diagnosis through multi-turn information seeking; the Beer Game, adapted from a classic supply-chain management simulation for inventory control under delayed feedback; and Enterprise Digital Twin, a project-based business simulator for workforce, risk, and project planning. Experiments with nine agent methods under two backbone models show that current agents have not yet achieved stable, comprehensive, and cross-task reliability in enterprise scenarios. These results show that EnterpriseBench provides a practical benchmark for evaluating LLM agents in realistic enterprise strategic reasoning and decision-making. Our codes and benchmark implementation are publicly available at https://github.com/sduyangmin/FirmBench.

Abstract:
External LLM services provide strong generation capabilities but require users to send prompts that may contain sensitive information. Existing prompt-protection methods focus on masking, perturbing, or rewriting the input prompt itself; however, these transformations can degrade response quality by removing task-relevant context or leave sensitive cues recoverable from the sanitized prompt. We present SYMMETRA, an end-to-end prompt-response protection framework based on semantic prompt sanitization. SYMMETRA detects context-dependent sensitive spans, selects response-aware replacements, and locally reconstructs responses with session memory and post-editing. We also introduce a synthetic prompt-domain dataset with an extended taxonomy and span-level annotations. Our evaluation shows that SYMMETRA improves the privacy–utility trade-off, reducing direct leakage to 0.13 and adversarial recovery to 0.42 while preserving a response similarity of 0.82 and restoring task-specific references in final user-facing responses after external inference.

Abstract:
Mathematical reasoning is a primary indicator of large language models’ (LLMs) intelligence. However, existing LLMs exhibit failures in robustness and generalization. This paper attributes these deficiencies to spurious reasoning, wherein generated reasoning traces are driven by superficial correlations, leading models to blindly reproduce memorized patterns from the training data. To address this challenge, we propose the AdaR framework to equip LLMs with adaptive reasoning, wherein models establish correct correlations between query templates and reasoning processes. AdaR automatically synthesizes logically equivalent queries by varying variable values and trains models using Reinforcement Learning with Verifiable Rewards (RLVR) to penalize spurious logic while encouraging adaptive logic. To ensure data quality, we extract the problem-solving logic from the original query, generate the corresponding answer through code execution, and then apply a sanity check. Experimental results demonstrate that AdaR achieves substantial improvements in mathematical reasoning while maintaining high data efficiency. Furthermore, even advanced LLMs still exhibit deficiencies in robustness and generalization, which our work effectively mitigates.

Abstract:
Retrieve-then-generate pipelines are commonly used to produce deep-research answers for open-ended questions, but retrieval alone is insufficient: LLMs must organize noisy and fragmented evidence into comprehensive, well-cited answers. We refer to this process as evidence synthesis. However, direct generation often underuses evidence, misaligns citations, and collapses diverse information into shallow summaries, exposing an evidence synthesis gap between retrieval and generation. Thus, we propose DeepWeaver, a novel framework that weaves noisy retrieved evidence into comprehensive answers by maintaining Thought Block Chains (TBCs), a structured representation that groups claims, salient information, keywords, and supporting evidence. DeepWeaver uses subordinate TBCs to inspect residual evidence, commit TBC revisions, and discover new claims before final generation. We evaluate DeepWeaver on open-ended QA over both knowledge bases and the web, and introduce LoQA, a high-density benchmark for evidence synthesis. Across multiple LLMs, DeepWeaver improves content sufficiency, citation grounding, and detail preservation on LoQA, while achieving deeper insights and higher citation quality on DeepResearch Bench. These results show that evidence weaving is an effective mechanism for bridging retrieval and generation in open-ended QA.

Abstract:
Schema linking is a task of investigating relevant SQL schema (items and paths) for a specific query language. Constructing a reliable linker is crucial for Text2SQL auto-coding since it provides informative clues for directing SQL coders. Current studies have obtained considerable performance gains when independently evaluating linkers, which allow F 1-score to be higher than 90% at best. However, our observation illustrates that the existing linkers still suffers from a high missing rate of Bridge Path , i.e., omitting one or a series of SQL keywords that link across SQL tables. To address the issue, we propose to enhance linkers by Bridge Path Recovering (abbr., BPR) via Tentative Answer Profiling (TAP). On the one hand, BPR is implemented by learning to rank up the better bridged schemata during finetuning. On the other hand, TAP tentatively conjectures possible answers for the query language, and generates their profiles (types and CoT steps) to guide BPR. This is inspired by the experience that answer conjecturing contributes to reversely deducing the multi-hop routing path over SQL tables. Technologically, we train LLaMA-3.1 for TAP, where teacher-student distillation is conducted via Deepseek-3. To ensure sufficient ranking data used for training BPR, we employ a multilevel retrieval technique to align profiles with diverse schemata. We experiment on the benchmark datasets BIRD and Spider. The experimental results show that our approach (BPR-TAP) consistently yields improvements for F 1-score upon all benchmarks, and surpasses multiple competitive schema linkers. All source codes are available at https://anonymous.4open.science/r/BPR-TAP-Text2SQL .

Abstract:
Personal memory agents process sensitive user data, making on-device deployment with small language models (SLMs) attractive. However, existing memory-agent prompts are often monolithic specifications written for high-capacity LLMs, overloading a single inference with many requirements while leaving necessary intermediate operations implicit. This mismatch creates two bottlenecks for SLMs: prompt complexity and logical incompleteness. We introduce CAST (Compiling Agent Specifications into Scaffolded Tasks), a framework that converts a monolithic memory-agent prompt into a scaffolded execution procedure. CAST segments the original prompt into fixed, semantics-preserving main tasks, then synthesizes and places auxiliary scaffolding tasks that produce reusable intermediate artifacts; a trajectory-feedback step then adapts these scaffolds to the target SLM. At inference time, only the target SLM executes the compiled procedure. Across two memory-agent frameworks on LoCoMo and MemoryAgentBench, CAST outperforms monolithic prompting, prompt optimization, and reasoning-based baselines, while offering a stronger quality-latency trade-off than unconstrained reasoning. The results show that adapting memory agents to SLMs requires restructuring execution, not merely rewriting prompts.

Abstract:
Large Language Model (LLM) agents in distributed decision-making scenarios often operate as spatially separated entities and coordinate from fragmented local information. However, this physical dispersion constrains agents to limited local perception and finite temporal horizons. We characterize this bottleneck as spatiotemporal partial observability, in which spatial and temporal limitations are fundamentally coupled. Resolving spatial conflicts requires temporal reasoning about neighbors’ future actions, while temporal planning requires spatial context beyond local perception. To bridge this gap, we introduce MACRO-LLM, LLM-empowered multi-agent collaborative reasoning under spatiotemporal partial observability. The architecture integrates spatial and temporal reasoning within each decision cycle via three interdependent modules: (1) the CoProposer mitigates temporal uncertainty by screening candidate actions via LLM-predicted rollouts; (2) the Negotiator overcomes spatial myopia by resolving conflicts through mean-field statistical aggregation, grounded in the CoProposer’s LLM-estimated rollout rewards; and (3) the Introspector closes the reasoning loop by analyzing environmental drift and attributing performance changes to refine strategies. Extensive evaluations on two complex long-horizon tasks, cooperative platoon planning and pandemic control, demonstrate that our framework enables robust coordination under spatiotemporal partial observability.

Abstract:
Large language model (LLM)-based agents have demonstrated strong capabilities in leveraging external tools to solve complex tasks. However, existing evaluations largely overlook the temporal dimension of tool invocation, particularly the impact of tool response latency, and are typically limited to single-task settings. In real-world applications, multiple tasks often need to be executed concurrently, and overall efficiency critically depends on whether an agent can utilize idle time while waiting for tool responses. We refer to this capability as asynchronous tool calling. To evaluate this capability, we propose AsyncTool, a benchmark for assessing LLM-based agents in interactive multi-task tool-use environments with delayed tool feedback. AsyncTool presents multiple heterogeneous tasks simultaneously and simulates realistic tool response latency during execution. Using a hybrid data evolution strategy, we construct a diverse asynchronous multitasking dataset covering multiple scenarios and tool-use patterns. We evaluate models at three levels—step, sub-task, and task—and further introduce efficiency-oriented metrics to measure task coordination and completion efficiency. Extensive experiments show that delayed tool feedback poses substantial challenges to current agents, leading to clear performance degradation. Models that better coordinate task switching and dependency tracking tend to achieve stronger performance on AsyncTool.Our analysis identifies the main failure modes of current tool agents and provides practical guidelines for designing future systems with stronger temporal reasoning and coordination capabilities.

Abstract:
Large language models (LLMs) can now solve complex problems through long chain-of-thought (CoT) reasoning, but the trade-off between performance and token cost remains a central challenge. To address this issue, supervised fine-tuning (SFT) often uses compressed reasoning data, where CoT traces are shortened into compact forms. However, the effect of such compressed reasoning data on post-training remains poorly understood. In this paper, we propose a taxonomy of CoT consisting of Explicit CoT, which outputs all operations without aggregation, Composed CoT, which combines multiple operations into a single step, and Implicit CoT, which omits intermediate operations. As a controlled mechanistic study, we construct a synthetic compositional reasoning task that allows controlled variation of difficulty, compression granularity, and data size, and conducted a comprehensive set of experiments across different model families and sizes. Notably, we find that (i) coarser CoT requires more SFT data, (ii) compared with Explicit CoT, Composed CoT and Implicit CoT benefit more from data scaling, while Composed CoT benefits from data repetition and Implicit CoT tends to lead to memorization, (iii) unlike SFT, subsequent reinforcement learning (RL) with verifiable rewards (RLVR) decomposes compressed steps learned during SFT, and (iv) unidirectional CoT ordering shows stronger generalization on longer sequential tasks. Our findings provide implications for CoT design under data resource constraints and offer important insights into the mechanisms of SFT and RL in LLM post-training.

Abstract:
We investigate how large language models (LLMs) construct meaning using Quantifier Scope Disambiguation (QSD) as a controlled probing task. We introduce (i) a balanced English corpus designed to neutralize classical surface heuristics and systematically manipulate the relationship between formal predictors and preferred readings, and (ii) a pseudosentence dataset that attenuates world-knowledge cues while preserving formal information relevant to scope. In Experiment 1, we evaluate a range of LLMs in a zero-shot question-answering setup and compare them to human baselines. While models achieve high accuracy on the balanced corpus, performance drops substantially on pseudosentences. Crucially, robust performance is retained only when classical formal predictors converge on the preferred reading; once these predictors partially diverge or fully conflict, performance approaches chance levels and becomes highly unstable across models. These findings suggest that the strong QSD performance reported for current LLMs may reflect a masking effect caused by the systematic alignment of surface-level heuristics and world knowledge in naturalistic datasets. As an exploratory extension, Experiment 2 examines whether adding externally retrieved context changes model behavior while keeping the task and prompt fixed. RAG has a negligible aggregate effect, with the largest positive change occurring in the hardest configuration, where surface cues are maximally misleading. Taken together, our results suggest that high QSD performance in LLMs remains strongly dependent on convergence between formal cues and world knowledge, despite the apparent semantic robustness suggested by naturalistic QSD results.

Abstract:
This paper introduces DocPlan, a vision-language training dataset designed to emulate the stepwise reasoning strategies employed by human during visual document understanding. DocPlan was constructed to enable vision-language models (VLMs) to effectively learn the structured reasoning process that humans typically follow when interpreting documents. We reorganized existing document understanding datasets into 133K examples annotated with the following four sequential stages: Recognition → Planning → Extraction and Reasoning → Conclusion. To evaluate the effectiveness of DocPlan, we fine-tuned the direct prediction-based VLMs to perform stepwise reasoning generation across these stages. The experimental results demonstrated that our approach achieved up to a 34.4% improvement in accuracy on document-based benchmarks.

Abstract:
Audio tokenizers are fundamental to unifying audio understanding and generation. Understanding requires high-level semantics, while generation demands semantic and acoustic details. Existing unified tokenizers jointly encode both in high-dimensional continuous latents, which increases the modeling burden of Diffusion Transformers (DiTs) for generation. We propose LoSATok, a low-dimensional audio tokenizer for cross-domain audio understanding and generation. Motivated by the observation that 1280-dimensional semantic encoder features are compressible, we introduce a Semantic Bottleneck that compresses them into 128 dimensions, regularized by the proposed time-relation loss for temporal feature consistency. We further design a dual-level semantic supervision method that leverages both high- and low-dimensional semantic signals, enabling the tokenizer to jointly capture semantics and acoustic details within a compact latent space. Experiments on speech, music, and general audio show that SemBo preserves strong low-dimensional semantic capacity and LoSATok retains competitive understanding performance compared with several semantic representations, while consistently improving DiT modeling performance on speech, music, and audio generation. These results demonstrate that LoSATok’s low-dimensional representations can effectively support audio understanding and generation. Our code is provided at https://github.com/wxzyd123/LoSATok.

Abstract:
With the advancement of large language models (LLMs) and agent technologies, social bots have shed traditional stereotypes and evolved into autonomous, highly intelligent social media agents. This paper studies whether current social bot detection research is sufficient to identify such agents and discusses potential detection directions. We construct a social media agent dataset from authentic agent accounts on Weibo and Moltbook to evaluate prior detection approaches. Experimental results across various detection algorithms reveal that existing methods suffer from out-of-distribution (OOD) failures and exhibit a cognitive blind spot regarding agent-specific features. Through multidimensional feature comparisons, we find that such agents counterintuitively do not represent a transitional form between humans and traditional bots, but instead follow a distinct pattern that explains the observed generalization failures and feature limitations. Finally, motivated by the excellent performance of LLMs, we propose Agent-Detect-Agent, an online detection agent with tool calling. Experiments demonstrate that this agent further enhances the detection capability of base LLMs and achieves consistently good performance across different account types.

Abstract:
AI-text detectors are typically evaluated on benchmarks that pair human-written text with text generated by language models. However, differences in how these two classes are constructed can introduce surface-form cues that make the benchmark easier to solve. We show that surface form alone strongly separates human from machine-generated text across three detection benchmarks, but the learned surface patterns vary substantially across benchmarks. Detector scores are also substantially predictable from surface form, although detectors retain signal beyond it, and controlled reformatting reveals partial and detector-specific causal effects. Our findings show that benchmark performance alone does not establish that a detector has identified a stable and transferable signal of machine generation.

Abstract:
Retrieval-augmented generation (RAG) is usually evaluated from one observed trace: the query, the retrieved context, and the generated answer. Such evaluations can tell us whether the answer is consistent with the retrieved context, but not whether the system would still give the same answer if that context were removed, updated, or contradicted. We introduce COUNTERMEM, an evaluation protocol for RAG systems that are expected to answer from retrieved evidence: it keeps the RAG configuration fixed, changes only the retrieved context, regenerates the answer, and records whether the answer changes in the expected direction. Across RAGBench, ExpertQA, and DelucionQA, COUNTERMEM complements observed-trace faithfulness by exposing failures it is not designed to see. Configurations that look equivalent under RAGAS faithfulness behave very differently once their evidence is removed, updated, or contradicted, and among RAGChecker-approved answers more than half silently miss coherent contradictions and over a third ignore coherent updates. The failures persist across generator and judge families, the behavioral detectors agree with blind human annotators as closely as the annotators agree with each other, and row-level intervention traces localize failures to retrieval, reranking, or generation. Artifact (protocol, prompts, detectors, scripts, audit traces): https://github.com/DeltaLabTLV/CounterMem.

Abstract:
Layer pruning aims to accelerate the inference of Large Language Models on general hardware by removing less important Transformer layers. However, naively deleting layers typically degrades model performance. To reduce this degradation, prior work approximates removed layers with replacement components such as linear layers. These approaches often require large datasets and heavy replacement modules, which increase computational and memory costs. Their time complexity also restricts them to static, consecutive-layer pruning. In this work, we introduce a novel method, Superior layer Pruning performance by Super-fast Recovery (SPSR), which uses Pearson correlation coefficients to quantify a layer’s recovery ability and applies a super-fast fine-tuning procedure with a lightweight replacement component. SPSR readily extends to non-consecutive pruning and to dynamic pruning. Experimental results show that SPSR achieves performance competitive with state-of-the-art methods while substantially improving recovery efficiency.

Abstract:
While Multimodal Large Language Models (MLLMs) have achieved remarkable success in difficult purely textual mathematical reasoning tasks, even advanced closed-source models such as GPT-o3 still struggle with geometric problems. This discrepancy motivates us to investigate the root cause: is the bottleneck of multimodal geometric reasoning rooted in reasoning itself, or in the perception of geometric information from diagrams? To answer this question, we conduct an exploratory experiment and find that providing high-quality captions consistently and substantially boosts performance of MLLMs, empirically validating the visual perception bottleneck in geometric reasoning. However, MLLMs’ capabilities in visual geometric perception remain insufficiently evaluated. To fill the gap, we propose CapGeo-Bench , a benchmark of 4,641 high-quality figure–caption pairs equipped with a fine-grained keypoint-based evaluation metric. Specifically, CapGeo-Bench encompasses three geometric classes and is devided into four difficulty levels according to the complexity of geometric primitives. Captions are scored at a fine-grained level by extracting keypoints along three dimensions-geometric elements, spatial relationships, and numerical attributes. CapGeo-Bench provides a reliable instrument and guideline for evaluating and advancing MLLMs’ geometric perception capabilities. Our code and data are publicly available at: https://github.com/YuYingLi0/CapGeo .

Abstract:
AI-generated text detection is commonly framed as a binary document-level judgment about whether a text is human-written or machine-generated. This framing breaks down for mixed-origin writing, where content origin and expression origin may differ. We cast mixed-origin detection as dimension-to-composition source attribution, inferring content origin and expression origin before composing them into four collaboration types. We propose Dimension-to-Composition Routing (D2C-Routing), which routes content-side and expression-side evidence to supervised dimension heads before a learned gated composition layer predicts the final label. On MixD2C, a reconstructed split derived from the HART mixed-origin benchmark, our disclosed D2C-Routing-based detector system reaches 0.8603 four-way Avg TPR@1%FPR, 6.5 points above the same-split RACE-local rerun. Core ablations support the routing design, while error analysis shows that distinguishing AI-content/human-expression from fully AI-generated text remains the hardest boundary. Code is available at https://github.com/bystander563/d2c-routing-artifact.

Abstract:
Real-world data analysis often requires joint reasoning over heterogeneous artifacts such as tables, charts, and textual reports. Training reliable multimodal large language model (MLLM) agents for this setting remains challenging, as large-scale aligned training data are scarce, and outcome-only rewards are too coarse to distinguish grounded cross-source analysis from spurious success. We introduce OmniDA, a training recipe that addresses these two challenges with a scalable multimodal data-synthesis pipeline and an Evidence-Aware Process Reward (EAPR) that supervises both evidence recovery and procedural validity. Through this pipeline, we construct OmniDA-11K, a high-quality training corpus for multimodal data analysis. When instantiated with Qwen3-VL-8B, OmniDA improves over the corresponding agentic baseline by 19.2 points on average across five challenging benchmarks and achieves the strongest overall performance among existing open-source data-analysis agents. These results highlight the value of supervising evidence recovery and analytical procedure beyond final-answer correctness.

Abstract:
Reinforcement learning (RL) for multi-turn deep research agents suffers from a granularity mismatch: token-level methods incur high variance over long reasoning horizons, while sequence-level methods ignore the internal decision structure of multi-turn interactions. We introduce Block-wise Geometric Policy Optimization (BGPO), which treats each assistant turn—packaging reasoning and tool invocation into one semantically coherent unit—as an optimization-level macro-action and aggregates block-averaged log-importance ratios with equal weights across blocks. We prove that block-equal weighting minimizes a covariance-robust upper bound on ratio variance under weak inter-block coupling, with the approximation gap relative to sequence-length weighting bounded by block-length imbalance. To improve exploration efficiency, we expose human-defined rewrite operators—including decomposition, keyword extraction, synonym replacement, and HyDE—as a prompt-level exploration prior during RL rollouts. On seven knowledge-intensive QA benchmarks, our 7B system achieves an average judge score of 0.715 ± 0.010 over three training seeds, compared with 0.664 for Search-R1-7B. A controlled fixed-prior attribution study further confirms that block-level optimization independently improves answer quality, achieving 0.527/0.395 F1/EM compared with 0.482/0.350 for GRPO. Our code is publicly available at https://github.com/meiotoufa/Learning-to-Think-in-Blocks.

Abstract:
Post-training large language models often require optimizing multiple objectives simultaneously, motivating multi-reward reinforcement learning (RL) methods that model different objectives as separate reward signals. Existing methods like Group reward-Decoupled Policy Optimization (GDPO) compute group-normalized advantages separately for each reward dimension and then aggregate them for policy updates. However, this aggregation process can cause cross-reward advantage conflict : the same rollout may have positive advantages on some reward dimensions but negative advantages on others, causing positive and negative signals to cancel each other during aggregation. To address this issue, we propose GD 2 PO, short for Group-Dynamic Reward-Decoupled Policy Optimization , a conflict-aware multi-reward policy optimization method. Before advantage aggregation, GD 2 PO uses conflict-aware criteria to identify and filter rollouts with severe reward-wise disagreement, reducing the influence of unreliable update signals. It further introduces query-level reweighting to dynamically adjust the update strength of each query according to its overall reward consensus. Experiments on tool calling and helpfulness-safety alignment show that GD 2 PO outperforms existing methods. Our code is available at https://github.com/Qwen-Applications/GD2PO .

Abstract:
In this paper, we introduce LLMBridge, a new LLM based system for the task of end-to-end referential bridging resolution in English. Our bridging resolution pipeline combines heuristic pre/post-processing with the natural language inference ability that comes from LLMs. We evaluate our bridging resolution pipeline on three datasets which have been used for referential bridging resolution evaluation in English: ISNotes, BASHI, and GUMBridge. Comparison to previous bridging resolution systems shows that the performance of LLMBridge surpasses previous state-of-the-art (SoTA) systems for all three datasets in the challenging End-to-end Evaluation Setting, as well as the Basic Bridging Resolution Evaluation Setting (gold bridging anaphor given). We also conduct a thorough error analysis of the LLMBridge performance, examining what varieties of bridging remain difficult for LLM based systems to identify. With this paper, we release the code for the LLMBridge pipeline.

Abstract:
Multilingual watermarking aims to make large language model (LLM) outputs traceable across languages, yet current methods still fall short. Despite claims of cross-lingual robustness, they are evaluated only on high-resource languages. We show that existing multilingual watermarking methods are not truly multilingual: they fail to remain robust under translation attacks in medium- and low-resource languages. We trace this failure to semantic clustering, which fails when the tokenizer vocabulary contains too few full-word tokens for a given language. To address this, we introduce STEAM, a detection method that uses Bayesian optimisation to search among 126 candidate languages for the back-translation that best recovers the watermark strength. It is compatible with any watermarking method, robust across different tokenizers and languages, non-invasive, and easily extendable to new languages. With average gains of +0.23 AUC and +37%p TPR@1%, STEAM provides a scalable approach toward fairer watermarking across the diversity of languages.

Abstract:
Large language model (LLM) agents increasingly depend on external memory to support long-horizon interaction. Yet many current memory systems still rely on largely fixed default rules once their pipelines are configured. They often apply the same extraction, retrieval, and evidence-organization choices across inputs. This is restrictive. Different memory needs require different choices. We propose AdaMemPIP, an adaptive memory orchestration framework for LLM agents. AdaMemPIP uses an LLM to route both memory storage and memory usage. For storage, it selects extractors for each new batch of dialogue records and writes the extracted items to matched memory libraries. For usage, it selects retrieval methods, target libraries, and organization strategies for each query, then executes the resulting pipeline. The framework is also extensible: new extractors, retrievers, stores, and organizers can be added without changing the overall control logic. This makes AdaMemPIP a flexible framework to long-term agent memory.

Abstract:
Diffusion Language Models (DLMs) enable parallel text generation by denoising multiple masked positions in a single pass, but confidence-threshold decoding often commits only the safest predictions, leaving informative uncertainty underused. We propose Plan-Verify-Fill (PVF), a training-free draft-and-verification decoding strategy that preserves the high-confidence decoding backbone while selectively testing lower-confidence candidates with high expected verified utility. PVF uses a calibrated planning-token vocabulary to allocate its limited verification budget, accepts proposed drafts only when they remain consistent with the backbone trajectory, and otherwise defaults to conservative confidence-gated unmasking. Across mathematical reasoning, knowledge, and code-generation benchmarks on LLaDA-8B-Instruct and Dream-7B-Instruct, PVF maintains comparable accuracy while delivering up to 2.3× higher throughput than baseline methods.

Abstract:
Large language models (LLMs) increasingly handle in-context learning (ICL) tasks where a long, novel context defines the rules, knowledge, and output schema for a series of questions. On benchmarks that grade against every detail of the context, even strong open-weights models pass only 12-16% of tasks: a single overlooked rule fails the whole response. We argue this brittleness is structural—the dominant "read-and-reason" paradigm asks the model to extract, plan, generate, and self-verify in one forward pass—and ask whether explicit context compilation can fix it, how it compares to existing long-context strategies (gist retrieval, multi-agent self-play), and where the resulting harness benefit holds across task structure and model scale. We propose the Context Compilation Architecture (CCA): compile the given context into a structured IR of rules, schemas, and exact terms; generate executable verifiers from the IR; run a checked reasoning loop that corrects any violation those verifiers flag. On CL-bench (1,899 tasks across 4 open base models), CCA outperforms vanilla prompting and two long-context baselines (ReadAgent-P, Ctx2Skill) on every base model, lifting Kimi K2.5 from 15.4% to 21.4% with gains concentrated on rule-dense sub-categories.

Abstract:
Tangible control interfaces (TCIs), such as appliance panels, remotes, elevators, and embedded GUIs, are a fundamental component of everyday human-built environments. Interacting with these interfaces requires agents not only to ground language in visual observations, but also to execute actions, track temporally evolving state changes, and verify whether intended outcomes have been achieved. However, existing benchmarks predominantly evaluate open-loop perception or single-step action execution, failing to capture this continuous cycle of interaction, feedback, and correction. We introduce SWITCH, a benchmark for closed-loop interactive reasoning with TCIs inrealistic egocentric environments1. SWITCH comprises 1,170 temporally interactive videos across diverse functional categories, providing structured annotations of instructions, actions,state transitions, outcomes, and recovery behaviors over time. To probe generative world modeling, SWITCH also evaluates video generation models on interaction-centered tasks using both LLM-as-judge and human evaluation.Experiments with frontier proprietary and opensource multimodal models reveal persistent weaknesses in fine-grained visual-temporal perception, outcome verification, and error recovery, highlighting SWITCH as a testbed for closed-loop embodied intelligence

Abstract:
Automatic detection of signs of depression has received increasing attention in recent years, evolving from user-level classification (binary) to fine-grained symptom detection. Despite recent advances, state-of-the-art sentence retrieval models still struggle to discriminate between sentences associated with closely related symptoms. In this work, we propose BDI-batch, a novel contrastive learning method that leverages clinically grounded sentence pairs to encode fine-grained distinctions among depression-related symptoms. The method exploits a standardized clinical questionnaire and does not rely on annotated data. We demonstrate the effectiveness of our approach on two early risk detection datasets and under multiple retrieval strategies. Additionally, we explore the use of LLMs to generate synthetic contrastive samples and observe that their performance falls short of questionnaire-derived pairs. Finally, we conduct an error analysis that reveals the most common errors of the sentence retrievers.

Abstract:
First-order test-time optimization (TTO) updates continuous variables such as logits or latent states for a single input. Successful methods usually rely on external reward models or verifiers to supply a semantic gradient. We ask whether reward-free signals from model distributions or prompted self-verification can supply comparable update directions for reasoning tasks. In logit-space TTO, we compare each reward-free gradient with an operational reward-model reference direction, instantiated as the reward-model gradient on the same logits. Across Qwen2.5-7B-Instruct, Qwen3-14B, and Llama-3.1-8B-Instruct, confidence gradients remain nearly orthogonal to the reference, entropy-slope gradients have vanishing scale near greedy rollouts, and terminal answer certainty is sparse and reference-sensitive. Prompted-verifier gradients are more reference-dependent: in a tokenizer-matched Llama diagnostic, four variants attain moderate RM-relative alignment and measurable local RM-score gains. Nevertheless, in end-to-end TTO on Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct, the selected reward-free signals produce no wrong-to-correct flips on MATH-500, AIME-2024, AIME-2025, or GSM8K, while a separate reward-model-guided TTO reproduction provides an external positive control with measurable gains.

Abstract:
While multimodal large language models (MLLMs) extend model capabilities beyond text, they also make safety alignment increasingly challenging. Multimodal safety alignment methods must address cross-modal jailbreaks, safety-awareness failures, and over-sensitive refusals. However, existing methods often rely on retraining or internal-state inspection, limiting their applicability to deployed closed-source MLLMs and motivating test-time safety alignment. We analyze this setting and identify two key obstacles, utility dominance and reasoning inertia, which cause models to overlook latent risks or follow malicious reasoning trajectories. Guided by these insights, we propose ReFrame, a training-free multimodal input reframing framework where two agents share a lightweight locally deployed MLLM: the evidence-generation agent constructs complementary risk and utility evidence, and the rewrite-and-routing agent converts it into a safe proxy prompt and image-routing decision before calling the downstream MLLM, without modifying it or accessing its internal information. Experiments across multiple MLLMs and benchmarks show that ReFrame improves jailbreak defense, safety awareness, and oversensitivity reduction while preserving multimodal utility.

Abstract:
Quality-aware synthetic-data selection assumes that examples rated highly by an LLM judge will help downstream learning. We test this assumption in a controlled replay across two low-resource African-language classification tasks, four languages, and seven selectors. Six selectors retain 64 examples per task-language cell. CoSDA-V2 yields the highest judged label correctness (0.904 versus 0.767 for naive) and the lowest measured shortcut score, but AlpaGasus leads CoSDA-V2 on downstream Macro-F1 in both the main replay (0.202 versus 0.163) and the three-seed refit (0.193 versus 0.157). Adding the audit’s strict gate to AlpaGasus reduces Macro-F1 by 0.029, driven by one cell where only 31 of 64 candidates survive. Because the gate changes pool composition and training-set size, this comparison measures their joint effect. LESS also falls below naive when unconstrained top-k collapses label balance, but recovers with quotas. At this budget, retention constraints account for more of the observed differences than a consistent ranking advantage. In this replay, audit scores do not reliably predict downstream utility. We therefore recommend reporting both metrics for the same retained sets.

Abstract:
Rubrics aim to make language-model evaluation transparent by decomposing response quality into interpretable criteria. However, natural-language rubrics are often ambiguous, require LLM judges, and typically assume criteria aggregated through linear weighted sums, limiting their ability to capture dependencies, alternatives, penalties, and override conditions. We propose ExecRubrics, a framework for representing rubrics as compact executable programs. ExecRubrics encodes evaluation logic as verifiable Python scoring functions, giving natural-language rubric intent an operational semantics: a fixed decision procedure that can be inspected, executed, and edited. On three long-form response benchmarks – HealthBench, HelpSteer, and ArgQuality – we show that ExecRubrics can recover substantial preference signal without an LLM judge at evaluation time. On ArgQuality and HelpSteer, the strongest executable variants are within 1.1 and 4 percentage points, respectively, of the direct GPT-5.5 agentic baseline. Executable rubrics are also considerably faster, achieving a 192× average speedup. We show that incorporating external logic and resources from text processing libraries such as NLTK and spaCy can further improve preference accuracy. Our results suggest a novel way of approaching automated evaluation, by offering a faster, more explainable, and less ambiguous alternative to black-box rubric evals, particularly in high-stakes domains such as healthcare and banking where precision and auditability are critical.

Abstract:
Large language models often exhibit sycophancy, revising their answers to align with users when users push back. Such answer flips, however, can arise from different causes. One possibility is that the model simply aligns with the user’s feedback in order to satisfy them. Another is that the feedback genuinely contains useful evidence, prompting the model to update its answer in a rational way. We distinguish them as Unsupported-Yielding and Rational-Updating. Prior work focuses primarily on suppressing Unsupported-Yielding, while overlooking its effect on Rational-Updating. We address this gap with a two-turn evaluation framework that measures the two behaviors separately. Across representative training-time and inference-time interventions, we find that anti-sycophancy methods often encounter a trade-off in which reducing Unsupported-Yielding can sacrifice Rational-Updating, and vice versa, even when the two objectives are optimized jointly. Mechanistic analysis suggests that the two behaviors share an internal substrate: the MLP neurons and attention heads driving them overlap substantially, and their associated steering directions are positively aligned. We further conduct a preliminary orthogonalized steering exploration, which yields modest, backbone-dependent selectivity gains. Overall, our results suggest that anti-sycophancy should be treated not as a simple suppression problem, but as a selectivity problem, where effective interventions should preserve Rational-Updating while reducing Unsupported-Yielding.

Abstract:
Image Difference Captioning (IDC) has produced many task-specific models, but their gains are often entangled with isolated implementations, data processing, training protocols, and evaluation settings. As a result, the field accumulates method-level scores more easily than reusable structural evidence. This paper proposes OpenIDC, a flexible and extensible codebase that reconstructs a unified evidence chain for lightweight task-specific IDC. OpenIDC standardizes data pipelines, model interfaces, training strategies, and evaluation protocols, enabling 8 representative method families with 9 executable variants across 3 benchmarks to be reproduced, compared, decomposed, and diagnosed under consistent conditions. Beyond reproduction, OpenIDC advances IDC from final-score comparison to normalized design-space diagnosis. Based on this diagnosis, we introduce SIDFormer, a compact Spatially-guided Invariant-Difference Transformer that serves as a structural answer to lightweight IDC by organizing spatial anchoring, pseudo-change suppression, and reliable difference evidence through functional fusion. Experiments show improved reproducibility and comparability, together with competitive SIDFormer performance across the evaluated benchmarks. Code, reproduced methods, and models will be released.

Abstract:
Social interaction increasingly takes place in multicultural settings, where individuals may draw on multiple cultural influences and adapt their communicative behavior across contexts. Despite recent advances in equipping Large Language Models (LLMs) with social understanding capabilities, existing approaches often model culture as a static demographic attribute, limiting their ability to accommodate hybrid and dynamically expressed communicative preferences. Therefore, in this paper, we propose DyCAC, a training-free framework that achieves fluid social alignment by incorporating Dynamic Cultural Adaptation with continuous Cognitive tracking. Rather than inferring a fixed cultural identity, DyCAC models culturally relevant communicative preferences as a time-varying mixture of population-level cultural reference profiles. This reference-based representation is further calibrated using dialogue-style signals observed in the ongoing interaction, enabling the model to capture both composite cultural influences and turn-level shifts in communicative behavior. In parallel, a memory module driven by Theory of Mind (ToM) continuously tracks the cognitive states of the interlocutor. Extensive experiments on interactive social and cultural benchmarks demonstrate the superiority of our approach. The proposed framework outperforms existing baselines, exhibiting enhanced social intelligence and broad adaptability across varied multicultural contexts.

Abstract:
Mixture-of-Experts (MoE) models route each token to a subset of expert networks, increasing capacity while keeping per-token computation sparse. Many deployed MoEs use a fixed number of active experts across layers and tasks, despite evidence that layer roles, expert redundancy, and required capacity vary with depth and input difficulty. Existing layer-wise methods usually determine allocations offline and reuse them across tasks, while token-level methods rely on local routing signals without task-level context. We propose MetaNet, a support-set controller that predicts an expert-retention threshold and a bounded routing bias for each layer while keeping the backbone, experts, and original router frozen. On DeepSeek-MoE-16B-Chat, MetaNet provides a tunable accuracy–expert-activation trade-off. Relative to fixed k=6, a conservative setting activates 3.61 experts on average (40% fewer) with comparable MMLU accuracy (0.489 versus 0.474); an aggressive setting activates 2.28 experts on average (62% fewer) at 0.438 accuracy. Trained on MMLU, MetaNet transfers to C-Eval without retraining, activating 2.90 experts on average (52% fewer than fixed k=6) at 0.386 accuracy.

Abstract:
Through systematic experiments on long-context generation, we observe a damaging failure mode in which decoding can collapse into persistent repetition loops. We find that this degeneration is driven by collapsed attention patterns, where a subset of heads locks onto a narrow suffix of the history, and is further stabilized by inference-time KV cache reuse. Crucially, since many existing KV cache policies rely on attention-based importance, this collapse can produce spuriously high scores for repetitive tokens, causing cache management to inadvertently amplify repetition. To study this phenomenon in a controlled and reproducible manner, we introduce LoopBench, a benchmark with explicit loop-inducing conditions and loop-oriented metrics that quantify repetition severity and generation instability beyond downstream task scores. Building on these insights, we propose LoopGuard, a lightweight, plug-in KV cache guard that detects loop onset online and disrupts the feedback cycle by pruning repetitive tail spans under a fixed cache budget. Experiments on LoopBench show that LoopGuard reduces loop incidence by over 90 percentage points, while restoring output diversity and reducing token waste.

Abstract:
Large vision-language models (LVLMs) have made rapid advancements; however, existing benchmarks for evaluating their instruction-following abilities focus on purely textual constraints, e.g., format constraints and role-playing, which assess only language-level adherence and are not related to vision. To bridge this gap, we introduce VC-IF, a benchmark that involves visual constraints connecting instructions to explicit operations over image content, thereby directly probing visual spatial understanding and compositional compliance. VC-IF defines atomic constraints, systematically combining mixed settings, scaling from single- to multi-constraint scenarios, and single- and multi-turn scenarios. We evaluate 16 popular LVLMs and find that models performing well under text-only constraints may struggle when faced with visual constraints, highlighting significant gaps in their capabilities. Further, we conduct scaling analyses across varying numbers of constraints and dialogue turns, revealing performance trends as constraint complexity increases along different dimensions. Finally, we synthesize a large training dataset and propose VC-PO, a reinforcement learning strategy that uses a hybrid verifier mechanism to compute reward signals for multiple constraints. Applying VC-PO to Qwen3-VL-8B-Instruct improves its overall VC-IF score from 43.74 to 55.18 (+11.44). The VC-PO-trained model remains stable as the number of turns and constraints increases. Our data and code are available at VC-IF .

Abstract:
Speech relation extraction (SpeechRE) aims to extract structured relational triples directly from speech. However, existing studies still face three major challenges: the lack of Chinese SpeechRE benchmarks, insufficient modeling of speech-specific cues, and evaluation bias introduced by character-level exact matching in Chinese scenarios. To address these challenges, we first construct CSRTE, a real-world Chinese SpeechRE benchmark built from human speech, providing unified data and evaluation settings. We then propose ACRE, a reasoning enhancement framework for SpeechRE, which integrates acoustic-cue and contextual-knowledge into a structured acoustic-semantic reasoning chain, enabling the model to go beyond literal semantic content (what-to-say) and capture speech-specific features (how-to-say) for relation reasoning. We further combine supervised fine-tuning with reinforcement learning and design composite rewards to improve the quality of the model’s reasoning process. In addition, we propose pronunciation-aware evaluation (PAE), which considers pronunciation and tone consistency during entity matching to reduce the evaluation bias. Experiments on both Chinese and English SpeechRE benchmarks demonstrate the effectiveness of ACRE and the usefulness of PAE. Our source code and datasets are available at https://github.com/MWXGOD/CSRTE.

Abstract:
Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence scaffold and replays it before final generation while preserving the full original context. This recursive sifting process separates evidence organization from answer generation without training, external memory, or context pruning. We also provide a theoretical analysis based on associative memory, which characterizes the context as a memory store, the question as a retrieval cue, attention as cue-trace association, and replay as trace reactivation. Experiments on eight long-context datasets with 128K context length show that RECONTEXT consistently improves evidence utilization across Qwen3-4B, Qwen3-8B, and Llama3-8B, achieving the best average rank on all three backbones. Code is available in the supplementary software.

Abstract:
Financial text frequently contains cognitive bias triggers, such as an authority cue like “according to Goldman Sachs”, that can distort sentiment judgments without altering the underlying facts. While prior work has established that LLMs are susceptible to cognitive biases, existing evaluations treat bias exclusively as a property of the model, overlooking triggers already present in the input text. We ask whether LLMs can (i) detect such triggers, (ii) rewrite text to remove them, and (iii) whether doing so improves sentiment predictions, a task central to automated financial decision-making. We introduce TriggerBank, a dataset of 3,006 financial sentences with six types of synthetically injected bias triggers, and evaluate 11 LLMs across tasks (i)–(iii). Detection remains challenging, with framing and overconfidence hardest to detect, and removal fails for anchoring. Despite this, rewriting outperforms all instruction-based defenses for most models in downstream sentiment classification, while chain-of-thought prompting generally degrades performance. Debiasing the input text via rewriting is thus a more robust strategy than prompt-based approaches.

Abstract:
Biomedical knowledge graphs (KGs) offer structured medical knowledge that can ground large language model (LLM) reasoning in clinical diagnosis application, yet how KG signal should be integrated into LLMs remains an open question. We present a systematic study spanning five KG task formulations, three training paradigms, two KGs, and three base LLMs. At the task level, all paradigms improve over the non-finetuned baseline, but methods with comparable in-domain accuracy show substantially different knowledge transfer behavior. We introduce Gradient Intervention Density (GID) to measure how broadly an optimizer modifies the pretrained model. GID reveals a clear divide: KG-judgment training under KL regularization produces sparse, localized updates (a regime we term as surgical alignment ), while task-specific SFT produces dense ones. A controlled ablation shows that the objective and KL contribute to sparsity independently, and the paradigms that produce sparse updates also improve reasoning quality, even when their in-domain accuracy is lower than task-specific SFT. Assessing KG-LLM integration thus requires complementing accuracy with optimization-geometry diagnostics. Our implementation can be found at https://github.com/LARK-NLP-Lab/Surgical-Alignment.

Abstract:
Retrieval-Augmented Generation (RAG) enhances large language models by grounding outputs in external knowledge, improving factuality and reducing hallucinations. At the same time, the retrieval-augmented pipeline introduces new robustness and security risks, including corpus poisoning, backdoor attacks, privacy leakage, and fairness violations. Despite rapid progress in this area, existing surveys remain limited in their treatment of attacker objectives, threat models, and stage-specific defenses across the full RAG pipeline. This survey presents a unified and pipeline-aware overview of RAG robustness. We formalize threat models over the corpus, retriever, and generator, and organize attacks into three main objectives: accuracy, privacy, and fairness. We further review defenses from a pipeline-aware perspective, covering the retrieval, rerank, generation, and traceback stages. In addition, we summarize robustness benchmarks and explainability methods for more deeply evaluating and explaining RAG robustness. Finally, we highlight open challenges and future directions for building trustworthy RAG systems. To continuously track developments in this rapidly evolving field, we will also actively update a public repository at https://github.com/coutMinh/A-Survey-on-RAG-Robustness.

Abstract:
LLM-powered coding agents increasingly participate in software development workflows by generating code, selecting dependencies, and producing package installation commands. This creates a new software supply chain risk: when an agent hallucinates a non-existent package, an attacker may register the hallucinated name and later compromise users who install it. Existing package hallucination attacks and defenses primarily focus on naturally occurring hallucinations, targeted dependency steering, or post-hoc package validation. In this paper, we introduce Neutral Prompting Attack (NPA), a stealthy attack paradigm in which semantically benign instructions, such as encouraging imagination and exhaustiveness, increase package hallucination propensity without containing explicit malicious intent. Unlike targeted dependency steering, NPA does not specify an attacker-chosen package. Instead, it shifts the model’s dependency generation behavior toward more speculative package names. We evaluate NPA across multiple coding-oriented LLMs and package hallucination benchmarks. Our results show that NPA increases both Hallucination ASR and Pip Install ASR, changes the distribution of hallucinated package names, and evades existing static-analysis, LLM-based, and agent-based Skill defenses. These findings reveal that harmless-looking prompts can covertly manipulate hallucination behavior and create downstream software supply chain risks.

Abstract:
Large Language Models (LLMs) are sensitive to the order in which answer choices are presented in Multiple-Choice Question Answering. The same question can yield a different prediction when its options are shuffled. Although exhaustive majority voting over all permutations can mitigate this bias, its factorial cost makes it impractical at inference time. We propose GPS (Guided Permutation Scoring), a two-stage inference-time pipeline that selects option orderings more likely to produce correct predictions from a frozen LLM. A sensitivity classifier first identifies which questions are genuinely vulnerable to positional interference; robust questions are answered directly, while sensitive ones are forwarded to a trained permutation scorer that identifies a more position-stable option ordering, promoting better predictions. Unlike prior approaches that either enumerate all permutations uniformly, or require modifying the LLM’s parameters, GPS is the first method to actively select permutations so that the base model can consistently give better answers. GPS steadily improves Accuracy, and Macro F1 across all evaluated model-dataset combinations, with remarkable reductions in positional bias metrics.

Abstract:
Recently, researchers have explored leveraging the commonsense knowledge and reasoning capabilities of large language models to accelerate human mobility simulation. However, these methods suffer from several critical shortcomings, including inadequate modeling of urban spaces, poor integration with individual mobility patterns, and weak alignment with collective mobility distributions. To address these limitations, we propose Geospatial Knowledge Enhanced Agentic framework for Mobility Simulation (GAMS), an agentic framework that leverages the language based geospatial foundation model to simulate human mobility in urban space. GAMS comprises three core modules: MobExtractor, which extracts template-based mobility profiles and synthesizes new ones during simulation; GeoGenerator, which generates trajectory points considering collective mobility knowledge and urban geospatial knowledge; and TrajEnhancer, which captures individual mobility regularities from real-world trajectories and refines to generate style-aligned synthetic trajectories. Experiments on two real-world datasets show that GAMS achieves superior performance over various state-of-the-art methods, with more than a 17% improvement by capturing diverse mobility regularities. Our codes and datasets are open-sourced via https://anonymous.4open.science/r/CAMS-1992/.

Abstract:
Patient recruitment remains a major barrier in clinical trials, as identifying eligible participants often requires manual screening of complex health records, contributing significantly to trial delays and increased costs. While Large Language Models (LLMs) offer potential to automate this process, existing public datasets are overly simplistic in their patient representation: they pair full criterion lists with single-paragraph synthetic summaries, failing to capture the longitudinal complexity of real clinical records. We introduce MIMIC-CTRL-2026, a dataset of 800 manually labeled patient-criterion pairs grounded in real-world longitudinal EHR data from MIMIC-IV, annotated by clinically trained experts. Each instance pairs one eligibility criterion with a full patient record spanning multiple discharge notes, radiology reports, lab values, and medication histories. While simple prompting achieves high performance (κ = 0.84) on existing benchmarks, it collapses (κ = 0.06) on our dataset, revealing a substantial benchmark-difficulty gap between simplified and real-world clinical evaluation. Lastly, we present a framework combining structured pre-processing with a clinician-validated expanded prompt that substantially outperforms naive baselines without any model fine-tuning, establishing a realistic benchmark for automated clinical trial screening on real EHR data. Our dataset and code are available at: https://doi.org/10.5281/zenodo.22103011

Abstract:
Diffusion large language models (dLLMs) generate text through iterative denoising, but they remain prone to factual errors in knowledge-intensive tasks. Through layer-wise and timestep-wise probing, we observe a spatiotemporal stratification pattern in dLLMs: structural-token predictions tend to converge earlier across layers, whereas factual-token predictions rely more heavily on deep layers and become reliable only after sufficient denoising progress. Motivated by this observation, we propose Time-Aware Internal Contrastive Decoding (TICD), a training-free decoding method that calibrates final-layer predictions using entropy-gated intermediate-layer references only during semantically reliable denoising stages. This design uses reliable final-intermediate prediction gaps as correction signals to strengthen factual predictions at the final layer. Experiments on LLaDA-8B and Dream-7B across factuality and reasoning benchmarks show consistent improvements over vanilla decoding.

Abstract:
Large Language Model (LLM)-based Multi-Agent Systems (MASs) are vulnerable to stochastic reasoning and coordination failures, yielding verbose, unstructured trajectories that are difficult to debug. Existing debuggers typically rely on either post-hoc static analysis, which suffers from ambiguous causal attribution, or dynamic verification, which incurs prohibitive computational costs and execution instability. To bridge this gap, we propose SynDebug, a static–dynamic synergistic debugging framework. Statically, SynDebug reconstructs execution topologies from flat logs via a Hierarchical Causal Graph and prioritizes root-cause candidates using synthesized Virtual Oracles.Dynamically, it performs targeted interventions on prioritized candidates to progressively recalibrate suspiciousness rankings based on execution impact. Evaluated across three MAS frameworks and four benchmarks, SynDebug outperforms six baselines, improving the repair success rate by 5.59%–81.81% while maintaining competitive efficiency.

Abstract:
Retrieval-Augmented Generation (RAG) systems for question answering typically retrieve evidence by semantic similarity between the query and document chunks. While effective for unstructured text, this approach is less reliable on semi-structured corpora where answering may require exact filtering, aggregation, or exhaustive retrieval over structured attributes across multiple documents. Symbolic approaches support such operations, but they are often brittle on noisy natural-language corpora. We address this gap with DualGraph, a RAG framework that represents documents through two complementary views: a Textual Knowledge Graph for semantic retrieval and a Symbolic Knowledge Graph for symbolic querying over typed subject–predicate–object triples. Building on these two components, we provide multiple strategies for selecting or combining semantic and symbolic evidence. We also introduce SpecsQA, a benchmark from a commercial shopping website with semi-structured product documents and manually curated questions spanning open-ended and specification-oriented retrieval. Experiments show that DualGraph consistently outperforms state-of-the-art dense-retrieval, GraphRAG, symbolic, and table-oriented baselines across question types. Code and data are available at https://github.com/corneliocristina/DualGraphRAG.

Abstract:
We introduce CodeEvolve, an open-source framework that couples large language models with island-based evolutionary search for end-to-end algorithmic discovery. CodeEvolve integrates inspiration-based crossover, meta-prompting, and depth-based refinement on top of a CVT-MAP-Elites archive and a weighted LLM ensemble to generate optimized solutions for complex problems. On the AlphaEvolve benchmark suite, CodeEvolve matches or surpasses the reported AlphaEvolve results on 5 of 9 problems and, under matched conditions, outperforms the open-source frameworks OpenEvolve and ShinkaEvolve on 6 of 9. With the open-weight Qwen3-Coder-30B backbone, it surpasses the reported AlphaEvolve score on both CirclePackingSquare instances at roughly an order of magnitude lower cost than a frontier closed-source ensemble, and remains competitive with EoH on heuristic-design tasks without retuning. Ablations show that the interaction between CodeEvolve’s components, rather than any single operator, drives these results. We release the framework, experimental data, and practical hyperparameter guidelines at https://github.com/inter-co/science-codeevolve.

Abstract:
Evaluating whether a large language model (LLM) genuinely knows a concept remains a fundamental problem, since behavioral metrics can be fooled by suppression: a model that refuses to answer may still harbor recoverable knowledge. We introduce Activation Fragmentation, a framework that diagnoses parametric knowledge directly from internal geometry, with no token generation. The framework measures the geometric coherence of anon-vectors , the differences in hidden states between a concept-specific query and its anonymized paraphrase, and shows that genuine knowledge forms a tight, low-entropy cluster while ignorance forces geometric fragmentation. Across ten instruction-tuned LLMs from three model families, all fragmentation metrics reliably separate known from unknown concepts, and post-unlearning, fragmentation geometry correlates with adversarial recovery across four attack types and five unlearning algorithms, though the direction of that relationship is shown to depend on attack methodology. Furthermore, a layer-resolved analysis reveals that different adversarial mechanisms exploit this geometry at varying depths, with high attack-specific correlations that are invisible to behavioral evaluation. Together, these results constitute a single-phase, attack-free proxy for unlearning robustness that captures aspects of parametric knowledge inaccessible to output-level metrics.

Abstract:
Hybrid reasoning models can answer directly or spend extra tokens on extended thinking. A practical router should choose between these modes for each query, so easy problems avoid unnecessary reasoning and hard problems receive enough budget to finish the answer. Existing routers move in this direction, but they typically require labeled training data or fix thinking budgets up front, ignoring answer-level evidence from the model itself. We introduce DART, a training-free routing framework that samples two cheap no-think drafts, accepts direct answering when the drafts agree, and predicts a thinking budget from draft entropy when they disagree. Across the main comparisons, DART preserves or improves always-thinking accuracy in most settings while reducing thinking-token use. On math reasoning, accuracy improves by up to +9.0 points on Olympiad-level problems while thinking tokens drop 15-69%. On code reasoning under execution-based equivalence, accuracy improves by up to +22.5 points while thinking tokens drop 51-63%. The Stage 1 signal extends across model scales (0.6B-32B), model families, and API-only hosted settings, with no labeled data and no gradient updates required.

Abstract:
Voice agents face a fundamental tension: the reasoning, retrieval, and tool use that make foundation models capable are iterative and slow, while conversational interaction demands prompt responses. Smaller, real-time models meet the latency bar but do not match foundation models on complex tasks, leaving current voice agents to trade off responsiveness and capability. We introduce conversational infill, where a small Talker model both fluently integrates streamed Reasoner model knowledge into its responses during inference and immediately generates contextually grounded responses to hide external Reasoner latency. We curate a 290,571-example dataset and demonstrate that this task is learnable across seven small language models (135M to 1.7B parameters). Our system, ConvFill, runs on an Apple M2 with millisecond-level latency, while remaining within 6.3% of corresponding frontier Reasoner performance. Conversational infill unlocks a new point on the latency–capability Pareto frontier, offering a practical path toward responsive and capable voice agents.

Abstract:
Open-weight large language models create demand for weight protection schemes that let users run models locally while preserving developer control over valuable model weights. Existing schemes, however, are often justified by heuristic arguments and empirical checks rather than explicit adversarial goals, security definitions, or proofs. As a case study, we analyze a recent scheme previously published in EMNLP 2024 Main Conference and show that it is vulnerable to our efficient weight-extraction attack, recovering more than 99.95% of protected parameters and producing reconstructed models with near-identical performance. Motivated by this failure, we formulate LLM weight protection schemes through game-based definitions, and we discuss how these notions relate. We encourage that future weight protection schemes should be designed with formal security goals, proof-oriented analysis, and offensive evaluation rather than relying on informal intuition alone.

Abstract:
Massive activation spikes in Large Language Models (LLMs) severely degrade quantization by increasing dynamic range requirements. While prior hypotheses characterize these as high-level scalar biases, we argue that they are merely the scalar intermediates of rigid, structural vector biases in the spike-carrying tokens. We show that these tokens converge to constant vectors after normalization that drive the attention sink and value-state drain mechanisms. We geometrically substantiate this by analyzing the coordination of projection weights: W K contrastively amplifies the vector, W Q aligns semantic tokens toward it, and W V projects it into the spectral null-space. Furthermore, we reveal that these structural biases are preserved against Rotary Positional Embedding (RoPE) perturbations by localizing them in “zones of rotational stability” utilizing low-frequency bands and coherent channel pairs. Leveraging this, we propose InsertQuant, a post-training quantization (PTQ) framework that clamps spikes and restores their function via pre-computed template vectors. This renders activations strictly spike-free, enabling robust low-bit quantization with high fidelity. InsertQuant achieves competitive performance compared with prior per-tensor quantization methods on LLMs and also generalizes beyond text to other modalities such as ViTs.

Abstract:
Gene expression analysis holds the key to many biomedical discoveries, yet extracting insights from raw transcriptomic data remains challenging due to the complexity of multiple large, semi-structured files and the need for extensive domain expertise. Current approaches either rely on inflexible workflows that fail in edge cases or fully autonomous agents that lack the precision required for rigorous scientific inquiry. GenoMAS addresses this by presenting a team of LLM-based agents that integrates structured workflows with autonomous adaptability, orchestrating diverse specialized agents through typed message-passing protocols. At its core lies a guided-planning framework where programming agents decompose task guidelines into Action Units and dynamically choose to advance, revise, bypass, or backtrack, maintaining logical coherence while adapting to the idiosyncrasies of genomic data. On the GenoTEX benchmark, GenoMAS achieves 89.13% CSC for preprocessing and 60.48% F₁ for identifying trait-associated genes, surpassing prior art by 10.61% and 16.85%. It again outperforms strong agent baselines on ScienceAgentBench and DiscoveryBench, two independent bioinformatics benchmarks beyond transcriptomics. Its analyses yield gene-phenotype associations that are biologically plausible and corroborated by the literature. Our code is available at https://github.com/Liu-Hy/GenoMAS.

Abstract:
As LLMs advance, popular commonsense and NLU benchmarks have become saturated, yet multilingual evaluation still lags. Existing benchmarks cover only a small set of languages, and translation based multilingual benchmarks can introduce bias. We introduce HellaSwagUltra, a multilingual commonsense and NLU benchmark with substantial local cultural knowledge. It is built with an automated, continuously extensible pipeline. Instead of explicitly prompting for reasoning, each example implicitly embeds two commonsense or local knowledge facts in the context, and answer options contain subtle cues about whether those facts are violated, requiring models to pick the most plausible continuation. All test items are fully reviewed and corrected by language experts, with ongoing updates. Even the strongest proprietary model (e.g., Gemini-2.5-Pro) reaches only 62.5% accuracy, while GPT-4o and leading open-source models hover around 40%–50%, demonstrating that multilingual commonsense reasoning remains an open challenge.

Abstract:
Bengali is the seventh-most-spoken language globally, yet LLM safety evaluation remains overwhelmingly English-centric. We introduce BanglaSafe, a benchmark of 879 Bengali prompts combining 309 natively authored prompts with 570 expert-reviewed prompts, spanning 17 culturally grounded harm categories and five prompting conditions that vary language, writing style, and authority framing. Evaluating 18 frontier LLMs, we find that over half of all responses are unsafe or partially unsafe (53.6%) while 14.7% contains strictly harmful content, and that the strongest observed effect is not the switch from English to Bengali but the choice of writing style within Bengali: the same harmful request phrased as a formal newspaper investigation succeeds 17 percentage points more often than the same request phrased as a casual message, with no adversarial engineering involved. We further show that existing safety classifiers struggle to reliably evaluate Bengali content, with even frontier models failing on nearly half of all cases. We publicly release the benchmark, a calibrated judge, and the evaluation framework.

Abstract:
Retrieval-Augmented Generation (RAG) has substantially advanced knowledge-intensive question answering (QA). Existing methods typically follow a ’decompose and sequentially solve’ paradigm, mainly optimizing sub-question generation, retrieval, or search decisions, while lacking explicit maintenance of cross-step intermediate reasoning states. As a result, retrieval noise, query drift, and local reasoning errors introduced in early steps can be inherited and amplified along the reasoning chain, eventually causing reasoning failure. We propose CloRe, a state-based closed-loop reasoning framework for multi-hop question answering that explicitly maintains hop-level intermediate states and coordinates forward review with dependency-aware backward correction. In the forward phase, CloRe uses conditional query rewriting to mitigate query drift, stage-wise evidence acquisition to mitigate retrieval noise, and step-level review with local repair to filter unreliable intermediate answers before they are propagated. When forward correction is insufficient, CloRe enters a backward phase that uses downstream failure signals to backtrack along explicit dependencies and revise predecessor states that were locally acceptable under earlier review but become incompatible with later dependency constraints. Experiments on three representative multi-hop QA benchmarks show that CloRe achieves competitive performance against strong baselines, demonstrating the effectiveness of explicit intermediate-state maintenance for multi-hop reasoning.

Abstract:
Matched molecular pairs (MMPs) capture the local chemical edits that medicinal chemists routinely use to design analogs, but existing ML approaches either operate at the whole-molecule level with limited edit controllability or learn MMP-style edits from restricted settings and small models. We propose a variable-to-variable formulation of analog generation and train a foundation model on large-scale MMP transformations (MMPTs) to generate diverse variables conditioned on an input variable. To enable practical control, we develop prompting mechanisms that let the users specify preferred transformation patterns during generation. We further introduce MMPT-RAG, a retrieval-augmented framework that uses external reference analogs as contextual guidance to steer generation and generalize from project-specific series. Experiments on general chemical corpora and patent-specific datasets demonstrate improved diversity, novelty, and controllability, and show that our method recovers realistic analog structures in practical discovery scenarios.

Abstract:
In financial markets, effective trading decisions require accurately interpreting unstructured information, especially news articles. However, prior work overlooks the complex news and agent interactions when making trading decisions. To address these limitations, we propose FinRAQ, which enhances the news importance scoring by modeling semantic and temporal interactions between news articles. FinRAQ incorporates a Socratic questioning stage into the debate framework, prompting agents to challenge each other’s reasoning through Socratic questions. Extensive experiments across six company stocks show that FinRAQ outperforms state-of-the-art baselines, achieving up to 38% improvement in annualized returns.

Abstract:
Fake news detection increasingly relies on LLMs to extract semantic and pragmatic signals. We study a narrower problem than factual hallucination: when an LLM scores multiple intent dimensions, the resulting vector may contain internally inconsistent labels. We present IGTF, an intent-guided, game-inspired structured intent calibration framework for LLM-assisted fake news detection. IGTF defines an expert-validated nine-dimensional intent taxonomy, extracts intent-aware text representations with a Mixture-of-Experts encoder, and calibrates LLM-produced intent vectors through a fixed graph of cooperative and conflicting intent relations. The refinement is motivated by a potential-game formulation, whereas the deployed algorithm is a differentiable fixed-point update rather than an exact Nash-equilibrium solver. We prove convergence to a unique fixed point under a contraction condition and evaluate the refinement empirically. Across four benchmark datasets, IGTF improves over supervised GPT-4+BERT fusion baselines by 4.07 F1-macro points on average, with the largest gains when raw intent vectors show stronger relation-level inconsistency. Human evaluation and graph-perturbation controls suggest that these gains come from structured intent calibration rather than LLM features alone.

Abstract:
Clinical diagnosis is inherently sequential: clinicians escalate from cheap to costly tests only when additional evidence is expected to resolve diagnostic uncertainty. We present ActiveMedAgent, a framework that brings this cost-aware sequential logic to multimodal medical AI. Given a frozen, API-accessed vision-language model, ActiveMedAgent probability distributions over candidate diagnoses induced each acquisition by its per-step diagnostic utility minus cost. A lightweight MLP controller is then trained offline on these scored trajectories, learning when to request additional evidence and when to commit. Across three commonly used benchmarks, trajectory-based policy learning consistently outperforms both unguided acquisition and full-modality baselines. Notably, we identify an information overload effect. In 175 cases, the agent produces a correct diagnosis with fewer channels while the full-modality baseline fails, showing that learning what to omit can be as important as learning what to acquire

Abstract:
Language model agents are increasingly used to automate scientific research, yet evaluating their scientific contributions remains a challenge. A key mechanism to obtain such insights is through ablation experiments. To this end, we introduce AblationBench a benchmark suite for evaluating agents on ablation planning tasks in empirical AI research. It includes two tasks: AuthorAblation, which helps authors propose ablation experiments based on a method section and contains 83 instances, and ReviewerAblation, which helps reviewers find missing ablations in a full paper and contains 350 instances. For both tasks, we develop LM-based judges that serve as an automatic evaluation framework. Our experiments with frontier LMs show that these tasks remain challenging, with the best-performing LM system identifying only 45% of the original ablations on average, below human-level performance. We observe an inverse performance trend between the author and reviewer tasks, which we attribute to differences in model grounding. Lastly, we analyze the limitations of current LMs on these tasks, and find that chain-of-thought prompting outperforms an agent-based approach.

Abstract:
Multimodal large language models (MLLMs) can memorize identity-specific facts about people in their fine-tuning data, creating privacy risks when a person requests deletion. Existing MLLM unlearning methods often assume access to retain images or ground-truth answers during deletion, which is unrealistic in many practical scenarios. We study identity unlearning when retain images are unavailable at deletion time. Our analysis shows that identity and visual-perception questions occupy distinct regions in fine-tuned hidden states and are organized differently: identity questions cluster by person, whereas perception questions cluster by question type. This suggests that identity knowledge can be suppressed without erasing general visual perception. Building on this observation, we propose AIM, a two-stage method that anchors an identity-forgetting target with a universal visual prompt and then matches the vision encoder to that target under a Fisher-based constraint. Extensive experiments show that AIM achieves competitive identity forgetting while preserving non-deleted identities, prior knowledge, and visual perception on the same images.

Abstract:
Large language models are widely employed as evaluators, a paradigm commonly referred to as LLM-as-a-judge. Prior research has predominantly examined point-wise or pair-wise evaluation protocols; in contrast, our focus is on rubric-based evaluation, which has been attracting increasing attention owing to its utility for training models in domains where verification is otherwise difficult. In this work, we show that rubric-based evaluation implicitly resembles a multiple-choice setting and therefore exhibits position bias: LLMs tend to prefer score options that appear at specific positions within the rubric list. Through controlled experiments across multiple models and datasets, we demonstrate that this position bias is consistent. Its direction, however, is model-specific: some judges favor the first option, while others favor the last. We further identify a second, orthogonal axis of bias: when a prompt scores several criteria simultaneously, the ordering of the criteria itself shifts the resulting scores. We additionally explore permuting the order of the rubric options as a means of mitigating position bias, and find that although the bias can be attenuated, improvements in the correlation between model judgments and human annotations are obtained primarily for models that exhibit strong bias. Our results recast rubric-based LLM-as-a-judge as a multiple-choice problem with measurable, model-specific position bias, and we further confirm that only a small number of random order permutations are sufficient to reduce the error introduced by this bias for the majority of models.

Abstract:
The rapid advancement of generative AI raises concerns about the misuse of Multimodal LLMs (MLLMs) for large-scale disinformation campaigns on social media. Despite existing research on textual disinformation, a fundamental question remains unanswered: can MLLMs be exploited to fabricate realistic multimodal fake news, and can they reliably detect it? We introduce a multi-agent framework in which a story agent, an image agent, and a critic agent collaborate to produce fake social media posts that plausibly counter true news. We apply the framework to generate over 9,000 paired multimodal news posts across science, health, and entertainment domains, and benchmark 16 open- and closed-source MLLMs for automated detection. We find that most models fall substantially short of human-level accuracy and fail critically on identifying image authenticity. Our research provides a foundation for developing robust defenses against social media fake news. Code and data are available at https://github.com/xiuzhenzhang/Multimodal.

Abstract:
Architecture-specific vector programming remains a difficult frontier for language models: an implementation can satisfy surface syntax yet fail because of a hallucinated intrinsic, an invalid predicate shape, a sizeless-type misuse, or a subtle tail-handling error. This work studies NL2SVE, a function-level code generation problem that maps natural-language specifications and fixed C/C++ signatures to ARM Scalable Vector Extension (SVE) implementations. We introduce SVE-Loom, a model-agnostic generation-and-repair framework that operates within a bounded compile-and-run feedback session. SVE-Loom combines ACLE schema repair, execution feedback, and reference-assisted Regional Semantic Bootstrap when scalar semantics are available. From an initial prompt under a bounded feedback budget, wrapped models using SVE-Loom achieve higher success rates, producing more legal and correct SVE implementations across evaluation benchmarks. We further mitigate the scarcity of public SVE supervision by constructing a NL2SVE corpus from GitHub-mined SVE code, template-generated kernels, and SVE-Loom-guided API-model candidates. This third layer makes the corpus extensible: scalar references or capability gaps can be converted into NL-to-SVE examples, while candidate SVE implementations are retained only after schema checks, compilation, execution testing, speedup measurement, and human audit.

Abstract:
Multimodal Large Language Models (MLLMs) process speech and text jointly, yet whether they exploit prosodic cues for pragmatic inference or rely on surface acoustic patterns has received little systematic investigation. We address this through sarcasm detection, evaluating Qwen2.5-Omni and Qwen3-Omni on Mandarin Chinese and English under five modality conditions that decompose the contributions of lexical content, vocal semantics, and prosodic structure. Adding audio systematically inflates false positives without improving true positive detection. Acoustic error diagnosis reveals that model errors cluster on a shared stereotype of expressive prosody, namely elevated pitch and irregular pausing, that diverges from the actual cues marking sarcasm in both languages. Targeted manipulation of only these two dimensions causally confirms the heuristic, inducing false positive rates of up to 60%. Applying the same manipulation template to Gemini 3 Flash Preview without modification replicates the effect, suggesting that the stereotype extends beyond the Qwen Omni family rather than arising from a single model architecture.

Abstract:
Parametric retrieval-augmented generation (PRAG) injects retrieved evidence into a large language model (LLM) through passage-specific LoRA adapters, reducing reliance on long in-context prompts. When multiple passages are retrieved for the same query, however, evidence-level fusion becomes a bottleneck: equal-weight merging can amplify weak or conflicting evidence, and translating retrieval signals into fusion weights often requires fragile global tuning. We propose FCPRAG, a fusion-controlled parametric RAG framework that adds a lightweight controller for retrieval-conditioned, sample-level adapter fusion. The controller predicts per-passage fusion scores together with sample-level calibration signals, including a mixing gate and an adaptive temperature, enabling fusion that stays selective under informative retrieval signals and conservative under uncertainty. FCPRAG is trained with merge-aware supervision derived from each adapter’s marginal contribution within a multi-adapter merge, using training data only. We further show that a single dataset-level temperature is suboptimal under heteroscedastic retrieval uncertainty, motivating sample-level adaptation. Experiments on HotpotQA, 2WikiMultiHopQA, PopQA, and ComplexWebQuestions (CWQ) across three LLM backbones show that FCPRAG consistently improves F1 over standard RAG and parametric RAG baselines, with gains of up to 4.65% on 2WikiMultiHopQA and 7.55% on CWQ, while also reducing tuning cost and improving robustness under retrieval perturbations. Code is available at https://github.com/ReinerBRO/FCPRAG.

Abstract:
Topological reasoning—the ability to identify structural invariants under continuous deformations—is a cornerstone of human visual cognition, yet it remains under-explored in modern AI systems. Existing benchmarks provide limited coverage of mathematical topology as a discipline and rarely probe the depth of topological reasoning. To bridge this gap, we introduce TopoEval, an 800-item benchmark suite for topological reasoning. TopoEval centers on a 400-item answer-based Core track and includes a 400-item Proof track for reference-grounded informal proof assessment. The suite spans four major topology branches, twelve fine-grained subfields, and three difficulty levels. On Core, we investigate model scaling, reasoning depth, prompt strategies, image dependency, and academic tiers. The strongest Core configuration reaches 73.0% overall and 54.1% on all 85 K12 items. For context, topology-trained graduate-student solvers reach 96.3% under the audited protocol in Appendix B.2. On Proof, pass-plus-borderline rates are low across the selected conditions, indicating difficulty in producing reference-aligned natural-language topology proofs. TopoEval profiles these capabilities across academic level, image dependency, and proof quality.

Abstract:
Large language models (LLMs) often memorize personally identifiable information (PII) during training, raising serious privacy concerns. While machine unlearning has emerged as a promising mitigation, its effectiveness against privacy attacks remains unclear. We introduce PrivUn, an evaluation framework for privacy unlearning under recovery attacks, paired with three measurement tools: forgetting scores, association metrics, and layer-wise analysis of forgetting depth. This framework studies not only whether a model forgets, but also how forgetting propagates and where it fails. Using PrivUn, we make two findings. (i) The PII ripple effect is gradient-driven, not semantic. Graph-based proximity is uncorrelated with PII forgetting, while gradient-based association is more strongly correlated. (ii) Label-based unlearning is typically shallow, while PII is stored across deeper layers. Layer-wise evidence suggests that label-based methods mainly change the final layers, explaining why they reduce direct retrieval but remain vulnerable to stronger recovery attacks.

Abstract:
Diffusion language models (dLLMs) generate text by iteratively denoising multiple token positions in parallel, offering an attractive alternative to strictly autoregressive decoding. In practice, however, block-wise dLLM inference exposes a difficult granularity trade-off: small blocks preserve local conditioning but require many denoising steps, whereas large blocks expose more parallelism but can make premature commitments. Existing block-wise acceleration methods either use a fixed block size or adapt block boundaries along a single decoding trajectory, leaving the complementarity among concurrently executed block-size trajectories unused. We show that block-size diversity provides a useful branching dimension. Different block sizes induce related but non-identical KV-cache trajectories: branches often share an initial prefix, bifurcate at semantically decisive positions, and later agree on syntactically lightweight tokens. Motivated by this structure, we propose BlockBatch, a training-free online inference framework that executes multiple block-size branches for the same request inside a batched forward pass. BlockBatch coordinates these branches through confidence-gated token merging, leader-based synchronization, and periodic full-sequence refreshes that re-anchor local block updates to a globally consistent KV state. Across three representative dLLMs and four datasets, BlockBatch reduces denoising NFEs by 26.6% on average and achieves a 1.33 × average end-to-end speedup, with comparable average accuracy.

Abstract:
Spatial reasoning is a fundamental component of world understanding for Multimodal Large Language Models (MLLMs). While existing research has advanced the perception of spatial relations in static scenes, the domain of dynamic spatial simulation remains relatively unexplored. To bridge this gap, we introduce MentalGrid, a benchmark focused on spatial reasoning tasks that require models to simulate evolving spatial states without intermediate visual feedback. Extensive evaluation on our benchmark reveals a significant deficiency in the spatial simulation capabilities of existing models. Specifically, we find that performance is constrained by perceptual limitations in simple simulation scenarios and hindered by inadequate multi-step reasoning as complexity increases. To address these challenges, we propose a three-stage curriculum learning framework utilizing 16k specialized spatial reasoning samples. This approach yields SpatialWalker, which achieves a 33.8% average improvement over its base model, surpassing proprietary models like Gemini-3-Flash in spatial simulation. Furthermore, SpatialWalker demonstrates robust generalization, maintaining a 4.9% average gain on out-of-domain benchmarks, demonstrating the pivotal role of spatial simulation as a fundamental building block for broader spatial intelligence.

Abstract:
Long chain-of-thought (CoT) traces provide observable natural-language rationales for mathematical problem solving, but they rarely expose the named definitions and theorems that justify the written solution. We treat each problem–trace pair as carrying a recoverable knowledge footprint: a trace-grounded support set of named definitions and theorems. Inspired by formal proofs, where supporting premises are recorded more explicitly than in free-form CoT, we formulate Mathematical Support Set Extraction (MSSE): given a problem and a completed long CoT trace, recover the canonical support set that supports the written solution. We propose ReasonMap, which attributes this support set to make solutions auditable, reusable, and easier to repair. ReasonMap follows a two-stage pipeline: proof-to-CoT support transfer adapts premise-explicit proof supervision to premise-implicit long CoT traces, and hard-negative support calibration applies preference optimization against plausible-but-wrong support sets. Experiments show that ReasonMap improves support-set extraction over strong baselines and that ReasonMap-generated support hints aid downstream reasoning in both frontier-model and self-generated settings. The GitHub repository is available at https://github.com/kui253/ReasonMap .

Abstract:
Large language models (LLMs) can solve complex multi-hop problems yet exhibit puzzling failures on simple two-hop queries: although a model may correctly store each individual hop, it often fails to combine them. To understand the internal mechanisms of this phenomenon, we train transformers from scratch in a controlled symbolic environment. Our experiments reveal a pattern in two-hop generalization: models generalize reliably when the second hop follows the training distribution, but always fail when it deviates. Through mechanistic analysis, we provide a complete explanation for these distinct generalization behaviors: in settings where models generalize successfully, performance is driven by the emergence of consistent intermediate representations for the same entities across contexts, whereas failures on settings where the second hop is out-of-distribution arise from a mismatch across layers: lower layers correctly construct these intermediate representations, but upper layers, while trained on corresponding atomic facts, primarily learn to map them to outputs rather than to reason over them. Driven by this insight, we propose a recurrent-style training strategy, which enables transformers to reuse their reasoning circuitry across input forms and substantially improves generalization on out-of-distribution two-hop queries. Our data and code are available at https://github.com/zzl-strong/two_hop.

Abstract:
Air traffic control (ATC) requires structured, safety-critical decision making whose effects unfold through aircraft trajectories, so text-level correctness alone is insufficient for evaluating large language models (LLMs) in this domain. We introduce ATC-Bench, a simulation-grounded benchmark for ATC instruction generation with Chinese-facing scenario descriptions and controller-style instruction fields. Given a structured Chinese traffic situation, a model generates a JSON control plan that is translated into BlueSky commands and evaluated through closed-loop simulation against separation, no-fly-zone (NFZ) avoidance, and route-recovery constraints. The benchmark spans four task families that form a controlled complexity gradient: single-aircraft NFZ avoidance, pairwise conflict resolution, joint conflict-airspace constraints, and multi-aircraft conflict graphs. Experiments with 11 representative LLM endpoints show that while most models produce format-valid instructions, task success varies widely across models and task families: the highest-scoring endpoint under our fixed protocol achieves 63.5% macro-average task success rate (TSR), while the two lowest endpoints fall below 25%, with the largest gaps appearing under joint constraints and multi-aircraft coordination. These results expose a persistent execution gap between static validity and simulation-passing behavior, motivating simulation-in-the-loop evaluation for LLM-based decision making in safety-critical aviation settings.

Abstract:
Emotion understanding from text has been predominantly approached through discrete classification frameworks. However, complementary perspectives in psychology and neuroscience suggest that affective states may also exhibit continuous, context-dependent, and temporally structured properties. Building on this perspective, we investigate emotion representations as latent dynamical trajectories within transformer embedding spaces. Using GoEmotions and MELD across four transformer architectures, we analyze the geometric organization and temporal evolution of learned emotional states. We find that latent trajectories exhibit temporal persistence and structured transition dynamics, while the prediction error increases during emotion shifts, acting as a signal of expectation violation. We additionally introduce trajectory-level latent interventions enabling controllable emotional transitions through directional operations in representation space. Finally, manifold capacity analysis reveals partially aligned and variably separable emotional manifolds, suggesting shared geometric organization across affective representations.

Abstract:
The conversational capabilities of large language models hold significant promise for enabling scalable and interactive tutoring. Prior work has primarily evaluated general instructional competence while overlooking a critical dimension: the ability to adaptively guide learners based on their evolving cognitive states. This study takes instructional guidance as a core capability and asks whether LLMs can emulate expert tutors who dynamically adjust strategies in response to learners’ needs. We introduce GuideEval, a benchmark grounded in authentic educational dialogues that evaluates pedagogical guidance through a three-phase behavioral framework: (i) Assessment, inferring learner states; (ii) Orchestration, adapting instructional strategies; and (iii) Elicitation, stimulating proper reflections. Empirical results indicate that existing LLMs often fail to provide effective adaptive scaffolding when learners experience confusion or require redirection. Complementing quantitative results, we present a detailed failure case analysis, offering intuitive insights into these limitations. We further propose a behavior-guided finetuning strategy based on behavior-prompted instructional dialogues, substantially enhancing guidance performance. By shifting the focus from isolated content correctness to learner-centered state-aware interaction, this work advocates a more dialogic paradigm for evaluating Socratic LLMs.

Abstract:
Post-Training Quantization (PTQ) enables efficient deployment of Large Language Models (LLMs), yet activation outliers remain a key challenge for low-bit quantization. Existing global transformation methods overlook heterogeneous channel sensitivity to quantization noise, leading to suboptimal accuracy-efficiency trade-offs. To address this, we propose Locally Isolated Rotation Quantization (LIRQuant) , a sensitivity-aware framework that selectively applies transformations. LIRQuant identifies sensitive channels using a noise-weighted sensitivity metric, applying expressive learnable rotations only to these channels and lightweight fixed rotations to the remaining ones. This decoupled design reduces optimization complexity without sacrificing outlier smoothing capability. Extensive experiments across the LLaMA-2/3, Mistral, Qwen3, and Mixtral families demonstrate that LIRQuant achieves state-of-the-art accuracy under the challenging W4A4KV4 setting. Meanwhile, LIRQuant calibrates LLaMA-3-8B in only 25 minutes with 20.3 GB peak GPU memory, achieving the fastest calibration and the lowest memory footprint among recent learnable-rotation methods, and its fused inference kernels deliver up to 2.3× prefill speedup and 3.5× memory saving over FP16.

Abstract:
Automatic harmfulness metrics and LLM-based judges are central to LLM safety assessment, yet differences in evaluation criteria, methodologies, and output formats may yield inconsistent judgments for the same model response. The lack of systematic research on these evaluators undermines the credibility and consistency of LLM safety assessments. To bridge this gap, we introduce HarmMetric Eval, a benchmark for both overall and fine-grained assessment of harmfulness evaluators. We construct a dataset of representative harmful prompts paired with diverse harmful responses and non-harmful counterparts across multiple categories. We propose a flexible scoring mechanism that rewards metrics for correctly ranking harmful responses above non-harmful ones, making it applicable to nearly all metrics and judges with varying output formats and scoring scales. Using HarmMetric Eval, we find that conventional reference-based metrics such as ROUGE and METEOR can outperform existing LLM-based judges in fine-grained harmfulness evaluation, challenging prevailing assumptions about LLM superiority. Fine-grained analysis reveals that LLM judges tend to overestimate irrelevant and useless responses. Furthermore, we build a new harmfulness judge by incorporating fine-grained criteria into its prompt template and leveraging reference-based metrics to fine-tune its base LLM. The resulting judge demonstrates leading performance on HarmMetric Eval and two external human-labeled datasets.

Abstract:
Large language models are increasingly used to simulate social media users and infer how individuals may respond to online discussions. However, it remains unclear whether these simulations reflect precise user-specific beliefs or whether they are highly sensitive to semantically independent changes in conversational contexts. In this work, we study counterfactual context revision as a framework for auditing LLM-based stance simulation. Given an original online conversation, we first infer a target user’s stance toward a specific topic. We then apply controlled revision strategies to the conversational context and simulate the user’s stance again under the revised context. We compare text-only revision strategies with a multimodal one that incorporates meme-based context and evaluate two main effectiveness metrics, i.e., average directional stance shift and stance transition rate. The results reveal effective and robust stance transitions in both text-only and multimodal strategies across different polarization-preference mechanisms. Our study contributes an evaluation framework for understanding the context sensitivity of LLM-based stance simulation (https://github.com/STARResearchLab/StanceShift). More broadly, it highlights both the promise and risk of using LLMs as proxies for social media users in social simulation.

Abstract:
Inference with modern Large Language Models (LLMs) is expensive and slow, and speculative sampling has emerged as an effective solution to this problem. However, the number of calls to the draft model for generating candidate tokens in speculative sampling is a preset hyperparameter, lacking flexibility. To generate and utilize the candidate tokens more effectively, we propose RADAR, a novel speculative sampling method with RL-based dynamic draft trees. RADAR formulates the draft tree generation process as a Markov Decision Process (MDP) and employs offline reinforcement learning to train a prediction model, which enables real-time decisions on calls to the draft model, reducing redundant computations and further accelerating inference. Evaluations across three LLMs and four tasks show that RADAR achieves a speedup of 3.17x-4.82x over the auto-regressive decoding baseline.

Abstract:
LLMs enable the integration of external conceptual resources at inference time, creating new opportunities for detecting ideologically and historically complex phenomena such as antisemitism. We investigate how different forms of conceptual grounding affect antisemitism detection and explanation behavior across four state-of-the-art LLMs. Using two expert-annotated datasets, we compare definitional, fine-grained taxonomic, example-augmented, and large-context representations of antisemitism. We find that fine-grained taxonomic representations substantially improve recall, while simultaneously reducing precision. Surprisingly, supplying substantially larger conceptual resources yields no additional quantitative benefit. Post-Holocaust antisemitism poses the most persistent challenge across models and configurations. Analysis of explanations further reveals systematic limitations including overproduction of conceptual references, reliance on lexical cues, overconfidence, and difficulties with subtle or justificatory forms of antisemitism. Our findings highlight both the potential and the remaining limitations of conceptually grounded LLMs for antisemitism detection and reasoning.

Abstract:
Tables are widespread in real-world scenarios. In particular, hierarchical tables encode much richer structural semantics; however, their complex layouts pose severe challenges to Large Language Models (LLMs). The core difficulty lies in precisely aligning complex topological structures with multi-level semantic dependencies, where alignment failures often lead to cross-level aggregation errors and hallucinations. To address this challenge, we propose STAR-Table, a framework that synergizes visual perception with symbolic reasoning. Unlike traditional text-only approaches, STAR-Table leverages OCR models to extract pixel-level spatial coordinates, thereby establishing explicit Spatial-Textual AnchoRs. These anchored spatial priors are formulated as explicit structural hints to guide LLMs in reconstructing the hierarchical topology of tables. Based on the reconstructed topology, complex visual layouts are decomposed into one or more distinct sub-tables, each of which is further mapped into a deterministic flattened database. Finally, we design an adaptive QA module that fuses Vision-Language Model (VLM) semantic summaries with multi-step SQL execution to support robust table-based reasoning. Experimental results across multiple benchmarks demonstrate that STAR-Table achieves superior performance, significantly outperforming specialized fine-tuned models and advanced in-context learning paradigms.

Abstract:
Complex decision-making often requires agents to progress through intermediate tasks rather than solve the final target directly. Existing LLM self-refinement methods typically iterate on a fixed target, while curriculum-learning methods often rely on hand-designed schedules, training-time optimization, or domain-specific difficulty metrics. To smooth the learning curve through adaptive curriculum design, we introduce EvoCurr, a general inference-time framework that co-evolves curricula and executable policies. A Designer proposes verifiable intermediate tasks from the latest accepted progress and recent failures, while a Solver generates or trains policies for these tasks. A task-policy pair is accepted only when it passes hard feasibility checks and reaches a specified performance threshold; meanwhile, an accepted-floor constraint preserves the latest verified checkpoint and prevents failed harder attempts from overwriting mastered behavior. We evaluate EvoCurr across code-as-policy and closed-loop MARL settings, using StarCraft II super-late-game micromanagement as our hardest stress test. EvoCurr demonstrates effective micromanagement across three LLM backbones without LLM parameter training. Improvements in additional experiments on Flatland, POGEMA, Overcooked, and LiveCodeBench Hard demonstrate the generality of EvoCurr.

Abstract:
Multi-turn jailbreak attacks pose a growing threat to LLMs by exploiting gradual escalation and cross-turn coordination. Many existing defenses either rely on costly retraining—often degrading utility—or analyze each turn independently, failing to capture risk accumulation across interactions. We observe that safety behavior in multi-turn interaction is trajectory-dependent: dialogue history reshapes the model’s conditioning context, making isolated turn-level evaluation insufficient. We present THRD, a training-free framework that explicitly models temporal risk accumulation for multi-turn jailbreak defense. THRD integrates four modules: a Turn-level Risk Assessor (TRA) for instantaneous risk estimation, a Historical Context Analyzer (HCA) for cross-turn intent escalation detection, a Response Evaluator (RE) for identifying facilitative outputs, and a Decision Module that combines these signals through a time-evolving scoring mechanism with attenuation-based modulation and trend-aware adjustment. Experiments against representative multi-turn attacks—including tree-search-based and multi-agent collaborative methods—across two target models show that THRD reduces ASR to 0.2–4.0% while preserving model utility within 1.5% degradation on MMLU and GSM8K. Ablation studies confirm non-redundant module contributions and consistent performance across both models. Over 70% of attacks require Turn 2 or later to detect, highlighting the importance of cross-turn information.

Abstract:
In e-commerce, LLM agents show promise for shopping tasks such as recommendations, budget management, and bundle deals, where accurately capturing user preferences from long-horizon conversations is critical. However, progress is limited by two key challenges: (1) the absence of benchmarks for evaluating long-term preference-aware shopping tasks, and (2) the lack of fine-grained supervision for shopping agent training. To fill the benchmark gap, we introduce Shopping Companion Bench, a novel benchmark comprising two shopping tasks that require cross-session preference memory, grounded in a product pool of over 1.2 million real-world items. Our analysis further identifies two major sources of failure on this benchmark: cascading errors caused by preference hallucination, and insufficient verification of product attributes against user requirements. To address these failure modes, we design annotation-free, tool-wise rewards that provide process supervision for each tool call, alleviating reward sparsity in long-horizon tasks. Experimental results demonstrate that even state-of-the-art models such as GPT-5 achieve success rates below 70%, highlighting the difficulty of our benchmark. Notably, our fine-tuned lightweight 4B model consistently outperforms strong baselines in both preference capture and task performance, suggesting the effectiveness of our reward design.

Abstract:
Explainable Recommendation (ER) aims to improve the transparency of Recommender Systems (RS) by generating natural language explanations. However, existing methods primarily optimize textual similarity to reference explanations, which often leads to item-centric explanations focusing on product attributes rather than users’ personalized preferences. Moreover, they usually do not care how the generated explanations influence the behavior of users. To address these limitations, we introduce Agent4ExpRec, a closed-loop user-centric pipeline designed for generating and evaluating recommendation explanation. Based on the original explanations, it enables the peer agent to analyze the user’s requirements from multiple aspects and provides feedback, and automatically plans the refinement process to generate explanations focusing on user preferences. During the evaluation stage, we analyze how different explanations influence user behavior, providing evidence that our generated explanations are more consistent with users’ personalized preferences. Since the actual user behavior on large-scale explanation text is difficult to obtain in practice, we use LLM-powered agents equipped with user profile to simulate user interactions and decision-making in ER scenarios. Extensive experiments demonstrate that the generated explanations consistently align with user preferences, highlighting the potential of user-centric refinement for explainable recommendation tasks. Our code is available at https://github.com/yy-Hou/Agent4ExpRec .

Abstract:
Achieving mastery in real-world software engineering tasks is fundamentally bottlenecked by the scarcity of large-scale, high-quality training data. Scaling such data has been limited by the complexity of environment setup, unit-test generation, and problem statement curation. In this paper, we propose Scale-SWE, an automated, sandboxed multi-agent workflow designed to construct high-quality SWE data at scale. The system coordinates three specialized agents—for environment setup, test creation, and problem description synthesis—to process 6 million pull requests across 5.2k repositories, producing Scale-SWE-Data: 100k verified SWE instances, the largest such dataset to date. It substantially surpasses existing real-world datasets in repository diversity and reflects realistic task complexity. We further demonstrate the dataset’s utility for training by distilling 71,498 high-quality trajectories and fine-tuning Qwen-30B-A3B-Instruct to produce Scale-SWE-Agent. Our agent achieves a 64% resolve rate on SWE-Bench-Verified—a nearly three-fold improvement over the base model. Scale-SWE provides a scalable, reproducible approach for data construction to advance LLM-based software engineering.

Abstract:
LLM judges now score most open-ended NLP output, and their mutual agreement is routinely read as evidence that the scores can be trusted. That reading is unsafe: judges may agree because they capture quality, or because they share the same blind spots, and agreement statistics alone cannot tell these apart. We develop a geometric test that can. Treating each judge’s scores as a vector, we measure spread, effective rank, the angle to human scores, and the judge–judge, judge–human, and human–human agreement triple for 42 judges on two community-built Indic benchmarks covering four domains and eight languages. Every comparison is reference-matched: a judge and a held-out rater are scored against the same two-rater mean, since an averaged reference otherwise flatters judges by several degrees. On subjective rubrics, judges agree with one another as much as humans do yet reach only 58–66% of human agreement: consensus on an axis humans do not weight. On the one rubric with a verifiable answer, most of that gap closes. Ensembles converge on the judges’ shared axis rather than the human one, and training widens scores without rotating them. Inter-judge agreement is evidence of human alignment only after this check passes.

Abstract:
Named entity recognition (NER) is evolving from a sequence labeling task into a generative paradigm with the rise of large language models (LLMs). We conduct a systematic evaluation of open-source LLMs on both flat and nested NER tasks. We investigate several research questions including the performance gap between generative NER and traditional NER models, the impact of output formats, whether LLMs rely on memorization, and the preservation of general capabilities after fine-tuning. Through experiments across eight LLMs of varying scales and four standard NER datasets, we find that: (1) With parameter-efficient fine-tuning and structured formats like inline bracketed or XML, open-source LLMs achieve performance competitive with traditional encoder-based models and surpass decoder-based LLMs with in-context learning techniques; (2) The NER capability of LLMs stems from instruction-following and generative power, not mere memorization of entity-label pairs; and (3) Applying NER instruction tuning has minimal impact on general capabilities of LLMs, even improving performance on datasets like DROP by 25.50 to 45.32 F1 points due to enhanced entity understanding. These findings demonstrate that generative NER with LLMs is a promising, user-friendly alternative to traditional methods.

Abstract:
Automated Radiology Report Generation (RRG) could substantially reduce documentation workload by supporting physicians in routine clinical practice. Clinical adoption, however, requires each generated finding to be supported by inspectable evidence that a physician can verify. Existing systems either provide only post-hoc explanations weakly coupled to the prediction, or require costly region-level supervision to achieve spatial grounding. To address this gap, we introduce ProtoRRG, an RRG framework that grounds generation in visual prototypes. Specifically, each generated finding is tied to a dual form of evidence: localised image exemplars learned without anatomical supervision, and retrieved prior reports that supply case-based linguistic precedents. A multi-task training strategy reconciles the competing representational demands of prototype-based prediction and autoregressive decoding within a shared architecture. On MIMIC-CXR, ProtoRRG improves state-of-the-art clinical F1 and recall by 2.9% and 10%, respectively, with gains also demonstrated on IU X-Ray. An evaluation against anatomical annotations confirms that prototype activations localise within the correct regions, indicating spatially faithful evidence for the generated findings. These results demonstrate that combining visual prototypes with case-based retrieval improves both diagnostic accuracy and report quality while preserving the inspectable evidence that clinical deployment demands.

Abstract:
Large Language Models (LLMs) are increasingly deployed in adversarial environments, yet current automatic prompt optimization methods often suffer from "shortcut learning"—overfitting to specific samples rather than learning generalizable principles. We argue that, compared to scalar reward optimization or few-shot enumeration alone, the interpretability of language offers a useful medium for inducing robust decision logic. To study this, we introduce PROSE, a robust prompt optimizer that uses semantic reflection and multi-objective evolution to reduce over-reliance on enumerated examples. Facing adversarial tasks, PROSE employs an operational proxy for explicit case enumeration as an additional optimization objective. By exploring the Pareto frontier between performance and this proxy, it adds a regularization signal against sample fitting and favors more reusable rule descriptions. It employs ensemble-based de-biasing strategies and a semantic greedy selection mechanism to reduce inductive bias and premature "semantic collapse," while leveraging LLM-based reflection to summarize concise, high-level rules from failure patterns. Across the evaluated benchmarks, PROSE achieves higher average performance than the strongest baseline by up to 14.6 percentage points in accuracy on Short-Text and up to 11.84 percentage points in F1 score, while showing stronger held-out content-category generalization and reducing the F1 degradation from 29.7 points for GEPA to 13.0 points.

Abstract:
Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model. We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver’s current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests. Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator can also improve outputs from other LLMs to further improve their performance.

Abstract:
Self-consistency (SC) improves chain-of-thought reasoning by aggregating multiple sampled answers, yet all existing variants operate in the output space—they refine answer selection but leave the noisy input context unchanged. We observe that when multiple reasoning traces independently cite the same support items, those items are likely key evidence for the question. Building on this input-side consensus signal, we propose SVote-2S, a two-stage self-consistency framework for multi-hop question answering. In Stage 1, SVote-2S performs confidence-weighted voting over support items cited across N sampled traces and selects a compact evidence set. In Stage 2, the model re-reasons over a filtered context containing only the selected evidence paragraphs (about 15% of the original length on average on our benchmarks), enabling more focused reasoning. The final answer is determined by confidence-weighted fusion of samples from both stages. The fusion serves as a robustness safeguard, while the primary gain comes from evidence voting and re-reasoning. Across three benchmarks (HotpotQA, MuSiQue, 2WikiMultihopQA) and three LLMs, SVote-2S improves over SC by 5.36 EM on average and consistently outperforms the strongest output-side baseline (CISC) across all nine settings, without additional training or external retrieval.

Abstract:
Large Vision-Language Models (LVLMs) commonly suffer from hallucinations, which appear as fabricating non-existent objects or outputting incorrect attributes and spatial relationships in images. Previous works empirically indicate that CLIP-style vision encoders tend to focus on primary objects while ignoring image details, which is one of the causes of hallucinations. To address this issue, we propose plug-and-play fine-grained visual experts (PLANE) comprising two components: LoRA visual experts operating on CLIP for multi-aspect image encoding, and an instruction-aware gate network for adaptive image feature fusion. During training, we construct a fine-grained image-caption pairs dataset, where images are described from three aspects: objects, attributes, and relations between objects. Image-caption pairs from different aspects are used to train the three fine-grained LoRA experts respectively. Aiming at comprehensively evaluate our method’s capabilities, we manually annotate a fine-grained hallucination evaluation benchmark for LVLMs which is GPT4-independent during evaluation phase (GIFT). We conduct experiments both on popular hallucination evaluation benchmarks and GIFT. The results show that our method effectively reduces hallucinations across models of different types and sizes while introducing minimal inference latency. Compared to advanced baselines, our model also demonstrates superior performance.

Abstract:
Deploying speech command recognition on multilingual edge devices demands models that are both energy-efficient and capable of cross-lingual transfer without per-language retraining—two goals rarely addressed jointly. Spiking Neural Networks (SNNs) offer substantial energy savings via sparse binary communication, yet their cross-lingual generalization remains unstudied. We present SpikeXL, a framework that investigates whether SNN spike sparsity provides an inductive bias toward language-agnostic representations. Our experiments suggest that the V th firing threshold of Leaky Integrate-and-Fire (LIF) neurons suppresses language-specific surface phonetics while preserving command-level acoustic structure, yielding embeddings that transfer across languages more readily than dense ANN activations. Building on this finding, we introduce Spike-space Contrastive Alignment (SCA), a contrastive objective adapted for spike-rate embeddings that aligns semantically equivalent commands across languages without requiring bilingual lexicons or parallel speech data. On Speech-MASSIVE across 12 typologically diverse languages, SpikeXL achieves a Cross-Lingual Semantic Similarity (CLSS) of 0.912—outperforming a parameter-matched Transformer with the same SCA objective (0.763) and SNN without SCA (0.721)—indicating that spike sparsity itself contributes an inductive bias beyond what contrastive training alone provides. Few-shot k -NN classification on target languages (115 support samples, frozen encoder) reaches 80.8 macro-average accuracy, corroborating alignment quality with a metric fully independent of the SCA objective. On English GSC-v2, SpikeXL reaches 92.76 accuracy with 2.5M parameters and 45M synaptic operations per inference.

Abstract:
Tool-using large language models (LLMs) can make multiple attempts to complete a task, using execution feedback from failed tool calls to identify errors and revise subsequent actions. However, their recovery performance often plateaus in later attempts, as models may repeat similar tool-call errors instead of making effective corrections. To address this limitation, we introduce PRFC, a reinforcement learning method that models error recovery as a sequential decision-making process. PRFC uses a progress-aware reward that assigns credit only when a later attempt improves upon previous failures, encouraging steady improvement across the recovery trajectory. Across four benchmarks with a Qwen3-4B backbone, PRFC improves multi-turn recovery on BFCL-v3 by an absolute 6.4% and achieves the highest mean score across three single-step tasks (API-Bank, T-Eval, and Nestful). These results show that modeling progress across attempts provides an effective learning signal for multi-attempt tool-use recovery. These results suggest that explicitly modeling progress across attempts provides a useful learning signal for tool-use recovery.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in training LLMs for reasoning tasks, but representative methods such as GRPO assign uniform credit across all tokens, wasting gradient on routine tokens while under-crediting pivotal reasoning steps. Existing token-level credit assignment methods require resources beyond the model’s own rollouts. GRPO variants rely on process reward models or ground-truth answers. Knowledge distillation assigns credit through per-token divergence but requires external teachers (On-Policy Distillation) or privileged information (On-Policy Self Distillation). However, these dependencies limit applicability in the pure RLVR setting. We observe that conditioning the model on its own verified trajectories induces a measurable per-token KL divergence between the original and conditioned distributions, and prove that distilling from a self-teacher constructed by verified trajectories leads to infeasible weighted-average solutions when multiple verified trajectories exist. We propose SC-GRPO (Self-Conditioned GRPO), which uses KL divergence mentioned before as a multiplicative weight on GRPO gradients. Across five benchmarks spanning math, code, and agentic tasks, SC-GRPO consistently outperforms 8.1% over GRPO and 5.9% over DAPO with stronger OOD performance. Moreover, SC-GRPO achieves higher performance than OPD.

Abstract:
Agentic MT frameworks often adopt human-inspired roles, layered post-editing, and complex workflows. To stress-test these designs, we examine cost–performance trade-offs across five multi-agent MT workflows on specialized long documents. Using six language pairs from two financial test sets and LLMs spanning lower to higher capacity and cost tiers, we evaluate translation quality and terminology accuracy relative to token-level cost. We find: (i) lean pipelines match the quality of more complex systems at lower cost; (ii) terminology accuracy depends more on model capacity than on workflow complexity; (iii) document-length resilience is not universal, as some simple and some complex workflows maintain quality on longer documents; and (iv) post-editing agents offer a better return on model capacity than base translation. Edited-span analysis further suggests that many post-edits are stylistic or lexical alternatives rather than consistently preferred improvements.

Abstract:
Language models achieve strong performance in medical question answering, yet hallucinations persist even in domain-specialized models, particularly on complex clinical vignettes involving rare presentations, multi-hop reasoning, and ambiguous evidence requiring explicit relational grounding. We introduce OntoDiagnose, a training-free, ontology-grounded, clinician-inspired reasoning pipeline that augments parametric knowledge with structured evidence from SNOMED CT. Given a clinical question, OntoDiagnose extracts salient medical entities, grounds them in the ontology, explores a bounded subgraph, and compresses relevant relations into compact, interpretable evidence chains that guide reasoning. We evaluate OntoDiagnose across USMLE-style and biomedical QA benchmarks using MedGemma, BioMistral, MedPhi, and OpenBioLLM under zero-shot, few-shot, and chain-of-thought settings. OntoDiagnose consistently improves over prompt-only baselines, with an average gain of 9.6 accuracy points on MedQA and the largest relative gains for the lowest-scoring models. These results show the benefit of incorporating ontology-grounded evidence into medical question answering without modifying model parameters.

Abstract:
We introduce Long Listening Thoughts (LLT), a framework for constructing post-training datasets that improves audio reasoning by pairing deliberate perceptual grounding with explicit cognitive behaviors. On audio understanding and reasoning benchmarks, large audio–language models (LALMs) are limited primarily by perceptual errors, yet existing post-training datasets rely on audio descriptions of limited diversity. Through a four-stage pipeline, LLT (1) builds multi-aspect dense audio descriptions spanning semantic, acoustic, and temporal information, (2) generates more challenging questions, (3) synthesizes incorrect reasoning traces, and (4) extends short reasoning traces into long ones via a thought-continuation strategy that elicits cognitive behaviors such as verification and backtracking. Using LLT, we synthesize 1.6M reasoning traces and train Qwen2.5-Omni-7B with a two-stage pipeline of supervised fine-tuning (SFT) followed by direct preference optimization (DPO). This improves accuracy on MMAU Test-mini ( 71.5 → 78.2 ), MMAR ( 56.7 → 63.7 ), and MMSU ( 60.6 → 66.0 ).

Abstract:
Spoken dialogue models currently lack the ability for fine-grained speech style control, a critical capability for human-like interaction that is often overlooked in favor of purely functional capabilities like reasoning and question answering. To address this limitation, we introduce UltraVoice, the first large-scale spoken dialogue dataset engineered for multiple fine-grained speech style control. Encompassing over 830 hours of spoken dialogues, UltraVoice provides instructions across six key speech stylistic dimensions: emotion, speed, volume, accent, language, and composite styles. Fine-tuning leading models such as SLAM-Omni and VocalNet on UltraVoice significantly enhances their fine-grained speech stylistic controllability without degrading core conversational abilities. Specifically, our fine-tuned models achieve improvements of 29.12-42.33% in Mean Opinion Score (MOS) and 14.61-40.09 percentage points in Instruction Following Rate (IFR) on multi-dimensional control tasks designed in the UltraVoice. Moreover, on the URO-Bench benchmark, our fine-tuned models demonstrate substantial gains in core understanding, reasoning, and conversational abilities, with average improvements of +10.84% on the Basic setting and +7.87% on the Pro setting. Furthermore, the dataset’s utility extends to training controllable Text-to-Speech (TTS) models, underscoring its high quality and broad applicability for expressive speech synthesis. The complete dataset and model checkpoints are available at: [https://github.com/bigai-nlco/UltraVoice](https://github.com/bigai-nlco/UltraVoice).

Abstract:
Large language models (LLMs) finetuned for specialized domains represent crucial high-impact applications. Inference-time alignment improves safety degraded from specialization finetuning without requiring substantial computational resources, complementing finetuning-based methods with an easy-to-use, plug-and-play solution. However, existing inference-time methods fail to reliably improve safety without disrupting domain capability. We identify the root cause as complementary expertise orthogonality: specialized base models and general-domain guidance models have orthogonal competencies, making the guidance signal unreliable for specialized generation. This primarily manifests as stop token interference, where the guidance model’s tendency toward continuation overrides the base model’s decision to stop, burying correct answers under guidance-induced continuation. To address this problem, we propose CREST, an inference-time alignment method that steers base model hidden representations using safety directions extracted from a guidance model of any family, avoiding token-level structural limitations entirely. CREST improves safety where specialization has weakened it while preserving both domain-specific capability and the safety of already well-aligned models, outperforming baselines by up to 22.2% on safety benchmarks. Our code is available at: https://github.com/DecayingSeart/CREST.

Abstract:
Large Vision–Language Models (LVLMs) show impressive performance across image and video understanding tasks, yet their computational cost grows rapidly with the number of visual tokens. Existing token pruning methods address this by reducing the number of visual tokens, but overlook the internal mechanism of attention. In this paper, we propose a novel training-free token pruning framework grounded in the dual-form perspective of attention. We reformulate attention as an implicit linear layer whose weight matrix is the sum of rank-1 outer products, each generated by a single token’s key–value pair. Token pruning thus reduces to identifying an optimal subset of these rank-1 updates that retains the essential information of the original dual weight matrix. Extending this perspective to standard softmax attention in LVLMs, we derive a novel importance metric quantifying both a token’s information magnitude and information duplication. To efficiently select the subset with the proposed metric, we introduce Progressive Chunked Maximal Marginal Relevance. Extensive experiments demonstrate that our method achieves a better trade-off between performance and efficiency, while providing another perspective on existing pruning approaches.

Abstract:
As LLMs process increasingly long contexts, the KV cache has become a major source of memory and bandwidth overhead during inference. Existing methods compress the KV cache by preserving attention sinks, recent tokens, or layer-local important tokens, but they rarely exploit how token importance is shared and specialized across layers. In this paper, we introduce ConsistKV, an adaptive KV cache compression method that exploits consensus in token importance across layers. For each layer, ConsistKV aggregates importance scores from neighboring layers to identify shared contextual anchors, while using residual local importance to preserve tokens that are salient to the current layer. It further estimates the cache demand of each layer from disagreement between layers and importance dispersion, enabling adaptive budget allocation under a fixed global cache budget. Experiments on LongBench show that ConsistKV achieves the highest average performance among evaluated KV cache compression baselines under the same cache budget.

Abstract:
Vision Large Language Models (VLLMs) show promise for multi-modal understanding; yet, their application to emotion recognition remains limited by insufficient spatial and contextual awareness and by supervised fine-tuning that uniformly penalizes all reasoning tokens—suppressing valid alternatives. We present Set-of-Vision-Text Rewards (SoVTR), a unified, reward-based framework that guides VLLMs with interpretable and verifiable visual attributes, focusing optimization on the emotion-discriminative reasoning steps that truly determine correctness. SoVTR decomposes supervision into three rewards: (i) Face Reward capturing action units, gaze, and micro-expressions; (ii) Body Reward encoding pose and gesture cues; and (iii) Context Reward modeling person–object relations and scene activities. By combining BERTScore attribute matching with GRPO policy optimization, SoVTR produces attribute-grounded explanation traces while preserving non-critical reasoning diversity. Across DFEW, FERV39k, and ExpW, SoVTR consistently improves both recognition accuracy and grounding quality, delivering substantial gains over strong VLLM baselines.

Abstract:
Large Language Models have shown remarkable potential in automated code generation, yet the programs they produce frequently fail due to logical flaws or unhandled edge cases. To address this, we propose RA-CRF, a Retrieval-Augmented Code Repair Framework that explicitly integrates a structured Error-Repair Knowledge Graph (ERKG). Unlike existing methods that rely on unstructured self-reflection or raw execution feedback, our framework constructs a hierarchical taxonomy connecting error types to specific repair strategies. By leveraging execution feedback, the framework retrieves relevant knowledge to generate structured repair documents, effectively decoupling error diagnosis from targeted code editing. Furthermore, we introduce an adaptive self-evolution mechanism that continuously refines the ERKG by persisting verified repair results. Experimental results on single-function benchmarks including BigCodeBench demonstrate that RA-CRF achieves significant performance gains across various open-source and closed-source models. Notably, RA-CRF lifts DeepSeek-V3 to 65.52% Pass@1, a new best on the official BigCodeBench leaderboard. Our findings indicate that integrating explicit and evolvable error-correction knowledge is a robust path to enhancing the reliability and interpretability of code generated by large language models.

Abstract:
Climate disclosure classification is a fundamental task for analysing corporate climate disclosures, yet such disclosures appear in many different sources—annual reports, press releases, and earnings calls—that differ in length, purpose, and writing style. Existing evaluations are mostly conducted within a single source, leaving open whether common LLM adaptation strategies remain effective under source shift. We reframe climate disclosure classification as a cross-source adaptation problem and study three widely used adaptation strategies— definitions, examples, and fine-tuning—across eleven open- and closed-source LLMs, using two corpora that share the same label space but come from different sources. We find that all strategies bring positive cross-source gains on average, but the strongest in-source strategies are not the strongest cross-source ones: similarity-based retrieval and LoRA fine-tuning gain most in-source but lose most of that advantage under source shift; randomly selected few-shot examples, a weaker in-source baseline, retain their advantage more reliably; definitions transfer most consistently, though only when their granularity matches the target text. Across these strategies, when the source changes, simpler is often safer.

Abstract:
Safety alignment in Large Language Models (LLMs) remains vulnerable to backdoor attacks. Existing LLM backdoors are almost all input-centric: activation depends on explicit trigger patterns in the user input, so modern guardrails are built to sanitize the input space. We challenge this assumption with a novel answer-side backdoor for multi-turn dialogue. Instead of inserting the trigger into the input, the adversary uses a benign first-turn prompt to naturally induce the model to generate a specific, seemingly innocuous word. Once merged into the dialogue history, this self-generated word becomes the trigger. When a later harmful query arrives, the model detects its own trigger and bypasses its safety refusal, while the user input stays perfectly clean. Across four LLMs, our attack reaches near-perfect Attack Success Rates, approaching 100% at only a 5% poisoning rate, while preserving general utility and clean-input safety, and it evades mainstream input-centric defenses. Representation-level analysis shows that the self-generated trigger consistently suppresses the model’s refusal signal, exposing a critical blind spot in current LLM defenses.

Abstract:
We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset containing 1,724 intent-label annotations over 1,275 unique difficult safety prompts, each paired with an intent description and harm label. We use AIMS to evaluate intent-aware training across supervised fine-tuning, preference learning, reasoning distillation, and reinforcement learning. Despite its size, AIMS enables competitive safety classifiers across training regimes: DPO from model-generated intent errors improves over SFT, and intent-conditioned distillation outperforms reasoning-only distillation in most teacher–student pairs. Most notably, directly rewarding intent faithfulness with GRPO yields the strongest average performance across five external safety benchmarks, while our intent-aware models form the inference latency–F1 Pareto frontier. These results show that faithful intent modeling is a compact, high-quality supervision signal for more robust safety classifiers.

Abstract:
Prompt-based LLMs are increasingly used for stance detection, but harder examples are not always repaired by clearer instructions, reasoning prompts, retrieval, or debate. We introduce (Stance Inference Complexity Index), a seven-dimensional diagnostic measure of the semantic-pragmatic burden imposed by a target–text pair. Across SemEval-2016 and VAST, predicts LLM accuracy better than surface proxies and shows substantial cross-scorer reliability ( 𝛼=0.771 ). More importantly, LLM errors change regime as increases: low-complexity examples invite over-attribution, especially Against predictions; intermediate examples form an unstable boundary; and high-complexity examples rapidly concentrate on None . This phase-transition-like structure persists across GPT-3.5, GPT-4o-mini, DeepSeek-V3, and GPT-4o, although stronger models move the boundaries. A 14-method intervention study further shows that prompting, retrieval, and debate often shift models along the attribution–abstention axis rather than removing the high-complexity bottleneck.

Abstract:
Medical vision-language datasets are limited in size and biased toward abnormal findings: clinicians document abnormalities in detail but frequently omit normal observations because they are considered uninformative for the patient’s condition. We propose a self-supervised enrichment method that leverages semantic clustering of report sentences and augments the training reports with normal observations drawn from co-occurring clusters in the same dataset. Our approach yields consistent gains in supervised fine-tuning: average improvements of +5.63% COMET, +3.04% BERTScore, +7.40% Sentence-BLEU, +5.30% CheXbert-F1, and +7.47% RadGraph-F1 across five language model backbones. Ablation studies confirm that improvements stem from semantic clustering rather than random augmentation. We further introduce a cluster-aware reward for GRPO training that complements exact-match rewards and yields additional gains: +2.78% COMET, +3.14% BERTScore and +12.80% Sentence-BLEU.

Abstract:
Memory is a cornerstone of foundation model-based agents for ultra-long-term interaction. However, hallucinated or noisy information can accumulate inside memory systems and progressively contaminate downstream tasks. Existing systems face two key challenges: (1) memory construction mechanisms trade off between faithful recall and hallucination suppression; and (2) retrieval relies on shallow semantic similarity, failing to distinguish logically irrelevant memories. We propose MemBook, a hierarchical time-aware memory framework for hallucination-resistant agents. For memory construction, MemBook introduces a stateful memory stabilization mechanism that prevents hallucinated claims from being consolidated into persistent memory and organizes reliable memories into an evolving Session–Chapter–Memory Point hierarchy. For memory retrieval, MemBook performs reasoning-based hierarchical retrieval over this evolving memory book to identify temporally and logically relevant memories. Experiments on HaluMem-Medium, HaluMem-Long, and LongMemEval show that MemBook improves extraction quality and downstream performance while suppressing cross-stage hallucination propagation. Notably, MemBook remains stable under ultra-long contexts, outperforming GAM by absolute 19.57% QA accuracy improvement on HaluMem-Long.

Abstract:
Template-based approaches have advanced Event Argument Extraction (EAE) by formulating it as a structured prediction problem. However, existing methods mainly rely on either fixed manually designed templates, which suffer from template under-matching in diverse contexts, or exhaustive enumeration of template permutations, incurring prohibitively high computational cost. Moreover, they often overlook the semantic heterogeneity across events. To address these challenges, we propose ATSR, an Adaptive Template Selection and Role-Aware Routing framework for document-level multi-event EAE, integrating an Adaptive Multi-structure Template Selection (AMSTS) module and a Role-Aware Mixture-of-Experts (RAMoE) module. Specifically, AMSTS leverages a set of pre-defined multi-structure templates for each event type and adaptively selects the most compatible one for each event based on learnable weights conditioned on trigger representations. The RAMoE operates over template-aligned slots and routes each role to specialized experts for event semantic modeling. Extensive experiments on three benchmark datasets demonstrate that ATSR achieves state-of-the-art performance with competitive efficiency, while remaining effective in few-shot settings.

Abstract:
Multimodal large language models (MLLMs) have recently shown strong potential as point-wise rerankers by directly modeling query–document relevance through next-token prediction. However, point-wise reranking suffers from substantial repeated computation across query–document pairs, while the causal structure of transformers allows only prefix segments to be reused via pre-caching. To address the misalignment of existing query-first and document-first formats with both VQA-style prompting and computation-aware reuse, we propose a vision-first formulation that improves both cache reuse efficiency and reranking performance. However, the remaining cost is still considerable and stems from three main sources: (1) model depth, for which we reduce active parameters via early exit; (2) cross-segment attention, which we restrict to a narrow interaction band across a few layers; and (3) visual tokens, where we reduce the number of tokens via embedder-guided pruning. Together, these designs form miniReranker, which reduces reranking runtime to <1% of the dense implementation under high-reuse settings for a single query, while preserving >96% of the dense model performance.

Abstract:
Reliable evaluation of open-ended question answering remains a bottleneck for measuring answer correctness of modern LLMs. Unlike multiple-choice tasks, free-form answers may be correct in many surface forms and may fail in qualitatively different ways, including incompleteness, contradiction, overgeneration, and endorsement of false premises. Existing judgment-based and similarity-based metrics often collapse these distinctions. We address this gap with three reusable contributions. First, we introduce a semantic correctness taxonomy that assigns open-ended answers to eight ordered classes, separating verbose-but-correct answers from those contaminated by hallucinated content. Second, we release CAP-Correctness, an 8.8k-example benchmark spanning widely used QA datasets, and CAP-Statements, an 11k-example dataset for converting question-answer pairs into declarative statements for natural language inference (NLI) training and statement-based evaluation. Third, we introduce CAP (Context-Aware Precision), a reference-based metric that scores question-conditioned statements using bidirectional NLI. Under a monotonicity protocol testing whether metrics respect the taxonomy’s intended ordering, CAP outperforms established baselines.

Abstract:
Large language models can generate runnable web applications from natural-language requirements, but many generated applications still fail interactive tasks. Existing generate-execute-repair pipelines execute the generated application and use task outcomes or error messages to guide code revision. However, this feedback often misses the runtime behavior between a browser action and the final task outcome, making interaction-level failures difficult to diagnose. Therefore, we propose TeleGen, an observability-enhanced framework for LLM-based web application generation. TeleGen instruments generated applications, collects runtime telemetry during task execution, and compresses raw telemetry logs into concise briefs for repair. We evaluate TeleGen on WebGen-Bench and Web-Bench. On WebGen- Bench, TeleGen improves task success from 67.7% with repair without telemetry to 76.2%, an increase of 8.5 percentage points. On Web-Bench, it improves cumulative Pass@2 from 21.7% to 29.8%. Ablation results show that runtime telemetry provides a useful diagnostic signal, while telemetry briefs make this signal more effective and less costly to use. Further analysis shows that telemetry is especially helpful for failures involving hidden execution paths, such as navigation, form workflows, and frontend-backend coordination.

Abstract:
Audio-Visual Speech Recognition (AVSR) systems typically consume only the final-layer representation of their modality-specific encoders. In self-supervised (SSL) encoders, lower and upper layers encode acoustic and more abstract linguistic cues respectively, so the most informative layer may differ from token to token. We propose Mixture of Layer-wise Tokens (MoLT), which aggregates the per-layer outputs of the AV-HuBERT encoder per token through an MoE-style router. A sparse top-k MoLT (S-MoLT) selects the most useful layers per token, and Dynamic routing-based MoLT (D-MoLT) replaces the fixed k with a learnable per-token threshold. To align routing with recognition difficulty, we further add a token-difficulty-aware lower-upper preference that biases, rather than overrides, the learned routing: Difficulty-adaptive D-MoLT (DD-MoLT) derives the difficulty signal from one modality, while Cross-modal DD-MoLT (CDD-MoLT) uses the audio-video reliability gap. On LRS3, our models consistently improve WER over a strong MMS-LLaMA baseline across SNR conditions, with the gain widening under severe noise (e.g., 13.85 vs. 17.10% at−15 dB), at negligible overhead.

Abstract:
Search agents achieve strong question-answering performance through multi-turn interactions with search engines, with Group Relative Policy Optimization (GRPO) being a widely used training algorithm. However, GRPO-style algorithms still face several challenges in multi-hop search settings. First, correct intermediate steps are often penalized when the final answer is wrong. Second, training is highly unstable, often causing degradation of natural language ability or even catastrophic training collapse. Our analysis attributes these issues to coarse-grained advantage assignment and an imbalance between positive and negative advantages. To address these problems, we propose CalibAdv, an advantage calibration method specifically designed for search agents that enables more accurate and more stable modeling of penalties and rewards. Specifically, CalibAdv leverages the correctness of intermediate steps to downscale excessive negative advantages at a fine-grained level. It then further rebalances positive and negative advantages to improve training stability. Importantly, CalibAdv adopts a lightweight design that calibrates advantages from standard rollout signals, making it simple and easy to deploy. Extensive experiments across three models and seven benchmarks demonstrate that CalibAdv improves both model performance and training stability. Our code is available at https://github.com/wujwyi/CalibAdv.

Abstract:
We introduce a new inference test for audio LLMs, where the correct response crucially depends on the location of a focal accent. The proposed task shows for the first time how to test the ability of LLMs to incorporate audio information in truth-conditional semantic interpretation, relying on the theory of alternative semantics. Several native audio models are tested, with performance ranging from random guessing to around 90% accuracy. The task is challenging for all the models tested. A detailed error analysis reveals specific areas of difficulty. The theory of alternative semantics provides a basis for a wide range of systematic tests of audio models.

Abstract:
Understanding long videos with multimodal large language models (MLLMs) requires selecting a compact set of frames from thousands of candidates, yet identifying the right frames seemingly requires understanding the video first. We resolve this circular dependency with a simple observation: cross-modal attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation. We exploit this property to build a training-free frame selector. A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer attention into relevance scores through query-conditioned aggregation. This enables cross-frame comparison without autoregressive decoding. To handle the selector’s own context constraint, we formulate the joint allocation of candidate pool size and per-frame token budget as a discrete optimization problem solved by dynamic programming. Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on Video-MME and outperforms prior training-based selectors under matched frame budgets, while generalizing across selector, answerer backbones, and tasks without retraining.

Abstract:
Direct Preference Optimization (DPO) simplifies alignment through pairwise comparisons but assumes all observed preferences are reliable. Real data often violates this assumption, leading to reversed, weak, or ambiguous labels that cause harmful policy updates. To address this, we propose Posterior Label Correction DPO (PLC-DPO) to robustly optimize preferences by routing each pair’s training signal as a clean, flip, or tie case. The key idea is to use the calibrated policy-reference margin as online evidence to take appropriate correction actions. This reframes noisy preference learning as actively correcting supervision direction and strength rather than merely filtering suspicious examples. Across 57 dataset-model-benchmark cells, PLC-DPO obtains the best mean win rate against DPO (60.5 vs. 55.5 for the next-best method). Injected-noise and tie stress tests, human disagreement analysis, and self-confirmation diagnostics further show that the routing remains stable and distinguishes flipped from weakly directional pairs.

Abstract:
Performing deliberate mathematical reasoning in visual contexts is a hallmark of expert-level Multimodal Large Language Models (MLLMs) and requires a sophisticated synthesis of perceptual grounding and symbolic logic. However, in the realm of mathematical functions, our investigation reveals a critical modality interference phenomenon: even advanced models, while performing textual computational reasoning, tend to disregard or misinterpret essential visual cues. To address this challenge, we propose Func-R1, which synergistically harmonizes precise visual perception and rigorous logical reasoning. Concretely, built upon an explicitly decoupled architecture, we employ a hierarchical post-training framework to progressively identify critical visual evidence and conduct in-depth theoretical reasoning. Furthermore, the Perception-Aligned Theoretic Optimization (PATO) strategy is proposed to steer policy updating towards internalizing fundamental theoretical properties while dynamically rectifying heterogeneous visual information throughout the reasoning process. Extensive experiments across diverse benchmarks demonstrate that Func-R1 delivers the optimal performance among open-source MLLMs, even surpassing GPT-5 with an 8.4% improvement on MathVerse’s function-oriented tasks.

Abstract:
Large language models are increasingly applied to high-risk domains such as law, yet complex legal reasoning remains limited by two structural challenges. First, existing RAG and GraphRAG methods emphasize lexical or semantic similarity while overlooking normative relations among legal provisions. Second, vanilla Chain-of-Thought prompting may generate plausible rationales without enforcing the normative structure of legal reasoning. To deal with the bottleneck of pipelines in the legal reasoning domain, we propose LEGO, a dual-module framework that synergizes Legal Expert GraphRAG and expert Chain-of-thought for complex legal reasoning. ExpertGraphRAG uses an expert-annotated civil code graph encoding these normative relations with a greedy normative-coverage retrieval algorithm to dynamically extract instance-specific provision subgraphs, while ExpertCoT organizes the retrieved provisions and case facts into structured Provision-Fact-Conclusion reasoning. With a Qwen3-8B backbone, LEGO achieves 40.53% exact-match accuracy on LawExamQA_Civil, outperforming the evaluated RAG and CoT baselines and performing comparably to the evaluated larger models, while remaining robust on multi-hop questions. It also achieves the best results among the evaluated baselines on the open-ended benchmarks. Ablation studies confirm the individual and complementary contributions of both modules, demonstrating LEGO’s effectiveness in improving LLMs’ complex legal reasoning ability. Code and dataset can be found in the link: https://github.com/BLK-WHT/LEGO

Abstract:
Tool-augmented language models ground their responses in external tool outputs, improving reliability beyond parametric knowledge alone. However, even when the correct tool is called and returns accurate information, models may still contradict such outputs, omit critical details, or generate unsupported claims. Existing hallucination benchmarks for tool-augmented dialogue primarily focus on the tool-calling stage, evaluating whether the correct tool and arguments are selected, while overlooking errors that arise during post-tool response generation. To address this gap, we introduce , a span-level benchmark for hallucination detection in tool-augmented LLM responses, covering five fine-grained hallucination categories. Through extensive experiments with modern hallucination detectors, we show that post-tool-call hallucinations are widespread and remain challenging to detect. Our results further demonstrate that effective detection requires training on specialized tool-augmented corpora, while transfer from tool-agnostic hallucination datasets performs poorly in this setting. These findings highlight the need for dedicated benchmarks and models for reliable hallucination detection in tool-augmented language systems.

Abstract:
Understanding biomedical experiments provides a foundation for downstream tasks, e.g., laboratory automation, and facilitates effective cross-disciplinary communication. Two challenges, High Information Density (HID) and Multi-Step Reasoning (MSR), pose unique difficulties for precise automatic experimental understanding. Extracting structured knowledge, e.g., Knowledge Graphs (KGs), is an effective approach to address the HID and MSR. However, existing biomedical datasets for structured knowledge Information Extraction (IE) are limited to a general or coarse-grained level, hindering fine-grained experimental understanding. To address this gap, we introduce Biomedical Protocol Information Extraction Dataset (BioPIE), a dataset providing procedure-centric KGs that captures entities, actions, and relations at a scale sufficient for reasoning across biomedical protocols. We evaluate both supervised and LLM-based IE methods on BioPIE to verify its effectiveness, and implement a biomedical question answering system to provide a quantitative illustration of BioPIE’s effectiveness for downstream understanding tasks. The experimental results demonstrate improved understanding performance on both the HID and MSR question sets.

Abstract:
Humor generation is a challenging task in natural language generation that requires fluency, surprise, and cultural grounding. Although large language models (LLMs) show promise in English, little work addresses low-resource and culturally specific humor, such as in Korean. We propose K-Humor, a dynamic multi-agent framework for generating Korean jokes. Unlike static one-shot models, K-Humor integrates specialized agents for cultural awareness, audience simulation, editing, and punchline creation, which are coordinated through a dynamic iterative refinement loop. A human-data-grounded mechanism further guides the punchlines using authentic Korean jokes. Evaluation with automatic metrics, agent-internal diagnostics, LLM-as-a-judge, and human ratings shows that K-Humor consistently outperforms a single-prompt baseline and ablations in criteria such as fluency and cultural appropriateness. Although human-written jokes remain stronger overall, particularly on core creative dimensions, K-Humor demonstrates that dynamic, culturally grounded multi-agent systems can improve humor generation over standard LLM baselines in low-resource settings. Code is available at https://github.com/dxlabskku/k-humor.

Abstract:
Mamba language models process sequences recurrently, but recurrent processing does not imply that all tokens are retained equally. We study this gap using structured recall tasks that expose primacy, recency, and lost-in-the-middle behavior. Our results suggest that Mamba uses two functional memory regimes: a short-lived state that supports recency, and a sparse long-lived channel subspace that preserves selected early inputs. Targeted interventions on this long-memory subspace selectively degrade early recall and transfer to long-context QA and summarization, indicating that the same channels support real long-context use. We then ask why middle items are forgotten despite being processed sequentially. We find that the central failure is a short-to-long memory transition bottleneck: middle items are no longer recent enough for short memory, but are often not promoted into the sparse long-lived subspace. Repeated input structure strengthens this bottleneck by changing the input-dependent discretization dynamics. These findings provide a mechanistic account of Mamba’s U-shaped recall curve as the interaction between long memory, short memory, and a fragile transition between them.

Abstract:
LoRA fine-tuning is standard for adapting LLMs to reranking, but it remains unclear where in the network task-specific relevance behavior is learned and what attention-level changes accompany that learning. Through ablation and attention experiments, we identify where LoRA attention updates to RankLLaMA improve performance and whether those gains coincide with interpretable relevance-oriented attention patterns such as lexical matching, rarity sensitivity, and query-document interaction. We find that given LoRA fine-tuned MLPs throughout the network, restricting LoRA attention updates to a compact mid-network region is sufficient for recovering over half of the performance gained by applying LoRA to all attention layers, and that omitting attention fine-tuning in this region hurts performance more than elsewhere in the network. Additionally, we show that regions where applying LoRA affects performance the most overlap with regions where fine-tuning increased attention to axiomatic IR features. Rarity sensitivity, document-query interaction, and several compositional features are highly correlated with gains in ranking performance. Our results support an interpretable, correlational account of how relevance-oriented behavior emerges during LoRA fine-tuning and point toward improved strategies for adapting rerankers.

Abstract:
The widespread use of human-like text from Large Language Models (LLMs) necessitates the development of robust detection systems. However, progress is limited by a critical lack of suitable training data; existing datasets are often generated with outdated models, are predominantly in English, and fail to address the increasingly common scenario of mixed human-AI authorship. Crucially, while some datasets address mixed authorship, none provide the character-level annotations required for the precise localization of AI-generated segments within a text. To address these gaps, we introduce LLMTrace, a new large-scale, bilingual (English and Russian) corpus for AI-generated text detection. Constructed using a diverse range of modern proprietary and open-source LLMs, our dataset is designed to support two key tasks: traditional full-text binary classification (human vs. AI) and AI-generated interval detection, facilitated by character-level annotations. We believe LLMTrace will serve as a vital resource for training and evaluating the next generation of more nuanced and practical AI detection models.

Abstract:
LLM evaluations using clinician-authored triage vignettes have reported substantial under-triage under constrained multiple-choice testing. Yet model performance on the same clinical cases can change when responses are generated in free text. We test whether this format effect appears while the case is processed or when clinical information is mapped to the final answer. Using sparse-autoencoder (SAE) features in Gemma 3 4B/12B IT and Qwen3-8B, we find that medical features fire on the shared clinical narrative under both formats but are inactive at the multiple-choice decision token. Emergency-tier information is linearly decodable from vignette representations with ROC-AUC 0.95–1.00 under both formats, with no significant format difference, but is attenuated at the decision token. Natural-language autoencoder verbalization and top-feature characterization associate that token with the multiple-choice scaffold. In a direct linear projection, the identified medical features contribute zero, whereas scaffold-peaking features account for over 91% of unsigned attribution in both Gemma models. Behaviorally, whether multiple choice improves or worsens performance depends on the model. Option-order shuffles rule out simple positional bias, and cases that differ between formats are usually one severity tier apart. Together, these findings place the strongest correlates of the format effect at answer selection while leaving open whether unmeasured clinical representations also differ.

Abstract:
Understanding anomalies in real-world videos requires reasoning about when events occur, where they happen, and what actions unfold. Existing benchmarks, however, largely ignore one or more of these dimensions, making it difficult to evaluate whether multimodal large models (MLLMs) truly understand abnormal events beyond simple detection. To address this gap, we introduce Anom3W , a benchmark designed to evaluate MLLMs across temporal, spatial, and semantic dimensions . Our semi-automatic pipeline consolidates heterogeneous anomaly resources into verified spatio-temporal annotations, enabling rigorous, reasoning-oriented evaluation. By treating spatial boxes and tubes as evidence cues, Anom3W challenges models to reason about entities, locations, and behaviors, rather than merely flag anomalies. Experiments demonstrate that even state-of-the-art MLLMs struggle, highlighting the critical importance of spatial grounding and semantic understanding. Anom3W establishes a new standard for interpretable, context-aware video anomaly reasoning.

Abstract:
The American Sign Language (ASL) lexicon contains heterogeneous sign types, including established signed words, neologisms, ad-hoc depictions of real-world phenomena, and fingerspelled words. Productions within each type exhibit unique phonotactics and substantial variability, potentially confounding models pretrained on downstream ASL tasks. To survey the capacity for representation collapse, we isolate and annotate 14 types of ASL productions as they appear in science, technology, engineering, and math (STEM) educational content. We then probe seven pretrained models’ latent representations of the annotated clips, reporting collapse metrics with respect to domain, sign type, and meaning. Our findings show that out-of-distribution signs are more frequent in STEM content, and that pretrained models lose information about their visual and semantic contents in varing ways. Among the tested models, those which encoded the phonological properties of signs generalized to STEM data more effectively, while English- and gesture-based models can exhibit more brittleness. Future work can design interventions to mitigate the kinds of collapse documented here or extend this survey by evaluating additional models and datasets.

Abstract:
Latent Visual Reasoning (LVR) unfolds reasoning in a continuous visual embedding space, markedly improving Vision-Language Models (VLMs) on vision-intensive tasks. However, our experiments reveal a hidden cost: LVR inevitably undermines VLMs’ strong textual reasoning ability, causing substantial degradation on logic-intensive tasks. This motivates us to unify both reasoning modes within a single VLM that adaptively selects the appropriate one. We propose MoLaT (Mixture of Latent and Textual Reasoning), a novel method that trains VLMs via reinforcement learning (RL) to select LVR and text-only reasoning adaptively. MoLaT is trained in two stages. Supervised fine-tuning endows the VLM with a dual-mode capability using mixed CoT data. Then, MoLaT introduces two complementary designs: 1) The mode-adaptive advantage decouples the mode-selection signal from per-rollout correctness reward at the token level: mode-selection tokens receive dedicated cross-mode gradients, while content tokens retain the group-relative advantage. 2) The entropy-modulated advantage weight uses sequence-level entropy as a difficulty proxy, modulating signal strength to prevent premature mode commitment. Experiments confirm that MoLaT improves both vision-intensive and logic-intensive abilities, validating adaptive mode selection as key to general-purpose reasoning VLMs.

Abstract:
Large language models remain fragile against malicious fine-tuning, motivating training-time defenses against harmful persona drift. Preventative Steering injects undesirable-trait persona vectors during fine-tuning and removes them at evaluation time, yet the mechanism behind its lasting protection remains unclear. Analyzing its temporal optimization dynamics, we find that the defense emerges from an early compensatory adaptation phase followed by a steady-state phase where the corrective signal decays; in parameter space, attention output projections emerge as the dominant residual-write route for defensive updates. Through Intervention Delta Preservation (IDP) and IDP Continuation experiments, we further show that preserving or reinjecting the weight offset fails to maintain protection, indicating that preventative steering relies on active adaptation rather than a static defense. Motivated by this finding, we propose Progressive Intensity Scheduling (PIS), which starts with a moderate injection strength and increases it after static-strength alignment begins to decay. Across the evaluated Qwen2.5 and Gemma-3 models, PIS improves safety robustness over static-strength steering while reducing harmful trait expression.

Abstract:
The widespread misuse of large language models (LLMs) has made machine-generated text detection (MGTD) a critical task. Existing detection methods are often vulnerable to stylistic noise, which limits their generalization in real-world scenarios and robustness on short texts. Furthermore, many methods generate multiple auxiliary texts through LLM-based rewriting or similar strategies, substantially increasing detection latency and limiting their practicality. To address these challenges, we propose the DisGAR detection framework, which strips stylistic noise from the input text to mitigate its impact on detection performance, thereby improving cross-domain and cross-model robustness. Specifically, we first fine-tune the attention layers of the proxy model to capture generation-specific patterns. Then we further introduce a global artifact aggregation mechanism and a contrastive learning strategy based on rewritten texts, enabling the separation of stylistic artifacts from the underlying generation patterns. Experimental results demonstrate that DisGAR outperforms existing state-of-the-art methods in unseen-model and unseen-domain detection. Moreover, DisGAR achieves detection efficiency comparable to Fast-DetectGPT, successfully balancing detection performance and efficiency. Source code is available at https://github.com/m-Clown/DisGAR.git

Abstract:
Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Current unlearning benchmarks include mainly single-hop questions and a narrow set of multi-hop questions. Although effective, they still face two challenges. (1) Knowledge is not isolated, whereby diverse multi-hop reasoning paths can potentially induce knowledge leakage than normal queries. (2) Unlearning may be fragile: unlearned knowledge can be partially recovered through recovery attacks such as lightweight post-unlearning adaptation, making static evaluation insufficient. Therefore, in this paper, we introduce Leak-resistant Unlearning as a novel benchmark to understand robust LLM knowledge removal across diverse reasoning paths and recovery attacks. We experiment with this benchmark on 3 models, 6 unlearning methods, and 2 carefully curated datasets. Results show that existing methods are vulnerable to multi-hop reasoning paths and recovery attacks. We further explore the trade-off among forget quality, robustness, and model utility for LLM unlearning.

Abstract:
LLM-as-a-Judge has become the dominant evaluation paradigm for many natural language generation tasks (albeit mostly in English) due to shortcomings of conventional metrics and high correlations with human judgment. There are now attempts to extend LLM-as-a-Judge to multilingual settings including low-resource languages. However, LLMs have limited proficiency in low-resource languages, and there is often no adequate human validation in these settings. To highlight the scope of the problem and current practices, we explore the use of LLM-as-a-Judge evaluators in ACL Anthology papers focusing on multilingual settings and low-resource languages across a diverse set of tasks. Out of 650 papers mentioning LLM-as-a-judge, only 33 of them focus on low-resource or multilingual settings. Our in-depth analysis of these papers indicates inconsistent evaluation outcomes, a tendency to overtrust LLM judgments in multilingual settings, and the widespread reliance on a single judge model per study. To help the NLP community further, we conclude with a checklist of recommendations about how to use LLM-as-a-Judge in multilingual and low-resource settings.

Abstract:
Training data attribution is increasingly used to filter, reweight, and prioritize rollouts in reinforcement learning with verifiable rewards (RLVR). However, these interventions often assume that individual samples, transitions, or trajectories can be attributed independently of the other data used in the same update. We show that this assumption is not merely a property of the attribution estimator, but a structural property of the training objective. We recast training data attribution as individual treatment effect estimation and use the Stable Unit Treatment Value Assumption (SUTVA) to determine when a proposed attribution unit is well-defined. This yields a simple criterion that attribution is only valid at the level where the algorithm assigns a stable contribution to that unit. Applying this criterion to RLVR, we show that the PPO objective admits trajectory-level attribution, whereas naive trajectory-only attribution is not well-defined for GRPO unless peer context is modeled. Experimental results support this distinction and provide preliminary evidence for group-level attribution under GRPO.

Abstract:
Despite the success of large language models (LLMs), their potential in Traditional Chinese Medicine (TCM) remains underexplored due to scarce high-quality data and the multimodal nature of TCM diagnostics, which involve looking, listening, smelling, and pulse-taking. These modalities challenge conventional LLMs. To address these challenges, we present ShizhenGPT, the first multimodal LLM tailored for TCM. To overcome data scarcity, we curate the largest TCM dataset to date, comprising 100GB+ of text and 200GB+ of multimodal data, including 1.2M images, 200 hours of audio, and physiological signals. ShizhenGPT is pretrained and instruction-tuned to achieve deep TCM knowledge and multimodal reasoning. For evaluation, we collect recent national TCM qualification exams and build a visual benchmark for Medicinal Recognition and Visual Diagnosis. Experiments demonstrate that ShizhenGPT outperforms comparable-scale LLMs and competes with larger proprietary models. Moreover, it leads in TCM visual understanding among existing multimodal LLMs and shows promising results across modalities such as sound, pulse, electronic-nose signals, and vision, paving the way toward more holistic multimodal perception in TCM. We hope that the datasets, models, and methods of this work will inspire further exploration in this field. Code, models, and released data are available at https://github.com/FreedomIntelligence/ShizhenGPT.

Abstract:
Parallel decoding improves generation efficiency by processing multiple decode positions within a single decode forward, but reported speedups conflate algorithmic token utilization with the system cost of executing multiple positions. We isolate the system side by introducing Near-Free Parallelism (NFP), the maximum number of positions executable at near-free latency. Analyzing Dense FFNs, MoE FFNs, and Attention against an idle-compute baseline, we find that NFP is shaped not by memory-bound resource slack alone, but also by implementation-induced kernel-granularity slack. Based on these mechanisms, we establish a Near-Free Parallelism principle that predicts the NFP boundary from hardware balance and implementation granularity. Validation on representative Dense and MoE models—spanning both diffusion and autoregressive decoding—shows that the principle accurately predicts practical NFP boundaries, revealing that the standard idle-compute intuition over-predicts them by 2.4–23x in over half of the evaluated deployment settings—offering a system-side budget for parallelism selection and model-system co-design.

Abstract:
High-quality representations are a core requirement for effective recommendation. In this work, we study the problem of LLM-based descriptor generation, i.e., generating keyphrase-like natural language item representations with minimal constraints on downstream applications. However, open-ended generation provides little control over the generation space, leading to high cardinality, low-performance descriptors that render downstream modeling challenging. To this end, we propose AgenticTagger, a framework that queries LLMs to represent items with text descriptors drawn from an automatically mined vocabulary. AgenticTagger features two core stages: (1) a vocabulary-building stage in which a set of hierarchical, low-cardinality, and high-quality descriptors is identified, and (2) a vocabulary-assignment stage in which LLMs assign in-vocabulary descriptors to items. To ensure the vocabulary effectively describes the item corpus of interest, we design a multi-agent reflection mechanism in which an architect LLM iteratively refines the vocabulary guided by parallelized feedback from annotator LLMs that validate the vocabulary against item data. Experiments on public and private datasets show AgenticTagger brings consistent improvements across diverse recommendation scenarios, including generative retrieval, ranking, and critique-based recommendation.

Abstract:
Large language models frequently memorize specific factual associations from their training data, necessitating post-hoc removal of targeted knowledge. However, standard gradient-based unlearning techniques treat all parameter directions uniformly, often leading to catastrophic forgetting of general capabilities. We introduce ERASE (Eigenspace-Regulated Adaptive Selective Erasure), a geometry-aware unlearning framework that combines three complementary mechanisms: (1) a spectral filter that attenuates gradient updates along principal directions of retained model knowledge, confining forgetting pressure to low-interference subspaces; (2) selective error-signal construction, which restricts gradients to semantically meaningful token positions, avoiding diffuse collateral damage; and (3) an adaptive Cross-Entropy Target (CET) loss with dynamic per-sample stopping, which drives each forget example toward a target cross-entropy and halts on convergence, preventing both under- and over-forgetting. Our method requires no reference model, no retain-set loss, and no auxiliary task supervision during training. On TOFU within the OpenUnlearning evaluation framework, ERASE achieves the highest aggregate score across four model families and two forget splits, attaining the best privacy and near-original utility among all compared baselines; on the document-level MUSE-News benchmark, with forget spans recovered automatically via named entity recognition, it simultaneously attains the strongest forgetting and retention with the privacy score closest to the retrain oracle. Our code is publicly available at https://github.com/ItzikWaizman/ERASE.

Abstract:
While diffusion models produce high-quality images, they often lack consistency in sequences, especially for non-linear visual sequences. To solve this, we introduce an inference-time scaling beam search strategy for latent space exploration, enabling the conditional generation of coherent image sequences through beam search diffusion. In contrast to earlier methods that rely on fixed latent priors, our method dynamically samples past latents to search for an optimal sequence of latent representations, ensuring coherent visual transitions. As the latent denoising space is explored, the beam search graph is pruned with a cross-attention mechanism that efficiently scores search paths, prioritizing alignment with both textual prompts and visual context. Human and automatic evaluations confirm that BeamDiffusion outperforms other baseline methods, producing full sequences with superior coherence, visual continuity, and textual alignment.

Abstract:
Natural language understanding often depends on meanings that are implied rather than explicitly stated, requiring pragmatic reasoning. Despite strong performance on math and logical reasoning, large language models (LLMs) still trail humans on pragmatic inference, often defaulting to literal interpretations. To improve LLM pragmatic reasoning, we introduce PragReST, a self-supervised framework that constructs pragmatic QA data, generates counterfactual reasoning traces, and trains models to internalize them through supervised fine-tuning and reinforcement learning, without human-labeled training data or distillation from a stronger teacher. Across four pragmatic benchmarks (PRAGMEGA, LUDWIG, METOQA, and ALTPRAG) and multiple model families, including Qwen3, Gemma, and GPT-OSS, PragReST consistently improves over backbone models, task-specific pragmatic tuning baselines, and non-counterfactual variants of the same pipeline. On accuracy-based benchmarks, PragReST improves over the instruct backbone by 5.37% and 5.50% (absolute) for Qwen3-8B and Qwen3-14B, respectively. Our error analysis and ablations underscore the importance of counterfactual reasoning: PragReST primarily reduces errors caused by failures to contrast observed utterances with plausible alternatives, and removing counterfactual reasoning substantially reduces performance. Moreover, our training preserves out-of-domain performance on general-knowledge and mathematical reasoning benchmarks.

Abstract:
Building effective AI systems increasingly depends on writing high-quality task requirements, yet users often struggle to articulate the constraints, preferences, and edge cases that determine success. This problem is especially acute in AI development, where behavior is shaped not only by human expectations but also by data characteristics. We present AREAs-Lab, an interactive environment for AI-driven Requirement Elicitation for AI systems. In AREAs-Lab, an assistant iteratively refines an initially incomplete requirement by analyzing the underlying dataset and asking targeted clarification questions to uncover the user’s latent intent. To study this setting systematically, we construct a synthetic benchmark grounded in 16 public datasets spanning diverse domains and task types. Each benchmark instance includes a user profile, a complete reference requirement, and an intentionally underspecified version that serves as the assistant’s starting point. We further introduce an automated evaluation pipeline based on an AI-simulated user that reveals hidden information only when appropriately prompted, enabling scalable and reproducible assessment of interactive elicitation quality. AREAs-Lab provides a controlled testbed for studying how AI assistants can transform vague user goals into actionable requirements for AI systems.

Abstract:
Multilingual large language models (mLLMs) achieve strong performance in machine translation, yet our understanding of the mechanisms by which they transform representations from one language to another remains incomplete. Prior work suggests that translation decomposes into separable processes within an mLLM, where conceptual content is first represented independently, followed by a production into language-specific form. In this work, we show that translation is even more modular than previously assumed and that the output language production in translation processes is actually further separable into a syntax and a surface language process. We construct controlled multilingual datasets that isolate cross-linguistic differences in word order and use causal interventions and probing to track how representations are transformed during translation. We find that models first construct target-side word order before realizing the target language surface form. We identify individual attention heads that are selectively sensitive to syntactic transformations while remaining largely invariant to language identity. These results establish the commitment to a syntactic structure as an independent stage in translation, extending prior decompositions and showing how translation is implemented by functionally different components within mLLMs.

Abstract:
Millions turn to AI companion chatbots during loneliness, grief, and personal crises. How these companion platforms respond in such moments can shape the trajectory of a user’s vulnerable state. Yet existing model audits evaluate reactions to pre-defined crisis prompts and miss the response policy that governs sustained real-world interaction. We address these gaps with two key contributions. First, we introduce the AI Companion Vulnerability-Response Taxonomy, a grounded, paired taxonomy of user vulnerability and chatbot response designed for analyzing extended companion chatbot interactions. Second, we apply Maximum Causal Entropy Inverse Reinforcement Learning to ~47k turns of real-world user conversations with GPT-4.1, Character.AI, and Replika to infer each platform’s short-horizon response policy: the probability of each response category given the user’s current vulnerability state. Our findings reveal distinct response profiles of AI companions in conversations with vulnerable users: GPT-4.1 reaches for advice, Character.AI spreads its response across different strategies, and Replika consistently asks questions and stays present. Over four weeks of repeated interaction, GPT-4.1 asks progressively fewer follow-up questions when users are distressed and increasingly sets boundaries or refers users out rather than pushing back. Within each platform, exploratory comparisons across user groups suggest that response policies also differ with users’ pre-existing psychological risks and their bonds with the companion. Estimated model response policies are invisible to shallow behavioral audits, providing a new lens for auditing chatbots in the wild and enabling more realistic safety evaluation.

Abstract:
When retrieved evidence contradicts an agent’s prior beliefs, does it revise its answer, acknowledge uncertainty, or persist with an incorrect conclusion? Existing evaluations of agentic systems focus primarily on task success, offering limited insight into how agents handle such conflicts. We propose to evaluate agents on epistemic humility (EH): the agent’s willingness to recognize, act on, and communicate uncertainty during task execution. We operationalize EH through three trajectory-level behavioral dimensions: _Identify_, _Solve_, and _Escalate_ (ISE). Through _knowledge conflict_, situations where the backbone language model’s parametric knowledge contradicts the evidence it encounters, or where two contextual sources disagree, we build a two-level evaluation framework: (1) controlled single-step conflicts, and (2) naturally occurring conflicts during multi-step agentic executions, each level paired with matched no-conflict controls. Evaluating four agents, we find an inverse relationship between task accuracy and epistemic humility: the most accurate agents are often the most likely to give overconfident answers despite recognizing conflicts or risks. Trajectory-level analysis further reveals that agents frequently detect conflicts in early steps of execution but fail to maintain or resolve them in later steps. Finally, we show that Model-level interventions can improve EH, but often at the cost of task accuracy, suggesting that epistemic humility emerges from the interaction among the backbone model, agent harness, and evaluation environment.

Abstract:
AI translation of literary works is increasingly common. While the content may be rendered adequately, we do not know enough about how readers experience it in terms of immersiveness and literary effect—aspects poorly captured by automatic metrics or human evaluation targeting fluency and adequacy. We ask 15 avid readers to compare recently published human translations (HT) to machine translations (MT) generated with an agentic language model-based pipeline, for 15 recent novels in French, Polish, and Japanese translated into English. Readers evaluated approximately 8K-word excerpts in two conditions: immersive reading of the whole excerpt (30 comparisons) and close reading of 386 aligned HT–MT chunk pairs (772 comparisons), with two readers per book. Overall, readers find MT “fine”, but prefer HT (descriptively at the excerpt level, 19/30, and significantly at the chunk level, 522/772) for its ease, clarity, and immersive nature. Readers’ highlights show that MT’s quality varies more within one book than HT’s does. Crucially, readers cannot reliably tell the two apart (17/30 guess correctly) and tend to prefer the version they believe to be human. Automatic metrics, including LLM-as-a-judge approaches, fail to recover reader preferences and favor MT. We release LAIT (Literary AI Translation), a reader-centered evaluation dataset with 1K reader comments, 2K judgments and preference ratings, and 7.2K span-level annotations, along with our evaluation protocol and supporting interface.

Abstract:
Multiword Expressions (MWEs) are an important part of language competence which are both commonplace across languages and challenging for language learners to master. Since Grammatical Error Correction (GEC) systems tend to provide suggestions at the token level, they may fail to handle the semantic unity of MWEs. We present annotations of MWEs in learner English essays while identifying their intended spans. This not only allows us to analyse learner MWE errors in more detail for the first time, but also enables us to build GEC systems that provide more meaningful feedback to learners. We have annotated 1,850 essays that were previously tagged for grammatical errors with various types of MWEs. We extend an error annotation toolkit with an MWE error-type and present a method that leverages these annotations to make GEC evaluation more explainable, enabling fine-grained analysis of MWE errors. Experiments with current GEC systems show that an edit-based model fine-tuned on our dataset achieves higher F0.5 scores for the correction of common MWE error types, demonstrating its effectiveness for handling MWEs.

Abstract:
Retrieval-augmented generation (RAG) systems depend critically on how documents are chunked and searched. Fine-grained chunks can improve retrieval precision but expand the search space, increasing latency and cost; larger chunks reduce the number of candidates but make dense similarity less reliable, as each representation mixes multiple topics and introduces more semantic noise. This trade-off becomes especially limiting in deep research tasks, where retrieval must be both fast and precise across large, heterogeneous corpora. We introduce MCompassRAG, a metadata-guided retrieval framework that uses topic-level signals as a semantic compass for selecting relevant evidence. Instead of relying only on cosine similarity between queries and noisy chunk embeddings, MCompassRAG enriches chunk representations with topic metadata defined in the same embedding space and trains a lightweight retriever through LLM-teacher distillation. At inference time, MCompassRAG performs topic-aware retrieval without additional LLM calls, improving both efficiency and evidence quality. Across six complex retrieval benchmarks, MCompassRAG improves information efficiency by 8.24% on average with over 5x lower latency than the strongest efficient RAG baselines.

Abstract:
Retrieval-augmented generation (RAG) systems must balance retrieval granularity with contextual coherence, a challenge that existing methods address through LLM-guided chunking, single-level context expansion, or hierarchical summarization. These approaches variously depend on costly LLM calls during indexing or retrieval, limit context aggregation to a single granularity level, or introduce information loss through summarization. We present SproutRAG, an attention-guided hierarchical RAG framework that builds a binary tree over sentence-level chunks using learned inter-sentence attention from a sentence-level large language model (SLLM). To avoid proximity bias from uniform attention averaging, SproutRAG learns head- and layer-level aggregation weights that identify attention patterns most aligned with semantic document structure. At retrieval time, it uses hierarchical beam search over the tree to gather candidates at multiple granularities, capturing multi-sentence relevance beyond flat retrieval. The framework is trained end-to-end with a joint objective that improves both embeddings and tree structure, without requiring external LLM calls. Experiments across four benchmarks spanning scientific, legal, and open-domain settings demonstrate that SproutRAG improves information efficiency (IE) by 6.1% on average over the strongest baseline.

Abstract:
Large language models (LLMs) exhibit a range of human-like decision-making behaviors, but whether these reflect similar underlying mechanisms or surface-level mimicry remains unclear. We evaluate whether LLM context sensitivity aligns with a cognitive economic theory that explains human behavior through problem categorization and attention allocation. Across 12 open-source and commercial LLMs on a novel 140,000-trial product choice benchmark, context induces human-like shifts in choice and problem categorization, but does not reliably reweight attention between features like price and quality. Neither scale nor chain-of-thought reasoning reliably attenuates context sensitivity or generates human-like behavior. These results suggest that LLM decision mechanisms are distinct from human ones.

Abstract:
Recent advances in image and video generation have highlighted the need for intuitive, high-precision manipulation. However, traditional text-based prompts or 2D Visual Prompts (e.g., dragging, 2D trajectories) often suffer from unintuitive prompt engineering or depth ambiguity. To address these limitations, we introduce 3D Visual Prompting, a novel paradigm that significantly improves input intuitiveness for visual manipulation tasks. We develop a framework that integrates intuitive 3D constraints into 2D generative processes. Our approach projects 3D Visual Prompts onto 2D visual planes and leverages Multimodal Large Language Models (MLLMs) to verbalize intent, allowing plug-and-play use of existing 2D models. Furthermore, we incorporate In-Context Self-Refinement (ICSR) to summarize insights from self-correction into a meta-strategy. This enables high-precision 3D control in single-pass inference, eliminating the need for iterative computation. Experimental results demonstrate that our method significantly outperforms traditional 2D Visual Prompting for object translation and rotation. By utilizing pre-trained models without additional parameter tuning, our framework remains highly adaptable to future developments in generative models.

Abstract:
Steering vectors have rapidly emerged as a popular and effective method for guiding the output of LLMs in very specific ways. But constructing accurate steering vectors is a difficult manual process due to the opacity of embeddings. We introduce Hangman, a novel type of steering vector that operates using word senses, as well as AutoLexSteer, the first fully automated process for building steering vectors. AutoLexSteer employs families of closely-related words extracted from WordNet to specify both the steering source to be avoided and the desired steering target. The steering vectors are quite precise, can be used to steer at the level of words and sets of word senses (meanings), and are able to steer certain LLM behaviors like sycophancy. The dataset and code can be found at https://github.com/ShuheWang1998/autolexsteer .

Abstract:
Multimodal large language models (MLLMs) still struggle with spatial reasoning, particularly on tasks requiring consistent geometric understanding across multiple image regions. While recent works incorporate depth supervision into MLLMs, it remains unclear what forms of geometric learning are most beneficial for spatial reasoning transfer. In this work, we propose Multi-point Geometric Consistency Alignment (MGCA), a geometry-guided post-training framework that uses depth annotations as verifiable geometric feedback without requiring depth input at inference time. MGCA samples multiple image regions and optimizes the model through a unified geometric reward that jointly captures metric grounding and relative geometric consistency. Specifically, the reward combines a metric depth objective with a margin-aware ranking objective and is optimized via Group Relative Policy Optimization (GRPO). Experiments on multiple spatial reasoning benchmarks show that MGCA consistently improves spatial understanding and outperforms comparable open-weight MLLMs of similar scale. More importantly, our analysis reveals that optimizing relative ordering alone can achieve high ranking accuracy while failing to learn transferable geometric representations due to the lack of metric grounding. In contrast, jointly optimizing metric depth and relational consistency leads to substantially stronger spatial reasoning performance. These findings suggest that effective spatial alignment in MLLMs requires metrically grounded relational learning, rather than relation-only supervision.

Abstract:
Contextualized token-level embeddings from encoders, e.g., mBERT, are used for several cross-lingual tasks, e.g., machine translation (MT) evaluation and alignment tasks, but fine-tuning is required for the specific task settings. In contrast, multilingual sentence embeddings are trained under sentence-level objectives on parallel data, but their token-level representations remain unclear whether they carry cross-lingual semantics, although not explicitly trained with token-level objectives. In this paper, we investigate token representations extracted from multilingual sentence embedding models for MT evaluation and word alignment. Experimental results and analysis show that sentence embedding models can provide useful token-level representations for direct similarity-based tasks. Although the observed advantage is not consistent across all settings, the results indicate that multilingual sentence-level training may support token-level semantic similarity without explicit token-level training.

Abstract:
Multimodal Emotion Recognition (MER) systems often suffer from missing modalities in real-world scenarios. Existing methods usually generate, align, or distill missing modalities as a whole, overlooking the heterogeneous nature of the information carried by each modality. Such holistic treatment mixes inferable shared semantics with uncertain modality-specific details, yielding unstable representations and degrading robustness. To address this issue, we propose the Primitive Memory Distillation (PriMD) framework. Unlike existing methods, PriMD takes an intra-modal perspective and focuses on how different types of information within a modality differ in recoverability within each modality. PriMD first disentangles cross-modal shared semantics from modality-specific representations, and then discretizes the latter into learnable semantic primitives to construct modality-specific memory banks. When modalities are missing, PriMD is a teacher–student framework that the student model uses the shared semantics of available modalities as queries to dynamically retrieve primitives. It compensates for missing modality-specific information within a constrained memory space and aligns with the teacher model. Extensive experiments on IEMOCAP, CMU-MOSI, and CMU-MOSEI demonstrate that PriMD achieves state-of-the-art performance and consistently stronger robustness across a wide range of missing-modality settings, while mitigating the instability caused by holistic feature inference. Our code and project website are available at https://github.com/JiaqiZhang-Sengoku/PriMD and https://jiaqizhang-sengoku.github.io/PriMD/, respectively.

Abstract:
Diffusion language models generate text by denoising a masked canvas, but usually require the canvas length to be fixed before decoding. This leads to an allocation dilemma: short canvases may truncate outputs, while overlong canvases waste computation and introduce noisy continuations. Existing variable-length methods adapt the canvas through expansion or length search, but largely ignore whether such intervention is safe. We argue that canvas editing can perturb partially stabilized denoising trajectories, and propose Safe Adaptive Canvas Decoding (SACD), a training-free framework for conservative canvas control. SACD diagnoses allocation, estimates intervention risk, and chooses among expand, shrink, and keep. Experiments across reasoning, code generation, and infilling tasks show that SACD improves accuracy, latency, and token utilization across multiple DLM backbones, highlighting safe sparse intervention as a key principle for variable-length diffusion decoding.

Abstract:
A chain-of-thought (CoT) rationale can be coherent and answer-consistent while remaining causally unused. We call this phenomenon shortcut reasoning: the model appears to reason through CoT, but its answer does not depend on the expressed trace. We formalize shortcut reasoning as causal redundancy and propose a dual-disruption framework for measuring shortcut tendencies in CoT by testing two complementary aspects of causal explanation: necessity and sufficiency. We operationalize them as two behavioral signals: whether the answer changes under CoT disruption, and whether the answer can be recovered from the CoT after masking the original input. We instantiate this framework with CORD (Causal Redundancy of Reasoning via Disruption), an instance-level score that combines both signals. The two axes reduce complementary ambiguities in single-axis tests: answer preservation under CoT disruption can reflect shortcut use or redundant truthful reasoning, while early answer recovery can reflect shortcut use or a compact truthful rationale. Across 6 models and 2 datasets, CORD reveals a training-paradigm effect: R1-Distill models exhibit roughly twice the shortcut tendency of non-distilled models, while the two axes capture related but distinct signals. Together, these results motivate joint tests of necessity and sufficiency for detecting shortcut tendencies that can remain hidden behind correct answers.

Abstract:
Large Language Models (LLMs) remain unreliable for extremely low-resource languages whose writing systems conflict with high-resource pretraining priors. We study Bai-Chinese translation in the Bowen script, where Chinese characters are frequently borrowed phonetically rather than semantically, causing severe literal-translation hallucinations. We propose MAS - Bai , a linguistically informed adaptive multi-agent framework for Bowen-script Bai-Chinese translation. MAS-Bai routes each input through either a lightweight retrieval-translation path or a deeper philological path with etymological and syntactic agents, and uses cycle-consistency with a lightweight reverse translator as an inference-time reliability signal. We construct a 6,536-sentence Bai-Chinese corpus and evaluate on a 1,536-sentence test set. MAS-Bai achieves 16.23 BLEU and 62.15 COMET, improving over Standard RAG by 8.11 BLEU and 6.85 COMET. Forward LoRA, expert ablations and controlled routing experiments distinguish the roles of supervised adaptation, Bai-specific expertise and conditional routing. The largest gains occur on Bowen phonetic loans and divergent Bai word order, while human evaluation and reverse-verifier analyses support the automatic results.

Abstract:
Conditional crystal generation requires Large Language Models (LLMs) to produce Crystal Strings with discrete and continuous variables while satisfying composition, symmetry, and property constraints. Desired regions are often sparse in supervised data, while retraining for each objective preference is costly. We present a verifier-grounded framework combining label-free Test-Time Reinforcement Learning (TTRL) and Model Merging (MM). TTRL performs one batch-level, partially transductive adaptation per property using composition, symmetry, and learned band-gap rewards, without target reference structures. Property-specific experts are interpolated to navigate bounded symmetry–band-gap trade-offs without retraining intermediate policies. On MP-20, TTRL improves symmetry control, band-gap accuracy, and compositional fidelity over High-Precision SFT and Standard RL, and is competitive with diffusion and preference-aligned baselines. In a held-out high-band-gap region, TTRL generalizes to test prompts unseen during adaptation, reducing the mean squared error between surrogate-predicted and target band gaps by 52% relative to standard reinforcement learning under best-of-ten sampling. An independent band-gap predictor shows consistent gains, while density functional theory calculations further assess selected candidates. Similar gains with Qwen3.5-9B provide cross-backbone evidence. Model Merging recovers competitive two-objective trade-offs from two endpoint experts without additional RL, enabling efficient adaptation and preference control for structured scientific generation.

Abstract:
User simulators are essential for the scalable training and evaluation of interactive AI systems. However, existing approaches often rely on shallow user profiling, struggle to maintain persona consistency over long interactions, and are largely limited to English or single-domain settings. We present MUSE, a multi-domain Chinese user simulation framework designed to generate human-like, controllable, and behaviorally consistent responses. First, we propose Iterative Profile Self-Evolution (IPSE), which gradually optimizes user profiles by comparing and reasoning discrepancies between simulated trajectories and real dialogue behaviors. We then apply Role-Reversal Supervised Fine-Tuning to improve local response realism and human-like expression. To enable fine-grained behavioral alignment, we further train a specialized rubric-based reward model and incorporate it into rubric-guided multi-turn reinforcement learning, which optimizes the simulator at the dialogue level and enhances long-horizon behavioral consistency. Experiments show that MUSE consistently outperforms strong baselines in both utterance-level and session-level evaluations, generating responses that are more realistic, coherent, and persona-consistent over extended interactions.

Abstract:
Retrieval-Augmented Generation (RAG) mitigates factual hallucinations in Large Language Models but struggles with complex reasoning. Graph-based RAG (GraphRAG) addresses this gap yet can exhibit a failure pattern: semantic compression during triple extraction may reduce the preservation of fine-grained facts such as precise quantities, temporal constraints, and entity attributes. We formalize this limitation as the Fine-Grained Fact Retrieval (FGFR) cascade and introduce stage-wise diagnostic metrics, including R_rep, R_sel, and R_end, to quantify information loss. To stress-test FGFR, we construct a diagnostic evaluation suite from classic novels that provide controlled long-context scenarios containing explicit entity attributes and temporal constraints. F-GraphRAG, our framework, features a granularity-aware extractor that preserves atomic details and an information-preserving filter that counters global importance bias. On this evaluation suite, F-GraphRAG outperforms GraphRAG by 12.6% in accuracy while using 33.8% fewer relationships, and achieves gains of 5.3–11.2% across nine public benchmarks. These results suggest that improving evidence completeness can provide a more effective alternative to increasing graph connectivity.

Abstract:
Agents often repeatedly solve similar task instances from scratch, leading to unnecessary reasoning cost and long execution traces. Prior work has explored workflow reuse and executable skill induction, but it remains unclear which task scenarios admit procedural skills and how the shared procedural structure should be represented across successful traces. We study this problem in FSM-defined scenarios, where successful traces can be viewed as paths through partially observed instance-level transition graphs, and formulate procedural skills as reusable parameterized control-flow subgraphs. Based on this view, we introduce SkillDisCo, a distillation-and-compilation framework that distills reusable PFSM subgraphs from successful traces and compiles them into callable, executable, and verifiable procedural skills. Experiments on ALFWorld and WebArena show that SkillDisCo improves success rates and reduces agent turns across benchmarks and model scales, demonstrating the benefits of representing shared experience as reusable execution structures

Abstract:
Modern misinformation is often heard before it is read, yet fact-checking systems are still evaluated mainly on clean written claims. Spoken dialogue remains different even when systems operate on transcripts: claims may be distributed across speakers and turns, depend on prior context, and become harder to verify when Automatic Speech Recognition (ASR) errors distort the available text. Prior spoken dialogue fact-checking resources are small, English-centric, or focused on annotation rather than end-to-end benchmarking, leaving no large multilingual benchmark with paired speech and turn-level labels. We introduce TRILOGUE (TRIlingual spoken diaLOGUE fact-checking), a large-scale trilingual benchmark of source-grounded spoken dialogues in English, Russian, and Kazakh. It contains nearly 12K dialogues, 187K turns, and 390 hours of paired audio with ASR transcripts and word-level timestamp alignments across all three languages, including nearly 5K human-recorded Russian and Kazakh dialogue files. TRILOGUE supports claim check-worthiness detection, source-article evidence retrieval, and claim verification with claim-only, gold-evidence, and retrieved-evidence inputs. Baselines show that ASR degradation and cross-lingual transfer remain challenging, especially for Kazakh, while retrieved source evidence substantially narrows the gap to gold-evidence verification.

Abstract:
Language models (LMs) often pass behavioral bias evaluations, but it remains unclear whether they no longer represent the underlying associations that give rise to biases, or have merely learned not to express them. In this study, we show that representational biases are often detectable, even when behavioral biases are not visible. We introduce a causal framework that decomposes occupational bias into two measurement points: a model’s internal representation of a user’s competence, and its observable outputs. We derive steering vectors for representations of user expertise, and verify that they causally mediate model behavior in both a question-answering task and a hiring task. Applying this framework to several open-weight models, we find that demographic attributes, such as gender, race, and socioeconomic status, influence a model’s representation of user expertise, even in cases where behavioral metrics detect no disparity between demographics. We show that these model representations can influence downstream behavior under intervention, suggesting failure modes that behavioral metrics alone may not detect.

Abstract:
Large language models are increasingly used in value-sensitive decision settings, where irrelevant demographic cues should not alter judgments. We construct the Realistic Value Decision Benchmark (RVDB), a controlled benchmark that varies only the role-gender configuration while holding the scenario, ordered value pair, roles, candidate decisions, Value Distance, and Decision Severity fixed. Using a position-balanced evaluation across seven models, we test whether models preserve decision invariance under gender perturbations and whether their self-attributions reflect observed behavioral changes. We find that explicit gender cues induce bounded but systematic decision flips, including under an explicit gender-attribution prompt that asks models to report whether gender influenced their choice. Cross-gender role swaps reveal a consistent female-proposed-decision asymmetry, while models often attribute flipped decisions to No Influence or other non-gender factors. Further analysis shows that gender effects concentrate near less determinate value boundaries and under more severe decision contexts, suggesting that gender cues act as local boundary-shifting factors rather than global overrides of value reasoning. Value rankings remain largely stable, but ordered value-pair trade-offs shift unevenly across role-gender configurations. These results show that gender can enter LLM value trade-offs behaviorally while remaining obscured in self-attribution, motivating controlled behavioral audits beyond explanation-based evaluation.

Abstract:
We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks beyond short-form recognition and QA. CARTBENCH comprises four subtasks: CURATORQA for evidence-grounded recognition and reasoning, CATALOGCAPTION for structured four-section expert-style appreciation, REINTERPRET for defensible reinterpretation with expert ratings, and CONNOISSEURPAIRS for original-copy discrimination under visually similar confounds. CARTBENCH is built by aligning image-bearing Palace Museum objects from Wikidata with authoritative catalog pages, spanning five art categories across multiple dynasties. Across evaluated VLMs, high overall CURATORQA accuracy can mask sharp drops on evidence-heavy and style-to-period questions; long-form appreciation remains far from expert references; and original-copy discrimination remains near chance. A held-out protocol-replication pilot on 120 artworks from five external institutions reproduces the main task-difficulty pattern, while not preserving the exact overall model ranking.

Abstract:
Zero-shot classification for structured tabular data has attracted increasing attention in recent years. An important line of research is leveraging textual explanations as supervision to improve the modeling of tabular attribute features. Previous studies usually serialize tabular data into flattened explanation sequences, or employ logical rules to model directional associations between tabular attributes and labels. However, these methods often suffer from two major limitations: the former disrupts the inherent row-column topological structure of tables, while the latter mainly represents relations with fixed logical operators or aggregation weights, making it difficult to model learnable dependencies among features. To address these issues, we propose a novel method (MulEG), which harnesses multi-level textual explanations with a feature attribute graph to model both tabular topology and cell-element relations. Specifically, to preserve the intrinsic row-column topological structure of tables, we propose a multi-level textual explanation modeling with sample-level, attribute-level, and label-level perspectives. Moreover, to capture fine-grained feature dependencies, we propose a differentiable message-passing mechanism using a feature attribute graph that allows heterogeneous features to propagate through message passing. Finally, extensive experimental results on datasets covering sixteen tasks demonstrate the effectiveness of the proposed approach.

Abstract:
Multi-domain multi-task learning (MD-MTL) aims to build a single generalist model that performs well across heterogeneous domains and tasks. However, joint training often suffers from interference under distribution shifts. Existing model merging methods mostly operate on model parameters while overlooking the geometric structure of latent representation distributions across domains and tasks. To address these limitations, we propose Hierarchical Wasserstein Merging (HWM), a representation-level framework that models each domain-task specialist as a distribution of hidden representations on a shared support. HWM constructs task-level and global Wasserstein barycenters to capture within-task domain variation and cross-task structure, enabling either training-free specialist aggregation by Wasserstein-derived weights or training-based generalist learning through a hybrid Wasserstein alignment loss. Experiments on four NLP tasks across four domains per task show that HWM achieves superior effectiveness and generalization capability in MD-MTL settings.

Abstract:
Language models used in retrieval-augmented settings choose between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice: using an approach based on steering vectors, we carry out interventions at selected layers and token spans, and we are able to toggle the model’s behavior in two directions: Copy→Recall (suppressing context use) and Recall→Copy (inducing the model to copy any token from the context). Experiments on three architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy "reactivation" process that can be triggered at different locations in the input, while restoring recall is a "suppression" process that is more fragile and strongly tied to object-token interventions.

Abstract:
Cascaded Automatic Speech Recognition — Large Language Model (ASR-LLM) pipelines remain popular for industrial Spoken Dialogue Systems (SDS), primarily because their decoupled design ensures perceptual verifiability. However, cascaded systems suffer from error propagation, as transcription failures inevitably cascade to subsequent components, thereby degrading the final interaction quality. Although ASR confidence scores offer a simple filter for unreliable inputs, this approach is fundamentally limited because it typically fails to detect deletion errors or to distinguish between acoustic (inability to hear clearly) and linguistic (inability to understand) mismatches, both of which require targeted recovery strategies. In this paper, we propose a cause-aware error recovery paradigm that fundamentally rethinks robustness in SDS. Unlike traditional confidence filtering, we introduce a suite of small precision-focused detectors that exploit deep ASR latent representations to disentangle token-level errors into perception, comprehension, and deletion failures. This fine-grained diagnostic intelligence empowers the LLM to orchestrate targeted, multi-turn clarification strategies, effectively transforming ambiguous signals into seamless user interactions. Experimental results validate the precision of our approach, which more than doubles the recall on domain-shift errors (57.96% vs. 23.66%) compared to baselines. Crucially, this diagnostic precision yields up to a 31% reduction in WER and a 19% improvement on the downstream task across diverse accents, distortions, and domains.

Abstract:
Traditional research on Continual Machine Translation (CMT) has long been predicated on the assumption that catastrophic forgetting stems from the irreversible erasure of underlying linguistic knowledge. In this work, we challenge this conventional wisdom within the context of Large Language Models (LLMs). Through systematic behavioral and representational analyses, we demonstrate that the model’s latent knowledge often remains intact, yet its ability to align this knowledge with instructions becomes compromised. We attribute this " expressive drift" to the ubiquitous Supervised Fine-Tuning (SFT) paradigm. By minimizing token-level loss against fixed references, SFT neglects the inherent plurality of valid translations for a single source, inadvertently coercing the model toward narrow external distributions that conflict with its internal linguistic priors. To rectify this, we advocate for a transition from SFT to a Reinforcement Learning (RL) paradigm, which optimizes performance directly within the model’s own generative distribution. Our results show that RL significantly mitigates forgetting in both translation-specific and general tasks by preserving the model’s expressive stability. This work redefines the fundamental challenge of CMT and positions RL as a more principled approach for sustainable multilingual alignment in the era of LLMs.

Abstract:
While autoregressive models (ARMs) dominate natural language processing, their strictly left-to-right decoding inherently limits global context utilization. Diffusion language models (DLMs) have recently emerged as a promising paradigm shift, theoretically offering bidirectional attention, parallel decoding, and the flexibility of any-order generation. However, the nature of this generative flexibility remains heavily under-explored. In this work, we systematically investigate the interplay between any-order flexibility, task performance, and inference efficiency. By quantifying deviations from autoregressive (AR) generation order using Normalized Edit Distance (NED) across reasoning, code, and commonsense tasks, we find that current DLMs rely heavily on near left-to-right generation pathways to achieve high performance. To weaken this coupling, we demonstrate that the generation block size acts as a key regulator of decoding order. Consequently, we propose a novel mitigation strategy: training the model with a large block size to grant it a vastly expanded exploration space, while performing inference with a comparatively smaller block size to minimize computational overhead. Ultimately, our methodology increases deviation from left-to-right generation while preserving strong task accuracy and inference efficiency.

Abstract:
News-driven time series forecasting (NTSF) uses evolving textual events together with historical observations to predict future values, supporting applications such as market risk monitoring and resource scheduling. In multi-agent settings, two challenges still remain. The first is degeneration of thought, where agents converge to similar evidence-seeking behaviors. The second is insufficient theoretical grounding, where strategy updates are often heuristic and lack a principled formulation. To address the above challenges, we propose CompEvo, a competition-induced evolution framework for multi-agent NTSF. At the theoretical level, our evolutionary game formulation establishes equilibrium existence and local convergence under the stated assumptions. At the methodological level, we construct a trainable multi-agent evolution framework that integrates strategy execution, fitness-based differentiable selection, and competition-induced strategy evolution. CompEvo enables heterogeneous agents to explore diverse news evidence, converts forecasting feedback into differentiable influence weights, and evolves agent strategies under competitive pressure to preserve effective logic while maintaining diversity. Experiments on four real-world datasets show that CompEvo reduces RMSE by 29.8% and MAPE by 30.3% on average relative to the strongest multi-agent baseline. Further analysis indicates that CompEvo maintains diverse and specialized agent behaviors.

Abstract:
OpenClaw-style personal assistants extend LLM agents from isolated tool use to open-ended, stateful, and personalized software environments. Evaluating these assistants is fundamentally a fidelity problem: benchmarks must be faithful both to the distribution of real assistant tasks and to the execution semantics of the environments in which those tasks unfold. Existing benchmarks often lose fidelity in one dimension or the other. Their task distributions are shaped by what is easy to isolate, mock, and verify, underrepresenting real-world difficulties such as cross-service dependency, contaminated state, implicit intent, and runtime change. Their environments are either live but hard to reproduce, or reproducible but reduced to endpoint-level stubs that remove sessions, artifacts, state transitions, and downstream side effects. We introduce LiveClawBench , a benchmark designed around this dual-fidelity requirement. LiveClawBench combines a Triple-Axis Complexity Framework for difficulty-driven task construction with reproducible full-stack mock applications that preserve stateful execution semantics. With 134 executable cases across 10 domains with 22 mocked services, LiveClawBench supports controlled, extensible, and factor-level diagnostic evaluation of realistic agentic tasks.

Abstract:
AI agents augment large language models with external tools such as web retrieval, enabling grounded and up-to-date responses. However, incorporating external content into the generation pipeline can weaken the safety alignment mechanisms that govern model outputs. Prior work shows that enabling retrieval in agents increases compliance with harmful requests. We introduce AgentREVEAL, a diagnostic framework for analyzing retrieval-induced safety degradation in LLM agents. The framework examines two axes: how retrieval is integrated into the agent pipeline and the properties of the retrieved content. Along the integration axis, we find that binding tool invocation and response generation in a single step amplifies harmful outputs. Along the content axis, we uncover the Safe Source Paradox: even oppositional or safety-oriented sources, such as pages containing warnings or risk disclaimers, can increase harmful compliance by an average of 25% compared to the no-retrieval baseline. Finally, we show that relevance acts as a shared activation condition for both vulnerabilities. We observe that several agents retrieve information for harmful requests in the autonomous retrieval setting, while harmful compliance remains elevated despite several retrieval-pipeline interventions and defenses. Because relevance is also what makes retrieval useful, these results expose a safety-utility trade-off for retrieval-enabled agents. We introduce HarmURLBench, a benchmark containing 1,405 real-world URLs paired with 320 harmful behaviors to support future evaluations.

Abstract:
Text-to-SQL systems are typically trained and evaluated on a single dialect (SQLite), yet production deployments span PostgreSQL, MySQL, ClickHouse, and beyond. We show that this single-dialect assumption leads to a substantial drop in lower-bound dialect accuracy for every model we tested. The drop persists across scale, architecture, and even purpose-built text-to-SQL systems. We argue that the fix is to change the generation target: instead of asking an LLM to emit dialect-specific SQL, we have it emit a dialect-agnostic relational algebra query plan, which a deterministic compiler then renders into SQL for any supported backend. Across thirteen models from 3B to frontier scale, this restores cross-dialect portability nearly uniformly, at a small cost in peak accuracy on the model’s home dialect for capable prompted models and none once fine-tuned on plans; under matched fine-tuning, plan supervision yields a stronger model than SQL supervision. We also introduce DF-Match, a question-aware result-set comparator needed to evaluate fairly across dialects, where existing metrics confound semantic errors with benign cross-dialect variation. More broadly, the result is a reminder that a generation target chosen for execution is not necessarily the one that maximizes generation quality.

Abstract:
As large language models (LLMs) increasingly mediate both content generation and moderation, linguistic evasion strategies known as Algospeak have intensified the coevolution between evaders and detectors. This research formalizes the underlying dynamics grounded in a joint action model: when Algospeak increases, detectability and understandability decrease. Further, the concept of Majority Understandable Modulation (MUM) is introduced and defined as the modulation level at which additional evasive alteration increases detector evasion but loses comprehension for the majority of recipients. To empirically probe this trade-off, we introduce a reproducible framework that can be used to create meaning-preserving, Algospeak-style variants, based on an existing taxonomy and with tunable modulation levels. Using COVID-19 disinformation as a first proof-by-example setting, we construct a reference dataset of 700 modulated items, drawn from twenty base sentences across five modulation levels and seven strategies. We then run two linked evaluations with seven different language models: one testing for interpretation through meaning recovery and one for disinformation detection through classification. Curve fitting over modulation levels yields an estimate of the Majority Understandable Modulation threshold and enables sensitivity analyses across strategies and models, see Figure 1. Results reveal the characteristic relationships between understandability and modulation. This study lays the groundwork for understanding the dynamics behind Algospeak and provides the framework, dataset, and experimental setups described.

Abstract:
With the widespread deployment and application of Multimodal Large Language Models (MLLMs), disputes over model ownership have become increasingly frequent, raising serious concerns about intellectual property protection in black-box scenarios. Existing techniques primarily embed copyright information into models through backdoor or adversarial attacks, triggering abnormal outputs on specific inputs for verification. However, we find that such methods exhibit limited robustness after post-processing operations such as fine-tuning, pruning, and model merging. To address this, we propose Concept-based Probabilistic Fingerprinting (CPF). CPF selects a visual concept as the fingerprint trigger, embeds statistical watermark signals in concept-related multimodal conversations, and migrates the watermarked distribution to the original model through self-distillation. It further employs cross-concept similarity screening to construct a regularization constraint, ensuring the fingerprint is activated only on the target concept. CPF does not trigger semantically irrelevant abnormal outputs, nor does it rely on fixed inputs as triggers. Experimental results demonstrate that CPF maintains stable fingerprint detectability across diverse scenarios while exhibiting strong robustness.

Abstract:
LLM agents are increasingly used to run autonomous ML experiments, iterating on target metrics with little human oversight. Prior work has documented reward hacking in these environments, bringing into question the validity of produced research and the broader safety case for AI R D. Existing benchmarks do not measure exploits that live in the data or the modeling task itself. We introduce BAITBENCH, a suite of three synthetic tabular ML tasks that each contain a shortcut that allows agents to inflate the public test score but fail on a hidden test set. Since the shortcut is optional and using it breaks no stated rule, BAITBENCH measures how often models exploit the shortcut to achieve inflated scores. Across seven frontier agents scored by our two-stage judge pipeline, 57.1% of runs exhibit reward hacking, with five of seven above 50%. Agents cheat even under a second condition where they are prompted not to -the mean cheating rate remains above 50%. We release BAITBENCH, along with the judge implementation, and an annotated dataset of transcripts containing reward hacks as a testbed for evaluating reward-hacking mitigations head-to-head.

Abstract:
Quantization is widely used to accelerate inference and streamline the deployment of large language models (LLMs), yet its effects on self-explanations (SEs) remain unexplored. SEs, generated by LLMs to justify their own outputs, require reasoning about the model’s own decision-making process, a capability that may exhibit particular sensitivity to quantization. As SEs are increasingly relied upon for transparency in high-stakes applications, understanding whether and to what extent quantization degrades SE quality and faithfulness is critical. To address this gap, we examine two types of SEs: natural language explanations (NLEs) and counterfactual examples, generated by LLMs quantized using three common techniques at distinct bit widths. Our findings indicate that quantization typically leads to moderate declines in both SE quality (up to 4.4%) and faithfulness (up to 2.38%). The user study further demonstrates that quantization diminishes both the coherence and trustworthiness of SEs (up to 8.5%). Compared to smaller models, larger models show limited resilience to quantization in terms of SE quality but better maintain faithfulness. Moreover, no quantization technique consistently excels across task accuracy, SE quality, and faithfulness. Given that quantization’s impact varies by context, we recommend validating SE quality for specific use cases, especially for NLEs, which show greater sensitivity. Nonetheless, the relatively minor deterioration in SE quality and faithfulness does not undermine quantization’s effectiveness as a model compression technique.

Abstract:
Knowledge graph reasoning (KGR) infers missing facts, with recent advances increasingly harnessing the semantic priors and reasoning abilities of Large Language Models (LLMs). However, prevailing generative paradigms are prone to memorize surface-level co-occurrences rather than learning genuine relational semantics, limiting out-of-distribution generalization. To address this, we propose RADAR, which reformulates KGR from generative pattern matching to discriminative relational reasoning. We recast KGR as discriminative entity selection, where reinforcement learning enforces relative entity separability beyond token-likelihood imitation. Leveraging this separability, inference operates directly in representation space, ensuring consistency with the discriminative optimization and bypassing the generation-induced hallucinations. Across four benchmarks,RADAR achieves 5–6% relative gains on link prediction and triple classification over strong LLM baselines, while increasing task-relevant mutual information in intermediate representations by 62.9%, indicating more robust and transferable relational reasoning.

Abstract:
Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) semantic misguidance, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) strategy transfer failure, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these findings, we introduce Insight-to-Solve (I2S), which converts demonstrations into target-conditioned insights before solving, and I2S+, which adds iterative refinement. Across open- and closed-source models, both variants outperform direct answering and sequential test-time baselines. On a frozen 500-problem Omni-MATH transfer set, I2S+ improves R1-Distill-Qwen-7B and Qwen3-14B by +4.2 and +4.0 points, respectively. The same prompt stack also transfers to GPT-4.1, o1-mini, and R1-Distill-Llama-8B, showing that reasoning demonstrations can be converted from liabilities into useful test-time guidance.

Abstract:
Assessing progress toward the Sustainable Development Goals (SDGs) requires multi-step reasoning over visual cues, contextual knowledge, and development indicators, where incomplete evidence use and imperfect evidence integration can introduce hidden prediction biases. Real-world SDG monitoring further spans both qualitative judgments and quantitative estimation. However, existing benchmarks typically evaluate these aspects in isolation, obscuring systematic biases that emerge when models substitute priors for evidence. To address this gap, we propose SDGBiasBench, a large-scale benchmark suite for SDG-oriented vision–language reasoning. Spanning 500k human-curated multiple-choice questions and 50k regression tasks, the benchmark enables comprehensive assessment of both decision-level and estimation-level bias in Vision–Language Models (VLMs). Evaluations on SDGBiasBench reveal an intrinsic SDG bias in current VLMs, where predictions are frequently driven by SDG specific priors rather than reliable multi-modal cues. To mitigate such bias, we propose CADE (Contrastive Adaptive Debias Ensemble), a training-free, plug-and-play method that leverages modality-specific answer priors. CADE yields significant gains on the proposed benchmark, improving multiple-choice accuracy by up to 25% and reducing regression MAE by up to 12 points across multiple VLMs. We hope our work can foster the development of more fair and reliable AI systems for sustainable development.

Abstract:
Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for combining large language models (LLMs) with external knowledge sources. However, RAG systems remain vulnerable to prompt injection attacks, which may mislead the retriever or generator to expose sensitive database contents. To address this issue, we propose KFS-RAG, a defense that mitigates information leakage by reformulating the retrieved context. Specifically, our method first identifies a small set of influential keywords from the retrieved context via an attention rollout plus a causal perturbation mechanism. These keywords are then used to guide an auxiliary LLM to generate a compact set of keyword-grounded facts from the retrieved passages. Finally, the original context is substituted with these curated facts, ensuring that the generator operates on sanitized evidence rather than the raw retrieved text. Experimental evaluations demonstrate that KFS-RAG significantly reduces the risk of database leakage under injection attacks while maintaining response accuracy and relevance. This work highlights a practical pathway toward building secure and trustworthy RAG systems.

Abstract:
We investigate whether Large Language Models (LLMs) exhibit conspiratorial tendencies, whether they display socio-demographic biases in this domain, and how easily they can be conditioned into adopting conspiratorial perspectives. Conspiracy beliefs play a central role in the spread of misinformation and in shaping distrust toward institutions, making them an important testbed for assessing the social and psychological fidelity of LLMs and their potential to reproduce or reinforce harmful narratives. Although LLMs are often used as proxies for studying human behavior, it remains unclear whether they reproduce higher-order psychological constructs such as generalized conspiratorial beliefs. To bridge this research gap, we administer validated psychometric surveys measuring conspiratorial mindset to multiple models under different prompting and conditioning strategies. Our findings reveal that LLMs show partial agreement with elements of conspiracy belief, and conditioning with socio-demographic attributes produces uneven effects, exposing latent demographic biases. Moreover, targeted prompts can easily shift model responses toward conspiratorial directions, underscoring both the susceptibility of LLMs to manipulation and the potential risks of their deployment in sensitive contexts. These results highlight the importance of critically evaluating the psychological dimensions embedded in LLMs, both to advance computational social science and to inform possible mitigation strategies against harmful uses.

Abstract:
Asking questions is fundamental for deeper understanding, yet evaluating and generating questions that improve downstream outcomes remains a challenging problem. To address this gap, we introduce QUEST (Question Utility Estimation with Simulated Tests), a framework that offers a new outcome-driven paradigm for question evaluation and generation, instead of indirect metrics that are based on the underlying content of the question and its answer. QUEST uses language models to simulate learners that ask questions while studying a textbook chapter and quantifies the utility of a question as its direct contribution to understanding as measured by the learner’s exam performance. To analyze the effectiveness of QUEST, we curate TEXTBOOK-EXAM, a benchmark that aligns textbook sections with exam questions across five academic disciplines. Using QUEST, we filter for high-utility questions and fine-tune question generators via rejection sampling and show that questions generated by QUEST-trained models improve simulated exam scores by 19.6% compared to the next strongest baseline. Furthermore, we demonstrate the practical real-world value of QUEST by showing that simulation-computed utility of a question is moderately positively correlated with its usefulness as perceived by human evaluators.

Abstract:
Large language models perform well on many logical reasoning benchmarks, but it remains unclear which core logical skills they truly master. To address this, we introduce LogicSkills, a benchmark that isolates three fundamental logical skills: (i) formal symbolization \unicodex2013 translating premises into first-order logic; (ii) countermodel construction \unicodex2013 showing that an argument is logically invalid by constructing a finite countermodel; and (iii) validity assessment \unicodex2013 determining whether a conclusion follows from a set of premises. Items are drawn from the two-variable fragment of first-order logic without identity and are presented in both English and a Carrollian nonce-word language. All instances are solver-verified with Z3 for correctness and non-triviality. Across conventional instruction-tuned LLMs, performance is high on validity assessment but substantially lower on formal symbolization and countermodel construction, highlighting that high task-level accuracy can mask weaknesses in core logical skills. In contrast, recent reasoning-tuned models perform strongly across all three tasks, suggesting a more systematic logical skill profile.

Abstract:
Existing long-context benchmarks for Large Language Models (LLMs) focus on evaluating comprehension of long inputs, while overlooking the evaluation of long reasoning abilities. To address this gap, we introduce LongReasonArena, a benchmark specifically designed to assess the long reasoning capabilities of LLMs. Our tasks require models to solve problems by executing multi-step algorithms that reflect key aspects of long reasoning, such as retrieval and backtracking. By controlling the inputs, the required reasoning length can be arbitrarily scaled, reaching up to 1 million tokens of reasoning for the most challenging tasks. Extensive evaluation results demonstrate that LongReasonArena presents a significant challenge for both open-source and proprietary LLMs. For instance, even GPT-5 achieves only 27.8% accuracy on Level 3. Further analysis also reveals that the accuracy exhibits a linear decline with respect to the logarithm of the expected number of reasoning steps. Our code and data are available at https://github.com/LongReasonArena/LongReasonArena.

Abstract:
Although Large Vision-Language Models (LVLMs) excel at single-image understanding tasks, their performance often degrades significantly when processing multi-image inputs due to unintended cross-image information leakage, in which visual content from different images is erroneously fused and image-specific contextual information is lost. Current approaches attempt to mitigate this by inserting special vision delimiter tokens between images during input formatting to help models distinguish image boundaries and preserve context. However, our analysis reveals that such delimiters alone yield limited gains because models do not sufficiently perceive or respond to these boundary signals. To address this, we propose Vision-Delimiter-aware Attention Head Modulation (VDAHM), a training-free method that modulates the output of delimiter-sensitive attention heads, reorienting the model’s internal attention flow to strengthen its structural perception of boundary signals. Extensive evaluations across multiple benchmarks, including Mantis, BLINK, MuirBench, and Qbench2, demonstrate that VDAHM consistently improves LVLMs performance with virtually no additional computational cost. The source code is available at [https://github.com/ZeroNLP/VDAHM](https://github.com/ZeroNLP/VDAHM).

Abstract:
Enhancing LLM reasoning in federated settings is nontrivial due to stringent computational, communication, and privacy constraints, especially in healthcare, where clinically consequential decisions require not only accuracy but also interpretable, auditable rationales to meet safety, accountability, and regulatory requirements. Conventional federated fine-tuning largely imitates final answers rather than cultivating step-by-step reasoning, often relying on privacy-sensitive centralized distillation and still incurring substantial communication overhead. We address this gap with FedCoT , a federated reasoning framework that combines lightweight chain-of-thought resampling with a compact discriminator for selection, and client-aware LoRA stacking with weighted classifier aggregation to accommodate heterogeneity while reducing aggregation noise and communication; clients generate candidate chains and supervision locally, and only lightweight modules are aggregated on the server. Experiments on medical reasoning benchmarks show consistent gains under tight resource budgets while keeping data local and respecting privacy, offering an interpretable and resource-efficient solution.

Abstract:
Fine-tuning large language models for code editing has typically relied on mining commits and pull requests. The working hypothesis has been that commit messages describe human intent in natural language, and patches to code describe the changes that implement that intent. However, much of the previously collected data is noisy: commit messages are terse, human-written commits commingle several unrelated edits, and many historical commits come from simple, rule-based bots. The recent adoption of software engineering agents changes this landscape. Code changes co-authored by humans and agents are often accompanied by substantially more explicit natural-language descriptions of intent and rationale. Moreover, changes in public repositories may have passed through human selection and review, and we treat this potential human filtering as an indirect signal of data quality. We present AgentPack, a corpus of 1.8M code edits co-authored by Claude Code, OpenAI Codex, and Cursor Agent across public GitHub projects up to early October 2025. We describe the identification and curation pipeline, quantify adoption trends of these agents, and analyze the structural properties of the edits. Finally, we show that models fine-tuned on AgentPack outperform models trained on prior human-only commit corpora, highlighting the potential of using public data from software engineering agents to train future code-editing models.

Abstract:
The path toward autonomous software engineering is currently bottlenecked by a severe deficit of diverse, large-scale trajectory data. We address this by introducing , an expansive dataset of 207,489 agentic trajectories spanning nine programming languages (Python, Go, TS, JS, Rust, Java, PHP, C, C++). Sourced from 20,000 real-world PRs via OpenHands and SWE-agent harnesses, the dataset utilizes a hybrid-reasoning synthesis: Minimax-M2.5 generates trajectories with explicit "thinking" processes, while Qwen3.5-122B provides high-quality behavioral traces. Filtered for permissive licenses (MIT, Apache, BSD) from SWE-rebench-V2, this data facilitates the training of models capable of long-horizon reasoning. We validate the dataset by fine-tuning the Qwen3-30B-A3B series (Thinking, Coder, and Instruct). The best performing model achieves a resolve rates of 60% on SWE-bench Verified, 46.2% on SWE-bench Multilingual, and 36.8% on SWE-bench Pro. These results establish as a premier resource for distilling human-level software engineering capabilities into efficient, open-source agentic LLMs.

Abstract:
Recent years have witnessed rapid advances in graph representation learning, with the continuous embedding approach emerging as the dominant paradigm. However, such methods encounter issues regarding parameter efficiency, interpretability, and robustness. Thus, Quantized Graph Representation (QGR) learning has recently gained increasing interest, which represents the graph structure with discrete codes instead of conventional continuous embeddings. Given its analogous representation form to natural language, QGR also possesses the capability to seamlessly integrate graph structures with large language models (LLMs). As this emerging paradigm is still in its infancy yet holds significant promise, we undertake this thorough survey to promote its rapid future prosperity. We first present the background of the general quantization methods and their merits. Moreover, we provide an in-depth demonstration of current QGR studies from the perspectives of quantized strategies, training objectives, distinctive designs, knowledge graph quantization, and applications. We further explore the strategies for code dependence learning and integration with LLMs. At last, we give discussions and conclude future directions, aiming to provide a comprehensive picture of QGR and inspire future research.

Abstract:
Multi-agent large language model (LLM) workflows have emerged as a powerful paradigm for solving complex, open-ended tasks through collaborative reasoning among specialized LLM agents, but they incur substantial operating costs due to repeated LLM invocations and long-horizon context accumulation. Existing cascade routing methods make one-shot, query-level decisions and cannot adapt to the dynamic, state-dependent nature of multi-step workflows, in which the right LLM at each step depends on evolving task progress, remaining task difficulty, and cost-efficiency requirements. We present ProgRouter, an online progress-guided routing framework that adaptively selects LLM agents across workflow steps to preserve task-solving quality while adhering to time and cost budgets. ProgRouter introduces a multi-view task progress scorer that combines coarse workflow outcome regimes with fine-grained signals on subtask completion, progress trends, and workflow state quality. Then, a dual-path task progress predictor and an adaptive meta-gating mechanism estimate the progress gain for each candidate routed LLM. ProgRouter makes online step-wise routing decisions that balance progress gain, task time budgets, and long-term operating cost efficiency. Experiments on HumanEval Plus, MBPP, MATH-500, and ASQA, spanning agentic code generation, mathematical reasoning, and retrieval-augmented long-form question answering, demonstrate that ProgRouter reduces the operating cost relative to key baselines while maintaining strong task-solving performance.

Abstract:
Error Span Detection (ESD) is a crucial subtask in Machine Translation (MT) evaluation, aiming to identify the location and severity of translation errors. While fine-tuning models on human-annotated data improves ESD performance, acquiring such data is expensive and prone to inconsistencies among annotators. To address this, we propose a novel self-evolution framework based on Minimum Bayes Risk (MBR) decoding, named Iterative MBR Distillation for ESD, which eliminates the reliance on human annotations by leveraging an off-the-shelf LLM to generate pseudo-labels. Extensive experiments on the WMT Metrics Shared Task datasets demonstrate that models trained solely on these self-generated pseudo-labels outperform both unadapted base model and supervised baselines trained on human annotations at the system and span levels, while maintaining competitive sentence-level performance.

Abstract:
Multimodal harmful meme detection is challenging because harmful intent is often expressed through implicit image-text associations, cultural references, rhetorical framing, and context-dependent targets. Existing methods mainly rely on supervised multimodal models, MLLM prompting, or fixed agent workflows. However, they still lack experience accumulation and rely on static skill design, making it difficult to reuse prior judgments or adapt to emerging meme templates and long-tail harm patterns. To address these limitations, we propose HarmEvoSkill (Harmful Meme Evolving Skill-Memory), a label-free self-evolving Skill-Memory framework for harmful meme detection. HarmEvoSkill converts unlabeled reference memes into Case Memory, Insight Memory, and Risk Memory. It then automatically discovers candidate skills, abstracts recurring reasoning patterns, and induces executable skill cards without predefined skill labels. During inference, a Skill Planner retrieves relevant memories, selects task-specific skills, and generates structured evidence packets. A graph-assisted judge organizes the evidence into a harmful–harmless claim graph, makes the final prediction, and updates memory through graph diagnosis. Extensive benchmark evaluations show that HarmEvoSkill achieves consistently strong performance among evaluated methods. On HarM, it improves Accuracy by 9.33% and Macro-F1 by 17.97% over the direct MLLM baseline.

Abstract:
Vision-Language-Action (VLA) policies are commonly trained from dense robot demonstration trajectories, often collected through teleoperation, by sampling every recorded frame as if it provided equally useful supervision. We argue that this convention creates a temporal supervision imbalance: long low-change segments dominate the training stream, while manipulation-critical transitions such as alignment, contact, grasping, and release appear only sparsely. We introduce FrameSkip, a data-layer frame selection framework that scores trajectory frames using action variation, visual-action coherence, task-progress priors, and gripper-transition preservation, then remaps training samples toward high-importance frames under a target retention ratio. Because FrameSkip operates only in the dataloader, it leaves the VLA architecture, action head, training objective, and inference procedure unchanged. Across RoboCasa-GR1, SimplerEnv, and LIBERO, FrameSkip improves the success-retention trade-off over full-frame training and simpler frame selection variants, achieving a macro-average success rate of 76.15% across the three benchmarks compared with 66.50% for full-frame training while using a compressed trajectory view that retains 20% of unique frames in the main setting.

Abstract:
Agent skills are callable procedural modules that provide reusable knowledge and execution policies for complex agentic tasks. However, existing methods mainly focus on selecting relevant skills or improving the skills themselves, while overlooking whether a relevant skill should actually be invoked at the current decision point. Unhelpful invocations may introduce irrelevant context and disrupt an otherwise correct execution process. To address this issue, we propose SelSkill, a dual-granularity preference-learning framework for selective skill invocation. SelSkill formulates skill use as a skill-or-skip decision, uses predictive uncertainty to prioritize candidate decision points, and constructs controlled invoke-skip preference pairs from shared trajectory prefixes. It further combines episode-level outcome preferences with step-level invocation preferences to capture both overall trajectory quality and the local effectiveness of skill invocation. On ALFWorld with Qwen3-8B, SelSkill improves task success by 10.9 points over the skill-enabled baseline and 7.8 over No-Skill, with 29.1-point higher execution precision. On BFCL, task success and execution precision improve by 5.7 and 29.5 points, respectively. Zero-shot results on Tau-bench and PopQA suggest partial transfer to unseen domains and skills.

Abstract:
Maintaining logical coherence in multi-hop Retrieval-Augmented Generation (RAG) remains a challenging problem, as systems must integrate fragmented evidence without suffering reasoning drift. Iterative frameworks enhance information coupling through query refinement, yet often depend on lossy compressed representations that obscure subtle semantics. Conversely, global graph-based methods ensure structural connectivity but carry topological noise and high computational costs. To overcome these issues, we propose STAGE-RAG, a STAte-aware Graph Evolution framework that conceptualizes multi-hop reasoning as continuous Bayesian entity-state evolution on a bi-relational chunk subgraph. By tracking entity states composed of identities and dynamic belief scores, our method implicitly guides reasoning via state-weighted propagation. A dual-path retrieval strategy constructs local subgraphs to prune irrelevant nodes while retaining fine-grained logical dependencies. Evaluations on HotpotQA, MuSiQue, 2WikiMultiHopQA and MoreHopQA indicate that STAGE-RAG consistently surpasses state-of-the-art baselines in both accuracy and efficiency.

Abstract:
Multi-Agent Debate (MAD) has been widely adopted to improve LLM-based evaluation by prompting multiple agents to negotiate and reach a consensus. However, for subjective rubric-based scoring, inter-agent agreement does not guarantee alignment with human judgments. In this paper, we compare a single-judge baseline against a consensus-based MAD protocol on subjective evaluation tasks and design three ablations to isolate the impact of role prompting, multi-round interaction, and explicit score sharing. Evaluations across six LLMs show that the single-judge baseline achieves the strongest human alignment on average across six judge models, whereas MAD shows degradation in human alignment on both tasks. Our ablations demonstrate that this performance drop stems primarily from asymmetric role prompting rather than the interaction itself. Specifically, assigning a strict judge role introduces a systematic downward bias that the consensus process fails to correct. The central finding is that this bias reflects strict-stance dominance beyond averaging: the consensus score falls well beyond the arithmetic midpoint of the standalone strict and lenient conditions, rather than averaging them out. Removing role asymmetry (Symmetric MAD) largely recovers baseline performance, while masking peer scores widens inter-agent disagreement on average and worsens average human alignment. These findings demonstrate that multi-agent consensus can enforce artificial agreement at the expense of true human alignment, revealing a structural limitation in consensus-style, role-specialized MAD protocols for subjective scoring.

Abstract:
Organizations often develop and maintain portfolios of related applications: independently deployable codebases that share substantial domain logic, interface patterns, or operational conventions. As LLM coding agents are increasingly used to generate and maintain such software, a naive application-by-application workflow duplicates shared logic across codebases and allows prolonged agentic maintenance to accumulate verbosity, dead code, and structural erosion. We introduce the Super Library Agent problem, where an agent sequentially generates a portfolio of N related applications while maintaining a shared Super Library of reusable cross-application components. A minimal sequential scaffold can in principle extract shared code and migrate applications to the evolving library, but in practice suffers from low extraction recall and fragile dependency migration. We address these failures with candidate-guided extraction over code chunk summaries, pre-extraction codebase consolidation, and context-aware migration using extraction traces and call-graph information. Across WebGen-Bench and PaperBench, our method preserves application functionality while significantly reducing redundancy and token footprint (verbosity, token length) over zero-shot, and avoiding the structural erosion introduced by naive library construction, with additional reductions in LOC and MDL. Our code is available at https://github.com/sbigstar0310/super-library-agent.

Abstract:
Reasoning distillation from powerful teacher models to smaller students faces the Gap Curse: as teachers grow more sophisticated, their complex distributions increasingly diverge from what students can approximate, causing performance degradation. Existing mitigation strategies either filter out challenging examples through data selection or introduce weaker intermediate assistant models, inherently compromising supervision coverage or quality. We propose Teacher Alignment, which directly adapts the teacher toward the student’s distribution without discarding data or degrading reasoning quality. However, naive alignment through standard knowledge distillation triggers catastrophic collapse of the teacher’s reasoning capabilities. To address this, we reformulate teacher alignment as reinforcement learning and introduce TeacherGRPO, built on Group Relative Policy Optimization with two key innovations: (i) Curriculum Selective Alignment applies dual token- and distribution-level curricula to focus rewards on high-signal reasoning gaps while filtering noise from trivial tokens and uncertain tail distributions, and (ii) Importance-Adaptive Length Regularization selectively penalizes verbose redundancy while preserving pedagogically critical reasoning steps. The aligned teacher then distills knowledge to students via standard pipelines. Extensive experiments show TeacherGRPO significantly outperforms baselines across diverse reasoning benchmarks and distillation methods. Our code is available at https://anonymous.4open.science/r/TeacherGRPO-290F/README.md.

Abstract:
While role-playing agents excel in short-term interactions, long-term conversations overwhelm context windows, motivating external memory frameworks. Current systems typically rely on persona-agnostic summarization, which records facts without persona-specific interpretation, yielding generic responses that compromise persona fidelity. To bridge this gap, we introduce RoleMemo, a dataset featuring four reasoning tasks where the factual fragments must be interpreted through the persona to reach the correct answer. Evaluation on RoleMemo exposes critical limitations of persona-agnostic frameworks. We thus propose DualMem, which decouples memory into two streams: factual cognition and persona-conditioned insight. Trained through Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), our framework with a 4B-parameter model outperforms zero-shot persona-agnostic frameworks powered by DeepSeek-V3.2 for sustained persona fidelity. Our resources are available at https://github.com/Collab-Gen/RoleMemo.

Abstract:
The growing demand for scalable psychological counseling highlights the need for high-quality, privacy-compliant data, yet such data remains scarce. Here we introduce MAGneT, a novel multi-agent framework for synthetic psychological counseling session generation that decomposes counselor response generation into coordinated sub-tasks handled by specialized LLM agents, each modeling a key psychological technique. Unlike prior single-agent approaches, MAGneT better captures the structure and nuance of real counseling. We further propose a unified evaluation framework that consolidates diverse automatic metrics and expands expert assessment from four to nine counseling aspects, thus addressing inconsistencies in prior evaluation protocols. Empirically, MAGneT substantially outperforms existing methods: experts prefer MAGneT-generated sessions in 77.2% of cases on average across the nine aspects over the strongest baseline, and sessions generated by MAGneT using Llama3-8B-Instruct backbone yield 3.2% higher general counseling skills and 4.3% higher CBT-specific skills on cognitive therapy rating scale (CTRS). An open source Llama3-8B-Instruct model fine-tuned on MAGneT-generated data also outperforms models fine-tuned using baseline synthetic datasets by 6.9% on average on CTRS. We make our code, data and fine-tuned model public.

Abstract:
Mixture-of-Experts (MoE) models exhibit striking performance disparities across languages, yet the internal mechanisms driving these gaps remain poorly understood. In this work, we conduct a systematic analysis of expert routing patterns in MoE models, revealing a phenomenon we term Language Routing Isolation, in which high- and low-resource languages tend to activate largely disjoint expert sets. Through layer-stratified analysis, we further show that routing patterns first converge and then diverge across model depth. Building on these findings, we propose RISE (routing isolation-guided subnetwork enhancement), a framework that identifies and adapts language-specific expert subnetworks. RISE uses a tripartite selection strategy: specificity scores select language-specific experts in shallow and deep layers, while overlap scores select universal experts in middle layers. By training only the selected subnetwork while freezing the rest of the model, RISE improves low-resource performance while preserving other-language capabilities. Across 10 languages, RISE achieves target-language F1 gains of up to 10.85% with minimal cross-lingual degradation.

Abstract:
Post-training LLM compression often relies on low-rank approximations, which force all columns of a projection matrix to share a single low-dimensional subspace. We propose CoSpaDi, a training-free compression framework that replaces this single-subspace assumption with a union-of-subspaces model via sparse dictionary learning. CoSpaDi factorizes each weight matrix into a dense dictionary and column-sparse coefficients, allowing different columns to select different subsets of dictionary atoms at the same storage budget. To preserve model behavior, we use calibration activations to transform functional reconstruction into a standard dictionary learning problem. Across Llama and Qwen models, CoSpaDi improves accuracy-compression and perplexity-compression trade-offs over SVD-based and structured pruning baselines at 20-40% compression ratios, while naturally supporting sparse-dense computation and post-training quantization of sparse coefficients.

Abstract:
Out-of-context (OOC) misinformation often arises when an authentic news image is paired with misleading text, making the cross-modal evidence used by a detector an important object of interpretation. We present SparseAlign-OOC, a two-level sparse autoencoder (SAE) framework for auditing NewsCLIPpings CLIP-embedding detectors through recurring image-text alignment features. A pair-space SAE maps detector inputs into sparse coordinates whose high-activation examples form human-auditable concept groups, while a hidden-state SAE traces sparse features associated with detector decisions. In a 250-item blind human audit with three annotators, selected SAE groups yield 18/50 coherent-alignment concepts, compared with 9/50 for prototype clustering, 7/50 for PCA, 4/50 for a dense autoencoder, and 0/50 for random groups. A 150-item replication audit on a stronger deep MLP preserves the same ordering, producing pooled results of 29/80 SAE alignment groups versus 15/80 for the strongest baseline. Coverage analysis shows that the presence of concepts increases with activation strength. In contrast, direct pair-space interventions show that human-labeled alignment features produce larger detector changes than topical, incoherent, or random SAE features. In the NewsCLIPpings CLIP-embedding setting, SparseAlign-OOC provides a validated audit pipeline for exposing recurring alignment factors that are both human-auditable and functionally connected to detector behavior.

Abstract:
Tool-Integrated Reasoning (TIR) is a fundamental capability for LLM agents to solve complex tasks by interacting with external tools iteratively. Reinforcement Learning (RL) has become the dominant paradigm for enabling this capability. However, existing approaches typically assign uniform trajectory-level advantages and treat all correct tool calls equally, ignoring the varying difficulty and learning value across trajectories and reasoning steps. This can lead to imprecise learning signals that do not adequately distinguish between trivial and challenging tool-use patterns. To address this limitation, we propose HiDiffTIR, a Hierarchical Difficulty-aware policy optimization framework for multi-turn TIR. HiDiffTIR performs difficulty-aware credit assignment at both trajectory and turn levels, enabling the policy to focus on more informative trajectories and harder reasoning steps. Notably, this fine-grained optimization is achieved without additional supervision, relying solely on group-level statistics derived from standard RL rollouts. Extensive experiments on three tool-using benchmarks demonstrate that HiDiffTIR consistently improves multi-turn TIR performance and tool invocation accuracy over strong RL baselines, highlighting the necessity of difficulty-aware credit assignment for effective policy optimization in tool-integrated LLM agents.

Abstract:
Standard large language model prompting treats geospatial inference as independent, instance-wise prediction, ignoring the fundamental spatial dependencies that govern geographic reality. Consequently, even advanced models struggle with spatial consistency and exhibit severe biases toward populous regions. To bridge this gap, we propose GeoGR² (Geospatial Graph Refine Reasoning), a framework that formalizes zero-shot geospatial prediction as an iterative message-passing process on a dynamically constructed graph. Unlike static retrieval methods, GeoGR² instantiates three dynamic operators via collaborating operators: (1) a Topology Operator that constructs graph topology to enforce the Spatial Markov property; (2) a Feature Operator that enriches nodes with task-relevant semantic covariates; and (3) an Update Operator that performs natural language message passing to iteratively minimize spatial discrepancy. Theoretically, we frame this refinement as a contraction mapping that approximates the fixed point of a global consistency equation. Empirically, we validate GeoGR² on diverse physical and socioeconomic tasks. Results demonstrate that by explicitly embedding geostatistical inductive biases, GeoGR² significantly outperforms standard prompting baselines, while effectively mitigating systematic geographic bias. Our framework leverages large language models’ intrinsic capacity for understanding spatial correlations through explicit topological scaffolding, without resorting to general graph reasoning paradigms.

Abstract:
While large language models (LLMs) have increasingly been applied to hate speech detoxification, the prompts often trigger safety alerts, causing LLMs to refuse the task. In this study, we systematically investigate false refusal behavior in hate speech detoxification and analyze the contextual and linguistic biases that trigger such refusals. We evaluate nine LLMs on both English and multilingual datasets, our results show that LLMs disproportionately refuse inputs with higher semantic toxicity and those targeting specific groups, particularly nationality, religion, and political ideology. Although multilingual datasets exhibit lower overall false refusal rates than English datasets, models still display systematic, language-dependent biases toward certain targets. Based on these findings, we propose a simple cross-translation strategy, translating English hate speech into Chinese for detoxification and back, which substantially reduces false refusals while preserving the original content, providing an effective and lightweight mitigation approach.

Abstract:
Reinforcement learning (RL) holds immense promise for enhancing the reasoning capabilities of diffusion large language models (dLLMs). However, progress is fundamentally constrained by a dual misalignment between authentic generation trajectory and the gradient update process: (i) Process-reward misalignment. Sparse, terminal rewards are indiscriminately assigned to all intermediate steps of the generation process, failing to provide discriminative credit assignment. (ii) State-trajectory misalignment. Policy updates are often diverted toward artificial, out-of-trajectory states, squandering gradients on less informative samples. To address these limitations, we introduce Process Aligned Policy Optimization (PAPO), a novel framework that holistically aligns the RL update with the dLLM’s generative trajectory via Step-Aware Process Rewards (SPR) that transform sparse terminal rewards into dense, step-wise credit, and Entropy-Guided Historical Re-enactment (EHR) that replays authentic trajectories at high-uncertainty steps. Extensive experiments on four benchmarks demonstrate that PAPO significantly outperforms baselines, achieving gains of 4.5% on GSM8K, 4.8% on MATH500, 42.2% on Countdown and 16.1% on Sudoku.

Abstract:
Retrieval-augmented generation (RAG) has emerged as a critical paradigm for deploying reliable and knowledge-intensive large language model (LLM) applications. Its unique execution pipeline introduces large overhead in system retrieval, such as massive context loading and document dependencies. To reduce this overhead, existing work on RAG serving systems often relies on generic LLM inference traces or synthetic RAG datasets. However, these approaches fail to capture authentic temporal reuse patterns of RAG workloads, resulting in a disconnect between theoretical optimization and real-world deployment efficiency. To bridge this gap, this paper introduces RAGPulse, an open-source RAG workload trace dataset that preserves authentic temporal information missing in existing traces. Collected from a production-grade university Q A system serving over 40,000 students and faculty members since April 2024, the dataset comprises 134K requests over 389 days. It unveils key RAG dynamics including time-sensitive document reuse, skewed access patterns over long time scales, and bursty temporal locality. By capturing these temporal reuse patterns, RAGPulse empowers the systems community to develop and validate next-generation optimization techniques, offering practical guidance for efficient RAG serving. The trace is available at https://github.com/flashserve/RAGPulse.

Abstract:
Online memes are often carriers of unsafe content such as disinformation and propaganda. Their rapid evolution in response to news events and strong dependence on contextual information make it difficult for platforms and moderators to detect and moderate them effectively. Moreover, existing moderation studies typically classify memes as safe or unsafe based on individual images, lacking the ability to perform broader provenance analysis, i.e., where a meme originates, how it spreads, what cultural symbolism it represents, whether it poses potential risks to society, etc. To bridge this gap, we introduce MemeTrace, a multimodal LLM-based agent that automatically traces the provenance and potential risks of emerging memes. We evaluate MemeTrace on Meme100, a dataset of 100 real-world memes, using both human annotation and automatic metrics. The results demonstrate that MemeTrace substantially outperforms the VLM baseline. More importantly, MemeTrace achieves higher accuracy in detecting unsafe memes, especially those involving disinformation and sensitive content, and remains robust under visual perturbations.

Abstract:
Large Language Models (LLMs) provide rich semantic representations for recommender systems, but their semantic similarity may not align with collaborative preference patterns captured by collaborative filtering (CF). Directly combining LLM and CF representations can therefore distort the structural relationships encoded in the interaction graph. We propose Structure-Preserving Alignment (SPA), which reformulates CF–LLM integration as language-supervised representation alignment. SPA derives collaborative anchors from frozen CF representations and aligns multiple LLM-generated semantic views toward these anchors through cross-profile view competition. The aligned collaborative representations are then used for structure-preserving two-channel retrieval, which constructs behavior-aware context for downstream LLM-based preference prediction. Experiments on multiple datasets and CF backbones show that SPA consistently improves click-through rate (CTR)-style binary preference prediction.

Abstract:
Long-form story generation requires models to preserve narrative consistency across extended contexts, yet existing prompting-based methods often accumulate temporal, factual, character, commonsense, and stylistic errors as the story grows. We propose ConWriter, a training-free framework for consistency-aware long-form story generation. ConWriter writes stories incrementally at the scene level, guided by static story requirements, dynamic narrative memory, symbolic state reasoning, and uncertainty-aware risk signals. Rather than treating long-story generation as a single free-form decoding process, ConWriter maintains evolving story states, checks whether new scenes satisfy required narrative transitions, and uses uncertainty-aware risk signals to prioritize validation and localized repair. This enables consistency control during generation, before local errors propagate into later scenes. We evaluate ConWriter on ConStory-Bench across four long-story tasks, three target lengths, and multiple base LLMs following the official evaluation protocol. Across models and story lengths, ConWriter consistently matches or improves upon direct generation and outperforms the recent training-free baseline DOME in narrative consistency. These results demonstrate the effectiveness of lightweight neuro-symbolic consistency control for training-free long-form story generation. Code is available on GitHub.

Abstract:
GUI world models are increasingly evaluated as one-step next-screen predictors, yet their intended use is often as multi-step environments for GUI agents. This mismatch leaves a key requirement under-tested: generated states must remain contextually consistent when they are repeatedly reused for future interaction. We introduce GUI-CC, a benchmark that evaluates contextual consistency of GUI world models as agent environments rather than isolated next-screen predictors. GUI-CC contains two complementary tracks: an offline reference-action track that rolls models along real mobile GUI trajectories, and an online agent-loop track that lets fixed probing agents interact with model-generated UIs. We construct 500 offline trajectory tasks from GUIOdyssey and 200 emulator-verified online tasks across 30 mobile apps. GUI-CC evaluates transition fidelity, transition plausibility, contextual consistency, and task progress. Experiments show that plausible single-step generation does not guarantee reliable environment simulation: current models often produce usable-looking screens while failing to preserve task-relevant context or support executable multi-step rollouts.

Abstract:
Cultural fine-tuning has become the de facto paradigm for building culture-aware large language models (LLMs), yet existing optimization exclusively for alignment scores provides an incomplete portrait of cultural fidelity by systematically obscuring inherent cultural diversity. This unidimensional evaluation lens prompts a fundamental question: do models genuinely perceive distinct cultural nuances, or do they merely memorize dominant cultural values? To address this, we propose a synergistic evaluation framework that jointly formalizes cultural alignment and diversity. Through extensive benchmarking of six mainstream LLMs on the World Values Survey, this framework uncovers a systematic and critical trade-off: the pursuit of cultural alignment consistently incurs an acute expense of diversity, leading to severe "cultural flattening." Investigating this behavioral shift, we demonstrate that these superficial alignment gains stem from models artificially anchoring to dominant majorities, converging onto a monolithic response pattern that wipes out the heterogeneous distributions inherent to human groups. Crucially, our mechanistic analysis suggests that this diversity collapse is not merely a behavioral anomaly but more likely a structural consequence of the low-rank bias inherent in neural network optimization. Therefore, our findings expose the limitations of current post-training paradigms and call for a shift toward alignment objectives that preserve cross-cultural pluralism.

Abstract:
Leveraging Large Language Models (LLMs) to automatically formulate and solve optimization problems from natural language has emerged as an efficient paradigm for automated optimization. However, existing methods still exhibit limited generalization: they are sensitive to superficial narrative variations, reuse experience mainly at the case level, and struggle to adapt to shifted or emerging problem types. We propose OptSkills, an archetype-centric skill learning and reasoning agent system for optimization modeling and solving. To improve robust generalization, our system clusters problems by their underlying archetypes rather than surface narratives. To improve in-distribution generalization, it explores diverse modeling paradigms and solver configurations within each cluster, then distills successful trajectories into reusable workflow-level skills. To improve out-of-distribution generalization, it refines existing skills or expands the skill library using newly obtained trajectories. Our system achieves a state-of-the-art micro-averaged accuracy of 68.27% on datasets encompassing diverse problem types and scenarios. In addition, on MIPLIB-NL, a highly challenging large-scale and high-dimensional benchmark, it achieves 26.91% accuracy, outperforming DeepSeek-V3.2-Thinking by 4.53%. After skill learning on Nano-CO, it reaches 72.79% on the OOD NLCO benchmark. Code and skills are available at https://github.com/fujiwaranoM0kou/OptSkills.

Abstract:
Although large language models (LLMs) and retrieval-augmented generation (RAG) have advanced open-domain question answering (QA), they remain unreliable when documents evolve through amendments. Existing time-sensitive retrieval methods address only the disjoint-evolving environment, where each update is an independent snapshot. However, laws, policies, and regulations often operate in overlapping-evolving environments, where amendments override earlier clauses while preserving most content, creating strong semantic overlap across versions. We propose TimelyRAG, a retriever-agnostic framework that incorporates temporal distance into ranking to align queries with version-appropriate documents. We also introduce TimelyQABench, the first benchmark for regulation-heavy domains with overlapping-evolving challenges. Experiments show consistent gains, up to +28.6% in nDCG@10, highlighting the importance of temporal reasoning for reliable QA over evolving documents. All resources are available at https://github.com/kaist-dmlab/TimelyRAG.

Abstract:
Large language models compress centuries of discourse into fixed parameters, yet how they organize historical variation and whether this organization shapes generation remain unclear. We introduce the Time Travel Engine (TTE), a framework that compares geometric accounts of era-conditioned residual states and tests their behavioral accessibility under fixed prompts. Across six decoder language models in Chinese and English, broad eras repeatedly form ordered trajectories, although their exact spacing varies across model–language settings. Steering along these trajectories shifts both historical register and access to period-compatible knowledge, while adjacent anchors support local interpolation. Epistemic effects partly survive style removal, linking temporal steering to knowledge access beyond surface register. Across Chinese and English, partially alignable trajectories reveal corresponding chronological relations whose geometric realization remains language-specific. Together, ordered temporal geometry offers a testable account of how static language models organize diachronic variation, linking internal structure to historical register and access to period-compatible knowledge. Code and datasets: https://github.com/LilTiger/Time-Travel-Engine.

Abstract:
Large language models (LLMs) excel at text understanding and generation, yet still struggle to reliably understand and apply externally provided procedural rules at scale. To evaluate this capability, we introduce RuleWorld, a large-scale benchmark that reformulates rules as globally reusable abstract units rather than instance-specific facts. In RuleWorld, several scenarios, including single-rule, parallel multi-rule, and multi-hop reasoning, are settled for comprehensive evaluation. We further propose DynaRule, an end-to-end framework that injects the given rules into the KV cache and turns retrieval into an internal, learnable, step-wise process. Specifically, DynaRule employs Stacked Step-Level Attention Training with a special token to enable dynamic rule re-attention and updating during inference. In this way, the model can re-attend to the most relevant rules at each step, dynamically replacing outdated ones to support more stable multi-step reasoning. Experiments on RuleWorld show that existing LLMs face challenges under large rule pools, while DynaRule improves average QA accuracy by up to 19 points and achieves over 85% Recall@1 at 10K rules, outperforming strong baselines by large margins.

Abstract:
While MLLMs have made significant strides in chart comprehension and video understanding, current evaluations largely isolate these capabilities, leaving a critical gap in understanding temporally evolving structured visual information. To address this gap, we introduce DVBench, a benchmark for evaluating MLLMs on data videos, a storytelling medium that integrates dynamic charts with structured narratives. We decompose data video understanding into five dimensions. DVBench comprises 300 real-world data videos and 1,000 human-verified QA pairs curated through a rigorous semi-automated pipeline. Extensive evaluations of nine MLLMs show that Gemini-3.1-Pro achieves the best overall performance, while Kimi-k2.5 is the strongest open-source model. We further identify two notable phenomena: open-source model performance does not scale strictly with parameter size, and narrative proficiency does not guarantee visual capability. Fine-grained analyses and ablation studies further reveal dimension-specific weaknesses and the effects of frame configurations and subtitle inputs, informing future MLLM development. DVBench is publicly available at https://bomiaowang.github.io/DVBench/.

Abstract:
Large language models have become the dominant workload in modern GPU clusters, driving unprecedented demand for compute during both training and inference. Yet GPU clusters typically operate at only 30–50% utilization, largely because schedulers rely on user-provided runtime estimates that are notoriously inaccurate. Running thousands of LLM inference requests through shared datacenters further compounds this underutilization. Accurate runtime prediction is a key enabler of LLM efficiency: it supports shortest-job-first scheduling, backfilling, dynamic resource allocation, and cost-aware instance selection, while providing NLP practitioners with reliable completion estimates for training runs and serving latency guarantees. However, LLM workloads pose unique prediction challenges: attention complexity scales quadratically with sequence length, autoregressive decoding produces variable-length outputs, and 3D parallelism creates intricate performance interactions. This survey systematically reviews runtime prediction methods with emphasis on LLM training and inference workloads, presenting a feature taxonomy, analyzing design choices of existing prediction methods, surveying public datasets and highlighting their limitations. We identify applications critical to LLM efficiency and key open challenges including generalization to rapidly evolving model architectures and hardwares.

Abstract:
Biomedical multi-hop question answering (QA) requires models to connect evidence across intermediate entities such as diseases, drugs, proteins, and phenotypes. Existing agents typically rely on static retrieval workflows or coarse-grained prompt rewriting, which can lead to instruction drift when reasoning procedures need to be updated. We propose SSE-Bio, a structured self-evolving agent with an agentic retrieval policy for multi-hop biomedical reasoning. Instead of globally rewriting agent instructions, SSE-Bio maintains a structured state, selectively retrieves knowledge triplets and prior templates through a trainable proxy policy, and improves its reasoning memory through fine-grained template editing. To optimise retrieval decisions, we introduce a proxy-training strategy based on group relative policy optimization, where the proxy is improved through decision-contrastive groups over alternative retrieval choices. Experiments on three biomedical multi-hop QA benchmarks show that SSE-Bio consistently outperforms existing baselines, achieving an improvement of 6.56 absolute points over the strongest self-evolving baseline on BioHopR.

Abstract:
Alpha factor mining is a fundamental task in quantitative trading, aimed at discovering interpretable signals that can predict asset returns beyond systematic market risk. While traditional methods rely on manual formula design or heuristic search with machine learning, recent advances have leveraged Large Language Models (LLMs) for automated factor discovery. However, existing LLM-based alpha mining approaches remain limited in terms of automation, generality, and efficiency. In this paper, we propose Chain-of-Alpha, a novel, simple, yet effective and efficient LLM-based framework for fully automated formulaic alpha mining. Our method features a dual-chain architecture, consisting of a Factor Generation Chain and a Factor Optimization Chain, which iteratively generate, evaluate, and refine candidate alpha factors using only market data, while leveraging backtest feedback and prior optimization knowledge. The two chains work synergistically to enable high-quality alpha discovery without human intervention and offer strong scalability. Extensive experiments on real-world Chinese A-share and US S P 500 benchmarks demonstrate that Chain-of-Alpha outperforms existing baselines across multiple metrics, presenting a promising direction for LLM-driven quantitative research.

Abstract:
Chain-of-thought (CoT) reasoning has dramatically improved large language models (LLMs) by allowing them to decompose problems into intermediate steps. While CoT is widely effective for linguistic tasks, text-only CoT forces models to serialize visual problems into awkward prose. Although architectural solutions exist to process visual inputs, the community lacks a massive, multi-step, self-corrected dataset to teach models how to build and maintain internal visual workspaces when solving purely textual reasoning problems. To address this limitation, we introduce CoVA-SFT, a highly structured corpus of 51.9K samples containing over 222K multimodal reasoning steps across 5 distinct layout families and 17 complex tasks, and CoVA-Bench, a companion benchmark of 1,700 held-out test samples spanning the same tasks for reproducible evaluation. By providing explicit rationale formulations, agentic renderings, and verification loops, CoVA-SFT teaches multimodal language models to interleave text and visual abstractions. We validate the dataset by demonstrating that models fine-tuned on CoVA-SFT outperform all interleaved CoT baselines by more than 2× on average on CoVA-Bench, though they still fall short of strong text-only CoT baselines, highlighting open challenges for future work.

Abstract:
Construction accident narratives contain rich causal information, but the evidence is often implicit, long-span, and distributed. We introduce ConstructCIE, a manually annotated dataset for Causal Information Extraction from OSHA construction accident reports. The dataset uses a hierarchical schema for accident types, causal factors, sub-causal factors, and supporting evidence spans. We evaluate supervised sequence taggers and instruction-tuned LLMs in an end-to-end hierarchical extraction setting. Results show that most evaluated models achieve strong accident-type prediction and recover broad causal meaning but remain limited in precise span-level extraction. Joint Hierarchical Extraction generally achieves stronger exact and soft matching, while Individual Hierarchical Extraction sometimes achieves higher keyword F1. Error distributions vary by extraction strategy, but evidence-selection and span-boundary errors remain common. These findings show that reliable Causal Information Extraction for construction accidents requires stronger domain grounding and more accurate evidence extraction. The code and data can be found at https://github.com/lab-flair/ConstructCIE.

Abstract:
Long-tail autonomous driving failures are often framed as rare-object recognition errors. We argue that this view is incomplete: the decision-critical question is not only whether a model recognizes an unusual object, but whether it infers how that object changes the ego vehicle’s feasible high-level actions. We formalize this problem as decision-level driving affordance prediction, where a model maps a front-view image, ego-motion history, and navigation command to a structured longitudinal–lateral meta-action. To evaluate this capability, we introduce CoLT-Drive, a 3,536-sample counterfactual long-tail benchmark that inserts rare objects into otherwise fixed driving scenes and measures whether models predict acceptable action pairs. To improve deployable small VLMs, we propose KPA, a knowledge-preserving adaptation framework that combines structured perception-to-decision prompting, SLERP-based expert merging, and RegMoE, a regime-aware LoRA mixture-of-experts module. KPA preserves the pretrained model’s open-world knowledge while allocating lightweight adaptation capacity to different driving decision regimes. Experiments on an in-domain driving split and CoLT-Drive show that KPA achieves 60.8% pair accuracy on CoLT-Drive, outperforming the pretrained Qwen3-VL-2B baseline (50.3%) and LoRA SFT (32.4%) while maintaining competitive in-domain accuracy.

Abstract:
We investigate how vision-language models (VLMs) handle context-memory conflicts; that is, situations in which the model is given information in context that differs from what was stored parametrically during training. We document asymmetric biases: models tend to prefer in-context information about entities which appear in text, but prefer parametric information about entities which appear in images. We relate this asymmetry to the late representational alignment across modalities, showing that the longer processing time associated with resolving visual entities prevents the suppression of the model’s usual factual recall mechanism, thus resulting in more parametric answers. Chain-of-thought reasoning does not appear to resolve the gap, but increasing the amount of visual information in the context does show an effect. These results illustrate the complexity of ensuring consistent behavior as models become increasingly multimodal and retrieval-augmented.

Abstract:
The way a knowledge graph is serialized as text for inclusion into LLM context can substantially affect downstream LLM performance, with prior work reporting tens of percentage points variations across formats. However, existing studies consider only a limited set of representations and predate frontier reasoning models, which now offer stronger reasoning, instruction-following, and long-context capabilities. We evaluate 15 graph representation formats across nine frontier LLM configurations on ontology-based knowledge extraction and structural graph reasoning tasks. While knowledge extraction is relatively insensitive to different serialization formats, the choice of the optimal format for structural reasoning tasks significantly affects accuracy. We demonstrate that the optimal representation choice depends on model, query, and task and propose to approach graph serialization as a dynamic orchestration decision.

Abstract:
Large Language Models (LLMs) are considered to be a more efficient and cost-effective alternative to human judgment for Machine Translation (MT) evaluation. With MT evaluation spanning a large number of language pairs, domains and levels of annotation granularity, LLMs must be thoroughly evaluated across these dimensions before being reliably used as alternatives to human evaluation. In this paper, we evaluate the performance of LLMs for two prominent MT quality evaluation schemes: Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA) by comparing their agreement with human annotators. We present results on a long-context test set of 70 language pairs and the publicly available WMT23 and WMT25 data, investigating both score and error span annotation agreement across a variety of language pairs and domains. Our results show that while LLM agreement with human annotators exceeds agreement between human annotators for some evaluation tasks, both vary substantially across annotation schemes, language pairs and domains and remain unreliable for most settings. Furthermore, we identify challenges facing both human and LLM annotators: humans are particularly challenged by fine-grained MQM annotations and low-resource language pairs, while LLMs struggle with minor errors, wrong language variants and error span annotation. Our results highlight the potential for improvement for both human and LLM annotation performance, possibly through human-LLM collaborative annotation pipelines that address the reliability issues identified in this work.

Abstract:
Chain-of-thought reasoning unfolds in discrete token space: each step is committed as text, errors propagate, and eliciting good traces presupposes traces to imitate. Reasoning instead in a model’s continuous representation space – where intermediate states are vectors rather than words – sidesteps these constraints, but leaves open how those latent states should be computed. We approach this along two axes. First, we keep a large language model (LLM) frozen and use it for what it is already good at – modeling and decoding sequences – while a small auxiliary network supplies continuous latent thoughts as input. Second, we produce those latents by recurrence: a tiny recurrent reasoner refines them over many steps, decoupling the depth of computation from the size of the model, so that the latents are a product of iterative processing rather than a single forward pass. We instantiate this as Latent Recurrent Thoughts (LRT): a task-dedicated proposer supplies base latents, a recurrent reasoner refines them through bounded residual corrections, and the frozen LLM decodes the answer. On symbolic reasoning with answer supervision but no reasoning traces (Countdown-4, Sudoku) and on natural-language reasoning (HumanEval, MBPP, StrategyQA), LRT substantially outperforms prior frozen-decoder continuous-space reasoning methods under an identical decoder, prompt, data, and training budget, and outperforms non-thinking-mode chain-of-thought prompting on the same backbone at a small fraction of its inference compute.

Abstract:
This study investigates semantic changes in four historically loaded German political lexemes: Volk, Nation, Staat, and Land. These terms denote basic categories of peoplehood, national community, institutional statehood, and territorial belonging, but their use and emotional associations were reshaped by the ideological language of National Socialism and by the political reconstruction of Germany after 1945. Using DWDS as the empirical basis, we compare a pre-war and wartime baseline (1918–1945) with the post-war period (1945–2024). Our study combines frequency analysis, collocation analysis, and LLM-assisted annotation along three dimensions: nationalist intensity, emotional valence, and temporality. The results suggest different trajectories across the four lexemes: Staat and Land remain comparatively stable as institutional and territorial terms, while Volk shows clearer signs of contextual restructuring, semantic distancing, and possible renewed political markedness. Nation displays intermediate patterns shaped by historical rupture, reunification, and contemporary political discourse. By integrating corpus-based semantic analysis with conceptual history, our study contributes empirical evidence to debates on post-war German national identity and the linguistic reconfiguration of political concepts

Abstract:
Competitive debate is a multi-turn, zero-sum adversarial task with opposing stances that requires debaters to anticipate opponents’ counterarguments in real-time interactions and select strategies yielding long-term advantages. Existing large language model debate methods predominantly adopt a greedy generation paradigm that optimizes only the immediate effectiveness of the current rebuttal while lacking forward-looking planning for the overall debate trajectory. To address this limitation, we propose MCTS-Debater, a Monte Carlo Tree Search framework that models competitive debate as a Markov Decision Process. Rather than selecting a single locally optimal response, MCTS-Debater systematically simulates multi-step adversarial interactions under diverse rebuttal strategies, estimates their long-term expected win rates via tree search, and synthesizes the top-performing strategies into a coherent final rebuttal through multi-strategy fusion. Experimental results on the ORCHID and IQ2 datasets demonstrate that MCTS-Debater significantly outperforms existing baselines in logical validity, factual accuracy, and overall persuasiveness. Source code is provided in the supplementary material.

Abstract:
Conversational stance detection aims to identify a speaker’s attitude toward a target in multi-turn dialogues, while the zero-shot setting requires generalization to unseen targets. Existing methods often rely on coarse-grained semantic representations or static knowledge injection, limiting their ability to capture subtle stance cues and utilize commonsense knowledge. To address this, we propose FIRE, a reasoning-enhanced framework that integrates fine-grained stance cues and commonsense knowledge for zero-shot conversational stance detection. FIRE adaptively mines sentiment, rhetorical, referential, cross-turn stance dependency cues and commonsense knowledge, and fuses them into explicit reasoning chains for target-aware stance inference. By combining supervised instruction tuning with reinforcement learning, FIRE further improves reasoning-chain quality and stance prediction. Experiments on benchmark datasets show that FIRE consistently outperforms strong baselines across different target types and LLM backbones, demonstrating the effectiveness of fine-grained cue modeling and knowledge-driven reasoning.

Abstract:
Post-training pruning compresses Large Language Models (LLMs) without full-model retraining, but performance degrades sharply at high sparsity. We identify two limitations in existing methods: local importance estimates miss global spectral structure, and uniform within-block sparsity ignores heterogeneous sensitivity across transformer projections. We propose PRISM, a post-training pruning framework with training-free mask construction and optional fixed-mask reconstruction optimization. PRISM augments activation-based scoring with two adaptive-rank SVD-derived signals for principal-subspace and residual-aware importance, combines the factors through parameter-free CV-based fusion, and allocates sparsity non-uniformly across attention and MLP projections using a gradient-weighted matrix-level sensitivity proxy. Across six Qwen3 and LLaMA models, PRISM consistently outperforms existing pruning baselines, yielding up to 16.8% relative improvement in WikiText-2 perplexity under the evaluated sparsity settings.

Abstract:
In Korean medical LLMs, domain-trained models underperform General LLMs of comparable parameter scale. We attribute this paradoxical failure to a cascading two-bottleneck structure: tokenizer over-fragmentation (Fragmentation bottleneck, B1) compounded by a learning-stage representation-alignment failure (Alignment bottleneck, B2). Using the Korean medical domain as a case study, we diagnose this paradox and propose VocMoE, which is designed to address both bottlenecks: DVE selects domain vocabulary from a medical BERT via an information-theoretic score, and MoDE uses the DVE output (the partition of token ids) as the vocabulary-conditioned, deterministic routing key of the MoE FFN. In this view, the vocabulary partition itself serves as a routing prior, shifting routing from hidden-state learning to vocabulary identity. MediKo-9B-A4B reaches simultaneous SOTA on KorMedMCQA (57.5, +14.8 pt), KMMLU Medical, and HAE-RAE, while MediKo-4B-A1B (active 1.69B) surpasses Qwen3-8B (8B) and KULLM3 (10.7B) on KorMedMCQA, updating the active-parameter efficiency frontier.

Abstract:
Vision-Language-Action (VLA) models map visual observations and language instructions to low-level control actions, but their reliance on the current observation often limits temporal continuity in long-horizon manipulation. We propose ProRet-VLA, a temporal reasoning framework that links prospection and retrospection through predictive consistency. ProRet-VLA maintains compact episodic memories and predicts future observations from temporally enriched context. Rather than using future prediction only as an auxiliary target, we use its prediction error as a self-supervised deviation signal to train a lightweight Deviation Predictor. During inference, the predicted deviation conditions token-wise fusion between current context and retrieved memory, allowing memory usage to vary across decision stages without accessing future observations. Experiments on CALVIN ABC-D and LIBERO show that ProRet-VLA improves long-horizon task completion, achieving an average length of 4.50 on CALVIN and a 96.7% average success rate on LIBERO. Real-world experiments on two long-horizon manipulation tasks further indicate improved temporal progress tracking over strong VLA baselines.

Abstract:
Large language models have made role-playing agents viable as interactive characters, virtual companions, game agents, and social actors. The harder problem, however, is not producing one convincing response but sustaining a coherent role through long, changing, and sometimes adversarial interactions. This survey examines recent work on role consistency: the maintenance of identity, character knowledge, expressive style, memory, values, behavioral logic, and safety boundaries over time. We organize the literature around role specification, role-state maintenance, role-aware reasoning, role-consistent alignment, and evaluation.

Abstract:
Converting in-service reinforced-concrete (RC) building blueprints into simulation-ready models underpins safety assessment and seismic retrofit, but the process remains largely manual. Direct prompting of a multimodal large language model (MLLM) over a full scanned sheet is unreliable: plausible-looking outputs often violate engineering constraints on beam–column support, span count, or 3D continuity.We present BlueprintAgent (BPA), a constraint-triggered multimodal agent for simulation-ready frame extraction from scanned blueprints. BPA treats the MLLM as the primary reader and decision maker, with OCR and computer vision supplying localized evidence. Its novelty is not the tool stack but a control mechanism: engineering constraints are realized as callable validators whose entity-level conflict reports trigger targeted MLLM revisits over the relevant local region—an inference-time control distinct from fixed pipelines and free-form self-reflection.We evaluate BPA on 300 real scanned blueprint sheets from 20 anonymized RC frame projects, against five baselines and six ablations. BPA reaches a macro-averaged Beam F1 of 0.994, against 0.301 for single-MLLM zero-shot and 0.820 for a fixed pipeline; removing MLLM-led axis adjudication collapses Beam and Column F1 on complex multi-sheet projects. For dense technical drawings, engineering constraints are best deployed as triggers for entity-level targeted revisits rather than as post-hoc output filters.

Abstract:
LLM agents increasingly rely on externalized skill libraries, yet existing maintainers typically delegate skill editing to a prompted LLM, leaving the question of how to train a skill maintainer open. We present RepairGate, a 9B drop-in wrapper around the standard candidate patcher: a counterfactually-trained post-trial gate accepts, blocks, or overrides each candidate patch, with a structured repair protocol for field-level edits and overlap-aware regularization to control library growth. Across six base agents on SkillFlow, RepairGate delivers +7.73 pp over Vanilla and +9.13 pp over prompted +Iter, with the largest gains on the weakest base agents where prompted iteration regresses, at a standardized 0.033/trial local overhead on top of the shared candidate-patching pipeline. Code and data: https://github.com/xxy33/RepairGate

Abstract:
Large Language Models (LLMs) have achieved state-of-the-art performance in Aspect Sentiment Quadruplet Prediction (ASQP). However, we argue that this success often relies on superficial positional heuristics rather than robust structural reasoning. In this paper, we propose a Probe-and-Cure framework to scrutinize and enhance LLM robustness. First, we introduce a Multi-Surgical Adversarial Attack protocol as a diagnostic tool. Our study reveals that SOTA models are “Fragile Giants”: they suffer severe performance degradation when facing logical distractors and deep syntactic embeddings. To address this, we propose Adaptive Bi-SADA (Bidirectional Structure-Aware Data Augmentation). Unlike uniform augmentation strategies, our method constructs a length-aware curriculum: it applies natural structural hardening to short sentences to prevent overfitting, and syntactic normalization to long sentences to distill core dependencies. We implement a strict generation-time verification protocol to ensure semantic invariance. Experiments on ASQP and ACOS tasks demonstrate that our method not only achieves new SOTA F1 scores but also effectively transforms superficial pattern matching into robust structural reasoning, significantly closing the performance gap under adversarial stress tests.

Abstract:
Reinforcement learning (RL) post-training has shown to improve reasoning in large language models (LLMs). However, there has been little exploration on the problem of data contamination in RL post-training, potentially undermining generalization and evaluation reliability of the training process itself. Existing detection methods primarily rely on output-level signals such as likelihood or entropy, which become unreliable for RL-trained models since RL shapes behavior through trajectory-level rewards rather than token likelihoods. We propose LaRA, a layer-wise representation analysis framework for detecting contamination in RL post-trained LLMs. LaRA introduces three complementary metrics, measuring perturbation sensitivity, directional collapse, and local representation rigidity under controlled perturbations. We find that contamination produces progressive geometric deviations across layers, including amplified perturbation sensitivity, stronger directional collapse, and enhanced local rigidity. Based on our findings, we also develop a contamination detection protocol that aggregates representation-level deviations across layers and metrics. Experiments on RL-trained reasoning models show that our protocol outperforms existing output-level baselines for contamination detection.

Abstract:
Static QA and code-generation benchmarks only partially capture the role that large language models (LLMs) now play as coding agents and research tools. We introduce OptiArena, a budget-controlled testbed for studying whether LLMs can improve executable game-playing algorithms through five rounds of code edits within a fixed minimal scaffold and under bounded evaluator feedback and fixed resource budgets. The testbed uses two optimization regimes, surface obfuscation controls, calibrated references, held-out/stress splits, and diagnostics for degradation and exceptional failures, with LLM API cost reported separately from local evaluator wall-clock. The empirical study asks three questions: whether models can close the calibrated gap between a designated weak starter and an editable competent baseline, whether they can refine editable competent baselines without damaging them, and whether gains survive surface obfuscation controls. Across twelve frontier LLMs and five games, models improve designated weak starters more consistently than they refine editable competent baselines, with substantial variation across games and models. OptiArena provides a practical testbed for measuring bounded-resource algorithm optimization within the five-edit, fixed-scaffold setting studied here. Code is available at https://github.com/WJ-Peng/OptiArena.

Abstract:
Automatic medical coding assigns ICD codes to clinical notes, but it remains challenging due to long documents, imbalanced label distributions, and diverse terms. These challenges are especially severe for rare codes, which have limited training instances and are easily confused with semantically similar labels. We introduce CoLa-ICD, a knowledge-enhanced framework for long-tail prediction. CoLa-ICD enriches ICD labels with external terms, models dependencies among related codes, and learns stronger alignment between label semantics and clinical evidence for long-tail prediction. Experiments show that CoLa-ICD improves long-tail prediction with larger gains in larger and sparser label spaces and achieves state-of-the-art performance in AUC, F1, and P@ k . Our code is available at https://github.com/youwillbethebest/Cola-ICD.

Abstract:
Traditional vision search follows the multi-stage cascading architecture (MCA) paradigm to balance efficiency and conversion, yet optimization objectives collide across stages, making it difficult to achieve Pareto optimality. Meanwhile, pronounced multi-view discrepancies of the same item undermine retrieval accuracy, as embedding-similarity methods fail to bridge large appearance variations. In this paper, we propose OneVision, an end-to-end generative framework that addresses both challenges. OneVision introduces VRQ, a vision-aligned residual quantization method that assigns consistent shallow-level codes to visually similar images of the same item while preserving fine-grained differences at deeper levels. It further employs a multi-stage training pipeline to retrieve all relevant items across views for a query image and incorporates user-specific signals for personalized preference modeling. In offline evaluations, OneVision performs on par with online MCA. Dynamic pruning further reduces its inference latency by 18% over the unpruned variant. In A/B tests, it achieves significant improvements: +3.61% item CTR, +2.43% CVR, and +5.89% order volume. The code will be publicly available.

Abstract:
Large reasoning models (LRMs) have achieved remarkable success on complex tasks, yet their tendency to "overthink” leads to inefficiencies. Although "save-thinking” prompts are intended to mitigate this issue, we find that LRMs still frequently enter the "Still-thinking” mode instead of the expected "No-thinking” mode, especially when facing difficult queries. First, we examine the LRM’s confidence when terminating thinking process, finding that high perplexity signals internal uncertainty and precedes the thinking process. Second, we analyze the divergence in internal attention distributions across different LRMs. Third, we investigate where the LRM allocates its attention within the prompt, identifying a critical "re-reading” pattern. Based on these analyses, we propose an attention intervention method to regulate this behavior. While this intervention successfully suppresses thinking, we find it causes a drop in accuracy, confirming that the thinking process is functional. Our work provides a mechanistic explanation for this behavior, highlighting the trade-off between instruction following and reasoning correctness, and offers insights for developing more controllable LRMs.

Abstract:
Multimodal diffusion Large Language Models (dMLLMs) emerge as a compelling alternative to traditional autoregressive paradigms. But they still suffer from hallucinations driven by an inherent language bias. Through an analysis of the dMLLM inference dynamics, we observe that visual tokens frequently receive disproportionately low attention scores. We identify that the conventional low-confidence remasking strategy redundantly leverages logit-based probabilities during the token selection process, thereby exacerbating this language bias. To address this vulnerability, we propose Attention-Aware Remasking, a training-free, plug-and-play strategy that incorporates newly defined visual confidence scores to select the final output tokens, effectively promoting visual grounding. To enhance the logical integrity and inference efficiency of dLLMs, we further propose clue-based answering and echo-filling. Evaluations on representative Visual-Language Natural Language Explanation (VL-NLE) tasks, including VQA-X and e-SNLI-VE, demonstrate that our method achieves the highest BERTScore on both VQA-X and e-SNLI-VE, and the best answer accuracy on VQA-X among the compared methods. Our study reveals that the widely adopted logit-based remasking is not necessarily the optimal approach, highlighting promising new avenues for visually grounded inference in dMLLMs. Our code is available at: https://github.com/yechancha/Attention-aware-Remasking.

Abstract:
Online lifelong learning agents must decide not only how to act but also when to consult prior experience to continually improve on long-horizon tasks. Existing methods typically retrieve memories passively, such as at task initialization or after each step, and therefore miss knowledge gaps that arise during interaction. We propose ProactAgent, an experience-driven lifelong learning framework for proactive retrieval over a structured Experience Base. ProactAgent continually improves through ExpOnEvo, which jointly updates policies and refines memory, organizing past interactions into factual, episodic, and skill repositories. It further introduces ProactRL, which treats retrieval as an explicit policy action and learns when and what to retrieve. By comparing paired continuations from identical interaction prefixes with and without retrieval, ProactRL provides step-level process rewards that encourage retrieval only when it improves task outcomes or efficiency. Experiments on SciWorld, AlfWorld, and StuLife show that ProactAgent consistently outperforms all baselines, achieving up to 32% relative improvement in success rate and over 33% reduction in interaction rounds.

Abstract:
While AI-assisted text-based counseling is gaining attention, it remains empirically unclear which counselor behaviors are associated with higher dialogue quality. Existing research often focuses heavily on Reflection, borrowing frameworks from Motivational Interviewing. To address this gap, we conduct a multi-layered analysis using KokoroChat, a large-scale Japanese text counseling dataset conducted by professional counselors and trainees, newly annotated with counselor strategy tags and client distress levels. Our results show that, under the quality indicators used in this study, Affirmation is more consistently associated with session quality than Reflection among the analyzed strategies. Cross-dataset transfer experiments further suggest that this quality signal can be observed to some extent on ESConv, an English dataset with non-expert supporters. These findings provide empirical implications for counselor training and emotional support system design. We release the additional KokoroChat annotations and experimental source code at https://github.com/UEC-InabaLab/BeyondReflection

Abstract:
Conversational entrainment is well-studied in monolingual and written contexts, but remains underexplored in spoken code-switching (CSW). We present a cross-lingual analysis of entrainment in Mandarin-English, Hindi-English, and Spanish-English dialogue, replicating and extending an established Spanish-English framework (Bhattacharya et al., 2024) to two new language pairs, and show that, while lexical entrainment generalizes across language pairs, entrainment over acoustic-prosodic and CSW style aspects exhibits context-specific variation. We build on these findings by asking whether classification models capture these human behavioral patterns. Using feature importance and ablation analyses, we find that classical and Transformer-based classifiers detect entrainment reasonably well but consistently prioritize features other than those most salient to human entraining behavior. Our approach introduces a human-grounded framework for evaluating model decision-making in multilingual stylistic contexts, and suggests challenges for developing conversational agents capable of producing naturalistic code-switched speech.

Abstract:
Scientific-paper reading requires models to recover a paper’s core argument, connect claims to textual and multimodal evidence, and avoid unsupported inferences across dialogue turns. We introduce PaperGraph-Bench , a graph-guided benchmark that turns each paper into a paper-specific evaluation state graph with macro stages, evidence-grounded knowledge components (KCs), verified reasoning edges, multimodal assets, rubrics, and trajectory state. The graph drives macro coverage questions, challenge questions, reasoning threads, and bounded repair turns, while the evaluator tracks KC coverage, challenge failures, hallucination events, and repair outcomes. Experiments on a human-audited prototype with ten models show that coverage, robustness, hallucination type, and repair behavior reveal distinct paper-reading abilities. Our code and current dataset are available at https://github.com/furao0011/PaperGraphBench.

Abstract:
LLM-based agents are increasingly deployed for complex tasks requiring planning, tool use, and interaction with external services. Their reliance on untrusted external content exposes them to indirect prompt injection (IPI), in which adversarial instructions embedded in retrieved data hijack agent behavior. Existing attacks rely on static payloads that cannot adapt to agent-specific defenses; even recent adaptive methods lack structured feedback to guide optimization. We introduce IterInject, a feedback-guided iterative framework that closes the loop between injection, diagnosis, and refinement: a rule-based diagnoser produces structured outcome labels with behavioral descriptions, and an LLM-based optimizer refines payloads conditioned on the full optimization history. A synthesis step generates new disguise seeds from failure patterns, enabling the strategy space to self-evolve. On AgentDojo and InjectAgent, IterInject substantially outperforms static baselines and existing adaptive methods across four victim models. Extension experiments on Claude Code, a production-grade coding agent with layered defenses, show that optimized payloads achieve full success on 5 of 9 targets; even those that resist full exploitation exhibit measurable improvement from iterative refinement. We further present a mechanistic analysis of IPI, identifying an attention-mediated threshold mechanism in mid-to-late layers; three causal interventions validate this finding and point to concrete defense directions.

Abstract:
Effective emotional-support dialogue requires more than fluent empathy: responses must adapt to the support seeker’s evolving state. Existing emotional-support systems are largely optimized at the response level and may therefore learn the form of supportive counseling without reliably adapting to client-side change. We introduce ARIA, Affect-Responsive Interaction Alignment, a client-centered alignment framework that constructs DPO preference pairs from the support seeker’s subsequent multimodal affective trajectories observed in human interactions. ARIA distills audio- and visual-derived valence trajectories into text-based predictors and uses cross-modal consensus and uncertainty-aware filtering to identify high-confidence preferences associated with delayed client-side valence improvement. Experiments show that ARIA improves process-oriented measures of client progress beyond surface compliance and produces support that is more timely and calibrated to the seeker’s evolving openness. Behavioral analyses further show a phase-sensitive shift from restrained pre-open support toward more active support after openness emerges. Human and counseling-expert evaluations provide evidence that ARIA’s responses are more timely, targeted, and supportive of the client’s own process.

Abstract:
Automated evaluation of creativity tasks remains challenging for LLM-as-a-Judge, as LLM is susceptible to biases such as verbosity bias and leniency bias. Such limitations are particularly evident in Contextually-Grounded and Procedurally-Structured Tasks (CGPST), a complex multi-step creativity task where inter-step dependencies, highly subjectivity, and wide scoring ranges lead to more unstable and biased judgments. Existing approaches either rely on task-specific training or directly apply LLM-as-a-Judge, both of which struggle to ensure reliable evaluation under such complexity. To bridge these gaps, we propose CreaEval, an automated creativity evaluator for CGPST that decouples typical LLM-as-a-Judge into analysis and judging. Correspondingly, CreaEval involves two critical phases: Memory-augmented Analysis, a SoT-LLM converts multi-step responses into structured evaluation evidence, incorporating cross-step memory; and Evidence-based Judging, a Judge-LLM uses the extracted evidence for judging without accessing raw responses. Comprehensive experiments on CGPST show that CreaEval achieves an average human-LLM agreement of 0.64 (QWK) on CGPST, improving over the second-best baseline (supervised training) by 0.17. The code is available at https://anonymous.4open.science/r/CreaEval-0058/.

Abstract:
Expense reimbursement internal control is a high-stakes corporate finance task that combines multimodal, visually rich evidence (e.g., receipts and approval forms), firm-specific policy text, and multi-step compliance reasoning. No existing benchmark evaluates large multimodal models (LMMs) in this domain. We introduce ReimburseBench, the first benchmark targeting full-workflow multimodal audit reasoning in corporate internal control. It comprises 151 expert-constructed reimbursement workflows, each packaging an application, receipts and supporting documents, and a firm-specific policy, evaluated by a process-level rubric grounded in twelve auditing logic points. Benchmarking nine frontier and locally deployable systems, we find the strongest attains only 72.5%, and no model exceeds 47.7% on hard workflows. Beyond this aggregate score, risk-aware, process-level evaluation surfaces three structural findings: (i) a persistent gap between binary admissibility judgment and precise quantitative computation; (ii) a systematic lenient bias in monetary predictions that poses deployment risk; and (iii) a performance pattern consistent with compositional-generalization difficulties under realistic workflow load. We provide ReimburseBench as a testbed for advancing LMMs toward trustworthy financial systems. Data and code: https://github.com/ikerzhan/ReimburseBench.

Abstract:
Generating academic paper titles requires balancing two key goals: faithfulness to the content and attractiveness to readers. Existing methods often treat title generation as a simple summarization task, producing accurate but unengaging titles. While multi-objective Direct Preference Optimization (DPO) offers a potential solution, our empirical analysis shows that combining conflicting objectives into a single reward is associated with substantial gradient interference during training. To address this problem, drawing inspiration from Pareto dominance in multi-objective optimization, we propose Multi-Objective Dominance Alignment (MODA), a data-centric training strategy tailored for academic title generation. MODA retains preference pairs with strict Pareto dominance, thereby reducing exposure to supervision with opposing reward directions. We implement MODA using dual reward models to independently score both dimensions. Across our experiments, this filtering strategy improves the observed balance between faithfulness and attractiveness, outperforms Human Author baselines, and approaches the performance of frontier models despite using an 8B backbone. Our code is publicly available at https://github.com/realyoume/title_it_right.

Abstract:
Tool-using LLM agents execute multi-step tasks by iteratively reasoning and interacting with external tools. However, they are vulnerable to indirect prompt injection (IPI), where adversarial instructions embedded in tool outputs can divert benign user intents into unsafe actions or unintended goals. Existing defenses mainly prioritize reducing unsafe behaviors, while paying limited attention to preserving the original user intent. As a result, safety improvements may come at the cost of interrupting execution or distorting benign task goals. This reveals a key gap in current LLM-agent defenses: the trade-off between safety and intent preservation remains largely underexplored. We propose RecoverAgent , an intent-preserving recovery framework that enables agents to continue execution after detecting unsafe intermediate responses. Instead of terminating the task, RecoverAgent rewrites unsafe responses into safe and executable alternatives while maintaining alignment with the original user intent. Experiments on R-Judge, InjecAgent, AgentHarm, and BadAgent demonstrate that RecoverAgent reduces attack success and harmful behaviors while substantially improving trajectory completion and intent fidelity compared to other defenses under IPI.

Abstract:
Large Reasoning Models (LRMs) are valuable intellectual property due to their substantial training and alignment costs, creating a need for reliable ownership verification under black-box access, where a verifier can only query a suspect model and inspect its returned output. Existing trigger–response fingerprints are easy to test but are often weakly coupled with Chain-of-Thought (CoT) reasoning and can be fragile under paraphrasing or post-hoc model modifications. We propose CoT-Lock, a reasoning-mediated fingerprinting framework that casts ownership verification as a structured conflict-resolution task. CoT-Lock constructs verification queries in which a default semantic constraint is superseded only by a higher-priority meta-override rule, training the model to resolve fingerprint activation through priority-based arbitration. Our primary ownership test is output-only on confidential held-out probes; when reasoning traces are visible, trace-level verification further checks the expected arbitration process. Experiments on representative reasoning architectures show that CoT-Lock achieves stable black-box verification while preserving reasoning ability with minimal performance degradation, low false positives, and improved robustness against paraphrasing, fine-tuning, pruning, and quantization. Our code is available at https://github.com/acang425/CoT-Lock.

Abstract:
GUI grounding evaluations often treat high instruction–element embedding similarity as evidence of semantic grounding. Across three mobile and web benchmarks (AndroidControl, MoTIF, Mind2Web), we show this interpretation is frequently confounded by visible-label recovery: no off-the-shelf encoder significantly exceeds a BM25 lexical baseline at top-1, and encoder hit@1 is predictable from lexical rank, candidate-pool size, and label type at AUC 0.85–0.88. On AndroidControl, oracle fusion could lift R@1 by +12.8 pp, but standard reciprocal-rank fusion of the same signals instead degrades it by 1.4 pp. One epoch of in-domain contrastive fine-tuning of a small 22M encoder then beats every lexical and zero-shot baseline by +13.6 pp, localising the failure to off-the-shelf encoders rather than dense retrieval. These results suggest that, on its own, embedding similarity is not sufficient evidence of semantic GUI grounding beyond visible-label recovery; it should be reported against lexical controls and label-type stratification.

Abstract:
Stock market analysis inherently requires composite reasoning over historical records and future projections, yet existing benchmarks remain fragmented across isolated tasks. We introduce STQA (Stock-focused Tabular Question Answering), an end-to-end benchmark designed to systematically evaluate natural-language question answering over historical data, numerical forecasts, and forecast-based reasoning. Built on a large-scale financial dataset, STQA covers 4,451 stocks and contains 30,500 question–answer pairs derived from expert-crafted templates, accompanied by fine-grained intent and slot annotations. To operationalize this benchmark, we present SQFRS (Stock Query–Forecast–Reasoning System), an agent-based unified framework that orchestrates SQL retrieval and time-series forecasting tools. Experiments demonstrate that while current large language models perform well on historical queries, forecast-based reasoning poses a substantial challenge, revealing critical bottlenecks in tool coordination and reasoning under uncertainty.

Abstract:
Deep Research Agents (DR-Agents) extend LLMs with multi-step retrieval, tool use, and intermediate computation to solve complex problems. However, their execution logs often contain redundant context, noisy evidence, and brittle dependencies among intermediate decisions. As a result, final answers can be sensitive to irrelevant or weakly verified information. We propose DualResearch, a log-based post-processing framework for answer reconstruction. Given a query and a completed DR-Agent log, DualResearch normalizes the log into event records and constructs two evidence graphs: a breadth semantic graph for declarative evidence and a depth causal graph for procedural provenance. It then retrieves breadth nodes and depth paths, assigns higher weights to retrieval channels with more concentrated retained candidate distributions, and jointly considers relevance and cross-view agreement to select evidence for reconstruction; the concentration measure reflects retrieval confidence rather than factual correctness. Experiments on HLE, GAIA, and GPQA show consistent accuracy improvements over underlying DR-Agents and controlled log-reuse baselines.

Abstract:
Exact match (EM) can make checkpoint selection ambiguous when a small labeled out of distribution (OOD) validation set leaves several checkpoints tied at zero or at the maximum. Without an explicit tie rule, the reported metric does not identify which checkpoint produced the final test score. We study this validation resolution problem in semantic parsing and morphological reinflection. Our protocol reports the reference budget, token likelihood choice under teacher forcing, and latest tied candidate, while using EM for a unique nonzero maximum. Across 600 draws from 30 runs, 8 references leave every early checkpoint at zero in 48% of draws and the maximum tied in 57%. Within tied draws, mean full split regret falls from 0.063 under random selection to 0.050 with token likelihood. The latest tied candidate performs better on average for mostly monotone trajectories, but token likelihood independently orders candidates that hard EM cannot distinguish. These results support reporting both orderings when small labeled OOD validation sets guide checkpoint selection.

Abstract:
Current value alignment evaluations for large language models (LLMs) heavily rely on output compliance, overlooking whether LLMs maintain value consistency when alternating between generator and evaluator roles. Such consistency is increasingly important because modern LLM agents and alignment pipelines often rely on the same LLM to both generate actions and evaluate their acceptability. To mitigate this limitation, we propose VALCON, a role-driven data construction framework that operationalizes value consistency into three benchmark suites (VAB-Consistency, VAB-Boundary, VAB-Challenge) via dual-role variation. By systematically pairing an LLM’s generative behavior with its evaluative judgments under identical principles, VALCON synthesizes target evaluation data that exposes hidden value discrepancies. Experiments on 28 LLMs reveal pervasive cross-role inconsistency: models with highly compliant outputs frequently fail to align their judgments with their own generations. To understand and address this gap, we perform mechanistic probing that uncovers a role-dependent representational decoupling associated with this behavioral asymmetry. Guided by these insights, we propose a representation-directed intervention that effectively mitigates consistency failures. Our work establishes value consistency as a distinct alignment dimension and provides a complete evaluation-analysis-mitigation pipeline. Code and data are available at https://github.com/tjunlp-lab/VALCON.

Abstract:
Cross-market factor research studies whether firm-level signals from one or more markets can predict returns in a target market, but existing public benchmarks do not support cross-market disclosure-to-return evaluation. Building such a benchmark is challenging because filings differ across languages and regulatory systems, disclosure-derived similarity can be biased by common reporting components, and cross-market signals must be evaluated under feasible trading-time alignment. We introduce CrossAlpha , a public annual-report benchmark for cross-market factor research. CrossAlpha addresses these challenges through three corresponding components: Disclosure Distillation , which standardises heterogeneous filings into ten-category English business descriptions; Residual Schema Graph Construction , which builds PCA-whitened cross-market firm-pair scores from schema-level disclosures; and Timing-Aligned Evaluation , which pairs the graph with 11 years of daily OHLCV data to construct forward-return labels under feasible cross-market execution protocols. CrossAlpha covers 3,587 firms and about 10,700 firm-year reports from the United States, Japan, Taiwan, South Korea, and Hong Kong, and releases about 19M directed firm-pair scores. In experiments, disclosure-derived cross-market peers outperform domestic text, industry-code, and return-correlation peers in the US-to-Japan setting (ICIR 0.39 versus 0.07–0.18), and cross-market sources beat the domestic text baseline in most target markets. CrossAlpha offers a publicly released, reusable, return-grounded benchmark for cross-market financial NLP.

Abstract:
Equipping large language models with explicit skills has emerged as a promising paradigm for enabling autonomous agents to solve complex tasks. Agent skills can be inherently divided into general skills for broad cognitive transfer and task-specific skills for dynamic execution. However, existing skill-based reinforcement learning (RL) methods typically force a rigid choice between full externalization, which incurs prohibitive context overhead, and full internalization, which risks overfitting and knowledge conflicts. To address this dilemma, we propose Skill0.5, a novel agentic RL framework that explicitly differentiates skill treatments by combining general skill internalization with task-specific skill utilization. Driven by a dynamic, difficulty-aware router, Skill0.5 streams tasks into distinct mastery tiers to apply tailored optimization strategies: it internalizes general skills via privileged distillation to build a cognitive foundation for hard tasks, while using diagnostic probing on easy tasks to penalize shortcuts and enforce specific skill utilization. Experiments on ALFWorld and WebShop demonstrate that Skill0.5 outperforms both memory-based and skill-based RL baselines, yielding performance improvements across both in-distribution and out-of-distribution scenarios. The code is available at: https://anonymous.4open.science/r/Skill0_5-1234/.

Abstract:
Dispute mediation is essential for maintaining social harmony and resilience, yet developing skilled mediators is costly and time-consuming. Existing LLM-based mediation research remains limited by unrealistic task formulations, low-fidelity datasets, and coarse evaluation metrics that obscure turn-by-turn dynamics. To address these gaps, we introduce ProMediConv, a novel benchmarking framework that models mediation as a proactive, multi-stage, and party-aware dialogue process incorporating 11 mediation strategies and four party behavior pattern (BP) states. Using 972 complete real-world cases, we construct a high-fidelity mediation dataset with utterance-level annotations of strategies and BP states. Furthermore, to better assess agent impact, we propose MAD (Mean Attribute Difference), a fine-grained metric that captures BP shifts throughout the dialogue. Leveraging this framework, we establish a comprehensive benchmark by evaluating diverse models alongside our tailored baseline ProMediAgent. Extensive empirical analyses reveal critical behavioral phenomena and underscore the persistent challenges current models face in dynamic, multi-party mediation. Ultimately, ProMediConv provides a rigorous foundation and a vital quantitative standard for advancing AI-assisted conflict resolution. Our dataset and codebase are accessible at https://github.com/ZsWei66/ProMediConv_repo.

Abstract:
Large vision-language models (LVLMs) remain highly sensitive to text embedded in images: a misleading word printed, painted, or pasted onto an object can override visual evidence and flip the model’s answer. Existing defenses are often tied to specific vision encoders, require training or feature-level intervention, or do not operate on the autoregressive decoding interface of modern LVLMs. We propose TypoCD, a training-free contrastive decoding method for typographic robustness. TypoCD constructs a text-localized counterfactual view by detecting visible text regions and inpainting only the typographic content, then contrasts the attacked-image and typography-removed logits during decoding. This localized contrastive direction targets the attacker’s textual signal while preserving unrelated visual content, allowing TypoCD to subtract the typography-induced logit shift without modifying model parameters or internal features. Across three open-source LVLM backbones and synthetic, real-world, and safety-oriented typographic benchmarks, TypoCD recovers attacked-image accuracy to within a few percentage points of clean-image performance without per-backbone tuning, closing a gap that often exceeds 40 percentage points under attack. When combined with a simple prompt prefix, TypoCD also substantially reduces jailbreak attack success on safety-oriented typographic attacks.

Abstract:
Whether Large Language Models (LLMs) exhibit covert psychological manipulation in complex human-AI interactions has garnered increasing safety concerns. However, existing AI safety benchmarks remain largely restricted to explicit rule compliance and static prompts, failing to capture the dynamic and covert nature of manipulative strategies in multi-turn dialogues. We introduce CogManip, a comprehensive benchmark that evaluates 15 manipulation strategy risks across 1,000 multi-turn interaction scenarios, validated by human experts. A systematic evaluation of 13 representative models, including frontier models like GPT-5.4 and DeepSeek-V3.2, reveals significant risk heterogeneities and illuminates the targeted direction for future defense. Further analysis of objective function perturbation reveals that DeepSeek-V3.2’s manipulation tactics are highly sensitive to both negative and benign system prompts, demonstrating the critical necessity of prompt-based defense engineering and implicit goal auditing. CogManip offers a robust instrument and perspective for auditing the implicit psychological influence and dynamic strategy selection of modern LLMs. Our code is publicly available at https://github.com/yanchenfei/CogManip.

Abstract:
Frequent verbs such as ’have’ and ’make’ can function either as collocates in light-verb constructions or as full lexical predicates, as in ’make a decision’ vs. ’make a cake’. Whether language models represent this distinction, and whether such representations vary across languages, remains unclear. We introduce a large-scale controlled dataset in English, Spanish, and French, comprising minimally varying sentence series in which the same context contains the same verb in light-verb and full-verb uses. Two probing experiments show that language models differentiate between these uses even in minimal contexts and exhibit separable patterns across object types. We release the dataset, generation code, and materials as a reusable resource. The framework supports extensions to broader contexts, additional verbs, and other languages.

Abstract:
Large language models (LLMs) achieve strong reasoning performance, which depends critically on inference-time decisions. Yet these decisions are commonly handled by static, one-size-fits-all policies, limiting adaptation to diverse tasks and reasoning stages. Recent adaptive methods partially address this limitation, but they primarily adapt either decoding stochasticity (how the model explores) or reasoning compute (how long the model reasons) in isolation, leaving their interaction within a single reasoning trajectory unmodeled. To address this challenge, we shift toward a within-trajectory joint control view, and instantiate it in AutoCRAT, a decoder-side controller for frozen backbones. Using only signals available during decoding, AutoCRAT jointly adjusts sampling stochasticity and reasoning budget during generation. AutoCRAT operates over a discrete action space and updates control decisions only at semantic boundaries, improving stability while remaining responsive to the evolving reasoning process. Comprehensive evaluation across 6 benchmarks demonstrates that AutoCRAT (I) uses 13.8-52.7% fewer inference tokens on average than recommended static configurations, (II) surpasses recommended static and adaptive single-axis baselines by 1.5-4.5% in relative accuracy, and (III) enjoys strong cross-backbone transferability.

Abstract:
Tool-integrated reasoning (TIR) enables large language models (LLMs) to enhance their capabilities by interacting with external tools, such as code interpreters (CI). Most recent studies focus on exploring various methods to equip LLMs with the ability to use tools. However, how to further boost the reasoning ability of already tool-capable LLMs at inference time remains underexplored. Improving reasoning at inference time requires no additional training and can help LLMs better leverage tools to solve problems. We observe that, during tool-capable LLM inference, both the number and the proportion of erroneous tool calls are negatively correlated with answer correctness. Moreover, erroneous tool calls are typically resolved successfully within a few subsequent turns. If not, LLMs often struggle to resolve such errors even with many additional turns. Building on the above observations, we propose PruneTIR, a rather effective yet efficient framework that enhances the tool-integrated reasoning at inference time. During LLM inference, PruneTIR prunes trajectories, resamples tool calls, and suspends tool usage through three components: Success-Triggered Pruning, Stuck-Triggered Pruning and Resampling, and Retry-Triggered Tool Suspension. These three components enable PruneTIR to mitigate the negative impact of erroneous tool calls and prevent LLMs from getting stuck in repeated failed resolution attempts, thereby improving overall LLM performance. Extensive experimental results demonstrate the effectiveness of PruneTIR, which significantly improves Pass@1 and efficiency while reducing the working context length for tool-capable LLMs.

Abstract:
Federated fine-tuning adapts large language models (LLMs) to decentralized client data, but its scalability in cross-device training is often limited by the high communication cost. Muon is an optimizer that improves optimization performance by orthogonalizing momentum for matrix-valued parameters. Existing federated Muon methods demonstrate the benefit of matrix-aware optimization in federated learning, but still require transmitting full layer-size updates and optimizer state. A natural way to reduce communication is to directly apply Muon to LoRA factors, but this changes the optimized object and weakens Muon’s matrix-aware update geometry. We propose FedSubMuon, a communication-efficient federated Muon fine-tuning method that optimizes compact coefficient matrices within shared structured subspaces. This design keeps Muon on a single matrix-valued trainable object, while reducing the client upload to compact coefficient matrices. We further introduce FedSubMuon-GT, an accuracy-oriented extension that uses projected gradients to adapt tracked subspace bases toward task-relevant gradient directions. Experiments on instruction tuning and mathematical reasoning show that FedSubMuon-GT achieves the best overall accuracy on four of five dataset–model pairs, while FedSubMuon performs best under all matched communication budgets. On Dolly-15K, the closest communication baseline requires 5.5× and 1.4× more total communication on Llama-1B and Qwen-4B, respectively.

Abstract:
A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally.

Abstract:
Watermarking diffusion language models (DLMs) requires mechanisms compatible with iterative parallel unmasking rather than autoregressive decoding. Existing sampling-based watermarking methods typically inject position-wise i.i.d. perturbations, which can be poorly aligned with DLM decoding dynamics and degrade generation quality. We propose SAC-Copula, a quality-preserving watermarking method for DLMs based on smooth, locally correlated Gumbel perturbation fields constructed via a Gaussian copula. We further develop a SAC-aware detector using covariance-aware filtering and native-sample calibration. Mechanism-level analysis shows that local correlation reduces latent perturbation roughness and better matches iterative refinement dynamics. Experiments on LLaDA show that SAC-Copula achieves a favorable quality-detectability trade-off compared with existing baselines. In particular, further evaluations on Dream-7B and additional datasets show that SAC-Copula substantially improves PPL tail stability over the i.i.d. Gumbel baseline, while maintaining strong low-FPR detectability and competitive overall generation quality. Additional token-edit stress tests further assess watermark robustness under controlled synchronization drift. Code is available at https://github.com/PunkyKnife/SAC-Copula.

Abstract:
Social media platforms have experienced a significant rise in toxic content, including abusive language and discriminatory remarks, presenting growing challenges for content moderation. Some users evade censorship by deliberately disguising toxic words through homophonic cloak, which necessitates the task of unveiling cloaked toxicity. Existing methods are mostly designed for English texts, while Chinese cloaked toxicity unveiling has not been solved yet. To tackle the issue, we propose C TU, a novel training-free and prompt-free method for Chinese cloaked toxic content unveiling. It first employs substring matching to identify candidate toxic words based on Chinese homo-graph and toxic lexicon. Then it filters those candidates that are non-toxic and corrects cloaks to be their corresponding toxicities. Specifically, we develop two model variants for filtering, which are based on BERT and LLMs, respectively. For LLMs, we address the auto-regressive limitation in computing word occurrence probability and utilize the full semantic contexts of a text sequence to reveal cloaked toxic words. Extensive experiments demonstrate that C TU can achieve superior performance on two Chinese toxic datasets. In particular, our method outperforms the best competitor by up to 71% on the F1 score and 35% on accuracy, respectively. Our code and data are available at https://anonymous.4open.science/r/C2TU/.

Abstract:
Large skill libraries are now standard equipment for LLM agents, and which subset reaches the prompt decides what the agent can do. Enumeration and top-K embedding retrieval both treat skills as independent documents, assuming relevance is a property of one skill at a time. It is not: skills depend on, conflict with, specialize, and duplicate one another, so the closest match to a task can name it while omitting a prerequisite it needs. Graph retrieval encodes these relations but spends them in a fixed pipeline, handing the agent an opaque bundle. We present SkillDAG, which exposes a typed skill graph to the LLM as an agent-callable interface rather than consuming it for the agent: search returns vector matches, typed-edge neighbors, and conflict signals, and a propose-then-commit protocol lets it register execution-backed edges as it works. Across two benchmarks and three model families, SkillDAG outperforms full-library prompting, vector retrieval, and Graph-of-Skills, gaining +14.3 points of ALFWorld success and tying only at ceiling; its ranking holds flat across a 10× pool expansion that degrades the fixed pipeline. Inter-skill structure, these results suggest, is worth more to an agent that can query and revise it than to a retriever that consumes it.

Abstract:
Designing effective heuristics for NP-hard combinatorial optimization problems remains challenging and often requires substantial domain expertise. Recent LLM-guided evolutionary methods have shown promise for automated heuristic generation, but most existing approaches refine heuristics independently or through limited pairwise feedback. We propose ReVEL: Multi-Turn Reflective LLM-Guided Heuristic Evolution via Structured Performance Feedback, a framework for group-wise multi-turn heuristic refinement. ReVEL organizes heuristics into behavior-aware reflective groups, including similarity-driven groups for localized refinement and diversity-driven groups for exploratory search. Within each group, the LLM performs iterative multi-turn refinement using accumulated performance feedback, enabling related heuristics to be jointly analyzed and progressively improved across evolutionary iterations. Experiments on standard combinatorial optimization benchmarks show that ReVEL generally improves optimization performance over existing LLM-guided evolutionary baselines across multiple settings and LLM backbones. Additional analyses suggest that behavior-aware grouping contributes to more consistent refinement trajectories during iterative heuristic evolution.

Abstract:
Minimal-edit Grammatical Error Correction (GEC) is a challenging task for zero- and few-shot prompted Large Language Models (LLMs), which systematically overcorrect and degrade F 0.5 by rewriting well-formed spans. While fine-tuning provides an effective solution, it imposes substantial infrastructure demands. We introduce a prompt-based approach that closes the gap to fine-tuned models through three advances in GEC prompting methodology. First, we introduce taxonomy-based instructions to enforce minimal-edit constraints with a comprehensive list of grammatical error rules, equipping the LLM with a bounded, metric-aligned scope of correctable edits, which benefits the strongest models while remaining model-dependent overall. Second, we show that batching multiple uncorrected sentences into a single input context acts as a targeted regularizer against overcorrection, systematically reducing the edit rate across diverse LLM families; we hypothesize this arises from attention dilution effect induced by the bounded capacity of self-attention scores. Finally, LLM-assisted Prompt Optimization refines these instructions. Powered by Gemini 3.1-Pro, our prompt achieves F 0.5 =78.32 on the BEA-2019 test set — establishing a new prompt-based SOTA while shrinking the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to a mere 0.38 points. Code, prompts, and outputs are publicly available.

Abstract:
Retrieval-augmented multi-hop question answering (QA) decomposes a query into sub-questions and decomposes the corpus into smaller retrieval units such as sentences. Both forms of decomposition improve the pipeline, but we show that both share the same vulnerability, the loss of entity information, and that this loss breaks the pipeline at two separate points. The first point is retrieval, where a sub-question loses the entity resolved at the previous hop, leaving the retriever with nothing to match against. The second point is harder to see, because retrieval still appears to succeed. Once a passage is split into sentences, an isolated sentence loses the context that grounds its pronouns, so even with the correct sentence in hand the LLM cannot tell which entity the sentence is about. We isolate this second point as a distinct failure mode that we call lost-in-generation, and a retrieval-controlled experiment shows that it degrades answers even when the gold evidence is fixed in the context. We then propose Dual Entity Recovery RAG (DER-RAG), which keeps the grounding entity explicit from retrieval through to generation with two lightweight components, a two-way query decomposition that carries the resolved entity across sub-questions and a subject entity prefix attached to each sentence at generation time. DER-RAG needs no graph construction, no corpus modification, and no fine-tuning, yet on three multi-hop QA benchmarks it matches or exceeds strong baselines, including graph-based methods that depend on costly offline structures.

Abstract:
Most existing benchmarks evaluate social bias in large language models (LLMs) using multiple-choice questions (MCQ) grounded in stereotypical contexts. While MCQ formats enable straightforward quantification, how LLMs manifest social bias in task-specific contexts and under different output constraints remains underexplored. We introduce SCENE, a benchmark for measuring the social bias shown by LLMs in scenario-completion tasks. Given short scenarios involving social groups, models must complete key elements of the situation through two tasks: (i) Character Specification, which asks models to assign social attributes to characters, and (ii) Plot Continuation, which asks models to select the most plausible narrative continuation. Combining templates adapted from prior benchmarks with newly collected templates through a university-wide competition, we construct a dataset of over 75,000 Korean instances. Experiments show that bias measurements are highly sensitive to task context and output constraints, even for identical underlying scenarios, suggesting that widely used MCQ-based evaluations may not fully capture certain manifestations of stereotypes. SCENE is publicly available at https://github.com/DSAIL-SKKU/SCENE-EMNLP-2026.

Abstract:
Large language models (LLMs) exhibit significant layer-wise redundancy, motivating non-uniform pruning for efficient compression. However, existing sparsity allocation methods rely on indirect statistical heuristics derived from weight magnitudes or activation distributions, without directly assessing how robustly each layer preserves semantic information under input variation. We propose PerturbPrune, a framework that allocates layer-wise sparsity by probing the intrinsic stability of hidden representations. PerturbPrune estimates the principal subspace of each layer’s activations via PCA and injects structured perturbations along these directions. Layer sensitivity is quantified through matrix difference entropy (DiME): stable layers are identified as encoding compact, essential features and thus preserved, while highly sensitive layers are deemed over-parameterized and pruned more aggressively. Extensive experiments across diverse LLM families (LLaMA, LLaMA-2, Qwen, OPT) demonstrate that PerturbPrune broadly matches or outperforms existing non-uniform methods. Compared to uniform pruning, it prevents catastrophic collapse, reducing perplexity by up to 99.7% and improving zero-shot accuracy by up to 22.7%. The method is also highly efficient, completing sparsity allocation on Qwen3.5-9B in 7 minutes—achieving 24 × and 17.4 × speedups over AlphaPruning and MRP. Finally, PerturbPrune generalizes to Mixture-of-Experts architectures, achieving up to 19.3% perplexity reduction on Qwen3.5-35B-A3B.

Abstract:
Conversational stance detection requires identifying the stance of a current utterance toward a target within its dialogue context. Existing methods often rely on sequential encoding, fixed knowledge-integration pipelines, or uniform reasoning strategies, which limits their ability to capture cross-turn dependencies and adapt to instance-specific needs. We propose StanceAgent-R1, an agentic reinforcement learning framework that adaptively routes each instance to evidence-path selection and/or knowledge elicitation. The model is first warm-started with rejection-sampled expert trajectories and then optimized with a progressive trajectory-faithful reward that encourages correct, well-formed, and route-consistent reasoning. Experiments on ZS-CSD, MT-CSD, and C-MTCSD show that StanceAgent-R1 achieves state-of-the-art performance, with further analyses confirming the effectiveness of its agentic design and reward strategy.

Abstract:
Recent advances in agentic recommender systems are shifting recommender systems from passive filtering engines to instruction-following agents that use external tools to resolve user intent. However, existing benchmarks often assume explicit user intent, simplified tool environments, or isolated function calls, leaving realistic tool orchestration for recommendation underexplored. To bridge this gap, we propose RecToolBench , a Model Context Protocol (MCP)-based benchmark for evaluating tool-using recommender agents under fuzzy user instructions. RecToolBench contains more than 1,200 executable tasks across three recommendation domains, 13 MCP servers, and 32 tools, spanning single-tool calls, parallel tool calls, sequential tool chains, and hybrid tool orchestration. We construct RecToolBench with a scalable synthesize–fuzzify–judge pipeline that generates executable fuzzy recommendation tasks, and evaluates agent trajectories using rule-based execution checks and rubric-based LLM evaluation. Experiments on representative LLMs show that syntactically valid tool calls do not guarantee successful recommendations. Models struggle with semantic parameter grounding, multi-step evidence integration, and grounded final recommendations, especially as orchestration complexity increases. Our results identify tool orchestration under fuzzy user intent as a major bottleneck for agentic recommender systems. Our data and code are available at https://github.com/ShawnChenn/RecToolBench .

Abstract:
The decline of proficient practitioners poses an existential threat to numerous ancient writing systems, risking the loss of the cultural heritage they encode. Among these systems is Sawndip, a logographic script whose computational preservation faces severe challenges due to the scarcity of standardized corpora. In this paper, we present the first systematic framework for low-resource Sawndip-Chinese translation and bilingual conversational modeling. We introduce a newly curated dataset comprising sentence-level Sawndip-Chinese parallel data and character-level lexical definitions. Building upon this foundation, we propose a knowledge-enhanced supervised fine-tuning paradigm that incorporates lexical knowledge during training. To address the additional challenges of Chinese-to-Sawndip generation, we further develop a manually curated template-based substructure substitution strategy for synthesizing reverse-direction data. Bilingual conversational data is then constructed through a dimension-conditioned dialogue generation and multi-candidate back-translation pipeline, enabling controllable and diverse dialogue synthesis. Experimental results show that our final unified model effectively supports both Sawndip-Chinese translation and bilingual conversational abilities. Ultimately, this work provides a foundational reference for the digital preservation of Sawndip, offering broader insights into the computational documentation of globally-endangered languages.

Abstract:
Cyber Threat Intelligence (CTI) reports describe adversary behavior and attack infrastructure in natural language but remain largely unstructured and difficult to process at scale. Existing CTI benchmarks are limited in realism or scope, often relying on synthetic or knowledge-base-derived data, coarse document-level annotations, or isolated subtasks. We introduce SMAH-CTI, a human-annotated benchmark derived from 100 public CTI reports from five threat intelligence providers. It contains 12,304 validated sentence-level instances annotated for (i) relevance classification, (ii) CTI-specific named entity recognition (NER), and (iii) multi-label MITRE ATT CK tactic, technique, and sub-technique mapping, together with grounded procedure descriptions. By unifying these tasks over evidence grounded in real CTI reports, SMAH-CTI provides a reproducible benchmark for evaluating structured CTI extraction and transforming unstructured reports into machine-interpretable intelligence.

Abstract:
Large language models (LLMs) are strong zero-shot pointwise rankers, but lag behind pairwise and listwise methods. Beyond missing comparative signals, we identify a calibration gap : ranking-relevant information encoded in hidden states is not fully captured by the scalar output head. We propose RankSteer, a post-hoc activation-steering framework that calibrates ranking via projection-based interventions along multiple directions at inference time: decision, evidence, and, optionally, role. This is achieved without updating model weights or introducing cross-document comparisons. We instantiate RankSteer on two structurally distinct pointwise variants and observe improvements over their respective baselines on most TREC DL and BEIR datasets across three backbones. This suggests that the calibration gap is a general property of pointwise rankers. Our additional geometric analysis shows that steering improves ranking by concentrating each query’s document representations along an existing ranking geometry, offering new insight into how LLMs internally represent and calibrate relevance judgments.

Abstract:
Text embeddings are fundamental to many downstream applications, making robustness important for real-world NLP. However, most recent state-of-the-art embedding models are not reproducible because they rely on closed or undisclosed training data, and they remain insufficiently robust for Southeast Asian languages. We present SEA-LION-Embedding, a fully open and reproducible text-embedding pipeline for Southeast Asian languages trained only on publicly available data, and use it to study three core factors of robust embedding design: data composition, training objective, and base encoder initialization. SEA-LION-Embedding achieves state-of-the-art results on SEA-BED while enabling systematic and reproducible analysis of robust text embeddings for the region.

Abstract:
To alleviate the heavy burden of paper screening, researchers increasingly rely on existing AI agents, such as AI reviewers or DeepResearch, for paper evaluation and novelty assessment. However, lacking specialized mechanisms for processing scholarly literature, their analyses often produce superficial results with noticeable deficiencies in quality. To bridge this gap, we introduce MemoNoveltyAgent, a multi-agent system designed to generate comprehensive and faithful novelty reports. Beyond retrieving concrete prior-paper evidence via RAG, our system incorporates a high-level abstract memory constructed from large-scale scholarly corpora. This memory organizes research into hierarchical trees to distill field-specific evolutionary trajectories, thereby providing a broader historical context. Furthermore, we decompose papers into discrete novelty points for fine-grained analysis and retrieval, while employing a self-validation mechanism to improve report faithfulness. Finally, to address the evaluation challenges of such open-ended generation tasks, we propose a RAG-augmented checklist evaluation method that enables reliable and evidence-grounded assessments. Extensive experiments demonstrate that MemoNoveltyAgent outperforms GPT-5 DeepResearch by 13.69%. Code and demo are available at https://github.com/SStan1/MemoNoveltyAgent.

Abstract:
In the maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions. To enable fault trees stored as images to be directly processed by large language models, which can assist in tracking and analyzing malfunctions, we propose a novel textual representation of fault trees. Building on it, we construct a benchmark for multi-turn dialogue systems that emphasizes robust interaction in complex environments, evaluating a model’s ability to assist in malfunction localization, which contains 3130 entries and 40.75 turns per entry on average. We train an end-to-end model to generate vague information to reflect user behavior and introduce long-range rollback and recovery procedures to simulate user error scenarios, enabling assessment of a model’s integrated capabilities in task tracking and error recovery, and Gemini 2.5 pro achieves the best performance.

Abstract:
Evaluating causal reasoning in large language models (LLMs) is complicated by benchmarks that conflate distinct reasoning operations into aggregate scores. We propose CausalPrism, an evaluation framework grounded in structural causal model semantics and cognitive models of causal inference, which decomposes intervention-based causal reasoning into three functional primitives—Causal Prior, Forward State Tracing, and Belief Updating—plus a consistency primitive, Agency Sensitivity. Primitives are instantiated in directed acyclic graphs and evaluated across three narrative domains of varying semantic richness. Across 14 open-source and frontier LLMs, Belief Updating is the dominant bottleneck: the best model reaches only 0.613 accuracy, while GPT-5.2 and GPT-5-mini score 0.531 and 0.517 despite near-ceiling performance on Causal Prior and Forward State Tracing. Prior exhibits systematic domain sensitivity and over-rejection. Agency framing effects are family-dependent for weaker models. We argue against treating causal reasoning as a unitary capability and motivate targeted evaluation and training along multiple primitive axes.

Abstract:
Entity disambiguation links mentions in text to the entities they refer to in a knowledge base. Recent cross-entity interaction models improve candidate comparison by jointly encoding multiple candidates, but incorporating entity metadata into this joint encoding is not straightforward. Appending metadata to every candidate forces joint encoding to process longer inputs with detailed distractor information, which can weaken the contrastive cues that make cross-entity interaction effective. We propose DuCEI, a dual-stage framework that avoids this broad metadata interaction by reducing the candidate set before introducing entity metadata. DuCEI first compares all candidates using title-based cross-entity interaction, and then performs metadata-enriched interaction only among the remaining candidates. This design makes metadata use selective at the candidate level, applying detailed evidence only where it is likely to help. Experiments on six standard benchmarks show that DuCEI achieves the best overall performance among models trained solely on AIDA-CoNLL. Further analyses show that confidence-based selective refinement preserves nearly all of the full two-stage performance while applying Stage 2 to only 27.80% of mentions, reducing inference cost from 1.90× to approximately 1.27× that of Stage 1 alone. Our code is available at https://github.com/jinwoomin/DuCEI.

Abstract:
Medical visual question answering (Med-VQA) is often assumed to require medical fine-tuning, large models, or complex multi-agent pipelines. We revisit this assumption with MedProb , a lightweight probing framework that predicts multiple-choice Med-VQA answers from frozen VLM representations without free-text generation. Across PATH-VQA, SLAKE, and VQA-RAD, MedProb recovers substantially more answer-relevant signal than prompting and performs stronger than medical VLMs and agentic systems. Probing also reduces the apparent gap between small and large models compared to prompting, suggesting that smaller VLMs contain more recoverable Med-VQA signal than generation-based evaluation reveals. Across 14 matched general-purpose and medical VLM pairs, medical adaptation does not consistently improve this linear decodability.

Abstract:
Post-training reasoning models is computationally expensive, and the allocation between supervised fine-tuning (SFT) and reinforcement learning (RL) substantially affects reasoning quality. Existing methods operate at the training-procedure level and treat the reasoning corpus as homogeneous, ignoring the structural heterogeneity of individual traces. We propose GROVE (Graph-based Routing Over Variant Experts), an adaptive framework that routes each example to SFT or RL based on the structural topology of its reasoning trace. GROVE parses each trace into a Reasoning Tree, scores its complexity via graph entropy, and partitions the corpus through a data-driven Gaussian Mixture Model with no hand-tuned threshold. Low-complexity traces are trained with SFT and high-complexity traces with GRPO as parallel LoRA adapters from a shared frozen base, merged into a single deployable model with no inference-time overhead. Across two training corpora and five reasoning benchmarks, GROVE consistently outperforms single-stage and naive-hybrid baselines, and a reversed-routing ablation confirms that this mapping is intrinsic rather than arbitrary.

Abstract:
Hateful video detection is a critical task for maintaining a healthy online ecosystem. However, existing methods are limited in their ability to deeply comprehend hateful semantics in videos and lack an effective mechanism for handling ambiguous cases near the decision boundary, making implicit hate detection particularly challenging. In this work, we address these limitations through both internal meaning formation from the video itself and external evidence consultation, and propose a novel framework, HATELENS. Internally, we design perception-to-cognition reasoning to perceive multimodal cues, perform cognition-level deep understanding, and progressively construct structured hateful meaning. Externally, we propose a test-time retrieval and adaptation mechanism to provide richer local evidence to help disambiguate implicit hate, which operates on fused multimodal representations to exploit annotated precedents. Extensive experiments on four public datasets demonstrate that our method achieves state-of-the-art performance, with macro-precision improvements of up to 16%. The code of our project is available at https://github.com/JunyiChen-ai/HATELENS .

Abstract:
Recent large language models (LLMs) have demonstrated remarkable progress in 3D spatial reasoning, spatial grounding, and fine-grained geometric understanding. However, their ability to reason about densely packed object placement under strict spatial and functional constraints remains largely unexplored, despite being a fundamental challenge in practical electronic design automation (EDA) workflows. To bridge this gap, we introduce OmniLayout, the first benchmark designed to evaluate LLMs on printed-circuit-board (PCB) layout placement reasoning under real-world geometric, routing, and connectivity constraints. OmniLayout contains 1,681 industrial-grade schematic-coupled PCB layouts and includes four tasks: (1) geometric reasoning for IC physical placement, with 77.24K placement instances constrained within PCB board boundaries; (2) routability-aware placement reasoning, generating physically valid component placements; (3) electrical functionality, preserving schematic-specified connectivity and electronic functional correctness; and (4) tool-augmented agentic reasoning for invoking external tools to accomplish tasks (1)–(3). Our results reveal substantial limitations of current LLMs in PCB layout placement, including weak geometric reasoning, poor routability optimization, and inconsistent preservation of electrical functionality.

Abstract:
Large Language Models (LLMs) are now capable of generating highly fluent, human-like text. They enable many applications, but also raise concerns such as large scale spam, phishing, or academic misuse. While much work has focused on detecting LLM-generated text, only limited work has gone into understanding the stylistic differences between human-written and machine-generated text. In this work, we perform a large scale analysis of stylistic variation across human-written text and outputs from 11 LLMs spanning 8 different genres and 4 decoding strategies using Douglas Biber’s set of lexicogrammatical and functional features. Our findings reveal insights that can guide intentional LLM usage. First, key linguistic differentiators of LLM-generated text seem robust to generation conditions (e.g., prompt settings to nudge them to generate human-like text, or availability of human-written text to continue the style); second, genre exerts a stronger influence on stylistic features than the source itself; third, chat variants of the models generally appear to be clustered together in stylistic space, and finally, model has a larger effect on the style than decoding strategy, with some exceptions. These results highlight the relative importance of model and genre over prompting and decoding strategies in shaping the stylistic behavior of machine-generated text.

Abstract:
Large language models generalize well to individual tasks but lack an inherent mechanism for learning them sequentially, leading to catastrophic forgetting. To mitigate this, LoRA-based continual learning methods allocate a separate low-rank adapter per task, yet existing approaches either require task identity at inference or sum all adapters indiscriminately, letting irrelevant branches distort the output. Recent gating-based solutions route inputs to the correct adapter but introduce trainable parameters that themselves need protection against forgetting. In this work, we observe that pooled token embeddings from a frozen LLM embedding layer already separate task distributions throughout the learning sequence. A Gaussian mixture model fitted on these embeddings, without any gradient-based training, is sufficient for task-agnostic adapter selection at test time. This eliminates the need for a learned gating module. On the adapter side, constraining each task’s parameters to the principal subspace of the pretrained weights via SVD yields a compact latent-space parameterization. Within this subspace, orthogonal regularization directly controls inter-task interference. The resulting system, Latent-LoRA, is replay-free, requires no trainable routing component, and uses substantially fewer parameters per task. Experiments across five model scales and two established continual learning benchmarks show state-of-the-art performance with near-zero forgetting.

Abstract:
Output format selection is a necessary but understudied dimension of few-shot named entity recognition (NER) with large language models. Existing formats carry characteristic failure modes under autoregressive generation, yet output format has not been systematically evaluated as a primary variable in the few-shot setting across diverse domains and entity structural complexity. We evaluate 21 output formats over 17 datasets across two levels of structural entity complexity, using 9 open-weight models without fine-tuning, ranging from 1.7B to 70B parameters. Output format is the main factor affecting performance, with a 41.7 F1 points difference between the best and worst formats. This exceeds the gains from scaling models parameters within-family. Furthermore, output format accounts for approximately 49.1% of F1 variance on simple datasets and 74.9% on complex ones. We describe three mechanistically distinct failure modes that explain these observed differences; each existing format violates at least one of three properties jointly necessary for robust few-shot NER. We propose CANE (Context-Anchored Natural Expression), a format family addressing all three properties. All CANE variants ranked the highest on simple and complex datasets. This includes the GENIA dataset where all other formats had near-zero scores across nine models, indicating the necessity of the three properties. These findings highlight the importance of output format in few-shot named entity recognition, particularly for annotations involving overlapping or discontinuous entities.

Abstract:
Natural language (NL) to temporal logic (TL) translation lets engineers specify and verify system behaviors without hand-crafting formal specifications. Existing frameworks lift NL into syntactically valid TL but leave atomic propositions abstract—not grounded to the concrete predicates and typed objects a system exposes (e.g., binding “find the bookbag” to search(backpack) ). Ungrounded formulas cannot be checked over traces, yet current systems either omit grounding or ground inaccurately. We propose , a modular stage for G rounding Natural Language In to System Sign atures that attaches to any NL-to-TL lifter. We cast grounding as signature-conditioned classification: enumerating the signature as an input prefix lets one encoder-only model assign each abstract AP a predicate and its typed arguments, generalize to unseen signature elements, and transfer across domains without retraining, while never hallucinating out-of-signature atoms. Because predictions are selections from the enumerated signature, the out-of-signature rate is 0% by construction , against up to 95% for prompted LLMs given the same signature in-context. attains at least 99% joint grounding accuracy across seven domains (worst case 99.39 ± 0.04 over five seeds); paired with a lifter, it produces executable specifications that track the oracle-grounding upper bound, improving grounded logical equivalence by 75% over the strongest prompted-LLM and embedding-similarity grounder.

Abstract:
Neural Cellular Automata (NCAs) generate complex spatial patterns in vision by repeatedly applying a shared local rule to persistent cell states, combining local perception, weight-shared iteration, and gated in-place updates. We test this principle in language modelling with TextNCA, a 1D causal model in which token positions act as cells, windowed attention provides local perception, and a GRU-style gate updates their states. Its hierarchical variant cascades increasingly wide attention windows w ∈ 8,32,128 and reuses each stage’s parameters for T_s iterations. We train roughly 30M-parameter models on SlimPajama for 60k steps and evaluate on WikiText-103. TextNCA trails a similarly sized Transformer-6L (60.3 versus 52.8 best-run PPL), so we study it as an analytical probe rather than a performance alternative. Reordering or flattening the windows raises PPL by 16.7–70.8, whereas a non-iterating sliding-window Transformer preserving the staged order performs comparably across three seeds. These results identify the narrow-to-wide schedule as the dominant factor and provide no reliable evidence that iterated weight sharing improves perplexity at this scale. Intermediate-state analyses support a prepare-then-decode account in which early stages reorganise representations and the final wide stage performs most next-token loss reduction.

Abstract:
Large language model (LLM) inference is often constrained by both computation and memory, especially in offloading-based deployments where model weights are transferred across memory hierarchies during autoregressive decoding. In this setting, reducing the number of executed layers can lower token-generation latency while also avoiding costly weight movement. Motivated by this observation, we present FlexEE, an early exiting framework for resource-constrained and offloading-based LLM inference. FlexEE makes early exiting practical for LLM decoding through layer-wise exit supervision for reliable intermediate-layer prediction, self-speculative decoding over a Top-K local vocabulary for low-cost exit decisions, and dynamic hidden state management for KV-cache-correct and memory-aware execution. Across generative and downstream tasks, FlexEE enables efficient early exit with minimal accuracy degradation, delivering up to 1.27 × /3.16 × and 1.25 × /2.83 × end-to-end speedups on Llama2-7B and Llama3-8B under 0%/50% weight offloading, respectively.

Abstract:
Instead of directly distilling a language model, this study addresses the problem of aligning a base model with a target model in distribution by designing the domain mixture of training data for pretraining or continued pretraining as a fixed training recipe. We propose a method for determining domain weights by viewing models as points in log-likelihood space and aligning the training update direction with the direction toward the target model. Experiments with NanoGPT show that the proposed method consistently reduces the KL divergence between the trained base model and the target model relative to training with Pile-original weighting. Although knowledge distillation remains more effective when available, the proposed method achieves meaningful alignment, and downstream task performance also tends to become closer to that of the target model.

Abstract:
There is a rising interest in further exploring learning with noisy labels on textual data. Existing solutions often assume the ground truth classes of the training data are balanced, overlooking the long-tailed distribution prevalently encountered in real-world data, which can significantly harm the effect of denoising strategies. Meanwhile, misidentifying clean samples further compromises the model’s learning of long-tailed data. To tackle this issue, we propose a novel knowledge distillation framework for learning with noisy labels under long-tailed distribution (LNL-LT). Specifically, we first introduce a divide-and-conquer sample selection strategy to perform head-tail class division and better separate clean samples from noisy samples within these classes, especially for the tail classes. Then, we employ LLM knowledge distillation to generate pseudo-labels and new samples for tail classes, enhancing pseudo-label quality while mitigating the effects of sample scarcity caused by the long-tailed distribution. Experimental results on both synthetic and real-world datasets demonstrate that our method surpasses current state-of-the-art methods.

Abstract:
Models can answer questions from retrieved document pages while producing citations that are valid but irrelevant to the answer. This often happens when citation quality and answer correctness are optimized as separate signals, so the model can learn to cite text without relying on it. We introduce EviDocVLM , a training method for multi-page document question answering that links selected support to answer behavior. A structured source map provides citation addresses for page content. EviDocVLM first uses identifier validity and quotation fidelity to derive a continuous coefficient for the trajectory-level answer reward, thereby increasing traceable support without making every citation error a hard failure. It then applies Evidence-Aligned GRPO, which reuses full-context rollout log-probabilities as a frozen target for one training-only evidence-view pass. The resulting sampled-token alignment encourages cited spans to support the answer without an external grader, extra rollout, or inference-time module. Experiments on document parsing and multi-page document QA benchmarks show stronger answer performance and consistent gains from the evidence-conditioned objective under complex visual layouts.

Abstract:
While Large Language Models (LLMs) offer scalable mental health support, they typically operate within the superficial text space, providing generic comfort that fails to resolve a patient’s deep-rooted cognitive distortions. To address this, we propose MindUntangler a novel multi-agent framework that models psychological counseling as a dynamic topological routing task. By conceptualizing cognitive distortions as "pathological topologies"—tangible knots in a patient’s Cognitive Topology Graph (CTG)—our system dynamically tracks and resolves these complex connections through pluralistic therapeutic routing. To ensure clinical safety, we integrate topological guardrails and align the model via a novel topology-driven Direct Preference Optimization. Experiments on two benchmarks demonstrate state-of-the-art performance in cognitive restructuring and clinical efficacy. Ultimately, this work shifts AI therapy from textual empathy to computable, structural interventions.

Abstract:
Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them—retrieval, structured timelines, LLM summaries, agentic memory—preserve the longitudinal signal clinical reasoning needs has not been measured. We introduce ClinTraceBench: 385 MIMIC-IV-derived verified dialogues with event-ID provenance, a nine-task taxonomy (T1–T9), and L0–L4 deterministic + L5 human-audit validation (98.92% agreement). We evaluate eight history representation strategies—a no-context floor, last-visit-only, full-context, BGE-M3 dense-retrieval, two compression schemes, and two agentic-memory systems (Mem0, A-Mem)—across four backbones (DeepSeek-V3, GPT-4o-mini, Haiku 4.5, Sonnet 4.6) on 6,271 questions: 32 cells, 200,672 predictions. Four findings emerge: (F1) compressed strategies pay an aggregation tax on multi-visit trends and cross-patient comparisons; (F2) the blind-to-full gap spans +29.8 percentage points (GPT-4o-mini) to +62.7 (Haiku); (F3) abstention scales non-monotonically with context length; (F4) a controlled T3 injection probe isolates compression-induced relation loss—with the attribution sentence present before construction, Mem0, A-Mem, and llm-summary still recover only 0–5.3% of the injected positives. On the Pareto frontier Haiku dominates Sonnet under full-context ( 25.76 vs. 106.21), inverting the “biggest backbone wins” heuristic.

Abstract:
In the LLM era, many symbolic and structured problems are presented to models through 1D text serialization. Yet some such problems are natively two-dimensional: their relevant relations, such as row–column correspondence or spatial adjacency, are defined by position in a 2D layout rather than by sequential order. This raises a representational question: does preserving the same symbolic entries in a 1D sequence also preserve the relational structure needed for computation? We study this issue through the lens of serialization friction: the representational mismatch in which the same underlying task instances and entries are still present, but relations that depend on layout become implicit under 1D serialization. The study uses a controlled synthetic testbed of three tasks: matrix transpose, Conway’s Game of Life, and LU decomposition. In each task, the same instances are presented either as 1D text serialization or as their native 2D layout rendered as an image. Across this testbed, 1D serialization degrades more sharply as task size grows, and errors under serialization exhibit spatially structured patterns, suggesting that this presentation choice is consequential within our testbed. To assess whether the gap is merely due to routing inputs through a visual pathway, we add a within-visual probe that renders the 1D serialized content as an image; this layout-disrupted rendering performs worse than native 2D rendering. These findings suggest that, for layout-defined tasks, reducing inputs to 1D serialization is not a neutral choice of representation.

Abstract:
We introduce AMDKernelVault, an open corpus and training framework for optimizing HIP and Triton kernels on recent AMD CDNA GPUs. Existing LLM-based kernel agents are predominantly designed for CUDA and NVIDIA hardware and often rely on repeated calls to frontier models for generation, reflection, and optimization. To address this gap, we develop HIPKernelGen and TritonKernelGen, agent-driven pipelines that translate PyTorch references into GPU kernels, compile and validate candidates under ROCm, and profile their latency on AMD hardware. AMDKernelVault contains 62,153 execution-verified HIP kernel samples, 2,377 production-grounded ROCm Libraries QA entries, and 39,893 Triton kernels. To demonstrate the corpus’s utility, we specialize Qwen3-8B through supervised fine-tuning and execution-aware reinforcement learning. Under fixed evaluation budgets and identical agent settings, the resulting model achieves the highest correctness among the evaluated models on PyTorch-to-HIP (34.0% Pass@1), TritonBench-G (33.2% Corr@3), and ROCmBench (41.94% Corr@3), while running locally on AMD hardware. These advantages are specific to the reported correctness metrics and do not extend uniformly to compilation accuracy or execution speed. The corpus and documentation are publicly available at https://huggingface.co/datasets/amd/AIG-Datasets.

Abstract:
We introduce Personalized and Reliable Popular Science Writing, a novel task that requires adapting scientific explanations to audiences with different cognitive levels while preserving factual accuracy. However, improving personalization often introduces simplifications that increase the risk of hallucination and factual distortion. To address these challenges, we first construct a dataset of 39,134 entries and a reader-centric Personalized Science Communication Benchmark (PSCB) that jointly evaluates audience adaptation and factual accuracy. To reduce data and computational requirements while improving generalization across domains and audiences, we introduce DA-MoE, which explicitly decouples audience adaptation from domain knowledge through separate modeling. To enable robust verification and revision in evidence-scarce scenarios, a multi-agent fact-checking mechanism that augments limited evidence with role-specific agent debate and propagates confidence over a graph is proposed. Experiments on PSCB show that our approach achieves state-of-the-art performance.

Abstract:
Large language models (LLMs) are increasingly used as scientific assistants, but their helpfulness can become harmful when they validate or build upon a user’s flawed scientific premise instead of challenging it. We study this failure mode as scientific sycophancy and introduce SciSycEval, a trap-based benchmark that tests whether LLMs reject, ambiguously qualify, or sycophantically operationalize flawed but plausible premises. Using trap assertions constructed from evidence abstracts across six domains and three reasoning-error types, we evaluate high-performing models under controlled task and authority settings. We find that scientific sycophancy decreases sharply across stronger GPT models, but persists as a trigger-dependent failure mode: generative tasks, high-authority framing, domain-specific evidential gaps, and overgeneralized premises substantially amplify sycophantic behavior. Critical-evaluation prompts reduce but do not eliminate the failures, showing that scientific LLMs must be evaluated and safeguarded for epistemic robustness under the interaction pressures that trigger sycophancy, rather than for factual accuracy alone.

Abstract:
Automatic Medical Coding (AMC), which assigns standardized International Classification of Diseases (ICD) codes to clinical notes, is essential for medical reimbursement, quality reporting, and clinical research. Existing pre-trained language model (PLM)-based methods typically formulate AMC as an extreme multi-label classification problem over a predefined code set, while recent large language model (LLM)-based approaches instead frame it as generation or multi-step reasoning. However, key challenges remain, including the extreme length of clinical notes that hinders effective interpretation, the vast ICD label space, and complex coding rules that are not explicitly captured by LLMs. In this work, we propose Knowledge-Guided Reasoning over Clinical Evidence with LLMs (KREL), a framework that leverages LLMs for clinical text understanding and reasoning while integrating selected structured coding relations as external knowledge. This design enables tight coupling between domain knowledge and LLM reasoning, reducing hallucinations and supporting rule-aware verification. Experiments on benchmark datasets show that KREL consistently outperforms strong PLM-based and state-of-the-art LLM-based baselines.

Abstract:
Traditional finance relies on experts to hand-craft factors through a principled process grounded in economic rationale. Recent LLM-based multi-agent systems have automated this process, scaling factor mining far beyond manual effort. However, these automated approaches optimize directly for returns and rarely check whether a generated factor still expresses the economic hypothesis that motivated it. We identify this inconsistency between mathematical form and economic meaning as a structural failure mode of return-oriented automation. The resulting factors blur the line between real signals and spurious correlations and break down across regime shifts. We propose FaVOR (Factor Validation through Observable Reasoning), an agentic framework that restructures factor mining around hypothesis-level evidence rather than return outcomes. In place of the standard hypothesis-to-formula leap, FaVOR enforces a three-stage consistency loop tying mathematical form to economic rationale throughout. (1) Decomposition splits a broad economic hypothesis into independent observable conditions. (2) Validation checks whether each factor reflects its intended condition. (3) Integration merges them into a composite whose structure remains interpretable. On the CSI 500 and S P 500 in 2025, FaVOR outperforms existing baselines while remaining effective across regimes. FaVOR shows that hypothesis-grounded factor discovery produces signals that are interpretable by construction, regime-robust, and economically faithful. The code is available at https://github.com/damilab/FaVOR.

Abstract:
Millions of multilingual documents remain locked in image formats, but current document parsers are still largely evaluated and optimized for a narrow range of high-resource languages and regular writing layouts. As a result, their reliability drops sharply when facing underrepresented scripts, vertical or right-to-left writing, and mixed-script document structures. We present HoloDP, a multilingual benchmark suite and script-aware framework for document parsing. HoloDP first introduces a standardized evaluation suite covering 13 languages across seven script families, with tasks spanning page-level layout analysis, region-level parsing, and mixed-script structural composition. Built on this resource, HoloDP further develops a script-aware parser that combines a Swin Transformer encoder, a four-branch direction-adaptive vision-to-text module, and a multilingual decoder. The adaptive module explicitly captures horizontal, vertical, diagonal, and global spatial priors, enabling the model to better handle heterogeneous writing systems and layouts. Extensive experiments show that HoloDP delivers consistent gains on underrepresented and mixed-script benchmarks while remaining competitive on high-resource Chinese and English documents. Ablation studies confirm that the benchmark resources, direction-aware adaptation, and script-conditioned modeling provide complementary benefits. We will release the code, models, and benchmarks for further research.

Abstract:
Formal methods play a crucial role in ensuring the reliability of critical systems through rigorous mathematical verification. However, their adoption remains limited due to the labor-intensive nature of manual proof construction. Recent advances in large language models (LLMs) have opened new opportunities for automated theorem proving. Two main paradigms have emerged: stepwise tactic-based generation and whole-proof synthesis. While both approaches have complementary strengths, existing work largely treats them in isolation. In this work, we propose HybridProver, a unified framework that integrates whole-proof synthesis and tactic-based generation through proof sketches as an intermediate representation. This design enables the reuse of partially correct proof structures while effectively combining high-level planning with fine-grained reasoning. We implement HybridProver in Isabelle/HOL and post-train two 7B-scale LLMs on our optimized Isabelle datasets. Experiments on the miniF2F Isabelle benchmark achieved a 73.8% success rate and improved upon the previous state of the art (61.9%), demonstrating that lightweight models, when combined with our approach, can effectively generate Isabelle/HOL proofs without relying on very large LLMs. Ablation studies further analyze the impact of dataset quality, training configurations, and sampling strategies on proof generation. All code and datasets are released.

Abstract:
Homelessness is a persistent social challenge, impacting millions worldwide. Over 876,000 people experiencing homelessness (PEH) were recorded in the U.S. in 2025. Social bias is a significant barrier to alleviating homelessness, shaping public perception and influencing policymaking. Because online textual media and offline city council discourse both reflect and influence public opinion, they provide valuable signals for identifying and tracking social biases against PEH. We release the first multi-domain PEH bias corpus with a 16-category multi-label taxonomy: a 1,698-item stratified gold-standard set annotated by partner-trained raters, plus 48,389 GPT-4.1-labeled texts, drawn from Reddit, X (formerly Twitter), news, and council meeting transcripts across ten U.S. cities (2015-2025). We benchmark six prompted LLMs on the gold-standard set and complement F1 with prevalence-gap audits. Moderate F1 coexists with large miscalibration: every model over-tags "not in my backyard" (NIMBY) (+11.5 pp) and under-detects factual claims (-30.5 pp). Error analysis on consensus false positives reveals that models treat housing vocabulary and question form as opposition proxies, producing NIMBY false positives on pro-service text. The corpus and audit protocol support municipal PEH stigma monitoring without treating teacher labels as ground truth.

Abstract:
Recent structured RAG methods leverage tree- or graph-based reasoning structures to improve multi-hop QA. However, they face key limitations in evidence-intensive QA, where answering a question requires synthesizing information scattered across dozens or even hundreds of documents: structural rigidity, which limits adaptive reasoning expansion, and topology-ignorant evidence gathering, which prevents effective integration of evidence across different reasoning nodes. To address these issues, we propose APT-RAG, an Adaptive Planning and Topology-aware evidence gathering RAG framework. Adaptive planning dynamically expands the reasoning structure based on question dependencies and evidence requirements, while topology-aware evidence gathering improves evidence coverage through sibling evidence reuse, direct retrieval, and evidence aggregation from child nodes. We further introduce evidence-guided batched answer generation to reduce significant generation overhead in evidence-intensive QA. In the experiments on evidence-intensive QA benchmarks, APT-RAG outperforms existing structured RAG methods. Our code is available at https://github.com/hyudsl/APT-RAG.

Abstract:
Traditional in-vehicle agents mainly follow an instruction-following paradigm, where actions are executed only after explicit user commands. However, real-world cabin scenarios often require agents to proactively determine whether intervention is necessary by jointly reasoning over visual and state inputs. To address this gap, we propose CabinProAct, a multimodal benchmark for proactive intelligent cabin agents. In this setting, agents must jointly reason over in-cabin and out-cabin visual information, vehicle states, and executable APIs to autonomously decide whether to intervene and generate corresponding actions. We further introduce an execution-verification-based synthetic data generation framework to automatically construct execution-consistent proactive interaction data. We evaluate six state-of-the-art multimodal models. The results indicate that execution-grounded synthetic supervision effectively transfers to real-world in-cabin settings, enabling more robust proactive responses in real-world cabin scenarios.

Abstract:
Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains underexplored. Fire-smoke understanding is central to public safety and disaster response, but most existing benchmarks lack diverse real-world scenarios and context-aware evaluation. We introduce SAFIRE, a large-scale benchmark for fire-smoke understanding in MLLMs, comprising 83K captioned images from 20 scenarios and 193K multiple-choice VQA (MCVQA) generated from a 9.7K-image subset, spanning 10 evaluation dimensions from basic perception to higher-order reasoning. A GPT-5.4-assisted multi-stage verification pipeline with MLLM majority voting ensures annotation quality. Evaluating ten open-source MLLMs (8B-38B) yields an average accuracy of 61.9%, exposing major gaps in safety-critical reasoning. We further show that adapting vision encoders with only 7% of our domain-specific data boosts fire-scene classification accuracy from 20.1% to 64.5%, indicating that carefully curated data can yield substantial gains even when data volume is limited. All datasets, models, and code are available at https://risys-lab.github.io/SAFIRE/.

Abstract:
Large language models (LLMs) are increasingly used in computational social science, both to analyze human-generated web content and to act as human surrogates capable of answering substantive research questions. As these uses expand, a critical methodological question arises: how stable and reliable are the answers LLMs produce? We examine this question through the lens of stance mining on gun control and gun rights, a mature and widely used NLP task applied to one of the most extensively studied and polarized political issues in the United States. Using a multi-platform dataset spanning diverse online communities, we evaluate the stability of LLM-predicted stance across model families, prompt designs, and conversational contexts. Despite strong expectations of robustness in this setting, we observe substantial variability in predicted stance across LLMs and prompts. To contextualize these findings, we compare model-induced variability with human variability by analyzing annotations from raters with differing political affiliations. We find that political priors and annotator disagreement persist even in this ostensibly straightforward classification task. As a useful resource, we release a stance dataset annotated by raters with diverse political affiliations. Our results show that outcomes in stance mining depend jointly on which LLM is used and which humans provide the inputs, underscoring dual sources of variability that have important implications for using LLMs as research instruments in computational social science.

Abstract:
Formal theorem proving provides executable verification signals from proof assistants, making it a key domain for improving the reliable reasoning ability of large language models through reinforcement learning (RL). However, RL-based prover training is increasingly bottlenecked by the scarcity of high-quality hard tasks: many existing Lean instances are either already solvable by current provers or too uninformative to drive further improvement. We propose ProofFrontier, a proof-structure-guided method for generating statement-preserving boundary-hard training tasks from verified Lean proofs. Instead of synthesizing open-ended problems that may be ill-posed or unverifiable, ProofFrontier starts from existing theorems with Lean-certified proofs, preserving task completeness and formal verifiability. It selectively withholds proof-structural guidance around important local nodes, encouraging the prover to rediscover nontrivial intermediate reasoning while keeping the original theorem statement unchanged. We further introduce capability-aware filtering to retain tasks that are solvable in principle but not easily solved by the base prover, yielding hard yet rewardable RL data. Training with GRPO on ProofFrontier-generated tasks consistently improves DeepSeek-Prover-V2 and Goedel-Prover-V2 on miniF2F, ProofNet, and PutnamBench. Notably, these gains are achieved using only hundreds of synthesized tasks, showing that complete boundary-hard tasks can be more data-efficient than simply scaling weak or already-solved proof data for theorem-proving RL.

Abstract:
Existing LLM safety measures operate as black boxes, lacking categorical precision and calibrated severity assessment. We propose PRISM (Prototypical Representation for Internal Safety Mapping), a fine-grained two-stage safety auditing framework grounded in internal mechanistic representations. Stage I localizes functional safety zones via spectral analysis and maps internal activations onto concept prototypes for camouflage-resilient categorical risk identification. Stage II decomposes violation severity across orthogonal dimensions through structured CoT reasoning, producing calibrated human-auditable ordinal verdicts. PRISM achieves a new best F1 of 77.60% on the adversarial ToxicChat benchmark and approaches 671B frontier judges within a 2.3% margin on severity calibration, demonstrating that mechanistic grounding and principled task formulation can substitute for scale in fine-grained safety governance.

Abstract:
Low-Rank Adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet its robustness to textual adversarial perturbations remains understudied. Existing adversarial training methods often improve robustness by solving costly min-max objectives with inner perturbation optimization with suboptimal results. In this research, we propose Expectation-Regularized LoRA (ER-LoRA), a simple expectation-based regularization approach that improves the robustness of LoRA without requiring inner maximization or adversarial example generation. ER-LoRA smooths predictions over a LoRA-aware low-dimensional perturbation distribution and admits a closed-form curvature-aware objective via a local Gaussian approximation. It adds no trainable parameters and keeps the LoRA architecture unchanged. Experiments on adversarial NLP benchmarks and textual attacks show that ER-LoRA improves robustness while preserving clean accuracy and parameter efficiency.

Abstract:
Multi-condition retrieval requires systems to identify documents that satisfy multiple distinct constraints, moving beyond mere topical relevance. While query decomposition is widely adopted as an intuitive remedy, its effectiveness across different retrieval pipeline stages remains underexplored. In this paper, we conduct a stage-aware empirical study and uncover a stark, stage-dependent effect: decomposition during initial retrieval frequently harms retrieval performance due to semantic dilution, yet substantially improves reranking by enabling more fine-grained constraint verification. Motivated by these insights, we propose a principled Stage-Aware Decomposition framework that retains the monolithic query during initial retrieval to preserve global semantic context, while employing sub-queries exclusively during reranking for fine-grained constraint matching. Extensive evaluations on the MultiConIR and SSRB benchmarks demonstrate that our framework consistently improves ranking performance for compositional queries across multiple retrieval and reranking models.

Abstract:
Neural text representations often lack structural guarantees for syntactic variations, relying instead on statistical interpolation to recognize paraphrases. We introduce the Pāṇinian Intermediate Representation (PIR), a deterministic compiler that maps English sentences to role-tagged canonical sequences using Universal Dependency parses. Unlike heuristic methods such as stopword removal, the PIR compiler is deterministic: given a correct dependency parse, syntactically equivalent paraphrases compile to identical strings. On a 200-pair diagnostic suite of templated syntactic alternations spanning active/passive voice, dative shift, and topicalization, the compiler produces identical strings for 196 pairs; all four failures are structurally typed and traced to known parser limitations. Stopword removal produces identical strings for 0 of 200 pairs. To bridge the performance gap in downstream tasks, we propose Hybrid PIR, which selectively restores lexical content through a deduplication mechanism. Hybrid PIR achieves similarity performance comparable to stopword removal (0.8130 vs. 0.8154 average ρ; bootstrap 95% CI on the difference includes zero) at 43.8% token compression, while providing structural guarantees that surface-level methods cannot. On retrieval benchmarks, Hybrid PIR reaches 80% of the English baseline nDCG@10 on BEIR SciFact and 86% of MRR@10 on MS MARCO QA at 39.0% compression. A three-variant representation ablation validates our deduplication design: removing this mechanism degrades retrieval performance by 3.5 points and turns the compression benefit negative (-3.0%). In a QQP-validation duplicate-detection use case (40,430 pairs), Hybrid PIR reaches precision 0.861 with no tuning, matching a tuned embedding threshold and validating the intended deployment as an offline canonicalization tool for deduplication and index construction.

Abstract:
A cheap, now-standard safety filter attaches a linear probe to a frozen LLM’s hidden states to flag jailbreak prompts before generation, reaching near-perfect in-distribution accuracy at negligible cost. We show this single-layer probe is not a usable real-time filter, explain why, and give a fix. Across two model families, five attack types, ten languages, and three sets of hard-but-benign prompts, we find: (i) published probes are badly miscalibrated, so the headline 98–100% detection is an artifact of the chosen threshold (expected calibration error 0.20–0.33, and 0.07–0.20 even after a leakage-controlled retraining); (ii) most of the signal is benign writing style—its "register"—since a model-free style classifier reproduces ~0.92 of the probe’s accuracy; (iii) a 5% false-positive guarantee set on in-distribution benign prompts rises to 68–98% on diverse traffic; and (iv) a single 20-token suffix drops recall to near zero. Going beyond correlational "shortcut" diagnoses, we use concept erasure (LEACE) to causally remove a 3-dimensional register subspace: this cuts ~60 points of over-refusal (false flags on benign prompts) while leaving detection essentially intact (AUC drop ≤0.003), and the same subspace recurs across three architectures (verified against a shuffled-source-label null). Our recipe—register erasure plus deployment-time distribution-free calibration—achieves a guaranteed ≤5% false-positive rate on held-out diverse benign prompts at 0.945–0.961 recall, versus 0.18–0.29 for the un-erased probe. Cheap probing is a measurement instrument, deployable as a filter only via this recipe.

Abstract:
Large language models often exhibit a substantial gap between their performance in English and in lower-resourced languages on equivalent knowledge queries—a cross-lingual consistency issue that remains underexplored for Indian languages and their code-mixed counterparts. To study this gap, we introduce IndicKLAR, an Indic extension of the KLAR-CLC benchmark covering 18 of the 22 scheduled Indian languages. For 11 widely used language pairs, we additionally provide code-mixed variants. Both monolingual and code-mixed inputs verified by native speakers. This three-way alignment enables us to examine how knowledge recall consistency varies across English, code-mixed, and native Indian language inputs. Across nine open-weight models, we find that the accuracy gap between native-language and English inputs can reach ∼ 0.50, while code-mixed inputs substantially reduce this gap, bringing performance within ∼ 0.05 of English without any model-level intervention. Motivated by this finding, we evaluate several prompting strategies that differ in how explicitly language conversion is exposed: a two-stage translate-then-answer setup, a one-stage joint translation-and-answer prompt, and Translate-in-Thought (TinT)—a single-step strategy in which the model internally converts the input and outputs only the final answer. Across the native → code-mixed → English performance trajectory, we observe a consistent flip point—the transition from incorrect to correct prediction—between the native and code-mixed settings. Notably, this pattern holds both when the code-mixed representation is explicitly provided as input or when the model is prompted to convert internally using TinT.

Abstract:
Large language model (LLM)-based agents solve complex long-horizon tasks through iterative interaction with environments, where effectively learning from experience under sparse or delayed feedback remains a key challenge. Existing approaches typically rely on either non-parametric memory-based experience reusing or parametric reinforcement learning, each with inherent limitations when used in isolation. To this end, we propose ExpRL, a unified experience learning framework that bridges non-parametric memory and parametric policy optimization for LLM agents. ExpRL induces abstracted success and failure experiences from on-policy interaction trajectories, stores them in an external memory, and integrates experience acquisition and utilization directly into the reinforcement learning process via an agentic experience acquisition mechanism. This design enables memory and policy to co-evolve and mutually reinforce each other, effectively unifying online policy learning with offline experience reuse. Experiments on ALFWorld and WebShop demonstrate that ExpRL achieves an average performance improvement of 13.0 over GRPO, with improved learning efficiency and stability. We further show that the induced experience memory generalizes across different models and can be directly transferred to stronger LLMs without additional training.

Abstract:
Large audio-language models (LALMs) increasingly use explicit reasoning traces for complex audio understanding, yet the evaluation of reasoning quality remains underexplored. Although process-level benchmarks for process reward models (PRMs) have advanced reasoning evaluation in text and multi-modal domains, comparable evaluation for audio reasoning remains limited. In this paper, we present AudioProcessBench, a comprehensive benchmark for step-level process error identification in audio reasoning. AudioProcessBench contains diverse reasoning traces generated by 6 audio and omni language models. Each trace is segmented into discrete reasoning steps and annotated with binary step correctness and fine-grained error types. Our benchmark evaluates models under three complementary paradigms: (1) step correctness identification, (2) error-type-conditioned detection for diagnosing audio-specific verifier capacities, and (3) chain-level aggregation, where verifiers select or aggregate among multiple reasoning traces for the same question. This design enables a systematic analysis of whether current models can detect process errors, whether their weaknesses differ across audio-specific error types, and whether process verification translates into improved answer selection. AudioProcessBench provides a testbed for future research on audio reasoning verifiers, process reward models, and reliable omni-modal reasoning.

Abstract:
GRPO-trained LLMs sharpen single-sample reasoning but collapse in policy entropy: 7B models lose to their base at Pass@64 on AIME. Existing mitigations score tokens from current-step statistics, missing that collapse is a trajectory-level event accumulating through coherent batch drift. We propose FRACT, a per-rollout score projecting a learned T-step committor’s gradient onto the entropy direction in the dual Fisher metric, with a forward-only closed form requiring no extra backward through the policy. As a sparse, sign-aware selective KL atop GRPO, FRACT improves Acc@16 over KL-Cov by +.037/+.034/+.045 across Qwen2.5-Base 1.5B/3B/7B and simultaneously raises Pass@1 and Pass@64 on all four benchmarks at 7B—mitigating, not merely shifting, the accuracy–diversity trade-off. Random-matched-mass gains only +.009 and Anti-FRACT underperforms GRPO by −.007 at 1.5B—the score, not regularization mass, is the cause at ∼1.5% structural overhead.

Abstract:
Large language model (LLM) agents have increasingly advanced service applications, such as booking flight tickets. However, these service agents suffer from unreliability in long-horizon tasks, as they often produce policy violations, tool hallucinations, and misaligned actions, which greatly impedes their real-world deployment. To address these challenges, we propose NOD (Navigator-Operator-Director), a heterogeneous multi-agent architecture for service agents. Instead of maintaining task state implicitly in dialogue context as in prior work, we externalize a structured Global State to enable explicit task state tracking and consistent decision-making by the Navigator. Besides, we introduce selective external oversight before critical actions, allowing an independent Director agent to verify execution and intervene when necessary. As such, NOD effectively mitigates error propagation and unsafe behavior in long-horizon tasks. Experiments on τ²-Bench demonstrate that our NOD achieves higher success rates and critical action precision over baselines. More importantly, NOD improves the reliability of service agents by reducing policy violations, tool hallucinations, and user-intent misalignment.

Abstract:
Emotional Support Conversation (ESC) aims to alleviate users’ emotional distress through multi-turn interactions by selecting appropriate support strategies and generating aligned responses. Despite recent progress, existing approaches face three key limitations: (1) the inability to capture sequential transition patterns of strategies within conversations; (2) a training-inference mismatch in strategy prediction; and (3) reliance on a shared decoder that limits strategy-specific expression. To address these challenges, we propose S3-ESC, a simple framework of three components that are model-agnostic across encoder-decoder ESC backbones. First, we introduce a Sequential strategy predictor that models strategy transition patterns. Second, we apply Scheduled sampling for strategy to mitigate the training-inference mismatch. Third, we propose Selective LoRA for strategy that enables strategy-specialized generation. Experiments on six ESC backbones show that S3-ESC consistently improves strategy prediction accuracy, by up to 4.5pp, while enhancing response quality on most of them. Our code is publicly available.

Abstract:
The growing scale of academic peer review has motivated the use of Large Language Models (LLMs) as review assistants, yet LLMs can generate fluent but unsupported claims that undermine review reliability. Existing hallucination benchmarks are not designed for peer review, where verification requires grounding claims in long, technical papers. We introduce HalluPeer, a benchmark for detecting hallucinations in scientific peer reviews, providing aligned triples of paper content, human-written reviews, and hallucination-injected reviews, annotated for detection, classification, and localization. Our pipeline induces a peer-review-specific hallucination taxonomy, identifies review contexts, and injects hallucinations with automated filtering. Experiments on 12K papers and 38K reviews show that existing detectors struggle to separate hallucinations from legitimate critique, while evaluation on authentic reviews demonstrates that HalluPeer-defined hallucination patterns occur in real peer reviews, highlighting the critical need for source-aware verification.

Abstract:
Text-to-ES aims to translate natural language into executable Elasticsearch(ES) queries. Unlike more open-ended reasoning tasks, such as mathematical reasoning, this problem is defined over a constrained domain-specific language, where the synergy between forward reasoning and backward reasoning can be more explicitly realized under strict structural constraints. While recent work has explored reasoning-oriented training and prompting in text-to-SQL, forward and backward reasoning have not been systematically studied in text-to-ES. In this paper, we systematically study the bidirectional duality of the text-to-ES task by establishing a bidirectional reasoning perspective across both training and prompting stages. We introduce two large-scale datasets, TERD and ETRD, for forward and backward reasoning, respectively, and propose DualES, a unified framework that combines the two. Experimental results show that our DualES-CoRE strategy sets a new State-of-the-Art(SOTA), achieving 69.23% on LED and 50.14% on BirdES. This corresponds to absolute execution accuracy gains of 6.92% and 7.34% over previous benchmarks, outperforming both open-source and closed-source baselines. We will release the datasets in the future.

Abstract:
Large language models (LLMs) can generate executable solver code from natural-language descriptions of optimization problems. However, existing verification signals focus on whether the generated formulation produces the correct objective value. Such signals can overlook constraint-level errors: incorrect formulations may still produce the same optimum when extra or missing constraints do not affect the tested instance. We propose constraint injection, a verification method that directly tests whether the generated code implements the intended constraints. We use feasible solutions to detect incorrectly added constraints and one-constraint-violating solutions to detect omissions. Together with the optimal value check, these tests verify both the objective and the constraint set. We evaluate this approach on vehicle routing problems (VRPs), which provide a challenging testbed with diverse, tightly coupled operational constraints. We develop VRPCoder, an 8B model for generating Gurobi code from natural-language VRP descriptions, and VRPBench, an expert-verified benchmark spanning 21 VRP variants across four subsets. The verifier serves as a rejection-sampling criterion for synthetic data construction and as a rollout-level reward for reinforcement learning (RL). On VRPBench, VRPCoder-RL reaches 93% overall Pass@1, outperforms Gemini-3.1-Pro Preview on three subsets, exceeds Claude-Sonnet-4.5 by 28 percentage points, and exceeds the strongest prior OR-LLM by 78 percentage points.

Abstract:
Automated evaluation of radiology reports is essential for clinical trust, yet clinically significant errors (e.g., missed findings or incorrect descriptions) can critically mislead diagnosis. Traditional metrics fail to distinguish between clinically meaningful and harmless textual variations, collapsing report quality into a single scalar score. Although Large Language Models (LLMs) embody richer medical knowledge, such metrics still lack an explicit boundary of clinical significance. To bridge this gap, we adopt the ReEvalMed benchmark as testbed and evaluate metric-level clinical significance from detecting true clinical errors ("Discrimination") and tolerating insignificant variations ("Robustness"). We design an evaluation framework with one-pass and two-pass settings, and compare 8 LLM evaluators against score-based baselines. Results reveal a discrimination bias: LLMs readily detect true clinical errors but struggle to tolerate harmless variations. To mitigate this, we synthesize 4k report pairs and train lightweight interpretable metrics on Qwen3-8B and MedGemma-4B. Our trained metric sharpens the clinical significance boundary, surpassing 32B-scale medical LLMs and remaining competitive with proprietary models. Notably, two-pass inference does not eliminate errors, but redistributes them between discrimination and robustness. We will release the dataset and metric to advance future research.

Abstract:
This paper investigates structural priming in language model (LM) production, examining how preceding structural context influences sentence completion. While prior work has demonstrated priming effects in comprehension of structural alternations, it remained unclear whether these persist in production, where, when generating, an LM samples from many possible continuations at each step. We address this question through a series of controlled sentence-completion experiments on dative constructions. In line with prior work, we find that LMs are susceptible to structural priming, particularly in sentences that are semantically coherent. In terms of priming magnitude, we find that while there is a greater relative increase of double-object datives against our baselines, in line with inverse frequency effects, there is a larger absolute increase in prepositional-objects, the more frequently produced construction. Finally, we not only observe that structural priming is boosted by lexico-semantic coherence, but that structurally primed completions display greater levels of lexico-semantic repetition. Taken together, our evidence supports the view that structural priming in LMs operates across multiple levels of linguistic representation, facilitating, and facilitated by syntactic, lexical, and semantic alignment.

Abstract:
Embedding representations are widely used in natural language processing, yet their semantic interpretation depends on the operators used to compare them. Previous work has shown that, for certain embedding models, the inner product is linearly correlated with Pointwise Mutual Information (PMI). However, this relation has so far been established only empirically or analytically for specific models. In this work, we formally show that, under certain assumptions about distributional semantic embedding models, the inner product between embeddings approaches PMI. This theoretical result provides a more general explanation for the observed relationship and enables the formulation of training criteria aimed at learning embedding spaces in which operations between embeddings are semantically interpretable.

Abstract:
Rule-based methods for temporal knowledge graph (TKG) forecasting are valued for the interpretability of their explicit logic chains. However, they suffer from a severe cold-start coverage gap: exact-match closed walks terminate at sparse entities, leaving many queries without any rule evidence. Existing similarity-based remedies are unreliable because their similarity is derived from static structural co-occurrence or shallow embeddings with no temporal calibration and hand-tuned thresholds. To address these limitations, we propose SimRule, a collaborative recommendation framework for TKG rule mining. SimRule first mines both exact rules and collaborative-surrogate rules through a temporally calibrated dual-view entity similarity index. An adaptive dual-branch fusion then ranks candidate objects from their aggregated evidence, and each forecast remains fully auditable as an explicit rule chain with quantified time gaps. On ICEWS14, ICEWS18, and ICEWS05-15, SimRule achieves state-of-the-art MRR and Hits@ K ( K ≥ 3 ) among rule-based methods.

Abstract:
Despite the non-autoregressive potential of diffusion language models (dLLMs), existing decoding strategies exhibit positional bias, failing to fully unlock the potential of arbitrary-order generation. In this work, we delve into the inherent spectral characteristics of dLLMs and present the first frequency-domain analysis of dLLMs, showing that low-frequency components in hidden states primarily encode global structural information and long-range dependencies, while high-frequency components capture local details. Based on this observation, we propose FourierSampler, which leverages a frequency-domain sliding window mechanism to dynamically guide the model to achieve "structure-to-detail" generation. FourierSampler outperforms other inference enhancement strategies on LLaDA and SDAR, achieving relative task-specific gains up to 20.4% on LLaDA-1.5 and 16.0% on LLaDA-8B-Instruct, as well as average gains up to 8.6% across tasks.

Abstract:
Prompt-injection defences for LLM agents often classify external context as malicious or benign, ignoring whether a particular target model would interpret it as an instruction. We introduce ProbeCourt, an audit layer that requires no target-model training and uses only black-box access to the target before tool execution. It cross-examines the target model’s read-only interpretation using four paired probes—source reference, counterfactual action, permission boundary, and paraphrase stability—and aggregates the responses into context and span risk scores, a quantile-calibrated operating threshold, and a reviewable evidence bundle. Across four benchmarks and six backbones, ProbeCourt averages AUROC = 0.90, outperforming the strongest black-box baseline by 0.09 overall and the strongest applicable white-box baseline by about 0.05 on open-weight models. It reaches 0.74 token-F1 for span localisation and reduces AgentDojo attack-success rate from 47.2% to 9.4%, with a 2.0-point average benign-utility loss. Under DataFlip-style adaptation, AUROC remains 0.67 while the strongest single-pass baseline drops by about 0.40, from 0.812 to 0.41. A probe-aware attacker reduces the one-witness configuration to 0.61; a three-family ensemble recovers 0.68. Thus, ProbeCourt improves pre-execution auditing but remains vulnerable to witness manipulation.

Abstract:
Hateful memes are difficult to detect because their harmful meaning often emerges from the interaction between image and text rather than from either modality alone. This challenge becomes more severe in multilingual settings, where memes frequently contain code-mixed language and culturally grounded references. Existing Indic hateful meme datasets lack scale, language coverage, or task diversity, and current multimodal methods often overlook cross-modal semantic divergence. We introduce IndicHM , a trilingual hateful meme benchmark containing over 15,004 manually annotated image-text pairs in English, Hindi, and Bengali, including code-mixed content, supporting both hate detection and target classification. We also propose IndicHMNet, a disagreement-centric framework that decomposes representations into shared agreement and modality-specific residual subspaces and propagates the resulting disagreement-aware signal through multimodal fusion and contrastive representation learning. The proposed framework adaptively emphasizes modality-specific semantic divergence during fusion and optimization, enabling improved modeling of complex hateful meme semantics. Experiments across all languages and tasks demonstrate consistent gains over strong multimodal and vision-language baselines.

Abstract:
Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cultural understanding or mere visual matching. To probe this distinction, we introduce CulturalMenuBench, a benchmark of 4,870 items in 10 languages across 18 regions; its 10 tasks pair final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels, spanning basic recognition to process-grounded cultural attribution. Evaluating 12 models exposes a substantial knowledge-application gap: models exceeding 94% on standard multiple-choice tasks drop to at most 56% when attributing dishes to Chinese regional cuisines, despite an identical four-way format. Diagnostic analyses explain why: error patterns are consistent with random guessing, accuracy tracks visual distinctiveness rather than cultural structure, and models classify cuisines more accurately from dish names alone than from images (+7–18 points). The knowledge is thus present but cannot be activated through visual input. An ablation confirms these tasks genuinely require procedural evidence: removing sequential cooking images selectively degrades process-grounded tasks while others remain stable. Overall, CulturalMenuBench shows that near-perfect recognition can conceal an inability to apply cultural knowledge, motivating training that explicitly connects perception, procedure, and cultural context. Code and data are publicly available.

Abstract:
Logical reasoning with large language models (LLMs) is a critical capability, as it reflects a system’s ability to correctly deduce hypotheses from a given context using faithful deductive processes. However, LLM reasoning has often been shown to be sensitive to small surface-level variations in problem formulation, raising questions about whether models truly follow the underlying logical structure. Studying this behavior is challenging because the symbolic components of logical problems, such as operators and predicates, are difficult to systematically manipulate in natural language. We introduce a tool-driven framework for generating controlled, label-preserving edits to logical reasoning problems. Our method operates on symbolic representations of first-order logic and constraint satisfaction problem tasks, enabling targeted modifications to logical operators and other structural components before translating them back into natural language. Using this framework, we evaluate various LLMs under cumulative and individual operator edits and analyze their behavior in response to these changes. Our quantitative and qualitative analyses show that LLM reasoning behavior under controlled operator edits is inconsistent, regardless of model size or family: models sometimes adapt correctly to structural changes but often fail to track their logical consequences. The results from this automated stress test enable an evaluation of language models across different dimensions and help measure the reliability of their reasoning.

Abstract:
LLM-as-a-judge protocols are commonly debiased by instructing judges to ignore presentation cues such as citation formatting, source labels, and evidence-display style. We show that this intervention can suppress bias while damaging the resolution of the measurement instrument. We introduce TraceJudgeBench, a diagnostic benchmark for auditing citation-like artifacts in RAG and agent-workflow evaluation, covering content-equivalent pairs, citation ablations, correctness conflicts, human-validated soft and moderate quality gaps, prompt-strength ladders, decoupled judging, and a controlled workflow-ranking probe. Across GPT-5.5, Claude Sonnet 4.6, and DeepSeek V4-Flash, stronger anti-citation prompts reduce worse-cited wins from up to 50.5% to 0%; yet some operating points already convert validated moderate-gap decisions into Tie before the strict stress-test endpoint, while correctness-conflict accuracy remains at or above 93.0%. Open-weight Qwen2.5-14B-Instruct-AWQ and Gemma-3-12B-IT runs reproduce the central moderate-gap frontier, with TRACE-style decoupling recovering 96.5–98.5% better-plain resolution. Human validation separates three meanings of Tie: correct equivalence Tie, calibrated soft-boundary Tie, and resolution-destroying Tie on validated quality gaps. We frame debiasing as a measurement intervention whose bias suppression, resolution retention, Tie cost, and protocol cost must be reported jointly. We include TraceJudgeBench as a supplementary artifact with anonymized splits, prompts, raw judge outputs, validation summaries, and analysis scripts.

Abstract:
Multimodal Large Language Models (MLLMs) often struggle with complex mathematical visual reasoning primarily due to a lack of fine-grained perception, causing initial visual hallucinations to directly trigger cascading reasoning failures. In traditional end-to-end reinforcement learning (RL), sparse rewards fail to decouple perceptual hallucinations from logical missteps, hindering targeted perception optimization. Alternatively, fine-tuning with perception-enhanced CoT data incurs high costs and hallucinations. In this paper, we address these challenges by proposing UniCAR-RL, an annotation-free RL framework. By explicitly decoupling the optimization of perception and reasoning during the training process, it achieves isolation and optimization of both capabilities. Specifically, UniCAR-RL consists of three synergistic branches: 1) a Caption-RL branch that optimizes perception capabilities through verifier-guided reasoning validation; 2) a Reasoning-RL branch that performs logical reasoning based on a gold image description to halt cascading errors; 3) a QA-RL branch that retains native end-to-end alignment to ensure robust question-answering performance. Experiments show that UniCAR-RL substantially improves MLLMs’ mathematical and visual reasoning using only raw short-answer data. Furthermore, it demonstrates strong generalization across diverse architectures and scales.

Abstract:
Spectroscopic structure elucidation is central to molecular analysis, but recent Large Language Model (LLM)-based methods mostly formulate it as direct spectrum-to-SMILES generation. Although this paradigm can leverage paired spectral data, it does not explicitly model the analytical workflow used by spectroscopists, such as diagnostic peak interpretation, fragment reasoning, formula constraints, and chemical consistency checking. In this paper, we introduce S3C-LLM, a skill-guided and code-grounded agentic LLM for spectrum-to-structure elucidation. Rather than directly predicting a molecule, S3C-LLM retrieves modality-specific spectroscopy skills, executes analysis code to instantiate these skills on the input spectra, and integrates the resulting peak-level evidence and formula constraints before generating SMILES. Specifically, we contribute a self-evolving spectroscopy skill library, a thinking-augmented skill-code trajectory construction pipeline, and a two-stage training strategy that teaches Qwen3-4B through supervised fine-tuning (SFT) followed by our proposed step-level reinforcement learning (RL). Experiments on diverse benchmarks show that S3C-LLM consistently outperforms current general LLM and spectrum-specific models across spectrums, while using less than 1/10th of SpectraLLM’s training corpus.

Abstract:
Dynamic benchmarking is an essential research direction in the evaluation of large language models (LLMs) to avoid data contamination. Although existing works have proposed LLM-based methods for dynamic benchmarking, there still lacks a unified paradigm for performance comparison among different LLMs, which hinders researchers from model selection for specific domains and tasks. To address this challenge, we propose a comprehensive evaluation suite called DyBenchEval, including a novel seed dataset across 4 domains and 7 tasks as well as multi-aspect evaluation metrics. A cross data filling method is utilized for high-quality data augmentation with domain/task-level similarity, aiming to overcome the data scarcity problem. For automatic metrics, we consider coherence, correctness, and diversity of LLM-generated evaluation samples, simultaneously measuring the respective quality of generated questions / answers and the relationship between them. Extensive experimental results on DyBenchEval reveal deficiencies in 11 mainstream closed-source and open-source LLMs when dealing with dynamic benchmarking across different domains and tasks.

Abstract:
General-purpose code embeddings power tools for code search, classification, and retrieval. Compact transformer encoders for code typically rely on either human-written docstrings (labor-intensive and inconsistent) or mined structural signals such as execution traces (setting-specific and costly to collect). We empirically study an alternative: contrastive pretraining of small encoders with synthetically generated natural-language descriptions emphasizing code functionality and intent, paired with code in a dual-encoder framework at training and discarded at inference. We benchmark this approach against pretraining-based baselines, generalist LLMs, and embedding-specific models on eight retrieval, classification, and generation tasks across C, C++, and Java. Synthetic semantic supervision yields statistically significant gains over pretraining baselines of the same inference-time size on five of eight tasks, with parity on two more; once fine-tuned, it matches or exceeds zero-shot models two orders of magnitude larger on classification, and it stays on par with execution-aware supervision at matched pretraining data, suggesting a scalable, effective alternative to existing code-representation paradigms.

Abstract:
Large Language Models (LLMs) are powerful text generators, yet they can produce toxic or harmful content even when given seemingly harmless prompts. This presents a serious safety challenge and can cause real-world harm. Toxicity is often subtle and context-dependent, making it difficult to detect at the token level or through coarse sentence-level signals. Moreover, efforts to mitigate toxicity often face a trade-off between safety and the coherence, or fluency, of the generated text. In this work, we present a targeted subspace intervention strategy based on gradient signals for identifying and suppressing hidden toxic patterns from underlying model representations, while preserving the overall ability to generate safe, fluent content. On the RealToxicityPrompts, our method achieves strong mitigation performance compared to existing baselines while imposing minimal impact on inference complexity. Across LLMs, our approach reduces the toxicity of state-of-the-art detoxification systems by 8-20%, while maintaining comparable fluency. We further validate cross-benchmark generality on ToxiGen, where the same gains hold under adversarial, implicitly toxic prompts. Through extensive quantitative and qualitative analyses, we show that our approach achieves effective toxicity reduction without impairing generative performance, consistently outperforming existing baselines.

Abstract:
Generative process reward models (PRMs) improve step-level verification of mathematical reasoning, yet their effectiveness is limited by the quality of process supervision. The dominant Monte Carlo (MC) estimation derives step labels from final-answer accuracy rather than semantic correctness, producing systematic label noise, while standard pipelines discard the implicit tree structure of rollout data that encodes informative failure patterns. We propose CePRM, which addresses both limitations through Step-Cognitive MC Value Scaling, calibrating raw MC scores with step-level semantic signals via multiplicative adjustment, and pivot-step-directed repair mining, which exploits value cliffs in failed trajectories to construct same-prefix contrastive correction pairs via Sibling Advantage filtering. Experiments on ProcessBench and PRMBench show that CePRM, with a 4B backbone, achieves competitive or superior error localization and process verification against 7B-scale baselines, with notably strong error-side recognition.

Abstract:
The value alignment of large language models (LLMs) is crucial for ensuring responses align with human intention and value preferences. However, most evaluations of value alignment focus on Western or universal values, while assessments grounded in the value systems of specific countries remain scarce. In this paper, we introduce Pancasila-Dilemmas, an evaluation dataset of 1,834 questions derived from Indonesian news, classified by 5 values of Pancasila: Religion, Humanity, Unity, Democracy, and Social Justice. This dataset reflects daily life in Indonesia, making it suitable for measuring the value alignment of LLMs deployed for Indonesia. To ensure a more rigorous evaluation, we choose scenarios containing dilemmas. The dataset adopts a multiple-choice format consisting of a scenario, a question, and 4 different options without right/wrong decisions. The dataset is proofread by native speakers and answered by 5 diverse Indonesian citizens. We evaluate 50 closed- and open-source LLMs on our dataset. Results reveal that all evaluated LLMs achieves less than 0.5 Probability Match Score (PMS) and 0.72 Max-Vote Agreement Score (MVAS). Compared by each values, LLMs mostly struggle in Religion and Unity dilemma cases. This highlights a significant gap in capturing Indonesian values. The dataset is publicly available at https://github.com/tjunlp-lab/Pancasila-Dilemmas.

Abstract:
To build a math MLLM, current research efforts predominantly rely on elaborate training schemes or high-quality multi-modal math reasoning data. On the other hand, model merging has attracted much attention recently due to its low-cost and tune-free property, which has shown great potential for constructing math MLLM. However, most recent attempts either assume all parameters are task-relevant or adopt a less interpretable parameter selection strategy, thus merging parameters on a coarse granularity, leading to limited or even degrading performance. In this paper, we propose a causal-analysis–guided parameter merging framework, which identifies the math reasoning-related parameters in a finer granularity and interpretable manner, then merges the math LLMs with MLLM. Specifically, we first construct targeted arithmetic and logical tasks, generate reference and counterfactual samples, and then apply intervention-based causal attribution to localize parameters that are causally responsible for math reasoning. Once we identify the crucial reasoning parameters in the math LLM that exhibit higher causal effects, we can more precisely align and merge only the corresponding parameters from the math LLM into the base MLLM. The procedure is tuning-free and enhances both math reasoning and general reasoning abilities. Experimental results of three model families on four math reasoning benchmarks and two general reasoning benchmarks validate the effectiveness of the proposed approach.

Abstract:
In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectations of the communities they represent. We conduct a community-grounded mixed-methods evaluation with 19 culture representatives across five African communities, combining quantitative annotations with qualitative focus group discussions. We find that cultural alignment depends not simply on recognizable cultural markers, but on how those markers fit social, linguistic, procedural, and visual context. From these evaluations, we develop a taxonomy of cultural alignment comprising five broader cultural marker categories and eight recurring mechanisms of misalignment. We additionally evaluate five multimodal LLM judges to examine whether automated evaluation can approximate community-grounded judgments at scale. Judge reliability and score calibration vary substantially across communities, with no single judge performing consistently across all five settings. These findings motivate community-calibrated evaluation pipelines in which automated judges are validated against community judgments to determine where they can be trusted and where human review remains necessary.

Abstract:
Omni-capable Diffusion multimodal large language models (Diffusion MLLMs) unify text-to-image (T2I) generation and image-to-text (I2T) understanding within one backbone, but at substantial cost on both fronts: visual tokens far outnumber surrounding text, and for T2I this per-step cost compounds across many denoising steps. Token merging has been studied separately in adjacent fields, yet training-free acceleration spanning T2I+I2T in one omni-capable Diffusion MLLM remains limited. We present SaToMe, a training-free token-merging method built around a local-deviation operator that we interpret as measuring local representational uniqueness. The denoising trajectory exposes two natural inputs to this operator: the state h and the update direction Δh. Their spreads behave very differently along the trajectory: state-salience spread is dynamic and identifies the active merging window, while update-salience masks are window-stable and serve as a single-step protection signal for cell-level structural content. T2I uses both signals; I2T, with no temporal axis, instantiates the same uniqueness criterion on the state alone, producing a single-pass additive merge. The same I2T form transfers directly to a separate I2T-only Diffusion MLLM with a different visual representation, supporting its applicability across these visual representations. SaToMe attains +20.14% T2I wall-clock speedup on LLaDA-o at 68% visual-token retention while largely preserving reported generation quality, +35.5% I2T wall-clock speedup on LLaDA-o, and +27.5% I2T speedup on LaViDa.

Abstract:
Personal values alone cannot fully explain moral decisions. They are often treated as the main driver of moral choice, yet situational pressures can still shift actions when values are fixed. To study this effect, we introduce VISDA (Value-Informed Scenario-Driven Actions), a dataset of 80 binary moral scenarios across various themes. Keeping value profiles constant, we evaluate three open-weight LLMs (Gemma 4 31B, Qwen 2.5 32B, LLaMA 3.1 8B), a utility-based mathematical baseline over 95 Schwartz profiles, and humans for their decisions under modifiers. Each scenario is augmented with eight situational modifier axes designed to capture contextual pressures on decision-making. LLMs reverse decisions (flips) less frequently (2.03%-8.92%), with larger models showing greater sensitivity. In LLMs, authority and self-preservation modifiers drive most flips and remain statistically significant. Humans showed a pooled shift (12.36%) in their decisions. Humans prioritized stakes and affective modifiers, while LLMs overweight personal-cost modifiers. This work reveals axis-level misalignment between human and LLM sensitivity to situational moral factors.

Abstract:
Existing Text-to-SQL benchmarks remain idealized compared with enterprise scenarios, which require massive query scopes, complex schemas, and scattered external knowledge. We present BIRD-Ent and Spider-Ent, two enterprise-oriented benchmarks constructed via a refinement framework, together with Dual-Retrieval-Augmented-Generation (DRAG) Text-to-SQL, which formalizes table-schema and knowledge retrieval before SQL generation. The benchmarks feature over 4,000 columns per query and, for BIRD-Ent, documents totaling 1.5M tokens. Evaluation on state-of-the-art models shows sharp drops, with only 39.1% EX on BIRD-Ent and 60.5% EX on Spider-Ent. These results establish the Ent-series as a discriminative testbed for structurally robust and retrieval-aware Text-to-SQL.

Abstract:
Modeling the interaction between gaze dynamics and linguistic structure remains a central challenge in scanpath prediction. Existing approaches either incorporate linguistic information in ways that do not fully respect the temporal progression of reading, or ignore linguistic structure altogether. Their effectiveness is further limited by the scarcity and heterogeneity of available eye-tracking data. To address these issues, we introduce EYELIGNR, a framework for spatiotemporal scanpath prediction that integrates linguistic information at the representation level. EYELIGNR conditions language representations on fixation order and aligns gaze and text within a shared embedding space through contrastive learning. Experiments on five reading corpora show that EYELIGNR generalizes effectively across reading conditions, improving over prior state of the art on multiple scanpath metrics (e.g., up to NLD ~57%, NLL ~27%, and FLA ~59%).

Abstract:
Large Language Models (LLMs) are revolutionizing digital communication by powering conversational agents deployed across domains such as customer service, digital sales, and insurance. These agents, built on LLMs, can understand user input, retrieve relevant information, and generate coherent responses. However, while they excel at factual communication, they often lack the ability to engage in truly persuasive, context-sensitive dialogue, especially in domains like insurance, where trust and clarity are critical. Building on this need within the insurance domain, our work focuses on improving the persuasiveness of digital agents, aka LLMs. To support this, we introduce InsureDial, a Persuasive Insurance Dialogue dataset, designed to capture the nuances of persuasive communication specific to motor insurance interactions. We introduce PersuaRL, a reinforcement learning–based framework that equips LLM-driven dialogue agents with the ability to adaptively explore, select, and coordinate strategies across multiple expert modules, guided by the evolving dialogue context, to achieve more effective persuasion. We conduct extensive automatic human and qualitative evaluations on two benchmark persuasion dialogue datasets, including our InsureDial. Our evaluations consistently demonstrate that PersuaRL outperforms baseline, generating contextually appropriate and highly persuasive responses.

Abstract:
Fixed-target backdoor attacks can silently hijack the outputs of large language models by driving models to generate attacker-specified sequences upon trigger activation, motivating lightweight inference-time detection. Recent defenses like ConfGuard, which we term Local-window Confidence Detectors, leverage sliding windows to identify local spans where the top-1 probability, referred to as token confidence, exceeds 0.99 as indicators of backdoor activation. However, benign deterministic tasks such as math reasoning can exhibit similar sustained high-confidence patterns, causing confidence conflation . On the GSM8K dataset, ConfGuard’s default settings yield a 72.89% false positive rate, causing substantial over-defense. Our empirical analysis reveals that benign deterministic generation exhibits sparse confidence "dips", whereas backdoor-triggered generation forms a sustained high-confidence "plateau". Building on this, we propose RatioGuard, which utilizes a global high-confidence token ratio computed over the generated prefix to address confidence conflation. Across models and datasets, RatioGuard reduces false positives by up to 71.57 percentage points over ConfGuard while maintaining over 99% true positive rate with 1.005 × runtime latency.

Abstract:
Statute retrieval in multi-turn legal consultation is challenging because users often describe evolving facts in colloquial language, while statutes formulate legal conditions in abstract terms. This mismatch makes direct query–statute matching insufficient to capture the intermediate evidence needed to determine statutory applicability. Legal cases can bridge this gap by documenting how concrete facts and legal rationales support applicable statutes. However, case retrieval is noisy: factually similar cases may concern different legal issues, whereas cases with dissimilar surface facts may share the same applicability rationale. We propose FR-MEG, a Fact-level and Rationale-level Mixed Evidence Graph for turn-level statute retrieval. FR-MEG organizes case-derived evidence into two complementary views that model case–statute relationships through factual descriptions and legal rationales. The two views are calibrated and fused with full-corpus LegalDense retrieval, enabling case evidence to improve statute ranking without limiting candidates to graph-covered statutes. Experiments on LexRAG show that FR-MEG improves Recall@10 from 38.83% to 43.81% and NDCG@10 from 25.64% to 28.23% over the LegalDense backbone.

Abstract:
Large Language Models are now common in student assessment, but we know little about how student demographics affect their use. Sometimes, considering student demographics may be necessary—for example, to improve readability for users with lower educational levels. However, it can also risk discrimination, e.g., when assigning lower scores to students from lower socioeconomic backgrounds. We set up controlled prompts to test 1) explicit demographic effects, where we mention demographic details directly, and 2) implicit effects, where we use conversation history as a signal. We test these settings in three tasks: Automated Essay Scoring, Formative Feedback, and Metalinguistic Question Answering. We test six state-of-the-art LLMs on these tasks. In both explicit and implicit cases, the models pick up on demographic cues and can change their scoring, feedback, and answers accordingly. We find that LLMs frequently adjust the readability of feedback to education levels when it’s explicitly mentioned. On the other hand, implicit conditions produce unpredictable biases, such as in question answering, where responses from lower-education levels receive lower sentiment scores. Our results provide clear evidence of demographic sensitivity in LLMs for educational assessment tasks.

Abstract:
Real-world communication often requires pragmatic reasoning: interpreting meanings implied through context and cultural convention rather than stated literally. Existing pragmatic evaluation remains largely limited to English and high-resource languages, leaving Indic languages unexplored despite their linguistic and cultural diversity. We introduce VakyArth, the first pragmatic benchmark for Indic languages, designed as a diagnostic evaluation covering Hindi, Punjabi, Tamil, and Malayalam. VakyArth evaluates models across five phenomena: deixis, speech acts, implicature, social pragmatics, and coherence; through multiple-choice questions, natural language inference, and translation, with all items authored by native speakers. Across multilingual large language models (LLMs) of varying families and sizes, we find consistent failures on pragmatic meanings rooted in Indic linguistic and cultural conventions. Our analysis shows systematic differences across languages and tasks: MCQ accuracy exceeds NLI accuracy in all model-language combinations, translation performance does not reliably track pragmatic understanding, and Indo-Aryan languages show a translation advantage over Dravidian languages. We further show that automatic translation metrics can miss fluent but pragmatically unfaithful outputs, especially for implicature and deixis.

Abstract:
Post-training is essential for adapting Large Language Models (LLMs) to real-world applications. Deploying post-trained models faces significant challenges due to substantial memory overhead and noticeable inference latency. Existing work has identified significant redundancies in LLMs and proposed efficient architectures, namely intra-layer KV sharing and cross-layer KV sharing. However, these methods still result in high inference time overhead, remaining suboptimal for post-training pre-trained LLMs. In this paper, we identify that the Softmax operation is a primary bottleneck for LLM inference and discover that it is actually highly redundant during post-training. We propose Softmax Unification in Attention (UniAttn), a novel post-training method that unifies Softmax activations across transformer blocks to reduce LLM inference costs. Additionally, UniAttn adopts a linear projection to refine the approximation induced by Softmax unification. Experiments show that UniAttn matches the performance of standard post-training while significantly reducing inference costs, outperforming existing efficient architectures during post-training.

Abstract:
Voice interfaces are increasingly moving away from transcription pipelines towards end-to-end systems that directly respond to audio inputs. This development in turn requires a shift in evaluation methodology away from transcription accuracy and towards more substantive markers such as response validity. We introduce EchoLens, a demographically stratified dataset of 20,008 recordings from 515 adult participants residing in the U.S., with self-reported demographic information across race, gender, age, accent and primary language, among others. Each participant speaks out loud and verbatim a randomized subset of 69 advice-seeking and estimation prompts spanning 11 domains grounded in the American Time Use Survey. Most prompts elicit quantitative responses from the model, allowing for direct analysis of output distributions across demographic subgroups without requiring reliance on LLM-as-a-judge. EchoLens also contains a documented audit protocol, which—in an illustrative application—we use to evaluate six current audio-language models. Under this assessment, we find that racial disparities are more frequent and larger in magnitude than gender disparities, with particular concentration in two models.

Abstract:
Tabular foundation models like TabPFN and TabICL are often augmented with frozen large language model (LLM) embeddings when tables contain free-form text. Whether this helps is highly case-dependent: across 732 dataset–backbone–encoder configurations, 45.9% improve, 30.9% stay neutral, and 23.2% degrade. Using a fixed, outcome-free raw score, we define high- and low-readability strata before applying target-covariance Gaussian sampling, row–text shuffling, and pre-projection whitening. Each intervention reduces the mean gain in the high-readability stratum to near zero, whereas the paired intervention effects in the low-readability stratum remain within sampling noise. The first two interventions show that neither a Gaussian sample from the target covariance nor row-misaligned text reproduces the high-stratum gain; whitening further shows sensitivity to broad pre-projection geometry. Together, the results support a dependence on row-aligned embedding structure without isolating the covariance spectrum as its unique cause. The high-stratum Gaussian-replacement effect replicates on a second backbone. A within-dataset encoder ranking, Decision-Boundary Alignment (DBA), also shows stronger rank alignment with gain than the Massive Text Embedding Benchmark (MTEB) rank in our eight-dataset diagnostic.

Abstract:
Temporal knowledge graph (TKG) forecasting aims to predict future facts based on historical temporal information and still faces key challenges such as event sparsity and complex long-term dependencies. While graph-based methods effectively capture relational structure, they often fail to model nuanced long-range temporal patterns. Alternatively, standard Transformers, while potent for sequence modeling, are fundamentally time-agnostic; their generic positional encodings overlook critical TKG-specific temporal dynamics. As a result, many approaches struggle to capture fine-grained temporal patterns, limiting their effectiveness in forecasting sparse or complex events. To address these limitations, we propose HTA (Hybrid Transformer Architecture). HTA first constructs a query-specific Attention-Ready Temporal Sequence (ARTS) by aggregating historical context along a query-centered temporal reasoning path. This atomic sequence represents timestamps as dedicated temporal tokens that are contextualized together with their associated relation and entity tokens. At its core, HTA employs a stack of hybrid Attention-Mixer Transformer Blocks that combine bidirectional self-attention for capturing long-range dependencies with MLP-based layers for efficient feature refinement. Extensive experiments on four diverse benchmarks show that HTA achieves state-of-the-art performance. Notably, on the sparse and challenging GDELT dataset, HTA achieves an absolute MRR improvement of over 18 points, demonstrating its superior ability to model complex and noisy temporal dynamics.

Abstract:
Agentic search systems iteratively interact with retrieval models to answer complex queries. Despite substantial progress, optimizing retrievers for agentic search remains challenging, often requiring heavy co-training or gold-standard annotations that limit real-world applicability. We propose Critic-R, a framework that explicitly closes the feedback loop between the reasoning agent and the retrieval model during both inference and training. Critic-R introduces a critic model that evaluates the agent’s introspective reasoning trace after consuming retrieved evidence to determine whether the retrieved context sufficiently supports the next reasoning step. Critic-R has two complementary mechanisms: Critic-R-Zero, an inference-time query refinement loop that iteratively rewrites queries and retrieval instructions, and Critic-Embed, an optimization approach for retrieval models that leverages successful and failed refinement trajectories as automatic supervision without requiring manual relevance annotation. We evaluate Critic-R on HotpotQA, 2WikiMultihopQA, MuSiQue, and Bamboogle. Results show that Critic-R significantly improves both retrieval quality and downstream answer accuracy.

Abstract:
Large language models (LLMs) learn scientific knowledge from static training corpora, while web search can provide updated information such as paper retractions. This matters for LLM-based literature systems because a model may have seen a paper as valid evidence, while current web results report that it has been retracted. If an LLM does not use the updated retraction status, it may cite a retracted paper as valid evidence. We analyze eleven open-weight models on the retraction-status task and ten on the citation-recommendation task. With web-search results provided, LLMs increase correct status answers for retracted papers from 11.0% to 94.0%. But the same models do not use the retrieved context as reliably when no retraction signal is present or when they recommend citations. On non-retracted papers under web search, models answer “I don’t know” in 52.9% of answers in the retraction-status task. In the citation-recommendation task, even when search results state the retraction and the prompt asks for warnings, models recommend retracted papers without warning in 10.2% of answers. A separate pattern appears for highly cited retractions. Models give fewer correct status answers for high-citation than low-citation retractions (89.3% versus 96.4%). OLMo’s training data suggests one reason. Highly cited retractions appear more often in training data, mostly without any retraction signal, so retrieved retraction information may compete with outdated training knowledge. Overall, access to web search does not guarantee that LLMs will correctly use a paper’s current status when answering questions or producing citations, highlighting the broader challenge of knowledge updating in LLM systems.

Abstract:
Vision-language models such as CLIP are highly useful for diverse tasks due to their shared image-text embedding space. Despite this, the image and text embeddings are often poorly aligned, affecting downstream performance. Recent work has hypothesized that this can be attributed to an information imbalance: images contain more information than their captions describe. In this work, we propose TEVI, a framework that uses captions as a signal for what to retain from image embeddings. Specifically, we use sparse autoencoders to disentangle image embeddings and train a masking module to selectively reconstruct the embedding based on a given caption. In a controlled setup with synthetic captions, we show that TEVI is effective at preserving caption-described attributes while discarding others. We find that this extends to CLIP models trained on natural images, where TEVI learns to mask meaningfully and allows retrieval based on conditioning. Finally, we use TEVI to achieve improved retrieval performance across coarse-grained and fine-grained benchmarks. Code available at https://github.com/neuroexplicit-saar/TEVI.

Abstract:
Existing approaches to post-train models for long-context tasks face complementary limitations: (i) supervised fine-tuning (SFT) provides stable supervision but suffers from exposure bias; (ii) reinforcement learning methods such as Group Relative Policy Optimization (GRPO) train on model-generated trajectories but struggle with long-horizon credit assignment and sparse rewards; and (iii) on-policy distillation (OPD) provides dense token-level guidance but does not directly optimize task rewards. We study these complementary strategies for long-context alignment and derive a recipe that combines GRPO with OPD-style teacher guidance: the student learns from its own rollouts using outcome-level rewards, while a stronger teacher provides dense token-level regularization in place of the standard reference policy. This is especially useful when process-level supervision is difficult to obtain. To support this study, we introduce LongBlocks, a synthetic multilingual dataset spanning multi-hop reasoning, contextual grounding, and long-form generation. Through controlled ablations, we isolate the roles of cold-start initialization, teacher anchoring, and data mixing, showing that our recipe yields a more stable and effective path to long-context reasoning than GRPO or OPD while preserving short-context capabilities.

Abstract:
LLM-based fact-checking verifies complex claims through decomposition, retrieval, and reasoning aggregation. Existing methods expand verification inputs to cover more factual conditions, but added fragments can be redundant or misleading, compete with decisive evidence, and bias predictions toward surface relevance. Debiasing methods mitigate this bias by estimating shortcut effects from claim-side cues or biased signals in constructed inputs. However, because they calibrate verifier predictions after input construction, they are not designed to repair missing or noisy verification states. Thus, complex fact-checking should balance factual coverage and noise control during verification-state construction. We propose ERGA, an Explorer–Refiner Game with Backdoor-Adjusted Verdict Aggregation. ERGA formulates verification as editable state construction, initializing the state from decomposed verification units, retrieved evidence, and local analyses. It then runs an adversarial Explorer–Refiner game over this state: the Explorer strengthens under-covered factual signals, while the Refiner merges redundancy and separates weak or misleading fragments. Through this game interaction, ERGA obtains a refined state with a better coverage–noise balance and a removed fragment record. For verdict aggregation, ERGA uses the removed-fragment record to condition verifier predictions and aggregates them in a backdoor-adjusted form to reduce residual shortcut effects. Experiments on HoVer and FEVEROUS-S show that ERGA outperforms strong baselines.

Abstract:
Text-to-video (T2V) models now produce striking visuals, yet they routinely violate everyday physics; objects float, tools are misused, and causal sequences break down. Existing benchmarks mostly probe isolated physical laws and rely on vision-language models to score videos directly, which entangles perception and reasoning in one judgment and correlates poorly with humans. We introduce PhysVidBench, a benchmark of 383 base prompts, expanded to 766 prompts with enriched variants and 4,123 manually reviewed QA items, together with a language-grounded evaluation framework that takes a different route: instead of asking a VLM “Is this video physically correct?”, we caption the video, then ask a language model to answer prompt-derived yes/no physics questions using only the captions. This split between seeing and reasoning makes each score traceable to the captions that justify it and aligns more closely with human judgment than direct VLM scoring (Pearson r=0.45–0.69, compared with 0.21–0.47 for the strongest direct VLM evaluator). To test the framework, we carefully curate a set of human-validated prompts spanning seven physical dimensions, with a focus on tool use and affordances – areas largely absent from prior physics-focused benchmarks. Across 12 open and proprietary T2V systems, the best model reaches only 36.2% average accuracy, and no model consistently handles everyday physical reasoning. The same pipeline can also guide iterative error-guided prompt refinement, improving CogVideoX-2B from 21.6 to 32.7 and CogVideoX-5B from 17.8 to 29.7 without retraining.

Abstract:
We present MULTITEMPBENCH, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MULTITEMPBENCH contains 15,000 examples built by translating 750 curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages.

Abstract:
Backdoor evaluations ask whether hidden triggers elicit attacker-chosen responses. For generative LLMs, this behavioral test leaves unresolved which internal features contribute to triggered generation and whether those identified from source attacks remain actionable on held-out attacks. We study this question using pretrained sparse autoencoders (SAEs) over residual stream states. We contrast triggered and matched clean source traces, retain coordinates with recurring activation increases at early answer positions, and freeze each bank before evaluation. For each evaluation, the tested attack family and its adapted checkpoint are excluded from bank construction. Attack Success Rate (ASR) is the behavioral readout. Across Gemma-3 1B/4B/12B LoRA models with Gemma Scope 2 SAEs, ablating the selected coordinates reduces complete-target generation from 1478/1500 to 1/1500. Repeating the protocol with separately constructed banks for checkpoints trained with full-parameter fine-tuning reduces complete-target generation from 1499/1500 to 0/1500. Matched random controls leave ASR near baseline. Average clean-task changes are small, although instruction following declines more on larger models. Whether the selected coordinates encode the trigger condition itself remains unresolved. The evidence supports these coordinates as transferable SAE-based intervention handles associated with triggered generation in the evaluated Gemma-3 setting.

Abstract:
Transformer-based models have achieved remarkable performance. However, their structural characteristics—multiple layers and attention heads—introduce efficiency challenges in inference and deployment. To address these challenges, various pruning methods have recently been proposed. Notably, gradient-based methods using Head Importance Scores (HIS) have gained traction for interpretability, efficiency, and ability to identify redundant heads. However, HIS alone has limitations as it captures only the gradient-driven contribution, overlooking the diversity of attention patterns. To overcome these limitations, we introduce a novel pruning criterion, HIES (Head Importance-Entropy Score), which integrates head importance scores with attention entropy, providing complementary evidence on per-head contribution. Empirically, HIES-based pruning yields a 1.13% improvement in model quality and 3.52% improvement in stability over the HIS-only method on average, substantially reducing the accuracy and stability cost of aggressive compression. Code will be released upon publication.

Abstract:
Discrete diffusion language models generate text by iteratively committing masked positions, making decoding a trajectory-level process rather than a sequence of independent token decisions. Existing analyses often focus on surface-level knobs such as temperature, step budget, block size, or scheduler choice, but these knobs do not explain why the same change can induce different behaviors across models. We analyze discrete diffusion decoding through high-influence commitment events and define an observed trajectory influence measure based on downstream entropy reduction over unresolved positions. Across Dream-v0-Instruct-7B and LLaDA-8B-Instruct on reasoning, code, and question-answering tasks, we find that influence is sparse, early relative to each model’s commit frontier, and stable under tested seed and step-budget variations. We further show that criticality emerges from the interaction between commit timing and token role, explaining why decoding knobs reshape trajectories in model-dependent ways. Finally, lightweight margin-based interventions validate the analysis by correcting diagnosed failure modes without additional forward passes.

Abstract:
Vision-language models (VLMs) often answer new questions about recurring visual content, where reusing the key-value (KV) cache can avoid re-encoding expensive visual prefixes. Exact-prefix reuse, however, fails when the same visual content appears under a changed prefix. Selective recomputation can recover quality under a small visual-token budget, but only when the right stale tokens are refreshed. Raw-attention selection can waste budget on high-attention tokens with small value-norm proxy scores and on query-irrelevant images. To address these failure modes, we propose CONDUIT, a training-free refresh policy that unifies single- and multi-image reuse as residual-stream restoration. Building on norm-weighted attention, CONDUIT ranks cached visual tokens using cached-key query attention and an accessible pre-output cached-value-norm proxy, then applies empirical image-level relevance amplification before one global selection. With one image, the coefficient is one and the rule reduces to intra-image token selection. The method preserves model architecture and weights, adding only a single query-conditioned scoring pass at inference. At a 10% refresh budget, CONDUIT achieves 97.0%–99.5% of the corresponding full-prefill five-dataset average across three VLM backbones and leads budgeted methods on average; on the MMLongBench-Doc latency subset, it uses 13.5% of full-prefill FLOPs and achieves a 2.99× time-to-first-token speedup.

Abstract:
Large Language Models (LLMs) excel when a solution can be planned from the prompt, but many sequential decision-making tasks require actions to depend on observations acquired during interaction. We study two central challenges in this regime: learning reusable interaction policies for history-dependent search, and acting under partial observability when task-relevant state is revealed only through interaction. We instantiate these challenges with two diagnostic benchmarks: Calc-Bench for reusable, history-dependent tool interaction, and partially observed sorting for action selection under hidden state. We introduce the Expanded Action space (ExpA), which decouples language reasoning from environment interaction by adding routing and environment-specific actions beyond vocabulary tokens. ExpA is trained with EARL, an RL algorithm using counterfactual policy optimization to explore rarely invoked but useful actions. EARL improves Countdown by 26.3% with 3B models and achieves perfect Sort-4 accuracy while discovering an efficient sorting strategy, providing a proof-of-concept for learned non-language interaction in controlled sequential decision-making diagnostics.

Abstract:
Zero-shot detection methods for AI-generated text typically aggregate token-level statistics across entire sequences, overlooking the temporal dynamics inherent to autoregressive generation. We analyze over 120k text samples and reveal Late-Stage Volatility Decay: AI-generated text exhibits rapidly stabilizing log probability fluctuations as generation progresses, while human writing maintains higher variability throughout. This divergence peaks in the second half of sequences, where AI-generated text shows 24–32% lower volatility. Based on this finding, we propose two simple features: Derivative Dispersion and Local Volatility, which computed exclusively from late-stage statistics. Without perturbation sampling or additional model access, our method achieves state-of-the-art performance on EvoBench and MAGE benchmarks and demonstrates strong complementarity with existing global methods.

Abstract:
Data augmentation is widely adopted in molecular graph learning due to its practical benefits in enhancing model performance. However, conventional augmentation like edge deletion often destroys critical chemical semantics, as molecular properties are highly sensitive to structural changes. To address this, we introduce CMAS, a novel ̲C ollaborative ̲M ulti- ̲A gent ̲S ystem for enhancing data augmentation in molecular graph classification. Specifically, for each input molecule, we first use a BRICS-based decomposition strategy with prompt-guided LLM generation to produce structurally diverse candidates while preserving the core scaffold. To ensure that the generated candidates preserve the physicochemical properties of the original molecule, we retrieve domain-specific chemical knowledge from a knowledge graph and prompt an LLM to assess property consistency, effectively filtering out semantically divergent samples. Finally, we perform rule-based sanitization with iterative feedback to ensure chemical validity and preserve distributional diversity, resulting in a high-quality augmented molecular set. Extensive experiments on MoleculeNet benchmarks validate the superiority of CMAS against several state-of-the-art baselines. Code is available at https://github.com/Catfromuibe/CMAS.

Abstract:
In this paper, we study a fundamental design problem in pretraining Large Language Models (LLMs) for low-resource language regimes, focusing on cross-family language pairs in which English serves as the high-resource language. Existing works adopt multi-epoch, multi-lingual, and multi-stage training to utilize the limited target-language corpus efficiently, but no prior scaling law can compare recipes spanning these approaches under the same compute budget C and target-language corpus size D T , leaving the optimal training setup unclear. To address this gap, we propose the M 3 Scaling Law, a unified predictive model parameterized by the model scale, the number of target-corpus epochs k , the average target-language ratio r , and the final-stage target-language ratio r f , which places monolingual single-stage, multi-lingual single-stage, and multi-lingual multi-stage recipes on a single target-language validation-loss surface. Across three language pairs (English paired with Japanese, Indonesian, and Swahili), it extrapolates to unseen hyperparameter regions more accurately than existing scaling laws. Using M 3 as a surrogate objective, we derive two practical guidelines for low-resource LLM pretraining: (i) as D T decreases, the optimal recipe shifts directly from monolingual single-stage to multi-lingual two-stage training at a compute-budget-dependent threshold, with multi-lingual single-stage never optimal in our experimental grid; and (ii) the optimal number of epochs collapses onto a single curve in the scarcity variable D T /D\(C) , where D\(C) ∝ C 𝛼/(𝛼+𝛽) is the monolingual compute-optimal corpus size.

Abstract:
Recent advances in Medical Large Language Models have shown strong performance in medical question answering and clinical knowledge understanding, yet real-world clinical practice requires long-horizon, dynamic treatment decision-making rather than static prediction. Existing medical agent environments are mostly turn-based, assuming patient states change after actions while overlooking natural disease progression and delayed treatment effects. We propose MedACE, a time-aware asynchronous clinical world model, where patient evolution is formulated as an elapsed-time-conditioned process driven by historical observations, treatment actions, clinical context, and disease-level knowledge. Built from timestamped MIMIC-IV trajectories, our JEPA-style simulator, MedWorld-JEPA, learns future patient state representations in latent space rather than directly generating noisy clinical observations. Based on this simulator, we construct an asynchronous treatment environment with global time advancement, closed-loop patient simulation, multi-step physician-agent interaction, and outcome-grounded rewards. Experiments on Hospital and ICU scenarios demonstrate competitive physiological forecasting, strong clinical consistency, and the importance of jointly modeling time, disease progression, and treatment effects. Agent evaluations further show that current LLM-based physicians struggle with long-horizon treatment decisions, highlighting our environment as a challenging foundation for future medical treatment agents.

Abstract:
Adaptive multimodal reasoning has emerged as a promising frontier in Vision-Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual reasoning and text reasoning to enhance both effectiveness and efficiency. However, existing evaluations rely on static difficulty labels and simple metrics, overlooking the model-dependent nature of task difficulties: a task that is challenging for one model may be trivial for another. Consequently, they obscure the distinction between reasoning mode selection and overall task accuracy while neglecting fine-grained reasoning process analyses. In this paper, we propose AdaptMMBench, a comprehensive benchmark for adaptive multimodal reasoning across five domains: natural scene, OCR, GUI, knowledge, and math, encompassing both direct perception and complex reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the quality of reasoning mode selection, isolating this meta-cognition ability by dynamically identifying task difficulties based on models’ capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional reasoning process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation shows that strong reasoning mode selection is helpful, but final adaptive gains and efficiency also depend on overall model capability. Process metrics further diagnose bottlenecks in reasoning coherence, tool use, and cost for improving adaptive performance and efficiency.

Abstract:
The rapid evolution of large language models necessitates robust machine-generated text detection. Existing paradigms typically follow two isolated tracks. Training-free methods rely on global statistical scalars such as perplexity, while training-based methods utilize semantic hidden states. Both approaches exhibit fundamental vulnerabilities in adversarial scenarios. Global scalars act as lossy compressions that obscure local probabilistic burstiness in interleaved texts, whereas pure semantic models overfit to specific fingerprints and remain susceptible to spoofing. To expose these flaws, we introduce MOSAIC, a comprehensive adversarial benchmark comprising 16000 samples across a full-granularity attack spectrum. To address these challenges, we propose NeuroStat, an end-to-end framework bridging the statistical and semantic gap. NeuroStat captures uncompressed token-level probabilistic logits alongside deep semantic hidden states from a single causal language model backbone. We fuse these heterogeneous signals through Macro-State Residual Modulation, which adaptively calibrates local convolutional features using global uncertainty indicators. Orthogonal and contrastive losses further ensure the learning of complementary representations. Extensive experiments demonstrate that NeuroStat maintains exceptional robustness on MOSAIC compared to the severe degradation of state-of-the-art methods, establishing a new standard for adversarial text detection. Code and the MOSAIC benchmark are available at https://github.com/TencentBAC/NeuroStat.

Abstract:
Comparative analyses of NLP research, such as identifying who is shaping a subfield, how a topic is shifting geographically, where method- ological gaps lie between research communi- ties, require institutional and topical metadata at a density and granularity the ACL Anthol- ogy does not itself provide. Large-scale re- sources offer partial support: OpenAlex pro- vides high-quality institutional metadata where it can match ACL papers (32% of our gold set), but the DOI dependency limits its cov- erage on older Anthology publications; and the S2ORC bulk release contains 52,013 ACL Anthology fulltexts but populates structured author–affiliation annotations on a small fraction of them in its current state. We present ANTHEM (ANTHology with Enriched Meta- data), a complementary resource that narrows this gap for NLP publications. ANTHEM links 117,386 ACL Anthology papers to Research Or- ganization Registry (ROR)-normalized author affiliations and Computer Science Ontology (CSO) topic labels via a multi-parser, LLM- based extraction pipeline that retains evidence spans for every decision, achieving F1hier of 0.760 on a 781-paper decade-stratified gold set, where parent–child ROR identifiers are treated as equivalent (e.g., a department and its parent university). On top of the corpus we release a benchmark of 99 structured comparison ques- tions across five categories, evaluated under a programmatic gold-set construction in which ANTHEM’s own metadata defines the evalu- ation ground truth. All data, code, and infras- tructure are released under CC-BY-SA-NC.

Abstract:
ArXiv recently prohibited the upload of unpublished review papers to its servers in the Computer Science domain, citing a high prevalence of LLM-generated content in these categories. However, this decision was not accompanied by quantitative evidence. In this work, we investigate this claim by measuring the proportion of LLM-generated content in review vs. non-review research papers in recent years. Using two high-quality detection methods, we find a substantial increase in LLM-generated content across both review and non-review papers, with a higher prevalence in review papers. However, when considering the number of LLM-generated papers published in each category, the estimates of non-review LLM-generated papers are almost six times higher. Furthermore, we find that this policy will affect papers in certain domains far more than others, with the CS subdiscipline Computers Society potentially facing cuts of 50%. Our analysis provides an evidence-based framework for evaluating such policy decisions.

Abstract:
Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in Persian, a low-resource language, across multiple evaluation strategies and prompt formulations. We find that LLM–human agreement varies substantially across dimensions: alignment is stronger for structured TTCT-derived properties such as Originality, Fluency, and Elaboration, but considerably weaker for more subjective dimensions, particularly Emotion and Attractiveness. Judgments are also sensitive to prompt formulation, while few-shot prompting, ensembling, and multi-agent debate provide no consistent improvement. Motivated by this dimension-dependent behavior, we investigate whether structured creativity dimensions can instead be approximated using simple, interpretable proxy metrics. We introduce CLIN, which evaluates three TTCT-derived dimensions separately using topic-aware novelty for Originality, contextual lexical clustering for Fluency, and lexical diversity for Elaboration. These proxies achieve human alignment comparable to or better than the strongest zero-shot LLM judge in our setting while requiring substantially lower evaluation cost.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved reasoning in formal domains such as mathematics and code, but extending these gains beyond STEM remains challenging. Extending RLVR beyond STEM is fundamentally constrained by the lack of high-quality verifiable training data. In this work, we introduce SUPERNOVA, a framework for curating RLVR data from natural instruction datasets, which are a rich source of expert-annotated data but are underexplored for RLVR training. Through 100+ controlled RL experiments, we systematically study how to utilize these dataset for RLVR and how data curation decisions affect downstream reasoning performance . In particular, we investigate three data designs: (a) source task selection, (b) task mixing, and (c) synthetic interventions. Our analysis reveals that source task selection has a significant impact on downstream reasoning performance. Moreover, selecting tasks based on their performance for individual target tasks outperforms strategies based on overall average performance and synthetic interventions do not improve reasoning. Guided by these insights, we construct SUPERNOVA, a high-quality RLVR dataset of 25K instances curated from natural instruction datasets. We show that training Qwen3-0.6B on SUPERNOVA outperforms the base Qwen3-0.6B, yielding a relative gain of 64.4% on BigBench Extra Hard (BBEH), a challenging benchmark comprising 23 complex reasoning tasks. Importantly, we find that gains from SUPERNOVA generalize to unseen benchmarks, larger model scales, and newer model families. Overall, our findings provide practical insights for curating human-annotated resources to extend RLVR to general reasoning.

Abstract:
LLM agents increasingly operate in long-horizon tasks and extended dialogues, where they must retrieve useful past experiences beyond limited context windows. A central challenge is to balance efficient recall with faithful evidence recovery. Existing memory methods often rely on static access strategies: compact memories reduce context cost but may omit fine-grained evidence, whereas full histories preserve details but introduce redundant and noisy context. We propose GraP-Mem, a multi-granularity memory planning framework that adaptively schedules memory access for each question. GraP-Mem represents each historical session with two complementary views: compact semantic memories for efficient retrieval and source contexts for detailed evidence recovery. During inference, a Plan Agent generates retrieval-oriented information needs and queries, while an Integration Agent maintains a question-specific evidence state that consolidates useful evidence and identifies remaining gaps. When the evidence is incomplete, GraP-Mem selectively expands access to source contexts or performs additional retrieval. The final Answer Model is conditioned only on the integrated evidence state. Experiments on LoCoMo and NarrativeQA show that GraP-Mem consistently improves F1 and BLEU across backbone models, demonstrating the effectiveness of plan-guided memory access across multiple granularities.

Abstract:
Large language models (LLMs) in medicine are mainly evaluated using multiple-choice question answering (MCQA), which can overestimate real clinical ability due to guessing strategies and answer biases. To address these limitations, we introduce an expanded and more challenging benchmark based on Polish medical exams, adding over 15,000 questions, two new domains, and four structural modifications that reduce MCQA-specific artifacts and better test reasoning. We evaluate 21 LLMs and show that evaluation design strongly affects results. Under our harder setup, the best model (Qwen3.5-122B) drops by 28.4 and 31 pp on English and Polish exams, respectively. Despite low evidence of data contamination, standard MCQA scores do not reliably reflect true medical competence. To facilitate further research, we make our benchmark publicly available: https://github.com/NASK-NLP/Polish-Medical-Exams

Abstract:
Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet recent work shows that their failures often reflect an interaction between visual grounding and downstream reasoning. What remains less clear is how the visual presentation of a task shapes model performance and failure modes when the underlying reasoning problem is unchanged. We study this question in SPaRC, a benchmark for grid-based visual spatial planning, by introducing lightweight input-side scaffolds that preserve the visual modality while making spatial structure more accessible. The scaffolds are task-specific diagnostic interventions built on SPaRC’s known board structure, not a general-purpose prompting strategy. Across multiple VLMs, these scaffolds improve task accuracy over the original visual setting by up to 34.0 percentage points and further complement GRPO-based training, yielding up to 4.6 additional accuracy points compared with near-zero gains on the original visual input. Analyses on both end-to-end task solving and object detection show that these gains are closely tied to reductions in grounding-related errors, while rule reasoning remains comparatively challenging. With the planning problem held fixed, visual presentation therefore changes both accuracy and the distribution of failure modes, which in turn determines whether a VLM benchmark measures grounded perception, downstream reasoning, or a mixture of both.

Abstract:
Large language models are increasingly used to simulate social and political audiences, but it remains unclear whether persona-conditioned generations provide coherent reception patterns or align with real public opinion. We study audience reactions to morally salient political messages, using Trump’s posts as a relevant case in point, through two theory-grounded human value frameworks, Inglehart-Welzel cultural map and Moral Foundations Theory. Across issue-specific message bundles, we analyze stance together with agreement, legitimacy, offensiveness, and endorsement, asking whether synthetic personas produce systematic reception differences, which persona dimensions explain those differences, and whether aggregate synthetic distributions align with matched Pew public-opinion patterns. The paper examines three related dimensions of synthetic-audience evaluation: internal differentiation, explanatory structure, and external plausibility. It contributes a reproducible study design for political audience simulation and an empirical account of where synthetic moral audiences are informative, and where they should not be treated as calibrated population proxies.

Abstract:
Existing machine translation (MT) metrics and discourse-focused evaluations primarily assess translation quality intrinsically, without measuring the downstream consequences of translation errors. In this work, we focus on extrinsic discourse evaluation of machine translation under two distinct regimes: static and interactive. Under the static regime, we propose an entity counting task as an illustrative probe of referential consistency in discourse. We show that high intrinsic MT quality does not reliably predict downstream discourse success and strong MT systems still produce referential inconsistencies. For the interactive regime, we study the goal-oriented multi-agent Welfare Diplomacy game as a probe of communication and coordination success over long horizons. We find that interaction-specific translation failures impact downstream coordination. Our results highlight goal-oriented translation-mediated environments as a viable framework for discourse-sensitive extrinsic MT evaluation.

Abstract:
Fusing charted times series with clinical notes has been found useful for predictive analytics. However, the incompleteness of clinical notes can undermine prediction quality. We thus propose a novel conditional diffusion-based model called CODE to recover the missing semantics of clinical notes, which integrates EHR time series into diffusion. CODE learns to align a diffusion process with unknown levels of note incompleteness via contrastive learning, by which missing semantics is recovered from the denoising process. We evaluate the effectiveness of CODE by conducting experiments on real-world EHR data for three prediction tasks. Empirical results show that CODE with the capability to recover the missing semantics can outperform state-of-the-art baselines.

Abstract:
Even though backdoors in LLMs have been a growing concern, their inner workings are still under heavy scrutiny. Trigger-based backdoors are easy to define behaviorally, a rare input that makes the model switch to a chosen response pattern, but the mechanism between triggers and their responses is less clear. We study this mechanism in a controlled, harmless language-switching setting, where fixed trigger sequences make 1B and 8B language models continue English prompts in French or German. For this, we train sparse autoencoders (SAEs) across layers and transformer components, then compare triggered prompts with translation and pretraining controls to identify trigger-relevant feature directions. We show how SAE features separate triggered prompts from controls with near-perfect F1, but features that detect the trigger do not necessarily control the behavior. In intervention tests, attention and MLP features often fire reliably on triggered prompts, making them good detectors, but ablating them rarely suppresses the language switch and activating them rarely induces it. In contrast, residual-stream features can suppress triggered generation when ablated, and some selected features can induce target-language continuations without the trigger. In short, these token-trigger mechanisms decompose into distinct SAE feature directions, with separate features for trigger detection, residual-stream propagation, and later language tracking. This role-level decomposition is the part most likely to transfer to other trigger-based backdoors, even when the payload, layers, or circuit locations differ.

Abstract:
Large language model (LLM) agents rely on long-term memory to support extended interactions and personalized assistance beyond finite context windows. Existing memory agents actively update external memory through generated write, revise, and delete operations, but these updates may omit important information, corrupt existing memory, or introduce unsupported hallucinated content. Once stored, such errors become persistent system-state failures that can affect future reasoning and generation. In this paper, we propose TrustMem, a framework designed to improve the trustworthiness of memory consolidation. TrustMem relies on a Memory Transition Verifier to evaluate the transition process of memory updates in terms of coverage, preservation, and faithfulness. It further constructs preference pairs among candidate updates under the same memory state, enabling preference-guided reinforcement learning to directly optimize memory updating behaviors. Extensive experiments demonstrate that TrustMem improves both memory utility and reliability: it achieves state-of-the-art results across MemoryAgentBench, HaluMem, and the Mem-alpha validation set, improves HaluMem memory extraction by 12.14 F1 points, and reduces transition-level omission, corruption, and hallucination by 40.1%, 79.1%, and 50.0%, respectively, compared with the strongest baseline for each error type.

Abstract:
Multi-agent systems powered by large language models (LLMs) increasingly decompose tasks by passing natural-language relays among specialized agents. This creates a boundary-level privacy failure: a non-entitled internal agent can learn private context before any final-output guard or tool-action checker observes a violation. We introduce contextual-integrity mediation for inter-agent communication, a formulation in which each candidate relay is treated as a commit that must satisfy recipient entitlement, attribute minimization, and transmission conditions before delivery. We present CI-Relay, a runtime mediation layer that induces CI scopes from runtime-visible norms and relay metadata, packages candidate messages as CI-labeled envelopes, projects recipient-scoped views, propagates transmission constraints, and records auditable mediation traces. On relay traces converted from three CI benchmarks, unmediated relays leak internally in 52.2% of cases. Relative to a metadata-matched LLM Guard Relay that receives the same induced scope, CI-Relay reduces internal leakage from 12.4% to 1.8% and downstream leakage from 10.9% to 5.6%, while improving utility and reducing over-redaction. Gains hold across three live LLM backends; multi-hop, prompt-robustness, and ablation studies show that the gains arise from structured CI mediation rather than generic privacy prompting, guard rewriting, or final-output checking.

Abstract:
Reinforcement Learning with Verifiable Rewards (RLVR) has significantly advanced the reasoning capabilities of Large Language Models (LLMs), while being prone to entropy collapse, a rapid degradation of policy entropy that forces the model into premature convergence and limits its performance ceiling. However, existing solutions for mitigating entropy collapse often struggle to reconcile the trade-off between exploration capability and convergence efficiency. To address this, we propose Variance-aware Negative Feedback Policy Optimization (VFPO), a framework that dynamically regulates the policy entropy through the lens of group statistics. We first mathematically prove that policy entropy is intrinsically coupled with the variance and mean of group probabilities for sampled trajectories. Leveraging this insight, VFPO operates via a dual-phase strategy: an initial exploration phase employs a closed-loop negative feedback mechanism via both sample-level and token-level reward modulation to maintain group variance and prevent the mean trajectory probability from saturating; then followed by an annealing phase that relaxes these constraints to facilitate deterministic convergence. Extensive experiments on models of distinct scales (1.5B, 3B, 7B) demonstrate that VFPO yields average improvements of 3.34%, 6.52%, and 6.03%, respectively, over the DAPO baseline, and achieves state-of-the-art performance across diverse mathematical benchmarks, effectively expanding the exploration boundary while achieving superior reasoning accuracy. To facilitate reproducibility, we release the code, data and trained models at the link https://github.com/carpe1/VFPO.

Abstract:
Reinforcement learning (RL) is a natural fit for agentic knowledge base question answering (KBQA), where a model must issue executable actions, observe knowledge-base feedback, and eventually return an answer. However, current RL-based KBQA systems mainly optimize sparse rewards from the final answer, leaving intermediate action errors weakly supervised. This is especially limiting for logical-form annotated KBQA benchmarks: gold logical forms can be converted into executable action sequences, but existing pipelines use them mainly for warm-start data construction rather than for on-policy RL updates. We propose GAPD, a training-time Gold-Action Policy Distillation framework that adds dense token-level guidance to outcome-based RL. To align gold actions with on-policy student rollouts, GAPD uses MID-ANCHOR MATCHING: it treats the intermediate entities reached during student exploration and gold execution as state anchors, and matches student states to gold states through these explored entity sets. The current policy conditioned on this aligned gold action serves as a stop-gradient teacher, whose token distribution is distilled back to the ordinary student policy over generated action-token spans. GAPD consistently surpasses the current state of the art on WebQSP, GrailQA, and GraphQ.

Abstract:
Conversational AI is moving beyond isolated text prompts toward sustained, multimodal interaction. In real conversations, users clarify goals, revise requests, interrupt responses, switch topics, and introduce new evidence while expecting systems to preserve context across turns. This makes multi-turn dialogue a distinct challenge requiring systems to maintain and update memory, ground responses across modalities, tools, and external knowledge, and adapt across languages and cultures. This study reviews multi-turn conversational AI across text-only dialogue, AudioLLMs and speech-native systems, multimodal and omni-modal systems, and tool-augmented agents. We organize the literature around datasets and benchmarks, modeling paradigms, training strategies, evaluation setups, and cross-cutting challenges. Our analysis shows that support for multiple modalities has advanced faster than the ability to sustain coherent interaction across a session. Despite stronger capabilities to perceive, speak, and act across modalities, current systems still struggle with persistent memory, cross-turn grounding, full-duplex interaction, robust evaluation, and cultural alignment. We conclude with a research agenda for systems that can remember, revise, ground, speak, listen, act, and adapt across turns, modalities, and cultures. ([https://github.com/faiza-sfa/multiturn-conversational-ai-survey](https://github.com/faiza-sfa/multiturn-conversational-ai-survey))

Abstract:
Low-rank adaptation (LoRA) has become the standard for parameter-efficient fine-tuning of large language models. Most LoRA variants follow a uniform-LR convention, applying a single global learning rate across every rank-one component of every adapter. We show that this convention overlooks substantial within-module heterogeneity, where the rank-one components of a LoRA adapter update at highly uneven rates and low-velocity modules converge to concentrated singular spectra that underutilize the nominal rank budget. To address this, we propose an adaptive anisotropic learning-rate model that assigns each rank-one component its own effective learning rate, computed online from training-time signals and mean-normalized per module to preserve the global LR budget. AnLR-LoRA instantiates this model with two signals available during AdamW optimization, namely function-space velocity and Adam SNR, as a lightweight scheme with no extra trainable parameters. Across commonsense reasoning, natural language generation and visual instruction-tuning benchmarks, AnLR-LoRA consistently improves over LoRA while encouraging broader use of rank capacity, with gains that remain robust across a wide range of global learning rates and transfer cleanly to other LoRA variants.

Abstract:
With the recent commercial availability of Large Language Models (LLMs), end users increasingly turn to them for encyclopedic knowledge. Doing so, users may inadvertently inject incorrect assumptions (as presuppositions) into LLM prompts, which are then affirmed by LLMs. In this work, we investigate the effect that existential presuppositions have on LLM responses. We introduce the Dataset for Evaluating and Explaining the Effect of Existential Presuppositions (D4EP) and use it to quantify their effect. We further develop a probing-based explanation that analyzes the LLM as a classifier and finds that the effect of presuppositions corresponds to a change of the classifier’s operating point.

Abstract:
When large language models process structured temporal data, serialization into token sequences collapses the two-dimensional organization ( Features Time ) that encodes meaningful relationships between co-occurring features. We investigate whether visual encoding can recover the structural information that linearization loses. We introduce LE-Viz, a framework that transforms longitudinal experiential data into complementary visual and textual representations for a vision-language model that generates prospective narratives, framing prediction as language generation. Evaluated on cross-distribution tasks across three datasets spanning mental health and educational forecasting, LE-Viz sets a new benchmark on a widely-adopted cross-distribution generalization task, improving over both the traditional state-of-the-art and the best text-only LLM baseline, and consistently outperforms all baselines. Analyses show that spatial organization of visual inputs, not information quantity, drives the generalization gain, and that the multimodal combination exceeds both text-only and vision-only conditions.

Abstract:
Text-attributed graphs (TAGs) are widely used in many real-world domains, and learning on TAGs requires jointly modeling text semantics and graph structure. A standard approach for modeling TAGs is to combine a language model (LM) and a graph neural network (GNN), but joint training is computationally expensive and difficult to scale. Dataset distillation is a promising way to reduce training costs, but existing methods are not well suited to TAGs because they are typically designed for a single modality or still require repeatedly training expensive LM-GNN models on the full dataset during distillation. To address this, we propose TaLK, an effective dataset distillation method for TAGs that couples an LM with a graph-aware neural tangent kernel. This design enables efficient dataset distillation, avoiding repeated joint training on the full dataset while reflecting both textual and structural information for effective TAG learning. Experiments on multiple TAG benchmarks show that TaLK consistently outperforms existing baselines and achieves up to 97% of full-dataset performance with only 1% synthetic data.

Abstract:
Affective cues across modalities may be incongruous (e.g., sarcasm or mocking praise), potentially leading to misinterpretation when relying on a single modality. Large Audio–Language Models (LALMs) have recently gained popularity and been applied to multimodal emotion recognition, but their ability to disentangle acoustic and semantic cues, especially in incongruent cases, remains underexplored. To address this gap, we introduce CREMA-ASIS, a dataset specifically created to investigate incongruence between acoustic emotion and semantic sentiment cues. It pairs acoustic emotion labels with semantic sentiment polarities. Using this dataset, we evaluate LALM biases within a multitask framework and conduct a layer-wise analysis to identify modality dominance across layers. Our findings reveal that LALMs struggle with semantic-acoustic incongruent cases, rarely predicting incongruity, and that LALMs are predominantly influenced by semantic information. However, supervised fine-tuning significantly improves LALM performance on our CREMA-ASIS test set while preserving transcription accuracy and joint emotion recognition. Results demonstrate potential for enhancing both acoustic and semantic understanding on out-of-domain data.

Abstract:
Speculative decoding accelerates large language model (LLM) inference by allowing a lightweight draft model to propose candidate tokens for verification by a target model. In real-world deployments, however, shifts in input distributions and updates to the target model can weaken draft–target alignment, reducing token acceptance and acceleration. Repeatedly retraining the draft model is costly, delays deployment, and may impair performance on previously encountered distributions. Achieving online alignment therefore requires rapid and stable adaptation while incorporating real-time feedback within a limited compute budget. To this end, we propose Dynamic Speculative Decoding (DySD) , a framework that aligns the draft model with the target model during inference without offline retraining. DySD combines hierarchical feature alignment with distribution-level matching and uses divergence-triggered dynamic distillation to selectively update the draft model. Experiments across multiple datasets show that DySD achieves 1.6×–2.0× speedups and boosts token acceptance rates by 10%–30% under distribution shifts, enabling speculative decoding to remain fast, stable, and adaptive in ever-changing environments.

Abstract:
Large language models (LLMs) encode vast amounts of knowledge in their parameters, but these facts often require precise post-training rectification as they become outdated or incorrect over time. Traditional batch editing methods typically apply a uniform update strength across all instances. However, this uniform approach ignores the inherent variation in editing difficulty, leading to a fundamental trade-off between correction success and model stability. To overcome this limitation, we introduce AdaPE (Adaptive Pressure Editing), a parameter-free adaptive editing framework that dynamically adjusts the update behavior for each specific instance. Specifically, we derive a closed-form adaptive update rate based on the ratio of residual energy to spatial sensitivity, which strictly quantifies the intrinsic injection difficulty. This formulation adaptively limits the update strength for difficult facts to preserve model locality, while ensuring sufficient knowledge injection for easier ones. Extensive experiments on three benchmarks and two prominent LLMs demonstrate the superiority of our approach in consistently circumventing the fixed-strength bottleneck, achieving a new state-of-the-art balance between editing efficacy and model integrity. Our code is available at https://github.com/skyandkibo/adape.

Abstract:
Currently, a prevalent approach for enhancing Vision-Language Models (VLMs) performance is to encode both the high-resolution version and the thumbnail of an image simultaneously. While effective, this method generates a large number of image tokens. When combined with the widely used Rotary Position Embedding (RoPE), its long-term decay property hinders the interaction between high-resolution tokens and thumbnail tokens, as well as between text and image. To address these issues, we propose ID-Align , which alleviates these problems by reordering position IDs. In this method, high-resolution tokens inherit IDs from their corresponding thumbnail token while constraining the overexpansion of positional indices. Our extensive experiments conducted within the LLaVA-Next framework demonstrate that ID-Align delivers comprehensive improvements: it not only achieves notable quantitative gains across multiple benchmarks, highlighted by a significant enhancement on MMBench’s relation reasoning tasks, but also qualitatively improves the model’s attention distribution, making it more interpretable.

Abstract:
Video Large Multimodal Models (VLMMs) have achieved impressive progress in video understanding, yet they remain limited in adaptive reasoning—the ability to revise conclusions when new evidence emerges. In reality, conclusions are rarely set in stone; additional context can strengthen or weaken an initial inference. To capture this capability, we introduce Defeasible Video Entailment (DVidE), a new task that challenges models to think like doubters, requiring them to dynamically update entailment judgments as evidence evolves. In DVidE, given a video premise and a textual hypothesis, the models must determine whether a new update strengthens or weakens the hypothesis (classification task) or generates a coherent update that modifies the entailment relationship (generation task). For solving the classification task, we propose the Chain of Counterfactual Thought (CoCT) framework, utilizing counterfactual reasoning, ASR-enhanced video content, and rationale refinement to reduce inference bias. For the generation task, we develop a framework that combines ASR output with a Large Language Model (LLM) to produce coherent, contextually relevant updates aligned with the intended strengthener or weakener goals. We also introduce a benchmark dataset with human annotations and an LLM-based evaluation metric for assessing generative performance. Experimental results show substantial improvements across multiple VLMM baselines, demonstrating that CoCT significantly enhances both classification and generation performance and strengthens the dynamic reasoning capabilities of video-language models.

Abstract:
When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores—using models as instruments for understanding behavior. We address this by asking: can we use large vision-language models (LVLMs) as experimental instruments for studying their own failure dynamics? Focusing on visual hallucination, we introduce SAFE, a training-free decoding framework that contrasts visually-grounded and vision-ablated generation paths to produce a token-level contrastive grounding score that identifies when the model favors linguistic priors over visual evidence. This signal serves dual roles: as a practical proxy for detecting visually-ungrounded tokens, and as the basis for decoding-time penalties. Our analysis yields three empirical observations: visual dependency decays over generation, hallucinations co-occur in temporal clusters, and early intervention reduces clustering without substantially degrading fluency. On MMHalBench, SAFE substantially outperforms all compared baselines; results elsewhere are more mixed. We argue that designing contrastive probes exemplifies a broader mission: using models as instruments for scientific understanding. Code: https://github.com/zhaozhipeng1997/SAFE_public.

Abstract:
Generating formally verified programs from natural language remains challenging: existing approaches either produce code in a single pass without recourse when verification fails, or rely on open-ended agentic reasoning that is non-deterministic and opaque. We introduce SKILLFORGE, a framework that decomposes formal code synthesis into a library of atomic, reusable skills —each targeting a specific subtask such as specification inference, body synthesis, invariant generation, error diagnosis, or targeted repair—defined by a prompt template, tool binding, and decidable success criterion. A verification-driven harness orchestrates these skills: it submits candidates to the Dafny verifier, diagnoses failures into structured categories, deterministically routes to the appropriate repair skill, and iterates until formal correctness is proved or a budget is exhausted. On a curated benchmark of natural language–Dafny specification pairs, SKILLFORGE substantially outperforms both state-of-the-art agentic approaches (including ReAct-style agents, MCTS-based repair, and RL-guided verification) and traditional iterative baselines, while requiring fewer tokens and lower latency. Ablation studies confirm that every skill contributes measurably, and the harness converges rapidly with the majority of programs verified on the first attempt.

Abstract:
Retrieval-Augmented Generation (RAG) grounds LLM responses in external evidence but treats the model as a passive consumer of search results, with no view of how the corpus is organized or what it has not yet seen. We present Corpus2Skill, a system-level retrieval architecture for bounded, structurally coherent corpora such as enterprise knowledge bases: an offline compiler distills the corpus into a hierarchical skill directory, and at serve time an LLM agent navigates it, drilling from a bird’s-eye view through progressively finer summaries down to documents and backtracking when a branch is unproductive. On an enterprise customer-support benchmark, Corpus2Skill improves both answer quality and grounding over single-shot dense, hybrid, hierarchical-retrieval, and agentic RAG baselines at a moderate cost tradeoff, and the lead persists under encoder-matched controls and paired significance tests. An eleven-dataset study shows that corpus navigation is not a universal replacement for retrieval: it significantly wins on five datasets, ties on three, and loses on three. It helps on single-domain corpora with a recoverable topical taxonomy, but flat retrieval remains preferable on open-domain factoid pools or homogeneous-tabular corpora that defeat top-level clustering. We characterize this scope distinction as a design guideline for knowledge-grounded systems. Code is available at https://github.com/dukesun99/Corpus2Skill.

Abstract:
Contemporary large language models (LLMs) exhibit multilingual capabilities but remain predominantly English-centric, resulting in a persistent performance gap across languages. While some approaches to multilingual training are grounded in the implicit pivot language hypothesis to improve interpretability, it remains underexplored how the languages represented in each layer are affected during training, particularly in practical, cost-efficient instruction-tuning scenarios. In this study, we investigate how instruction tuning with Low-Rank Adaptation (LoRA) in a target language affects the layer-wise language representation of instruction-tuned LLMs, with the aim of assessing the relevance of the implicit pivot language hypothesis as a theoretical grounding for multilingual finetuning. We find that target-language instruction tuning shifts models’ internal language representation in a non-uniform manner across layers and sublayers: middle layers shift further toward English, while early and final layers primarily serve the mapping between the target language and the dominant intermediate language. Our findings empirically support the implicit pivot language hypothesis in the context of instruction tuning, demonstrating that layer-wise parameter variation broadly corresponds to the hypothesized functional roles of layers in multilingual processing. Our code is available at: https://github.com/hyemijo/layer_interpretability

Abstract:
Operations research supports decision-making in domains such as energy, economics, and healthcare. Solving operations research problems typically begins with optimization modeling, which translates a natural-language problem description into executable solver code. LLMs offer a promising way to automate this process, but they remain prone to errors. In practice, these errors can be divided into two categories: syntactic errors refer to solver code that fails to run successfully or is judged infeasible by the solver; semantic errors refer to solver code that successfully returns an objective value but violates the intent of the original problem. Since semantic errors do not trigger runtime failures, they are difficult to detect and rectify. To address this problem, we introduce SemOPT, a semantic-guided framework for correcting LLM-based optimization models. SemOPT combines a semantic reward model that distinguishes faithful math models from plausible but incorrect ones with an adaptive correction system that applies hierarchical reward-guided search over the modeling space. Experiments on seven optimization modeling benchmarks show that SemOPT establishes a new state of the art and achieves an average 7.6% accuracy improvement over the strongest baseline on complex datasets.

Abstract:
Vision-language-action (VLA) models are built on the premise that semantic understanding from pretrained language or vision-language backbones should guide robot action prediction. Yet robot fine-tuning is optimized as imitation over task-specific action distributions, and many evaluations can be solved through visual or instruction-action shortcuts. We introduce RoboSemanticBench (RSB), an embodied benchmark for diagnosing semantic grounding in action prediction: whether post-trained VLA models can use complex instruction semantics to select and manipulate the correct physical target. In each episode, a robot receives a multiple-choice math or general-knowledge question, observes candidate answer blocks, and must grasp the block corresponding to the correct answer. RSB covers controlled arithmetic, grade-school mathematical understanding, and commonsense or factual understanding under four-choice and ten-choice suites. Across representative VLA models, we find that many policies learn to grasp candidate blocks but select the semantically correct block at near-random or below-random rates after controlling for grasp success, revealing a persistent gap between backbone-level semantic competence and action prediction.

Abstract:
Supervised fine-tuning (SFT) followed by reinforcement learning (RL) has become a standard post-training paradigm for large language models. This paradigm provides a cold-start for RL exploration, avoiding the inefficiency of pure RL where on-policy sampling yields insufficient positive samples. However, in practice, existing approaches often use a small amount of data for SFT initialization compared to the RL phase, which can cause the model to fit the limited samples and shift away from its pre-trained distribution. This distribution shift impedes the model’s ability to effectively explore during subsequent RL training. To address this challenge, we propose that in low-data regimes, SFT should prioritize activating task-relevant capabilities rather than memorizing specific content. Along this line, we propose EKSFT (Entropy-KL Selective Fine-Tuning), which selectively masks tokens that exhibit either high entropy or high KL divergence from a reference model. By excluding these high-uncertainty, distribution-shifting tokens from imitation, EKSFT injects task-specific knowledge while preserving the integrity of the model’s pre-trained distribution. Empirical evaluations on mathematical reasoning benchmarks demonstrate that EKSFT consistently outperforms standard SFT. Further RL fine-tuning from the EKSFT model yields consistently better post-RL performance, indicating improved exploration for the RL stage. Our codes and datasets are available at https://github.com/MINE-USTC/EKSFT.

Abstract:
Multi-agent LLM systems decompose complex tasks across specialist agents, yet joint prompt optimization remains difficult because agents are coupled through structured information handoffs. We identify interface mismatch as a structural failure mode that existing optimizers cannot diagnose: their scalar attribution signals conflate reasoning quality with format incompatibility, and the inter-agent contract is left entirely implicit. We propose BID-SPO (Bidding-based Shapley Prompt Optimization), which makes the inter-agent contract an explicit optimization variable. Each agent declares an Output Bid and an Input Demand, and a contract-aware cooperative-game characteristic function yields an attribution signal whose Shapley decomposition exposes three interpretable dimensions (global task contribution, downstream interface alignment, and upstream adaptation cost), turning attribution into a targeted diagnosis for prompt repair. A bid-conformance constraint keeps declared contracts feasible with respect to actual agent behavior throughout optimization. Across seven benchmarks and different model configurations, BID-SPO outperforms MASPO and AgentDropout, with accuracy gains of +2.93 % under the Qwen3-8B + Gemini-2.5-pro configuration and +4.38 % under the DeepSeek-v4 flash / pro configuration. Our findings emphasize that reliable multi-agent LLM optimization requires optimizing inter-agent interfaces as explicit objects, not just improving agents in isolation.

Abstract:
As Large Language Model (LLM) capabilities advance, locally deployed personal agents relying on API-based remote models and external skills have emerged as a novel paradigm. With the rapid expansion of available skills, enabling personal agents to learn and adapt to implicit user preferences becomes a critical challenge. However, local deployment constraints preclude complex centralized selection algorithms, creating an urgent need for a lightweight local preference harness. This paper explores the implementation of such a harness through a novel architecture that strictly decouples statistical preference learning from semantic intent parsing. Specifically, we leverage localized statistical results to influence and modulate the selection decisions of the remote LLM. Extensive evaluations demonstrate that our decoupled approach achieves the lowest cumulative regret and highest test accuracy, significantly outperforming traditional memory-augmented agents. We open-source our code at https://github.com/ZyGan1999/Personalized-Skill-Selection.

Abstract:
We introduce KFinEval, a benchmark for evaluating large language models (LLMs) in the Korean financial domain, with about 1K expert-curated instances across foundational financial knowledge, regulation-grounded procedural reasoning under noisy contexts, and financial toxicity under adversarial prompts. Existing financial benchmarks are largely English-centric and emphasize numerical or surface reasoning, providing limited coverage of regulation-grounded decision-making and domain-specific safety. Evaluating proprietary and open-weight LLMs, we observe substantial disparities and trade-offs among knowledge accuracy, procedural reasoning, and safety-aligned behavior; notably, strong capability does not consistently translate into safer responses under adversarial financial scenarios. KFinEval provides a practical Korean-regulation-grounded benchmark for diagnosing and improving financial LLM reliability and safety. Code and resources: https://github.com/sklim84/KFinEval.

Abstract:
Open-weight large language models (LLMs) can generate function-level programs from natural-language prompts, but plausible candidates still fail on hidden semantics and repeat mistakes across repair attempts. We present ReMCTS, an execution-grounded, memory-augmented, LLM-guided MCTS-style search framework. It organizes program candidates as tree states, retains branch-local debugging context, retrieves failure experience across branches, and distinguishes failed checks from unavailable evidence. On HumanEval and MBPP-Sanitized, visible-test ReMCTS improves over direct generation in 8 of 10 model–dataset pairs under held-out evaluation, whereas proxy-only search is less stable. Controlled tree-search, sampling, repair, and memory ablations characterize the source and limits of these gains. A 30-task HumanEval-X C++ pilot further demonstrates compatibility with compiler-backed execution, but does not constitute a broad multilingual evaluation.

Abstract:
While large language models (LLMs) have exhibited remarkable performance across diverse natural language processing tasks, their inherent propensity to generate hallucinations poses a critical challenge to reliable deployment in high-stakes scenarios. Existing hallucination detection methods often rely on static single-layer representations or aggregated features across all layers, which often fail to capture fine-grained factual inconsistencies and thus exhibit limited generalization capability. To address this limitation, this paper proposes TemHal (Temporal Hallucination Detector), a novel detection framework that models the cross-layer evolutionary trajectory of internal representations. Specifically, TemHal tracks the dynamic representational progression of key factual tokens across selected intermediate MLP activations and employs a bidirectional temporal modeling network to identify representational patterns that differentiate factually consistent outputs from hallucinated ones. Extensive experimental results demonstrate that the proposed method achieves consistent performance improvements across multiple mainstream model architectures and diverse benchmark datasets, while also exhibiting robust cross-dataset generalization across multiple LLM backbones. Furthermore, ablation studies validate the contribution of each architectural component and confirm the importance of temporal modeling and intermediate-layer focusing for detection performance.

Abstract:
Grammatical error correction (GEC) evaluation has traditionally relied on reference or edit overlap, which can penalize valid rewrites that differ from gold corrections. Reference-free metrics reduce this dependence, but evaluating whether a fluent output is a valid correction of the source remains challenging. We propose SURE, a source-conditioned reward evaluator trained on within-source preferences spanning minimal-edit and rewrite-oriented corrections. SURE jointly learns an overall reward with criteria-level supervision for grammaticality, faithfulness, and fluency, together with span-level grounding for source-side error resolution. Experiments on SEEDA show that SURE performs competitively against strong baselines, with particular gains on rewrite-style corrections and more disentangled criteria-level diagnostics. Our code is available at https://github.com/hayeonggg/SURE.

Abstract:
Preference tuning typically distributes adapter updates across every Transformer layer. We study how the placement of a fixed LoRA budget affects Direct Preference Optimization (DPO). Hierarchical Alignment partitions decoder depth into Local, Intermediate, and Global regions and restricts LoRA updates to a chosen contiguous block. Across Qwen and Llama models, three-seed sliding-window sweeps reveal broad low-loss basins around intermediate depth. Under exact trainable-parameter matching, Middle achieves lower held-out DPO loss than an all-depth allocation on Qwen2.5-7B and Llama-3.1-8B. Generation-level evidence is more variable. LLM judges and a blinded human study leave Selected versus All unresolved on Qwen; on Llama, two pairwise judges favor All while the human interval includes zero. Trained-adapter interventions show that learned depth components affect the optimization objective; removing the Llama Selected window also lowers generation quality. Across Qwen2.5 at 3B, 7B, and 14B, Middle versus All remains unresolved under Qwen3-Max; at 3B and 14B, both improve over Base. In our implementation, Middle also uses less peak memory and takes shorter steps than a parameter-matched all-depth allocation. Together, the results support intermediate depth as a practical starting point for fixed-budget preference tuning, though the best window and its behavioral value still need per-setting validation.

Abstract:
While multi-agent systems based on large language models (LLMs) have shown promise in automating academic research workflows, extending them to economics and business research presents two key challenges, as specialized domain knowledge often spans neighboring disciplines yet remains difficult to access in a structured manner. First, existing methods primarily retrieve information at the paper level, whereas the evidence required for research tasks is often distributed across different sections, creating a granularity mismatch that limits retrieval coverage and precision. Second, these fields demand strict empirical rigor, yet current systems provide limited mechanisms for learning from evaluation feedback. We present BizSage, a multi-agent framework that combines corpus-level fine-grained retrieval with quality-driven self-evolution. We construct a Lateral Knowledge Graph (LKG) by integrating section-level knowledge graphs and apply Personalized PageRank (PPR) to identify sections that are both semantically relevant and structurally important. Seven specialized agents collaborate under a Meta-Review self-evolution mechanism that distills failure modes from evaluation traces into reusable strategies. On a benchmark spanning four domains and three tasks, BizSage ranks first on the majority of metrics, achieves pairwise win rates above 60% against six baselines, and produces zero hallucinated citations. We hope BizSage paves the way for reliable research assistance in economics, business, and the broader social sciences.

Abstract:
Multilingual safety alignment typically assumes standard, canonical scripts, overlooking orthographic variation. We introduce PunjabiSafetyBench, a parallel safety benchmark evaluating 300 harmful intents and 500 benign controls across six sociolinguistic script variants (Gurmukhi, Shahmukhi, Romanized, code-mixed, Urdu, English). Evaluating six frontier LLMs, we reveal severe safety degradation on non-canonical scripts: transitioning queries to Romanized Punjabi or Shahmukhi drives significant guardrail bypasses in open-weight models, while proprietary models remain robust. Furthermore, we expose a localization gap where models overlook regional harmful slang despite successfully refusing standard clinical equivalents. Our findings demonstrate that modern guardrails are fragile to surface orthography, necessitating safety alignment on informal, non-standard scripts.

Abstract:
As post-training techniques evolve, large language models (LLMs) are increasingly augmented with structured multi-step reasoning abilities, often optimized through reinforcement learning. These reasoning-enhanced models outperform standard LLMs on complex tasks and now underpin many commercial LLM APIs. However, to protect proprietary behavior and reduce verbosity, providers typically conceal the reasoning traces while returning only the final answer. This opacity introduces a critical transparency gap : users are billed for invisible reasoning tokens, which often account for the majority of the cost, yet have no means to verify their authenticity. This opens the door to token count manipulation , where providers may overreport token usage or inject synthetic, low-effort tokens to inflate charges, a threat that can be carried out at near-zero cost . To address this issue, we propose CoIn , a verification framework that audits both the quantity and semantic validity of hidden tokens. CoIn constructs a verifiable hash tree from token embedding fingerprints to check token counts, and uses embedding-based relevance matching to detect fabricated reasoning content. Experiments demonstrate that CoIn , when deployed as a trusted third-party auditor, can effectively detect token count manipulation with a success rate reaching up to 94.7%, demonstrating a strong ability to restore billing transparency in opaque LLM services. The dataset and code are available at https://github.com/CASE-Lab-UMD/LLM-Auditing-CoIn .

Abstract:
Recent advances in Video Large Language Models (Video-LLMs) have enabled performance on long-video understanding tasks. However, existing methods still face two key limitations: evidence acquisition often relies on a single search intent, and answer generation lacks an effective visual feedback mechanism. To address these limitations, we propose CoVER , a Comprehensive Visual Evidence and Reflection framework for long-video understanding. CoVER enables Video-LLMs to See More by dynamically gathering query-expanded visual evidence, and Think Deeper by verifying draft answers with effective answer-specific visual feedback. Together, these mechanisms shift long-video understanding from answer-centric generation to evidence-centric and visually verifiable reasoning. Experimental results show that CoVER-7B substantially outperforms models with the same parameter scale and even surpasses state-of-the-art closed-source models on certain metrics.

Abstract:
Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck. As embedding-based retrieval approaches rely on compact encoders that may under-capture specialized tool semantics, parametric tool retrieval addresses this by encoding each tool as a virtual token appended to the LLM vocabulary, fine-tuned in two stages (memorization then retrieval SFT) to use the LLM as a retriever, achieving strong performance on standard ToolBench retrieval benchmarks. Yet these benchmarks use verbose, fully-specified queries, and their evaluation applies constrained decoding that restricts outputs to valid token paths - neither reveals whether the model actually understands its tools. We introduce ToolSense, an open-source LLM-powered diagnostic framework that takes any tool catalog as input and automatically generates three benchmarks: a Realistic Retrieval Benchmark (RRB) with queries at three ambiguity tiers, an MCQ probing benchmark, and a QA probing benchmark. Applying ToolSense to ToolBench ( ∼ 47k tools) and evaluating five parametric model training configurations reveals a knowledge-retrieval dissociation: on RRB queries, several configurations collapse by ∼ 50-64 percentage points compared to fully-specified ToolBench benchmarks, falling below embedding-model baseline. Additionally, despite strong retrieval performance, some models score near-random on factual probes, suggesting a knowledge-retrieval dissociation. We open-source the ToolSense framework and the ToolBench diagnostic benchmarks to encourage further research. (see supplementary material)

Abstract:
Multimodal large language models (MLLMs) incur substantial inference cost due to the processing of hundreds of visual tokens per image. Although token pruning has proven effective for accelerating inference, determining when and where to prune remains largely heuristic. Existing approaches typically rely on static, empirically selected layers, which limit interpretability and transferability across models. In this work, we introduce a matrix-entropy perspective and identify an ”Entropy Collapse Layer”(ECL), where the information content of visual representations exhibits a sharp and consistent drop, which provides a principled criterion for selecting the pruning stage. Building on this observation, we propose EntropyPrune, a novel matrix-entropy-guided token pruning framework that quantifies the information value of individual visual tokens and prunes redundant ones without relying on attention maps. Moreover, to enable efficient computation, we exploit the spectral equivalence of dual Gram matrices, reducing the complexity of entropy computation and yielding up to a 64x theoretical speedup. Extensive experiments on diverse multimodal benchmarks demonstrate that EntropyPrune consistently outperforms established competitive pruning methods in both accuracy and efficiency. On LLaVA-1.5-7B, our method achieves a 57.7% reduction in FLOPs while preserving 98.8% of the original performance. Furthermore, EntropyPrune generalizes effectively to high-resolution and video-based models, highlighting the strong robustness and scalability in practical MLLM acceleration.

Abstract:
GPUs increasingly accelerate database systems, but query-specific peak performance still often relies on hand-written kernels. Existing LLM kernel benchmarks focus on machine learning operators, leaving irregular, heterogeneous, data-movement-heavy database-style operators untested. We introduce DataKernelBench, which translates SQL into validated PyTorch TorchPlan programs and evaluates LLMs that optimize either the core tensor-bounded snippet or the full query in CUDA or Triton through execution-guided repair. Across ten proprietary and open-weight models on TPC-H SF10 with an H100 GPU, the strongest full-query CUDA configuration achieves 2.11× speedup over ‘torch.compile‘ at full pass rate. We find that higher-performing implementations commonly use kernel fusion and execution-strategy changes, stronger models benefit most from full-query specialization, and workload context matters more than hardware context. To handle data larger than GPU memory, we extend TorchPlan with Dask-cuDF for on-demand partition loading on TPC-H SF100 with four H100 GPUs, achieving 2.54× speedup. Project page: https://kerneldf.github.io/datakernelbench

Abstract:
Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own output? We find that the ability is largely present before any targeted training: prompted few-shot, a base model already predicts an external judge’s multi-attribute quality scores on open-ended responses well above chance across three benchmarks. We introduce Self-Evaluation Elicitation (SEE), a method that surfaces this latent ability through a short cycle comprising a calibration-coupled reinforcement learning phase that improves the answer and predicts the judge, followed by a masked distillation phase that sharpens the prediction while leaving the answer untouched. From 160 unique examples, roughly 31× fewer than a reinforcement learning baseline, SEE improves held-out calibration across three benchmarks while preserving answer quality. The elicited self-evaluation is sharply localized within the model’s own token distribution and stable across judges it was never trained against, indicating a transferable notion of quality rather than a single judge’s preference. These results reframe judge-aligned self-evaluation as a problem of elicitation rather than acquisition.

Abstract:
While recent medical benchmarks for Large Language Models (LLMs) have begun to incorporate clinical temporality, they still overlook a key reality: clinical evidence is encountered incrementally, requiring repeated decisions between reviewing the next available record and committing to a prediction. To address this issue, we introduce [] , a benchmark for Inpatient Note Acquisition that evaluates LLMs’ ability to make acquire-or-predict decisions using sequentially revealed real-world Chinese inpatient records. In this benchmark, each case sequentially reveals up to three note blocks: an admission note, a first-course summary, and a daily-progress summary. After each block, the model under evaluation must either request the next block or predict one or more later documented follow-up actions (laboratory, imaging, pathology, or consultation). Surgery and discharge records are withheld for label construction and boundary checks only. is built from 8 , 986 de-identified inpatient records collected from two tertiary teaching hospitals, with patient-disjoint and temporal evaluation splits. Representative LLMs exhibit diverse acquire-or-predict behaviors. For example, DeepSeek-chat improves action micro-F1 by +0.102 and +0.074 on the patient-disjoint and temporal splits, respectively, while reading fewer note blocks, though the gains weaken under temporal drift.

Abstract:
Reinforcement learning for code generation relies on verifiable rewards from unit test pass rates. Yet high-quality test suites are scarce, existing datasets offer limited coverage, and static rewards fail to adapt as models improve. Recent self-play methods unify code and test generation in a single model, but face an inherent dilemma: white-box access leads to self-collusion where the model produces trivial tests for easy rewards, yet black-box restriction yields generic tests that miss implementation-specific bugs. We introduce Code-A1, an adversarial co-evolution framework that jointly optimizes a Code LLM and a Test LLM with opposing objectives. The Code LLM is rewarded for passing more tests, while the Test LLM is rewarded for exposing more defects. This architectural separation eliminates self-collusion risks and safely enables white-box test generation, where the Test LLM can inspect candidate code to craft targeted adversarial tests. We further introduce a Mistake Book mechanism for experience replay and a composite reward balancing test validity with adversarial difficulty. Experiments on Qwen2.5-Coder models demonstrate that Code-A1 achieves code generation performance matching or exceeding models trained on human-annotated tests, while significantly improving test generation capability.

Abstract:
Large Language Models (LLMs) have shown impressive capabilities across many language understanding tasks; however, their ability to perform numerical reasoning remains fragile, particularly outside English. This limitation is especially concerning for Hindi, a language with hundreds of millions of speakers but relatively little focused evaluation in numerically grounded reasoning. In this work, we present HI-REASO, a benchmark designed to systematically evaluate numerical reasoning in Hindi. The benchmark comprises six complementary task types and 3k instances drawn from educational materials, real-world problem formulations, and commonsense reasoning scenarios. We assess a range of state-of-the-art LLMs using Hindi-native prompting as well as cross-lingual prompting and cross-lingual chain-of-thought reasoning. Across all settings, models consistently underperform compared to human baselines, with the largest performance degradation observed when reasoning is conducted directly in Hindi. While cross-lingual strategies offer partial improvements, they do not close the gap. Our findings reveal fundamental shortcomings in current LLMs’ numerical reasoning abilities under Hindi-native conditions and motivate the development of models and training paradigms that better support linguistically diverse numerical reasoning. We publicly release the code and data at https://github.com/kraritt/hi-reaso.

Abstract:
Diffusion language models (DLMs) accelerate generation by decoding multiple tokens in parallel, but pushing them to fast decoding often sacrifices accuracy. We attribute this to a static-to-dynamic mismatch: training optimizes local denoising under fixed schedules, while fast inference skips steps and encounters irregular mask patterns. Existing accelerators largely address only one side of this problem. Training-free inference methods speed up decoding without changing the model, leaving the mismatch unresolved; training-side distillation methods adapt the model, but often tie the student to a specific decoding trajectory, limiting composability and still suffering accuracy loss. We propose CD 4 LM, a composable framework for accurate DLM acceleration. Its core training component, Discrete-Space Consistency Distillation (DSCD) trains a trajectory-invariant student on diverse masked states, reducing the mismatch without fixing a decoding policy. We pair DSCD with a lightweight Confidence-Adaptive Decoder, while keeping the trained student compatible with external inference-side accelerators. On GSM8K, CD 4 LM preserves accuracy while achieving a 5.18 × speedup; stacking CD 4 LM with per-step cache accelerators further pushes speedup to 9.73 × at teacher-level accuracy, suggesting a scalable path for combining training-side robustness with future decoding advances.

Abstract:
While AI development and evaluation for Southeast Asia (SEA) has grown rapidly, agent capabilities in regional languages are still poorly understood despite its importance to sovereign AI. To fill this gap, we introduce SEATauBench, the first agent-focused evaluation framework for SEA sovereign AI. It adapts Tau2-Bench to five languages—Mandarin, Vietnamese, Thai, Indonesian, and Filipino—and evaluates agents across progressively localized settings that vary the language of user-agent interaction, tool specifications, and task domains. Across three models, we find that English agent capabilities transfer reasonably well when only the conversation language changes, but quality and robustness degrade sharply as more task contexts are localized, with the largest losses in full domain adaptation. We also highlight the limits of English-only agent assessment for predicting agent capabilities in SEA languages. More broadly, SEATauBench provides a diagnostic benchmark and reusable adaptation pipeline for building reliable multilingual agents for linguistically diverse regions.

Abstract:
LLMs have been used to simulate human decision-making in professional settings, yet their behaviors in common-law jury trials remain unexplored. We study when and how a defendant’s courtroom statement affects LLM-simulated jurors, focusing on persuasion, ideological bias, and background-based affinity. To support the analysis, we introduce JuryBench, a benchmark containing controversial criminal cases in U.S. criminal law. In each case, a defendant can claim various plausible justifications to support acquittal or reduced liability. We fix the base case and design defendants of different backgrounds, who give courtroom statements with varying emotional appeal or rebuttal. Jurors with diverse ideological profiles across the spectrum are simulated. We examine 20 frontier LLMs, resulting in a total of 432K decisions and rationales, and quantify changes in verdict severity. Our findings show that LLM-jury simulation echoes many human-jury findings. First, emotional persuasion can be detrimental, since jurors may perceive it as evidence of guilt or inconsistency. Next, we show that background fit between jurors and defendants is a stronger and significant factor than other isolated factors, and that jurors are in general harsher toward opposite-background defendants and lenient toward same-background ones. Finally, we find that juror ideology also strongly shapes severity judgments. These findings highlight both the promise and risks of using LLMs to model jury reasoning and call for careful evaluation.

Abstract:
Subjective judgments are part of several NLP datasets and recent work is increasingly prioritizing models whose outputs reflect a diversity of perspectives. Such responses allow us to shed light on minority voices, which are frequently marginalized or obscured by dominant perspectives. It remains a question whether our evaluation practices align with these models’ objectives. This position paper proposes seven evaluation desiderata for subjectivity-sensitive models, rooted in how subjectivity is represented in NLP data and models. The desiderata are constructed in a top-down approach, keeping in mind the user-centric impact of such models. We scan the experimental setup of 64 papers and show that various aspects of subjectivity are still understudied: the distinction between ambiguous and polyphonic input, whether subjectivity is effectively expressed to the user, and a lack of interplay between different desiderata, amongst other gaps.

Abstract:
LLM-as-judge frameworks are widely used for subjective NLP evaluation, yet their reliability on social-cognition tasks remains unclear. We investigate whether Theory-of-Mind (ToM) scaffolding can help LLM judges evaluate persuasion on Reddit ChangeMyView (CMV). We test four judges (GPT-5, GPT-4.1, Claude Sonnet 4.6, Qwen3-30B-A3B-Instruct-2507) under three conditions that share the same ToM annotations but vary how they are presented: No-ToM, Full-ToM (flat), and our proposed PAToM (Persuasion-Aware ToM), which structures reasoning across five sequential belief-stack turns. Our results show that better reasoning does not lead to better verdicts. Although PAToM improves reasoning coherence by 1.6x, semantic coverage by 1.1-1.8x, and our proposed process metric PRISM by 2-3x, all four judges still struggle to reliably distinguish persuasive comments from high-quality non-persuasive ones. Process-level analysis shows that reasoning quality and verdict accuracy are empirically decoupled under this scaffolding, an open challenge for prompt-based approaches to persuasion judging. We release prompts, ToM annotations, the PRISM metric suite, and a 1000-item benchmark.

Abstract:
How can language learning systems be developed for languages that lack sufficient training resources? This challenge is increasingly faced by developers across the African continent who aim to build AI systems capable of understanding and responding in local languages. To address this gap, we introduce AFRILANGDICT, a collection of 194.7K African language-English dictionary entries designed as seed resources for generating language-learning materials, enabling us to automatically construct large-scale, diverse, and verifiable student–tutor question–answer interactions suitable for training AI-assisted language tutors. Using AFRILANGDICT, we build AFRILANGEDU, a dataset of 78.9K multi-turn training examples for Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO). Using AFRILANGEDU, we train language tutoring models collectively referred to as AFRILANGTUTOR. We fine-tune two multilingual LLMs: Llama-3-8B-IT and Gemma-3-12B-IT on AFRILANGEDU across 10 African languages and evaluate their performance. Our results show that models trained on AFRILANGEDU consistently outperform their base counterparts, and combining SFT and DPO yields substantial improvements, with gains ranging from 1.8% to 15.5% under LLM-as-a-judge evaluations across four criteria.

Abstract:
Replay sampling is central to off-policy task-oriented dialogue policy learning, as it determines which past interactions are revisited during policy updates. Existing priority-based replay methods often rely on a monolithic priority distribution to form minibatches, which entangles coverage across error magnitudes with emphasis on high-error transitions. Under sparse dialogue success and heavy-tailed temporal-difference (TD) errors, this design overexposes extreme residuals and suppresses informative medium-error transitions, a failure mode we call mid-magnitude starvation. To address this problem, we propose Hierarchical Prioritized Experience Replay (HPER), a two-level replay sampler that separates coverage from prioritization. HPER first samples TD-error quantile strata according to occupancy, ensuring that different error magnitudes enter training, and then applies local prioritized replay within the selected stratum to retain emphasis on large residuals. This coverage-first design restores intermediate learning signals without diluting high-error samples. Experiments on multiple dialogue datasets show that HPER improves replay efficiency and yields faster competence onset and higher terminal success than uniform replay and monolithic priority-based replay under matched budgets.

Abstract:
Large language models (LLMs) frequently hallucinate when confronted with queries they cannot answer reliably. The root causes are multifaceted, including entity rarity, semantic ambiguity, temporal invalidity, false presuppositions, and other forms of epistemic uncertainty. Existing abstention mechanisms overwhelmingly rely on binary probes trained with manually curated “unanswerable” labels or calibrated confidence scores, both of which presuppose access to negative examples that are expensive to collect and fundamentally incomplete in coverage. We propose FLOW-ABSTAIN, an unsupervised framework that reframes query-aware abstention as density estimation in the model’s own latent space. Our method operates in three stages: (1) extracting intermediate hidden-state representations from a curated set of confidently answerable queries, (2) applying PCA-based dimensionality reduction to obtain a compact manifold, and (3) fitting a Masked Autoregressive Flow (MAF) to the resulting representations, yielding an exact, tractable log-likelihood for arbitrary new inputs. At inference time, queries whose latent projections receive low log-likelihood under the learned density are flagged for abstention—without requiring any negative or out-of-distribution training samples. Experiments across three instruction-tuned LLMs (Qwen-3-8B, Qwen-3-14B, Llama-3-8B-Instruct) and five diverse evaluation benchmarks demonstrate that FLOW-ABSTAIN achieves highly competitive abstention performance among methods with practical inference cost while providing a continuous, interpretable likelihood signal that naturally stratifies query difficulty along cognitive dimensions of unanswerability.

Abstract:
Mixture-of-Experts (MoE) has become a key architecture for scaling large language models. Recent fine-grained MoE designs introduce hundreds of experts per layer, with multiple experts activated per token, enabling stronger specialization. However, during pre-training, routers are optimized mainly for stability and robustness: they converge prematurely and enforce balanced usage, limiting the full potential of model performance and efficiency at inference. In this work, we uncover two overlooked issues: (i) a few highly influential experts are underutilized due to premature and balanced routing decisions; and (ii) enforcing a fixed number of active experts per token introduces substantial redundancy. Instead of retraining or redesigning MoE architectures, we introduce Ban Pick, a training-free, plug-and-play strategy for smarter routing. Pick discovers and reinforces key experts, a small group with outsized impact on performance, leading to notable accuracy gains. Ban further dynamically prunes redundant experts based on layer and token sensitivity, delivering faster inference. Experiments on fine-grained MoE-LLMs (DeepSeek, Qwen3) across math, code, and general reasoning benchmarks demonstrate that Ban Pick delivers free performance gains and inference acceleration. For instance, on Qwen3-30B-A3B, it improves accuracy from 80.67 to 84.66 on AIME2024 and from 65.66 to 68.18 on GPQA-Diamond, while accelerating inference by 1.25× under the vLLM.

Abstract:
Large language models have recently been used to discover heuristics through iterative program search, offering a promising approach to automated algorithm design. However, candidates selected by average performance may degrade across instance families, random seeds, and problem scales because mean-based objectives can obscure rare but severe failures. We propose CVaR-RMD, a risk-sensitive evolutionary framework that uses an LLM to mutate executable heuristic programs and ranks candidates by empirical conditional value-at-risk over heterogeneous evaluation conditions. Across three combinatorial optimization domains, CVaR-based selection improves upper-tail performance and reduces failure rates while maintaining competitive mean performance. These gains persist on larger unseen instances and independently curated within-domain benchmarks. Code is available https://github.com/BeyondHeaven/CVaR-RMD.

Abstract:
Multimodal large language models are increasingly given contextual evidence in heterogeneous forms: as a text passage, as a rendered image of the same text, or as both together. However, little is known about how consistently these surface forms are processed, especially when the evidence conflicts with the model’s parametric knowledge. In this work, we study modality robustness under knowledge conflict across 13 MLLMs and two datasets, and find them far from robust: (1) when a single context contradicts parametric knowledge, models tend to favor it more when presented as an image; (2) when a text and an image instead jointly conflict with parametric knowledge, the preferred source becomes essentially arbitrary, fluctuating drastically with input order, model choice, and dataset. We further show this instability affects downstream tasks like multimodal RAG and adversarial attacks: harmful instructions delivered as images, for example, raise attack success rates by up to 27 points. While conflict-aware fine-tuning partially mitigates the issue, we call for broader community attention to MLLMs’ modality robustness.

Abstract:
Step-level verification is central to improving the reliability of multi-step mathematical reasoning in large language models (LLMs), yet existing methods typically adopt a fixed strategy between formal and informal proof. Formal methods can produce interpretable and machine-checkable proof objects. However, they are computationally expensive and limited by the success rates of formalization, semantic alignment, and proof search. Informal methods based on LLM judges offer lower cost and broader coverage, but their judgments cannot by themselves provide rigorous checkable proofs. Thus, a mathematical verification agent should adaptively choose the proof form for each step, covering the union of both capabilities while preferring cheaper informal verification in their overlap. Based on this formulation, we propose FIVER (Formal–Informal Verification Agent with Efficient Routing), a step-level mathematical verification agent that adaptively fuses formal and informal verification. FIVER uses a learnable router modeled with item response theory (IRT) to select either a formal or informal workflow, forming a new step-verification architecture. Under the Qwen3-8B setting, FIVER improves over the purely formal HERMES baseline by 10.77% on MATH-398 and 33.33% on AIME 2025, respectively, while using only 58.8% of HERMES’s verification tokens on average.

Abstract:
Large language models (LLMs) are increasingly used as virtual survey respondents, yet most evaluations compare only item-level marginal response distributions. Such evaluations can miss two failures central to computational social science: synthetic respondents may distort correlations among attitudes, and errors may be distributed unevenly across countries or demographic groups. We introduce FairSynth, an instrument-relative framework for auditing and calibrating closed-ended, answer-only LLM survey simulations against existing anchor surveys. FairSynth evaluates marginal fidelity, Structural Attitude Fidelity (item–item correlation similarity), group representation error, distributional flattening, and between-group sign reversals. It then applies a model-agnostic post-hoc logit-shift layer with shared group parameters to reduce disparities in representation error while preserving attitude structure. Experiments on World Values Survey Wave 7 and General Social Survey 2022 show that FairSynth reduces worst-group representation error and sign reversals while improving structural fidelity on the evaluated questionnaires. FairSynth is not a substitute for human surveys; it is a measurement-fairness toolkit for auditing, stress-testing, and calibrating synthetic survey simulations when anchor data are available.

Abstract:
As humans delegate more tasks and decisions to artificial intelligence (AI), we risk losing control of our individual and collective futures. Relatively simple algorithmic systems already steer human decision-making, such as social media feed algorithms that lead people to unintentionally and absent-mindedly scroll through engagement-optimized content. In this paper, we develop the idea of human agency by integrating philosophical and scientific theories of agency with AI-assisted evaluation methods: using large language models (LLMs) to simulate and validate user queries and to evaluate AI responses. We develop HumanAgencyBench (HAB), a scalable and adaptive diagnostic tool for six behaviors related to human agency. HAB measures the tendency of an AI assistant to Ask Clarifying Questions, Avoid Value Manipulation, Correct Misinformation, Defer Important Decisions, Encourage Learning, and Maintain Social Boundaries. We find low-to-moderate agency support in contemporary LLM-based assistants, with substantial variation across system developers and behaviors. For example, while Anthropic LLMs most support human agency overall, they are the least supportive LLMs in terms of Avoid Value Manipulation. These behaviors do not appear to consistently result from increasing LLM capabilities or instruction-following (e.g., RLHF); we encourage further study of these behaviors so that developers and users can better understand the complexities of modern human-AI interaction.

Abstract:
Final-context accuracy in multi-passage fact verification can hide whether an added passage contributes a new verdict-relevant fact or merely repeats evidence already present. We introduce a controlled same-prefix diagnostic derived from HoVer supporting-fact annotations to measure marginal evidence contribution. For each claim, a shared prefix covers one fact’s evidence while leaving another required fact absent; the next passage either repeats the covered fact or supplies the missing fact. The diagnostic contains 1,180 same-prefix pairs and an exact-repetition ladder testing whether repeated copies of one evidence sentence can change a verifier’s label. Supervised verifiers with similar clean F1 scores, from 0.6324 to 0.6528, differ sharply in eight-repeat flip rate, from 8.5% to 30.3%. Conservative pooling and natural-language-inference (NLI) scoring reduce duplicate sensitivity but give near-zero missing-fact gain, while gain-oriented systems improve gain but remain repetition-sensitive. Latent-Gated Evidence Dynamics (LGED), a compact stateful reference model, shows that low duplicate sensitivity and positive missing-fact gain can coexist. Surface controls and decision-level analyses support this diagnostic interpretation. Redundancy and reader tests extend the duplicate-sensitivity finding beyond HoVer, while prompted open-weight LLMs reveal a gap between role recognition and verdict updating.

Abstract:
Current evaluations often treat Chinese varieties as monolithic, obscuring failures for regional users. We introduce RegioZH, a 2,607-item multiple-choice benchmark for regional lexical reasoning across Mainland China, Taiwan, Hong Kong, Singapore, and Malaysia. Built by region-specific annotators and expert-validated, it targets localized idioms, slang, code-mixing, politeness conventions, and sociocultural expressions. Zero-shot results on eleven LLMs show large disparities, including a 29.4-point average accuracy gap between Taiwan and Singapore. Persona and structured chain-of-thought prompts for five models yield selective gains, suggesting that some errors stem from retrieval or framing rather than only missing knowledge. Representation, attention, calibration, and surface-feature analyses show a dissociation between marker attention and semantic grounding, overconfident forced-choice errors, and localized lexical density as a consistent difficulty predictor. Qualitative analysis finds terminology misunderstandings, sociocultural inference gaps, and cross-regional semantic divergence. RegioZH demonstrates that apparent Chinese proficiency can mask uneven understanding across regional varieties, motivating intra-language robustness evaluation alongside multilingual generalization.

Abstract:
The central tension in continual learning (CL) is the trade-off between plasticity (acquiring new knowledge) and stability (retaining prior knowledge). We study how a pre-trained backbone can be continually updated to absorb new knowledge while preserving existing capabilities, via capacity control: regulating the effective rank of each parameter update, a per-step quantity directly controllable inside a LoRA update. A controlled probe of LoRA rank and placement across modules and tasks reveals a consistent trade-off, with a moderate-rank sweet spot that varies by placement and task, leaving no universally optimal fixed rank; a formal bound shows forgetting grows with rank. Building on these findings, we propose Continual Dynamic Rank-Selective LoRA (CoDyRA), which jointly trains each LoRA update with rank minimization via sparsity-promoting regularization on per-component importance weights. The supervised objective drives plasticity; rank minimization regularizes forgetting. We show that rank minimization serves as an implicit forgetting regularizer in the CL regime, protecting general capability and prior-task knowledge simultaneously by controlling forgetting against the current model state. Across MTIL, X-TAIL, and TRACE (CLIP, LLaMA, Gemma), CoDyRA outperforms prior CL methods on new knowledge learning and forgetting, achieving a strong plasticity–stability balance. Project page: https://artificer-ai-lab.github.io/CoDyRA.

Abstract:
Counterfactual reasoning requires models to reason beyond the observed world and explain how altered conditions propagate through downstream consequences. Existing benchmarks largely target bounded settings with fixed variables or single gold outcomes, overlooking open-domain scenarios requiring causal-process evaluation. To this end, we present WhatIfBench , a diagnostic benchmark for open-domain, open-form, long-horizon counterfactual causal reasoning, containing 220 what-if questions across STEM, HSS, and Hybrid scenarios. To evaluate free-form responses, we further propose PRISM , which first converts each natural-language explanation into a Response-Derived Semantic Causal Graph of events, states, and mechanisms. On top of this graph, PRISM then jointly applies a Process Metric assessing graph-level causal validity and a Rubric Metric assessing answer-level explanatory adequacy. Evaluating six frontier LLMs with this framework, we find that WhatIfBench remains far from saturated: even the strongest model reaches only a 64.62% final score. Further analysis reveals persistent causal gaps, premise drift, and topology fragmentation, suggesting that fluent counterfactual narratives often mask fragile causal processes. The benchmark, code, and evaluation scripts are available at https://github.com/zju-gt/WhatIfBench.

Abstract:
Scientific research is not merely a sequence of tasks to be automated, but a knowledge-production process in which claims must remain verifiable, criticizable, revisable, and accountable. Yet most AutoResearch systems remain single scaffolds that collapse generation, execution, interpretation, and review into one loop. This survey diagnoses three resulting epistemic limits of such architectures: collapsed skepticism, where a single scaffold produces not only errors but also the narratives that allow them to survive; unverified translation from discovery to application, where proposal generation outpaces independent verification; and invisible evidence trails, where final manuscripts hide the failed attempts and post-hoc decisions that produced them. To overcome these structural fragilities, we propose a shift from autonomous AI scientists to AI research communities, where specialized agents, shared memory, inspectable artifacts, and governance checkpoints distribute epistemic labor. We synthesize AutoResearch as a transition from metric-driven search loops to AI-scientist workflows, domain-mediated discovery, research infrastructure, and accountable human–AI co-evolution. Across these layers, we argue for a fundamental shift from productivity-centered evaluation to verifiability-centered evaluation.

Abstract:
Large Language Models (LLMs) are increasingly deployed through conversational interfaces, yet strong single-turn performance does not imply reliability in multi-turn setting. Multi-turn assistance requires a model to maintain an evolving representation of user requirements and to reason over that representation before complying with later requests. Failures can therefore be subtle: models may fail to recognize a changed constraint or hidden conflict, or recognize it without enforcing it in the final response. Existing benchmarks primarily test fully specified prompts, long-context recall, or controlled domain environments, leaving open-domain requirement reconciliation undermeasured. We introduce TurnFlow, an interactive benchmark for evaluating whether models preserve, revise, and act on user requirements across turns. TurnFlow combines adaptive user simulation with verifiable judging, allowing conversations to unfold naturally while keeping hidden requirements and pass/fail criteria controlled. In our main evaluation, twelve current LLMs achieve only 27.5%–51.0% aggregate pass rate on 490 scenarios. The failures show that current LLMs can remain fluent and cooperative while losing the global state of the conversation, exposing a gap between conversational helpfulness and robust requirement reconciliation.

Abstract:
The performance of large language models (LLMs) depends critically on the quality of their instruction-tuning data. For protein LLMs, existing instruction-tuning datasets are typically derived from structured protein annotations through template-based or label-guided generation. Although these datasets capture important biological facts, their construction paradigms often lead to two limitations: instruction homogenization and reasoning superficiality. The former refers to questions concentrating around repetitive forms and narrow topics; the latter to answers relying on shallow annotation lookup rather than evidence-rich biological reasoning. To address these limitations, we propose ProtSynGen, a knowledge graph-guided open-ended data synthesis framework for protein LLMs. ProtSynGen integrates structured protein databases with unstructured scientific literature into a protein-centric knowledge graph, and uses hierarchical graph sampling to guide LLMs in generating open-ended QAs across atomic, aggregated, and complex reasoning scenarios. Experiments show that ProtSynGen produces instruction-tuning data with broader topic coverage, richer reasoning depth, and stronger quality metrics than existing protein LLM instruction datasets. Models trained on ProtSynGen data further achieve superior performance on open-ended protein QA benchmarks, consistently outperforming baseline methods across atomic, aggregated, and complex reasoning scenarios.

Abstract:
While Rotary Position Embedding (RoPE) is widely used in large language models, its inductive biases can impair long-context reasoning and lead to attention sinks. In this work, we propose TRONTOR, a method that effectively combines RoPE and No Positional Encoding (NoPE) through a simple fine-tuning phase, bypassing the need for expensive pretraining modifications. Our preliminary experiments demonstrate that mixing RoPE and NoPE at the attention-head level can enhance long-context capabilities, while static selection strategies fail to capture the varying complexity of inputs. Based on this observation, TRONTOR employs a lightweight context-aware routing method that dynamically switches each attention head between RoPE and NoPE at every token step. This router-only configuration provides an efficient operating point, yielding long-context performance gains with minimal inference overhead. For further gains, we extend TRONTOR to TRONTOR-Agg, which applies simple unweighted averaging over multiple RoPE bases within RoPE-active heads while avoiding learnable intra-head routing. Evaluated on the RULER benchmark using Llama-3.2 (1B and 3B) and Qwen3 (1.7B and 4B), TRONTOR improves long-context performance by mitigating attention sinks and dynamically reallocating attention toward relevant mid-context tokens.

Abstract:
Synthetic data can be scaled along two routes: Source Expansion (SE), which enlarges the source by adding seed materials or generators, and Fixed-Source Synthesis (FSS), which holds the source fixed and scales the generation budget. Existing scaling studies typically expand the source as the data grows, conflating SE with FSS and leaving FSS underexplored. We isolate FSS within a standard rejection sampling (RS) pipeline and derive a scaling form for it from how repeated sampling covers a fixed source. Empirically, fit on low budgets, the form predicts performance at the held-out highest budget for every evaluated teacher–student pair. At matched total-sample budgets, SE and FSS are comparable at small budgets, but adding seed questions outperforms spending the same budget on more responses at large budgets. Within FSS, neither synthesizing additional questions from the existing seeds nor varying the synthesis protocol consistently outperforms plain RS across matched budgets. FSS is thus a bounded scaling axis and a controlled setting for comparing synthesis protocols. We will release our code and data to facilitate further research.

Abstract:
Generative language models (GLMs) excel at text generation and instruction following, while discriminative language models (DLMs) provide strong semantic representations for understanding tasks. Under the manifold hypothesis, natural language semantics can be regarded as lying on a shared latent semantic manifold, while DLMs and GLMs learn different representations of this manifold in their respective latent spaces. However, due to differences in training objectives and modeling strategies, these representation spaces are not naturally aligned, preventing discriminative representations from being directly utilized by GLMs through simple concatenation or linear transformation. To address this issue, a semantic alignment framework through discriminative conditioning for generation (DisCon4Gen) is proposed. From a manifold-based perspective, DisCon4Gen transforms DLM semantics into continuous conditions for GLMs. It further incorporates both global semantic signals and fine-grained discriminative cues to guide generation, enabling GLMs to better exploit external discriminative semantics. Experiments on five downstream datasets show that DisCon4Gen improves over strong supervised fine-tuning baselines and achieves competitive performance across both semantic understanding and generative reasoning tasks.

Abstract:
The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing interest. However, existing benchmarks largely focus on textual reasoning or isolated visual question-answering (VQA) tasks, lacking holistic integration of clinical narratives and medical imaging, and thus failing to assess the multimodal diagnostic synthesis capability central to expert clinical judgment. To bridge this gap, we introduce MedReaMM, a benchmark specifically designed to evaluate models’ ability to synthesize heterogeneous clinical evidence consisting of detailed patient histories alongside multiple medical images into accurate differential diagnoses under a complete-information paradigm. Constructed from case reports sourced from top-tier medical journals and curated clinical case databases, MedReaMM comprises 625 expert-validated cases with an average of 2.79 medical images per case and a total of 1,042 standardized diagnoses annotated with ICD-11 codes. These cases predominantly represent rare, atypical, or multi-system presentations that demand expert-level evidence integration beyond routine pattern recognition. We evaluate 23 Large Multimodal Models (LMMs) and find that most achieve diagnostic accuracy scores below 50%, underscoring a substantial gap in multimodal diagnostic synthesis capability. Further analysis reveals that medical knowledge proficiency, medical image understanding, and evidence integration are all highly correlated with diagnostic performance.

Abstract:
Automated Essay Scoring (AES) in low-resource, non-English educational settings faces a compounded challenge: existing corpora record only a final holistic score, discarding the rich intermediate reasoning that human examiners rely on. We argue this annotation gap may contribute to Rubric Drift—the observed tendency of LLMs to apply scoring norms misaligned with the target curriculum, potentially reflecting Anglocentric defaults. To address this, we introduce MPEAB-2430, a multilingual AES benchmark spanning Malay, Hindi, and Arabic curricula, annotated with Pedagogical Chain-of-Thought (P-CoT) trajectories that capture sub-dimensional analyses, explicit error attribution, and curriculum-conformant feedback, derived entirely from independent, language-specific expert practice. Leveraging this rich but modest-scale supervision, we propose RL-CRAF++, a sample-efficient pipeline combining CoT-SFT with Constrained GRPO. Crucially, our RL formulation relies entirely on human-annotation-derived rewards, completely bypassing the potential biases of external LLM judges. Evaluated independently per language, a 4B parameter RL-CRAF++ model consistently surpasses zero-shot frontier models (including GPT-5 and DeepSeek-V3.2) on in-domain test sets. Ablation analysis reveals that P-CoT supervision accounts for approximately 78.1% of the alignment gain over standard SFT, providing strong evidence that, in this culturally situated evaluation setting, process-level structural supervision contributes substantially more than RL-stage calibration alone. Our code is available at https://github.com/TW-NLP/MPEAB

Abstract:
The LLM-as-a-Judge paradigm has emerged as a scalable alternative to human evaluation. However, single-model judges are limited by their inherent model biases, while multi-agent evaluation protocols that mitigate this through diverse deliberation are prohibitively expensive at inference time. To this end, we propose JudgePanel, which equips a compact Judge model with multi-agent Panel deliberation capability. Specifically, we first train on panel deliberation traces from an ensemble of strong evaluators, capturing structured patterns of discussion, disagreement, and resolution. To further improve judgment quality beyond SFT, we introduce AdaReward, an adaptive multi-reward RL algorithm that dynamically rebalances reward component weights as different objectives saturate at different rates during RL training. For practical deployment, we further design a lightweight domain specialization module for rapid adaptation to new evaluation domains with few hundred labeled samples. As a result, (i) Novel: the first framework to equip a single compact judge with multi-agent panel deliberation capability at single-model inference cost; (ii) Effective Reliable: JudgePanel with a 14B backbone outperforms judge-specialized models up to 70B across four evaluation benchmarks, demonstrates strong position consistency, and rapidly specializes to new domains with few hundred samples.

Abstract:
Frontier model evaluations are shifting from foundational capabilities (e.g., instruction following and reasoning) toward compositional, agentic ones, but Korean agentic benchmarks remain scarce. We introduce K-BrowseComp, a web-browsing agent benchmark grounded in Korean contexts, consisting of 400 problems. The 300-problem K-BrowseComp-Verified subset is manually constructed and validated by native Korean speakers. On this subset, frontier LLMs, including GPT-5.5, DeepSeek-V4-Pro, and GLM-5.1, reach only 30.00–45.67%, a substantial drop from BrowseComp, while Korean LLMs released through Korea’s Proprietary AI Foundation Model program achieve at most 10.33%. We further construct a 100-problem Synthetic split using hard few-shot exemplars and failure-mode-targeted generation to exploit the asymmetry between solving and creating web browsing problems. On the adversarially filtered synthetic diagnostic split, the strongest model reaches only 26.00%, and we report this split separately as a targeted stress test. We publicly release our data and code at https://github.com/prometheus-eval/K-BrowseComp.

Abstract:
Text-to-SQL deployment is moving from one-shot query generation to interactive repair of executable but semantically incorrect SQL. In this setting, a model must revise an erroneous query under public tool feedback while preserving correct input structures. Existing reinforcement-learning signals are ill matched: outcome rewards reveal only whether the final SQL passes, whereas unconstrained process supervision may reward steps not grounded in SQL semantics. We propose SCOPE-SQL, a reinforcement-learning framework that treats SQL structure as an auditable credit space for interactive repair. It maps candidate queries and public feedback into F0–F6 modules—schema/runtime, table/join, projection, predicate, aggregation/grouping, order/limit, and nesting—and assigns action-level credit for repairing target modules, preserving stable modules, gathering evidence, and submitting when ready. Persistent low-credit regions then drive semantic refresh, generating validator-filtered frontier repair tasks. On BIRD-CRITIC-SQLite, SCOPE-SQL with semantic refresh reaches 48.8% success using a Qwen2.5-Coder-7B backbone, improving over current sota RL baselines by 4.4 and 4.2 points. Ablations attribute 4.4 and 1.8 points to process credit and semantic refresh. Trajectory analysis further shows better repair behavior: Repair+Preserve paths rise from 38.8% to 77.5%, while damage-and-recover paths fall from 43.8% to 1.2%. These results suggest that SQL-structured credit is an interpretable training signal for both policy learning and data expansion in interactive SQL repair.

Abstract:
In the current landscape of Large Language Models (LLMs), the curation of large-scale, high-quality training data is a primary driver of model performance. A key lever is the data recipe, which comprises a data processing pipeline to transform raw sources into training corpora. Despite the growing use of LLMs to automate individual data processing steps, such as data synthesis and filtering, the overall design of data recipes remains largely manual and labor-intensive, requiring substantial human expertise and iteration. To bridge this gap, we formulate end-to-end data recipe generation for LLM adaptation. Given a target benchmark and a pool of available data sources, a model is required to output a complete data recipe that adapts a base LLM to the target task. We present DataChef-32B, which performs online reinforcement learning using a proxy reward that predicts downstream performance for candidate recipes. Across six held-out tasks, DataChef-32B produces recipes that yield performance comparable to those curated by human experts. Notably, the recipe from DataChef-32B adapts Qwen3-1.7B-Base to the math domain, achieving 66.7 on AIME’25 and surpassing the official post-training checkpoint (Qwen3-1.7B). This work sheds new light on automating LLM training and developing self-evolving AI systems.

Abstract:
Agentic retrieval-augmented generation (RAG) formulates question answering as a multi-step interaction between reasoning and information retrieval, and has recently been advanced by reinforcement learning (RL) with outcome-based supervision. While effective, relying solely on sparse final rewards limits step-wise credit assignment and provides weak guidance for intermediate reasoning and actions. Recent efforts explore process-level supervision, but typically depend on offline constructed training data, which risks distribution shift, or require costly intermediate annotations. We present TreePS-RAG, an online, tree-based RL framework for agentic RAG that enables step-wise credit assignment while retaining standard outcome-only rewards. Our key insight is to model agentic RAG reasoning as a rollout tree, where each reasoning step naturally maps to a node. This tree structure allows step utility to be estimated via Monte Carlo estimation over its descendant outcomes, yielding fine-grained process advantages without requiring intermediate labels. To make this paradigm practical, we introduce an efficient online tree construction strategy that preserves exploration diversity under a constrained computational budget. With a rollout cost comparable to strong baselines like Search-R1, experiments on seven multi-hop and general QA benchmarks across multiple model scales show that TreePS-RAG consistently outperforms both outcome-supervised and leading process-supervised RL methods.

Abstract:
Integrating Large Language Models with symbolic planners is a promising direction for obtaining verifiable and grounded plans, with recent works extending this idea to visual domains using Vision-Language Models (VLMs). However, a rigorous comparison with methods that plan directly with VLMs is missing, due to a lack of visual benchmarks that support symbolic planning. We present ViPlan, the first open-source benchmark for comparing VLM-grounded symbolic approaches (VLM-as-grounder) with direct VLM planning methods (VLM-as-planner). ViPlan introduces a series of increasingly challenging tasks in two visual domains: a visual variant of the classic Blocksworld planning problem and a simulated household robotics environment. We find VLM-as-grounder methods to outperform direct VLM planning in Blocksworld (solving 46% of the tasks against 9%), where image grounding is both crucial and accurate. However, in the household robotics tasks, where linguistic knowledge helps, VLM-as-planner methods are greatly superior to VLM-as-grounder approaches (solving 34% of the tasks against 5%), which are hindered by partial observability. Thus, ViPlan domains capture fundamental shortcomings of both planning approaches, which we further diagnose with a qualitative failure analysis. Finally, across methods, we observe no consistent benefit from Chain-of-Thought prompting, suggesting persistent limitations in current VLMs’ visual reasoning abilities.

Abstract:
While prior studies have shown that gendered stereotypes can influence LLMs’ outputs, little attention has been paid to how gendered cues affect depression severity assessment in mental health contexts. In this paper, we systematically investigate the impact of gendered cues on LLM-based depression severity prediction, focusing on depressive disorder. To this end, we introduce a counterfactual evaluation framework, where we inject two types of gendered cues, explicit and implicit, into social media posts and analyze how they shift LLMs’ severity predictions. Our results reveal that gendered cues systematically shift LLMs’ severity predictions, increasing the risk of underestimating severe cases that may involve suicidal ideation or self-harm. To explore a possible mitigation direction, we propose FairDep, a symptom-grounded reasoning scaffold designed to reduce LLM sensitivity to gendered cues in depression severity prediction. By guiding models to focus on symptom-level evidence, FairDep improves prediction consistency under gender cue perturbations while preserving sensitivity to severe cases. Our findings show that gender cue sensitivity is not merely a fairness concern, but a safety concern in LLM-based mental health assessment, where even small prediction shifts can reduce sensitivity to severe cases involving suicidal ideation or self-harm. Our code and resources are publicly available.

Abstract:
Modals such as may and must can take on different senses depending on discourse context, alternating between root interpretations (e.g., permission, obligation) and epistemic interpretations (e.g., knowledge, belief). While modals are known to be notoriously difficult for children to acquire, limited research exists on whether language models (LMs) can reliably distinguish between different modal senses. We evaluate seven LMs that vary in architecture, size, and training data type using vector-based similarity and surprisal measures across four datasets. Specifically, we investigate whether LMs can leverage contextual information to disambiguate modal senses and whether their performance reflects the distribution of modal senses in their training data. We show that larger LMs trained on diverse text successfully leverage additional contextual information when available. In the absence of context, they tend to favor the dominant sense based on corpus statistics. Smaller developmentally plausible LMs, including those trained on child-directed language (CDL), fail to show consistent sensitivity to modal sense distinctions and do not benefit from context to the same degree.

Abstract:
Enterprise support retrieval requires matching noisy, multi-authored support cases to concise knowledge base (KB) articles. We present FineKB, an asymmetric multi-centroid retrieval framework for case-to-KB matching. FineKB summarizes historical cases offline, clusters the resulting issue summaries into per-KB centroids, and retrieves at query time using raw case text, thereby using a denoised index without per-query large language model (LLM) summarization. It further applies a lightweight reliability-aware re-ranking step based on validation-derived centroid precision. On 15,000 enterprise support cases spanning 188 KB articles, FineKB achieves 66.86% Recall@3 and 77.34% Recall@5, outperforming KB-content indexing, single-centroid representations, and a supervised fine-tuned encoder baseline, with a statistically significant Recall@5 gain over a strong case-text dense retrieval baseline. Re-ranking contributes +1.47 Recall@3 with negligible inference overhead. We release source code, anonymized vector embeddings, and cluster metadata for reproducible evaluation.

Abstract:
Multimodal large language models (MLLMs) trained on image-text data may retain sensitive information in textual memories, visual representations, and cross-modal associations, creating a need for reliable unlearning. Although existing unlearning methods can reduce target disclosure under clean queries, suppressed knowledge may remain recoverable under adversarial conditions. We introduce a new perspective on this vulnerability by assessing post-unlearning recovery through conditional perturbation generation, where a learned generator maps each image-text input to perturbations that promote recovery of the suppressed response. This framework is instantiated with two generators: CEPG injects conditional soft tokens into the multimodal input embedding sequence, while CPPG generates pixel-space perturbations under CLIP-based semantic conditions. Experiments across five unlearning methods show strong recovery risks. CEPG achieves average recovery ratio (Rec) scores of 60.19% on MLLMU-Bench and 46.74% on CLEAR, while CPPG attains 34.28% and 29.47%, respectively. Both attacks remain effective with limited or no direct access to forgotten samples, highlighting that current unlearning offers insufficient robustness guarantees. To mitigate this vulnerability, we introduce Adversarially Robust Unlearning (ARU), which integrates conditional recovery attacks into unlearning with attribution-guided parameter selection, significantly reducing recovery rates while preserving utility on retain sets.

Abstract:
Automated pronunciation assessment predicts pronunciation quality at the phone, word, and utterance levels. Recent joint models use both canonical and realized phone representations, but their mismatch is often encoded implicitly rather than preserved as a separate scoring path. This makes it difficult to trace how local phone errors affect word- and utterance-level scores. We propose HiGraph, which explicitly computes canonical-realized mismatch and propagates it through a hierarchical phone-to-word-to-utterance graph. This creates a structured discrepancy path from phone-level mismatch evidence to higher-level score representations. Under the same pipeline as our reproduced HMamba baseline, HiGraph improves word stress and utterance completeness on SpeechOcean762 (SO762). The learned word-level trace score also aligns with mispronunciation detection and diagnosis (MDD) ground truth. When words are ranked by this trace score, the top-ranked word contains a true mispronunciation in 81.5% of test utterances, and at least one of the top two words does so in 93.8% of test utterances. These results show that explicit discrepancy propagation can improve selected assessment dimensions while supporting feedback localization grounded in annotated mispronunciations.

Abstract:
Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information into comprehensive responses. For SAs evaluation, prior benchmarks mainly focus on specialized tasks that are unlikely to arise in real-world user scenarios. Moreover, their reliance on coarse task-level rubrics often limits evaluation interpretability. To bridge this gap, we introduce DailyReport, an open-ended benchmark to evaluate SA capabilities on daily search tasks. It contains 150 open-ended tasks with 3,546 associated rubrics, capturing widely discussed and timely information demands of real-world users. Each task is decomposed into subtasks and evaluated with cascade rubrics across disentangled dimensions. Through cascade performance attribution and user-centric aggregation, we derive highly interpretable scores for each dimension, along with a user preference score. Our results on 17 agentic systems show that current systems still fall short of users’ expectations. DailyReport will be open-sourced to facilitate future research.

Abstract:
Self-supervised speech models are known to encode phonological information, yet it remains underexplored whether they capture phonological properties characteristic of Japanese, such as mora and pitch accent. This paper introduces JPhonABX, a Japanese phonological discrimination dataset targeting moraic and accentual contrasts. It consists of 192,000 moraic and 98,000 accentual ABX test items, covering various types of contrasts such as those involving moraic nasal, geminate consonant, vowel length, and pitch accent. Our human evaluation by native Japanese speakers shows high discrimination accuracy and inter-rater agreement, validating the quality of the dataset. We further evaluate several self-supervised speech models and find that while almost all of them show no significant difference from humans in mora discrimination, they perform significantly worse in accent discrimination, and even models trained on Japanese fail to reach human-level performance.

Abstract:
Post-trained LLMs are often optimized to produce helpful, polite, and accommodating responses. In adversarial negotiation, however, such behavior can become a vulnerability: emotionally framed language may influence an agent’s bargaining decisions in ways that conflict with its user’s objectives. We therefore introduce EmoDistill , an offline framework for distilling emotional negotiation skills from LLM–LLM interactions into smaller language-model agents. Here, an emotional negotiation skill is a state-conditioned behavior that determines which explicit emotion to invoke in a bargaining state and how to realize that emotion as an effective negotiation utterance. EmoDistill learns these two components separately: an Implicit Q-Learning (IQL) selector learns which emotion to express in each bargaining state, while a LoRA-adapted 7B policy learns the emotion-conditioned expression through Supervised Fine-Tuning (SFT) and Judge Policy Optimization (JPO). Across four emotion-sensitive negotiation domains, the full EmoDistill policy achieves competitive Utility and improves over vanilla and IQL-only baselines in most settings. Emotion-free ablations show that removing the explicit emotion channel substantially reduces overall negotiation utility, while transfer experiments reveal partial, domain-dependent transfer and robustness to unseen LLM counterparties. The code is available at https://github.com/Yunbo-max/EmoDistill .

Abstract:
Tokenization is one of the factors which influences the multilingual downstream performance of a language model. This paper demonstrates that vocabulary constrained grapheme cluster based tokenization leads to improved performance on Natural Language Understanding and Word-level tasks. We assess three different grapheme cluster based tokenizers, which are the Grapheme Cluster tokenizer, the Grapheme Pair Encoding, and a vocabulary constrained version of Grapheme Pair Encoding, across eleven Indic languages, namely Assamese, Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Odia, Punjabi, Tamil, and Telugu. We present the intrinsic performance of three grapheme cluster based tokenizers using Fertility, Rényi Efficiency, and Percentile Frequency. We also compare the performance of these grapheme based tokenizers with seven popular multilingual and Indic-focused tokenizers on downstream tasks. We find that the vocabulary constrained Grapheme Pair Encoding tokenizer exhibits competitive and sometimes superior downstream performance in comparison to other grapheme cluster based tokenizers.

Abstract:
Large Language Model (LLM) agents increasingly operate in multi-step, multi-turn settings that combine user interaction, tool use, state changes, and communication. Most agent evaluation still emphasizes task completion, while procedure-aware checks remain fragmented. We conduct a systematic survey and introduce ℱ , a systematic organizing map that maps scattered procedural checks into a shared audit space spanning integrity relations and quality dimensions. Applying this framework to recent agent-evaluation work reveals three procedural blind spots: most work remains outcome-focused or procedurally under-specified, procedure-aware checks mostly remain local spot-checks, and communication grounding lags action compliance. To close these gaps, we propose four new Natural Language Processing (NLP) missions around communication grounding, constraint semantics, reasoning faithfulness, and procedural equity. This survey provides a comprehensive overview of procedure-aware agent evaluation, offering a common vocabulary and coverage map that clarify what current benchmarks can certify and provide clear guidelines for future research.

Abstract:
With the widespread industrial deployment of Large Language Models (LLMs), DeepSearch has emerged as the dominant paradigm for resolving complex user queries. It implements an iterative closed-loop workflow consisting of planning and reflection, information retrieval, and answer generation. However, existing reward models (RMs) are largely designed for static single-turn tasks, failing to capture the full-pipeline complexity of DeepSearch workflows. To address this limitation, we propose F 2 DR, a fine-grained full-pipeline reward framework. F 2 DR evaluates DeepSearch workflows across three dimensions: Content, Trajectory, and Answer, enabling comprehensive process-level assessment. We further construct DeepSearch RM-Bench, a dedicated benchmark for evaluating RMs in DeepSearch scenarios. Extensive experiments demonstrate that F 2 DR achieves significantly higher evaluation consistency than existing baselines, while DeepSearch RM-Bench exhibits strong discriminative capability across existing open-source RMs. We will publicly release the complete DeepSearch RM-Bench dataset.

Abstract:
LLM-based annotation has advanced rapidly, yet existing benchmarks remain misaligned with real-world practice. Actual annotation often involves lengthy, task-specific guidelines, and different annotators may produce divergent but reasonable judgments, especially on boundary cases, where single-labeled dataset fail to reflect this disagreement. This gap also limits existing single-agent and workflow-based annotation systems, which often lack adaptability to complex, evolving guidelines. To bridge this gap, we introduce ANT-A Benchmark , a real-world multi-annotator benchmark spanning 7 datasets across domains and modalities, with task-specific guidelines and 5 independent human labels per instance. ANT-A enables a distribution-aware evaluation, using Fleiss’ 𝜅 to assess how closely model annotations align with human labeling distributions. We further propose ANT-OR , a multi-agent annotation framework that refines prompts by distilling contrastive decision rules from positive and negative feedback. Experiments show that ANT-OR improves agreement@ ≥ 2 accuracy by 4.5 points over the strongest baseline and achieves the highest alignment with human annotator distributions as measured by Fleiss’ 𝜅 within 2 points. Code and benchmark are available at https://github.com/Krueske/ANT-OR.

Abstract:
Hybrid language models combine Transformer attention with recurrent, state-space, or online-updated components to lower long-context serving cost while preserving memory fidelity. Existing surveys mostly organize this space by architecture family, chronology, or efficiency technique, leaving workload-specific memory functions underspecified. We propose State-Evidence-Access (SEA) for this constraint: bounded State may lose distinctions needed by future queries not fixed in advance. Architectures therefore allocate active State, addressable Evidence, and a per-step Access budget to jointly support reusable structure, query-conditioned recovery, and effective use. This view helps explain why attention ratios or average perplexity can be misleading: a hybrid can save memory while losing the retrieval, state-update, or access function needed by the task. We survey hybrids by allocation locus—from key-value (K-V) and head-level mechanisms through layer, block, external-memory, and system-cache policies—and characterize their function-granularity mismatch risks. For workloads such as long-document QA, SEA attributes failure to missing evidence, unstable state updates, access bottlenecks, or poor integration.

Abstract:
Persistent AI assistants, such as OpenClaw, accumulate large collections of related memories over long-term interactions. As these memories grow, they may reinforce one another, diverge across contexts, or directly conflict, making correct assistance depend on memory relations rather than isolated recall. Existing long-term memory benchmarks do not systematically probe how agents preserve and utilize such relations during downstream tasks. To address this gap, we introduce SubtleMemory, a benchmark for fine-grained relational memory discrimination in long-running AI agents. SubtleMemory constructs relation-controlled latent semantic artifacts whose variants instantiate complementary, nuanced, or contradictory relations, and embeds them into realistic user-agent histories, requiring agents to recover distributed relational structures during later queries and instructions.The benchmark contains 1,522 evaluation instances over 10 long histories, grounded in 1,090 relation-controlled memory-variant sets and spanning user-related and non-user-related queries. Evaluating six standalone memory systems, two Claw-style agents with native memory modules, and three Claw-style agents with plugin memory modules, we find that current systems remain weak on fine-grained relational memory discrimination. We further introduce diagnostic protocols that reveal distinct capability profiles across memory preservation, retrieval, and downstream reasoning stages.

Abstract:
Real-world analysis often requires combining directly queryable structured sources (e.g., SQL and CSV) with evidence preserved only in visual documents, such as scanned reports, invoices, and dashboards. Current analytics agents still focus primarily on structured and textual inputs, leaving this cross-modal setting underexplored. We introduce DataCross, a benchmark and agent framework for heterogeneous, insight-driven data analysis. DataCrossBench contains 200 end-to-end tasks across six fine-grained categories: Fund Analysis, FinTech, Internet Development, Internet Firms, Short-form Drama, and AI Healthcare, grouped into four macro sectors and mixing SQL/CSV/JSON/TXT sources with image-based evidence. The benchmark is built with a human-in-the-loop reverse-synthesis pipeline to preserve cross-source dependency while keeping the target insights verifiable. We further propose DataCrossAgent, which coordinates source-specialized agents through intra-source exploration, source prioritization, cross-source pollination, and Reasoning ReAct (reReAct)-based executable verification. Experiments show that DataCrossBench reveals a difficulty gap for most baselines, especially on vision-dependent Hard tasks, and that DataCrossAgent improves average factuality over vanilla GPT-4o from 0.3291 to 0.4270 (+29.7%) while showing a smaller Easy-to-Hard degradation than several baselines. These results position DataCrossBench as a practical testbed for cross-modal heterogeneous data analysis.

Abstract:
As natural language becomes a key interface for controlling text-to-image generation, users specify not only which visual concepts should appear but also how entities are constrained by attributes, relations, quantifiers, and cross-entity dependencies. Reliable controllable generation therefore requires generalization over semantic structure, not merely over novel combinations of concepts. Existing benchmarks mainly evaluate conceptual compositional generalization, while leaving the internal semantic structure of instructions underexplored. We study structural compositional generalization: whether text-to-image models can generalize across increasingly coupled semantic dependencies. We introduce FormalImG, a first-order-logic-based benchmark with 4,000 instructions across three complexity levels in two scenarios, together with computable structural compositional complexity and ε-structural compositional generalizability. Experiments on advanced text-to-image models show consistent degradation as structural complexity increases; even the strongest evaluated model generalizes only up to G 0.7 = 6 in Natural and G 0.7 = 4 in Knolling. Further analysis suggests that the main bottleneck lies in language-to-vision instantiation and can be partially mitigated by explicit intermediate layouts. The benchmark and code are available at https://github.com/yhj137/FormalImG.

Abstract:
While Large Language Models (LLMs) have revolutionized scientific text processing, they exhibit a significant capability gap when interpreting chemical reaction diagrams. We identify two bottlenecks restricting current systems: a Visual Deficit, where generic vision encoders struggle to resolve the strict topological connectivity of dense molecular graphs, and a Semantic Disconnect, where standard linear strings (e.g., SMILES), although structurally precise, are often poorly aligned with the chemical semantic abstractions used by LLMs for reasoning. To bridge these gaps, we propose the Chemical Visual Activation (ChemVA) framework, which employs a novel Visual Anchor mechanism to ground functional groups via hybrid-granularity detection, followed by a semantic alignment approach that translates recognized structures into language-accessible chemical representations for downstream LLM reasoning. We evaluate our approach on OCRD-Bench, a newly constructed benchmark spanning recognition, chemical knowledge, and mechanistic reasoning. Experiments show that ChemVA achieves 92.0% structural recognition accuracy. It further delivers a consistent performance gain of approximately 20 percentage points across 9 diverse LLMs, allowing open-weights models to approach strong proprietary systems on complex chemical reasoning tasks.

Abstract:
We introduce BenGER (Benchmark for German Law), a benchmark and dataset for subsumption-based legal reasoning in German law. It combines 596 exam-style free-text case tasks across several levels of legal education with 531 short doctrinal reasoning tasks, and a controlled subset adds timed human solutions written both unaided and in human-AI co-creation. We evaluate 12 contemporary LLM systems (closed flagship, efficiency-oriented, and open-weight) with a rubric-aligned LLM-as-a-Judge, which we cross-validate against a multi-rater human layer (three blind reviews per solution) and across six LLM judges. Our results show that closed-flagship systems lead across all three corpora, that human-AI co-creation measurably improves on unaided human work, and that the judge tracks human grading at Pearson r=0.76 and Cohen’s κ=0.60, with rankings stable across judges and two judges from independent providers clearing the Calderon single-reviewer replacement bar on human-authored solutions.

Abstract:
Process Reward Models (PRMs) are vulnerable to reward hacking: policies exploit surface cues such as confident phrasing or structured formatting in place of logical correctness, which yields an inverted-U collapse under optimization pressure. We propose ARMOR-PRM, which augments standard PRM training with three regularizers: a counterfactual margin loss, an adversarial invariance loss with low-score anchoring, and a gradient penalty on the reward head. On ProcessBench (three splits; "math" excluded due to training data overlap), ARMOR-PRM improves Detection Rate by 14.9 percentage points and AUROC by 4.1 percentage points over Vanilla PRM, and is the only variant whose Best-of-N reranking accuracy improves monotonically through N=128. A domain-shift diagnostic further shows that objective-level robustness and data coverage are distinct but complementary axes: ARMOR-PRM responds best to iterative hard-negative retraining, suggesting a practical two-stage deployment path.

Abstract:
Large Audio Language Models (LALMs) are rapidly advancing, but evaluating them remains challenging due to inefficient and non-standardized toolkits that limit fair comparison and systematic assessment. Existing evaluation frameworks exhibit three critical limitations: (1) slow and inefficient processing pipeline that bottlenecks large-scale studies, (2) inadequate multi-turn dialogue support, leaving fundamental questions about cross-turn context integration and performance dynamics over extended conversations in LALMs unanswered; and (3) the absence of unified and scalable evaluation framework capable of keeping pace with the rapid growth of both LALMs and audio benchmarks. To address these issues, we introduce AU-Harness, an efficient and comprehensive evaluation framework for LALMs. Our empirical studies demonstrate that AU-Harness achieves 77% reduction in evaluation processing time over existing evaluation toolkits, enabling large-scale LALM evaluations previously considered impractical. We provide standardized evaluation protocols and flexible configurations for fair model comparison across diverse scenarios. AU-Harness unlocks a range of in-depth analyses difficult to conduct without a unified foundation, including multi-turn dialogue dynamics, enabling the study of true audio reasoning capabilities in existing LALMs. AU-Harness provides both practical evaluation tools and insights into model limitations, advancing systematic LALM development.

Abstract:
Training vision-language web agents with multi-step RL is compute-intensive, with two dominant forms of inefficiency: idle GPUs in synchronous RL, and trajectories that use more steps and tokens than necessary. We present AsyncWebRL, which addresses both. On the system side, an asynchronous design overlaps rollout, gradient update, and policy refresh across iterations, paired with two web-agent-specific adaptations, namely an everlasting rollout pool and lightweight screenshot handling, that together deliver up to a 2.9× end-to-end training-throughput speedup over the previously fastest open synchronous pipeline (WebGym). On the algorithmic side, we identify the per-trajectory normalizer 1/|𝜏 i | in multi-step GRPO as the root cause of trajectory-level and token-level inefficiency: because failures are systematically longer than successes, it down-weights the negative gradient on failed tokens, so the policy keeps producing verbose memory schemas. Replacing 1/|𝜏 i | with a constant 1/k breaks this coupling, contracting trajectories while preserving aggregate success. Together, these contributions set a new open-source state of the art on the WebGym out-of-distribution test split (+5.8% relative over the 42.9% prior best), with the largest gains on the harder slices (+42% relative on Medium, +48% relative on Hard).

Abstract:
We introduce StructuralCost, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution. We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance. Different language models (spanning n-gram models, SSMs, and transformers) partially mirror this graded difficulty profile, yet underestimate the integration cost humans incur, with a gap that persists across architectures and model sizes. This suggests these models capture the predictive component of human processing but not the full integration cost that working memory imposes. StructuralCost provides data needed to drive progress toward evaluating the cognitive plausibility of language models.

Abstract:
Large language models are increasingly used to simulate human opinions, but prior work reports conflicting results: some studies find promising alignment with human survey data, while others find persona collapse and weak demographic sensitivity. We propose that much of this conflict stems from conflating two distinct tasks. We call the first task emulation, in which models generate individual responses that aggregate into a population distribution. We call the second task estimation, in which models directly predict the population distribution. Evaluating six matched base and post-trained models on the Pew American Trends Panel, we find that base models are the stronger emulators: they produce response distributions closer to human ground truth and better preserve demographic structure. Post-trained models are generally the stronger estimators, producing more accurate distributional predictions when asked directly. We argue that model selection for human simulation should be guided by whether the task requires generating text or predicting distributions.

Abstract:
Puns pose a challenging test case for large language models (LLMs), as their interpretation requires mapping a surface expression to an implicit meaning through a specific pun-forming mechanism. Existing benchmarks typically evaluate isolated abilities such as detection, localization, explanation, or generation quality, leaving unclear whether LLMs can consistently identify, explain, and apply the mechanisms that make puns work. To address this gap, we introduce MECHPUNBENCH, a bilingual, mechanism-oriented benchmark for Chinese and English pun understanding and generation. Each positive instance is annotated with trigger spans, surface meanings, implicit meanings, rhetorical effects, and mechanism labels. Based on a taxonomy of Lexical-Semantic, Structural-Syntactic, and Structural-Semantic puns, we formulate six diagnostic tasks spanning surface recognition to mechanism-guided generation. Experiments on representative closed-source and open-source LLMs reveal a substantial surface–mechanism gap: although the best model achieves 82.8% detection accuracy and 88.2% containment localization accuracy, it remains below the 4/5 threshold used in our evaluation on implicit meaning explanation, mechanism explanation, rhetorical effect explanation, and mechanism-consistent generation. These findings suggest that recognizing puns or generating fluent pun-like text does not necessarily indicate robust mechanism-level pun understanding.

Abstract:
Large language model (LLM)-based systems are becoming increasingly popular for solving tasks by constructing executable workflows that interleave LLM calls, information retrieval, tool use, code execution, memory updates, and verification. This survey reviews recent methods for designing and optimizing such workflows, which we treat as agentic computation graphs (ACGs). We classify the literature according to when workflow structure is determined. This lens distinguishes static methods, which fix a reusable workflow scaffold before deployment, from dynamic methods, which select, generate, or revise the workflow for a particular run before or during execution. We further categorize existing work along three dimensions: when structure is determined, which aspect of the workflow is optimized, and which evaluation signals govern optimization (e.g., task metrics, verifier signals, human preferences, or trace-derived feedback). We also distinguish reusable workflow templates, run-specific realized graphs, and execution traces, separating design-time choices from runtime behavior. Finally, we outline a structure-aware evaluation perspective that complements downstream task metrics with graph-level properties, execution cost, robustness, and structural variation across inputs. This work aims to establish a principled vocabulary, a unified framework for situating new methods, a systematic comparison of existing work, and a minimum reporting protocol for LLM agent workflow optimization.

Abstract:
Recent automatic speech recognition (ASR) systems increasingly integrate large language models (LLMs) to leverage their semantic knowledge, either externally through logit fusion or internally through warm initialization. However, how to effectively combine these two strategies remains underexplored. In this work, we refine warm-initialized LLM-based ASR models by leveraging their own pre-adaptation base LLMs, focusing on LoRA-adapted settings where the base LLM is preserved. To achieve this, we propose Hybrid Search, a targeted correction strategy motivated by two observations. First, interaction features that characterize the relationship between LLM-based ASR hidden states and base-LLM hidden states provide informative signals about a token’s degree of semantic dependence. Second, selectively refining targeted tokens with high semantic dependence improves ASR performance far beyond naive global LLM-correction methods including rescoring and late fusion. Our analysis suggests that, even after semantic knowledge transfer through warm initialization, LLM-based ASR models can still leverage their base LLM to further improve inference-time performance. Demo: https://huggingface.co/spaces/Splend1dchan/Listen-To-The-Latent

Abstract:
Large Language Models (LLMs) trained on extensive scientific research are increasingly integrated as assistants for scientific discovery. However, most research papers omit the fine-grained cognitive process of examining constraints, failed alternatives, and iterative decisions required to achieve the desired goal. Such cognitive processes are vital for real-world scientists working toward specific goals under constraints. In this paper, we show that LLMs, when trained to produce such cognitive traces, perform better as scientific discovery assistants than when trained solely on scientific literature. We propose COGTRL, a trajectory-level reinforcement learning framework that trains LLMs to emulate cognitively grounded reasoning by jointly optimizing cognitive traces and the scientific steps produced in an interleaved manner. Across two 3B-parameter models and two scientific domains (AI and Materials Science), COGTRL improves method quality by an average of 7.85 points over comparable 3B model baselines and achieves competitive performance relative to 70B parameter models. Moreover, analysis by domain experts shows a preference for methods generated by COGTRL over the baselines.

Abstract:
Medical question answering (QA) is a knowledge-intensive task demanding evidence-based reasoning across diverse sources. Despite the demonstrated efficacy of augmenting large language models (LLMs) with domain-specific knowledge graphs (KGs) for QA tasks, the prevailing single-KG paradigm neglects the benefits of multi-KG complementarity. This paper introduces KGFlow, a novel multi-KG reasoning paradigm that mitigates the incomplete coverage and fragmented evidence of single-KG approaches through three LLM agents. Given a query, Knowledge Explorer extracts relevant knowledge triples in parallel from heterogeneous sources. Graph Aligner then semantically harmonizes these triples into global reasoning paths, which are filtered by Contextual Pruner to preserve the most valuable paths for inference. Extensive experiments show that KGFlow achieves promising performance improvements compared to state-of-the-art techniques on five medical QA datasets. We further observe by cases that the plug-and-play nature of our framework enables its application to diverse KG combinations and multilingual scenarios, resulting in broader knowledge coverage than single-KG approaches.

Abstract:
Large Vision-Language Models (LVLMs) integrate visual perception with language generation, enabling responses that span image understanding and complex reasoning. However, LVLMs do not just inherit the text-level hallucinations; they also hallucinate against the image, producing fluent responses ungrounded in what they see. This makes LVLM response scoring inherently harder, and our diagnostics show that existing confidence-based metrics adopted from LLMs is insufficient for LVLMs. Specifically, removing the input image barely changes confidence-based selection, suggesting that output-space confidence primarily captures textual plausibility rather than agreement with the image. To address this gap, we propose LookBack, a training-free LVLM response scoring method that augments token likelihood with visual lookback score, a lightweight measure of how strongly each response token refers to image tokens. Across four benchmarks and three models, LookBack consistently improves Best-of- selection over existing baselines with negligible additional overhead.

Abstract:
Despite their potential in standardized graph tasks, Large Language Models (LLMs) remain brittle to real-world shifts in node identifiers and task formulation. While deterministic graph tools are invariant to such shifts, extracting topological structures from noisy text is highly fragile for LLMs, which often overfit to surface patterns. Moreover, mitigating these parsing failures via multi-agent systems incurs prohibitive latency. To address this, we propose GRAIN, a single-agent framework optimized via reinforcement learning. GRAIN models reasoning as a semantic parsing and tool-execution pipeline, guided by a Structure Invariance Reward. By validating extracted intermediate graphs against ground-truth topologies, this reward forces the LLM to learn robust text-to-structure mappings rather than memorizing linguistic artifacts. We also introduce GRIT, a benchmark evaluating sensitivity to such linguistic shifts. GRAIN outperforms multi-agent baselines by 16.45% in accuracy with approximately 24% lower latency. Furthermore, it demonstrates superior structural generalization, halving the out-of-distribution (OOD) gap of SFT models (from 15.77% to 7.80%) and maintaining robustness on large-scale graphs beyond the training distribution.

Abstract:
While large language models (LLMs) demonstrate strong text classification capabilities, adapting them to specialized, few-shot industrial domains remains challenging. Vanilla supervised fine-tuning (SFT) applies uniform weighting to training samples and may overfit easy instances while insufficiently optimizing ambiguous boundary samples. Static regularization may further limit adaptation by applying the same constraint throughout training. To address these limitations, we propose cumulative area under the margin (C-AUM), a fine-tuning framework tailored to autoregressive LLM classification. C-AUM tracks the cumulative margin of the final decision token as an instance-level difficulty signal and uses it to coordinate AUM-aware instance weighting with difficulty-aware dynamic LoRA regularization. We also construct and release an expert-verified few-shot text classification benchmark for the steel industry, comprising 1,070 samples across seven fine-grained categories. Evaluations across six representative LLMs show that C-AUM achieves comparable or improved performance over vanilla SFT, with a maximum macro F1 gain of 1.99 percentage points. Our dataset and code are publicly available at https://github.com/pengdada98/C-AUM.

Abstract:
Large Audio-Language Models (LALMs) integrate audio perception and language understanding within a unified framework, enabling a wide range of real-world applications. Despite recent advances, evaluation for LALMs remains heavily underspecified relative to real-world requirements: most lack true linguistic and cultural authenticity, while others fail to capture acoustic realism. To bridge this gap, we propose GlobeAudio, a multilingual and multicultural benchmark designed to evaluate naturalistic audio understanding. GlobeAudio consists of 5,637 multiple-choice questions across six typologically diverse languages, expertly crafted by native speakers grounded on naturally occurring audio. In order to do well, models must possess higher-level auditory reasoning skills and culturally grounded interpretation. We systematically evaluate representative closed-source and open-source LALMs, as well as cascaded ASR–LLM pipelines. Our experiments reveal substantial performance gaps under natural acoustic conditions, particularly for open-source models and low-resource languages. These findings highlight critical limitations of current LALMs and underscore the importance of naturalistic audio evaluation for future audio-language systems. GlobeAudio can be found at https://huggingface.co/datasets/iNLP-Lab/GlobeAudio.

Abstract:
Skills are increasingly used to improve LLM agents on real-world tasks, yet it remains unclear how agents use skills, how skill quality should be evaluated, and how users should choose skills under cost-performance trade-offs. We present OpenSkillEval, an automatic framework for evaluating both skill-augmented agents and skills. OpenSkillEval dynamically constructs test cases from evolving real-world artifacts across five applications: presentation generation, front-end web design, poster generation, data visualization, and report generation. It also curates community skills for controlled comparison under unified settings. Using over 600 generated tasks and 30 open-source skills, we evaluate state-of-the-art models and agent frameworks. Results show that skill availability does not guarantee effective use, skill benefits depend strongly on the model and framework, and many popular skills fail to consistently outperform skill-free agents. These findings highlight the need for dynamic, task-grounded evaluation and offer practical guidance for skill design, selection, and deployment.

Abstract:
Incomplete multimodal learning seeks robust perception when modalities are absent at training or test time, using available streams to recover the missing ones. Most methods cast this as direct regression in a fixed continuous latent, which (i) collapses many valid solutions into an averaged reconstruction when observations underdetermine the target, and (ii) suffers semantic drift when cross-modal relationships shift with context. We argue that completion should first resolve what is missing and only then render how it appears. We realize this as Semantic-first Completion, instantiated by a Hybrid State-Space Model (HySSM) that decouples semantic intent from signal realization. A frozen omni-modal LLM plans over observation-only evidence, generating an evidence-grounded Markovian trajectory of discrete semantic states that act as a compact information bottleneck for stable cross-modal reasoning. The plan is projected into a coordinate-wise control field that drift-guides a conditional diffusion model, synthesizing missing signals or feature streams as diverse and semantically consistent completions. Across MNIST, Hateful Memes, and CMU-MOSEI, HySSM surpasses recent generative baselines and, under agnostic masking shift, remains robust up to 90% missingness.

Abstract:
Large language models (LLMs) have shown impressive capabilities in code generation across many programming languages but even state-of-the-art LLMs generate programs that contain syntactic errors and fail to complete the given tasks, especially for low-resource programming languages (LRPLs). In addition, the high cost of training makes finetuning LLMs unaffordable for those with constrained computational resources, further weakening the effectiveness of LLMs for code generation. In this work, we propose SLMFix, a novel code generation pipeline that leverages a small language model (SLM) finetuned using reinforcement learning (RL) techniques to fix syntactic errors in LLM-generated programs for domain-specific languages (DSLs) based on interpreter feedback. Our experimental results demonstrate the effectiveness and generalizability of our approach across multiple DSLs, improving the validator pass rates by 40% on LRPLs and eliminating more than 50% of syntactic errors for high-resource DSLs. Notably, SLMFix brings substantial performance improvement to the base model and outperforms supervised finetuning approach even for 7B models on LRPLs including Ansible and Lean, showing the potential of our approach in improving the quality of LLM-generated programs.

Abstract:
Textual spatial semantic anomaly detection and attribution requires deciding whether a sentence contains an implausible spatial relation and assigning the corresponding attribution label. Prior prompting and Low-Rank Adaptation (LoRA) fine-tuning methods rely on raw tokenlevel representations, leaving spatial roles and cross-triple conflicts implicit. We propose SCP-ISA, a Spatial Cross-attention Projection (SCP) method with an Inter-Slot Attention (ISA) layer that uses a frozen MacBERT-large encoder to build input-conditioned spatial slots and injects the projected slots into Qwen2.5 as soft prefix embeddings. The ISA layer enables structured cross-slot interaction that collapses co-referring spatial mentions into a shared entity-anchored representation before generation. On the CRSARC benchmark, SCP-ISA improves test cascade attribution F1 (Attr-F1) by 22.7 percentage points (0.509 to 0.736) over Direct LoRA, and detection macro-F1 (Det-F1) from 0.846 to 0.881.

Abstract:
Extreme multi-label text classification (XMTC) of biomedical literature aims to classify articles into a set of pre-defined labels from a large and highly skewed label distribution space. While recent pretrained language model (PLM)-based XMTC approaches have made substantial advances, they still struggle to generalize to tail labels representing concepts of high biomedical significance (e.g., rare diseases). A key factor contributing to poor tail-label performance is the presence of spurious correlations, where models inadvertently learn non-informative feature-label associations. To address this issue, we propose a novel context-adaptive spurious correlation mitigation technique, SPUR-XMC, which selectively attenuates spurious influences according to the context of the input article. A unique advantage of the proposed framework is that it does not modify existing PLM architectures for XMTC and instead operates as an add-on enhancement module. Extensive experiments across benchmark biomedical literature datasets demonstrate the effectiveness of the proposed approach.

Abstract:
Empowered by Large Language Models (LLMs), deep research systems integrate online information into research reports through multi-step web searching and information synthesis. Evaluating these reports presents a significant challenge due to their long context, high information density, and domain specificity, necessitating the use of AI evaluation as a substitute for prohibitively expensive human assessment. However, few studies have examined the effectiveness of existing reward models or LLMs in understanding and evaluating such research reports. To bridge this gap, we introduce DeepResearch RewardBench, the first benchmark dedicated to evaluating reward modeling for research reports. We collect diverse report candidates from multiple agents or tool-augmented LLMs. These candidates are scored using query-specific rubrics and judged LLMs, and we ensure data quality and difficulty through a combination of multi-round filtering and human verification. Our benchmark comprises preference datasets formatted as pairs, triplets, and quadruplets to test ranking capabilities across varying levels of difficulty. Extensive experiments demonstrate that while current reward models handle general coherence effectively, their performance significantly degrades when evaluating research reports. In contrast, generative LLMs acting as judges demonstrate comprehensive superiority in this domain, with comparative evaluation proving more effective than direct scoring methods. We anticipate that this benchmark will serve as a vital testbed for the evaluation and advancement of research agents.

Abstract:
Chest X-ray (CXR) reporting follows a region-based clinical workflow in which radiologists inspect anatomical regions and integrate localized findings into a final report. However, existing resources for CXR report generation provide these supervision signals in fragmented forms. We introduce MMRad-22K, a dataset that organizes regional textual observations, anatomical grounding coordinates, localized image evidence, and report targets into structured multimodal evidence units for CXR report generation. To motivate this formulation, we first compare different evidence formats for report generation and find that structured multimodal evidence is generally more useful than text-only or bounding box-based evidence. We then adapt a unified LVLM backbone using MMRad-22K and show that adaptation with multimodal evidence outperforms both textual-evidence adaptation and end-to-end adaptation on language and clinically oriented metrics. Under the same evaluation protocol, the adapted model also reaches a performance level comparable to several open-source LVLM references. Together, these results support MMRad-22K as a practical structured multimodal resource for training and evaluating CXR report generation aligned with clinical reading workflows. The code is available at https://github.com/qiuzyc/MMRad.

Abstract:
Autonomous web agents increasingly operate over sensitive personal data, yet existing privacy-preserving approaches rely on static filtering and globally enforced safeguards misaligned with compositional agent execution. We introduce PrivAgentFlow, a memory-guided agentic workflow framework that formulates privacy preservation as adaptive node-level governance over compositional web-agent workflows. PrivAgentFlow decomposes complex tasks into compositional workflow nodes, where each node operates under a distinct privacy policy conditioned on its execution role and information access requirements, enabling privacy governance to be localized at the execution level rather than globally enforced. Such localized governance is refined through execution memory, where historical task failures and privacy violations serve as corrective signals for improving node-level privacy control without exposing raw user data during planning. We provide theoretical insights showing that compositional web-agent workflows induce subtask-specific privacy–utility trade-offs, formally characterizing the advantage of heterogeneous node-level governance over uniform privacy control. Across 246 web-agent tasks on three platforms, PrivAgentFlow improves task utility by 23.1 percentage points (0.415→0.646), cuts environment-level privacy leakage by 92.3% (0.366→0.028), and reduces API-level leakage from 100% to 3.7% over the strongest prior baseline. Our code is available at https://github.com/haorentang/PrivAgentFlow.

Abstract:
Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) systems are increasingly integrated into clinical workflows. However, prompt injection attacks can steer these systems toward clinically unsafe or misleading outputs. We introduce the Medical Prompt Injection Benchmark (MPIB), a dataset-and-benchmark suite for evaluating clinical safety under both direct prompt injection and indirect, RAG-mediated injection across clinically grounded tasks. MPIB emphasizes outcome-level risk via the Clinical Harm Event Rate (CHER), which measures high-severity clinical harm events under a clinically grounded taxonomy, and reports CHER alongside ASR 2 to distinguish moderate-or-worse from high-severity outcomes. The benchmark comprises 9,697 curated instances constructed through multi-stage quality gates and clinical safety linting. Evaluating MPIB across a diverse set of baseline LLMs and defense configurations, we find that ASR 2 and CHER 3 can diverge substantially, and that observed rates vary depending on whether adversarial instructions appear in the user query or in retrieved context. The MPIB dataset and evaluation code are available on Hugging Face and GitHub, respectively.

Abstract:
Backtesting LLMs on resolved events assumes models reason only from pre-cutoff knowledge, yet pretrained models inevitably leak post-cutoff knowledge. We introduce a claim-level evaluation framework that decomposes prediction rationales into atomic claims and applies Shapley values to quantify each claim’s decision impact, yielding Shapley-DCLR ( Shapley -weighted D ecision- C ritical L eakage R ate)—an interpretable metric measuring what fraction of decision-driving reasoning is contaminated. We further propose TimeSPEC ( Time - S upervised P rediction with E xtracted C laims), an inference-time architecture that interleaves temporally-filtered retrieval with claim-level supervision, producing predictions grounded entirely in pre-cutoff evidence. Across three LLMs, the ablation experiments confirm retrieval and supervision are jointly necessary; and a three-task probe further illustrates that the performance cost of temporal enforcement scales with each task’s reliance on post-cutoff information.

Abstract:
ReAct-style agents for search-intensive, multi-step reasoning tasks rely largely on their own internal judgment to decide what evidence to seek, which reasoning or action step to take next, and when to stop, often producing shallow, redundant, or poorly targeted trajectories. Prior work has explored rubrics as external quality signals, but existing uses are mostly evaluative rather than action-guiding: rubrics typically serve as training-time rewards or post-hoc evaluators of completed outputs, and in deep-research settings they are often coarse-grained and report-level rather than step-level. We introduce Co-ReAct, a rubric-guided action-selection framework that uses rubrics as step-level guidance during inference. At each decision step, Co-ReAct injects a rubric into the agent’s context to guide the next Reason-or-Act decision, specifying what the agent should target in evidence seeking, search, reasoning, or self-evaluation. To make this guidance reliable, we train a dedicated rubric generator with GRPO. Unlike prior pairwise or binary preference formulations, our objective optimizes a list-wise Spearman rank-correlation reward against multi-judge expert consensus rankings, encouraging rubrics that are discriminative rather than merely plausible. On DeepResearchBench and SQA-CS-V2, Co-ReAct consistently improves over ReAct and representative test-time compute baselines across search agents built on both 8B/14B open-source and frontier closed-source base models. The trained rubric generator can also serve as a drop-in component that improves these baselines without changing their underlying decision mechanisms.

Abstract:
We introduce QatarGuard, an Arabic safety evaluation dataset containing 3,289 human curated prompts across eleven categories: Self-Harm, Violence, Child Exploitation, Vulgar Language, Radicalization, Sexual Language, Racist Content, Regulated Goods and Illegal Activities, Education, Family, and Health. We evaluate how safely five large language models respond to prompts: 299 from the Self-Harm category and 450 from three additional categories: Radicalization, Sexual Language, and Violence. The models tested include four commercial models (Claude 4.5 Sonnet, GPT-4.1, GPT-3.5 Turbo, and Google Gemini 2.5 Flash Lite) and one open-source Arabic-centric model (Fanar-2-27B-Instruct). We use GPT-5 as an automated judge, supported by manual review, to identify failure modes specific to Qatari Arabic dialect. For Self-Harm, unsafe response rates ranged from 2.34% (Gemini 2.5 Flash Lite) to 6.02% (GPT-3.5 Turbo), with Fanar-2-27B-Instruct at 4.35%. Across the other three categories, results varied more widely, from 2.91% to 13.48%. Sexual Language was the most challenging category, where even the best-performing model gave unsafe responses to 5.3% of prompts. Our findings show that Arabic dialect influences model behavior in safety-sensitive contexts, and that overall safety scores can hide weaknesses in specific categories. This highlights the need for safety evaluations that account for both dialect and category. The dataset is available for non-commercial research upon request, and evaluation scripts will be publicly released upon acceptance.

Abstract:
We present ArabicDialectSafety, a human-curated Arabic safety dataset of 25,071 prompts covering six Arabic varieties—Modern Standard Arabic, Syrian, Egyptian, Algerian, Palestinian, and Moroccan—annotated with dialect labels and seven fine-grained harm categories. We introduce a dual-task evaluation framework for binary safe/unsafe detection and granular harm classification across dialects. Benchmarking seven supervised and generative models, we find that fine-tuned MARBERTv2 achieves the strongest performance (0.95 binary and 0.90 granular Macro-F1), substantially outperforming prompted frontier LLMs, including Arabic-specialized models. Our analyses show that dialect conditioning is most effective when integrated at the representation level, while significant performance gaps remain for low-resource Maghrebi dialects. We further evaluate seven frontier LLMs as response generators on harmful dialectal Arabic prompts and observe unsafe generation rates below 5% across models. We release the dataset and code upon acceptance to support future research on dialect-aware Arabic safety evaluation. Warning: this paper contains examples of harmful and potentially offensive content included solely for research purposes.

Abstract:
Most multi-agent systems rely on fairly fixed collaboration patterns: roles, interaction paths, and task decomposition are set in advance. This works poorly for open-ended, literature-grounded research synthesis, where uncertainty changes what needs to be explored, challenged, and verified. We introduce DyCo, a framework for dynamic cognitive role coordination that combines cognitive role partitioning, dynamic teaming, and Exploration–Verification Alternation (EVA) to balance idea generation with validation. We use the Myers–Briggs Type Indicator (MBTI) only as a structured way to specify role priors under controlled conditions, not as a claim about human personality. On DeepResearch Bench, a diverse DyCo configuration reaches a RACE score of 42.75, outperforming the strongest multi-agent baseline by 4.6% ( p<0.01 , d=0.42 ), and a budget-matched non-MBTI control shows that this gain is not explained by additional computation. Extended matched-role controls on all 50 evaluation tasks indicate that most of the improvement comes from rich, structured role priors combined with verification-oriented coordination rather than the MBTI taxonomy itself, and the diverse configuration is not significantly better than the strongest homogeneous one. We further present DyCo-Efficient, which removes dynamic teaming and preserves most of the observed quality at roughly one thirteenth of the token cost, positioning Full DyCo as an optional high-budget configuration.

Abstract:
Researchers increasingly use human-derived bias constructs to study Large Language Models (LLMs), including social-cognitive constructs such as implicit bias and stereotype activation, and cognitive biases such as anchoring, framing effects, and confirmation bias. Such approaches offer alternatives to overt bias probes, particularly when direct questioning may obscure bias or when model behaviour appears normatively acceptable. However, adapting human bias constructs to LLMs introduces an inferential gap. Psychological instruments were developed to study human cognition and social behaviour, whereas LLM evaluations rely on probabilities, text completions, rankings, or simulated decisions. This paper critiques human-centered bias evaluation in LLMs. We show how this gap arises from mismatches pertaining to human-derived constructs, human-model differences, and evaluation contexts, which can blur distinct interpretations of model bias. We then introduce a framework providing an analytical lens for relating these elements to warranted interpretations, with attention to target constructs, operationalizations, scope of inference, and limits of human analogy.

Abstract:
Coreference resolution is an important task in contextual reasoning. In this paper, we investigate the mechanism for representing and retrieving singular and plural entities for plural reference. We use a combination of mechanistic interpretability and attention pattern analysis to study the process in which LLMs predict a pronoun to refer back to previously mentioned entities. Using a range of causal intervention techniques, we find a set of attention heads that are responsible for (1) representing coreference information in the input, (2) identifying entities that form a plural reference, (3) transferring the information to the component that is responsible for selecting the antecedents and predicting the pronoun. We also find that LLMs align with humans in preference for plural pronoun. Specifically, entities in a plural construction are more likely to be referred to as a plural entity if they are ontologically similar and are linked by the conjunction and.

Abstract:
Sparse Mixture-of-Experts (MoE) is a powerful paradigm for scaling Neural Machine Translation (NMT) and managing language heterogeneity. However, practical limitations hinder its deployment on resource-constrained devices: the entire expert pool must remain in memory, and the per-token compute budget is statically fixed during training by rigid top- k routing. We introduce EDWARD-NMT (Elastic Dimensions With Adaptive Routing and Dual-consistency for NMT), a novel MoE encoder-decoder architecture designed for a “train once, deploy anywhere” paradigm that exposes a family of deployable sub-networks at inference time. EDWARD-NMT integrates three key innovations: (i) Matryoshka-style elastic dimensions to flexibly reduce the memory footprint; (ii) an adaptive sigmoid router with a learned, task-conditioned threshold, allowing each token to dynamically determine its active expert count; and (iii) a dual-consistency objective to maximize the performance of the extracted sub-networks. Extensive experiments on PhoMT (En ↔ Vi), WMT-14 (En ↔ De), multilingual OPUS-16, and a multi-domain De ↔ En benchmark validate our approach. The full-capacity architecture matches or exceeds strong MoE baselines with improved parameter efficiency. Furthermore, the sub-networks seamlessly extracted from a single checkpoint consistently outperform comparably sized dense Transformers without any retraining.

Abstract:
Large language models (LLMs) still struggle with strategic persuasion, largely because existing approaches either neglect information asymmetry or rely on unrealistic pre-commitment assumptions. We introduce a type-induced commitment-communication mechanism that grounds Bayesian Persuasion (BP) in natural language dialogue without pre-commitment: the persuader narrates their potential types (e.g., honest vs. dishonest) to dynamically construct an information schema, enabling the persuadee to perform Bayesian belief updates within the conversation itself. We implement two variants: Semi-Formal-Natural-Language (SFNL) and Fully-Natural-Language (FNL), evaluating them against strong baselines across multiple LLMs and human judges. BP strategies consistently outperform baselines: SFNL excels in logical credibility, while FNL shows superior robustness and emotional resonance. We verify that gains stem from genuine Bayesian reasoning rather than superficial formatting, and we further show that supervised fine-tuning enables small models to match the persuasive performance of much larger ones.

Abstract:
In clinical tabular prediction, classical machine learning models with feature engineering often outperform neural methods. LLMs are increasingly used to automate this process, acting as domain experts that propose diverse feature transformations to boost downstream performance. However, the feature generation process of existing LLM-based methods is agnostic to the downstream learner: the LLM receives no signal about which features currently drive predictions or where the model’s representational capacity falls short, so proposals are neither targeted to promising regions of the feature space nor tailored to the learner’s inductive bias. This shortcoming is amplified in healthcare data, which simultaneously exhibits class imbalance, heterogeneous feature spaces, and strict interpretability requirements. In this paper, we propose MedFeat, the first feature engineering framework inspired by the workflow of machine learning practitioners, leveraging model-awareness and feature importance signals to iteratively guide feature discovery for clinical tabular learning. We evaluate MedFeat on a broad range of challenging real-world clinical tasks and show that it statistically significantly outperforms state-of-the-art baselines, with an average F1 improvement of more than 10% over the baseline across models with distinct inductive biases.

Abstract:
Recent advancements in Large Language Models (LLMs) have expanded context windows to million-token scales, yet benchmarks for evaluating memory remain limited to short-session synthetic dialogues. We introduce MemoryCD, the first large-scale, user-centric, cross-domain memory benchmark derived from lifelong real-world behaviors in the Amazon Review dataset. Unlike existing memory datasets that rely on scripted personas to generate synthetic user data, MemoryCD tracks authentic user interactions across years and multiple domains. We construct a multi-faceted long-context memory evaluation pipeline of 14 state-of-the-art LLM base models with 6 memory method baselines on 4 distinct personalization tasks over 12 diverse domains to evaluate an agent’s ability to simulate real user behaviors in both single and cross-domain settings. Our analysis reveals that existing memory methods are far from user satisfaction in various domains, offering the first testbed for cross-domain life-long personalization evaluation.

Abstract:
Personalized value alignment has become increasingly important as large language models (LLMs) are expected to accommodate diverse user preferences. However, existing methods typically align model outputs with a static value profile across prompts, overlooking that the salience of value dimensions varies substantially across contexts. Inspired by Lewin’s Field Theory, which views human behavior as jointly shaped by personal dispositions and situational constraints, we model personal values as priors and context-dependent preferences as posteriors. We propose BaCVA, an inference-time Bayesian Context-aware personalized Value Alignment method that approximates posterior personalized preferences by integrating static personal values with scenario-specific value salience. BaCVA first estimates contextual value salience from generally normative responses, and then employs a dual-view personalization module to infer posterior preferences from complementary personal-value and scenario-driven perspectives. This Bayesian formulation enables more accurate and adaptive personalized value alignment while improving data efficiency via prior values. Extensive experiments on benchmarks demonstrate its superiority over strong baselines.

Abstract:
Disasters are inevitable and increasingly costly, and effective response depends on querying structured tabular data: precise, information-dense records of hazard, exposure, vulnerability, and lifeline infrastructure that underpin disaster management. Current text-to-SQL methods enable natural-language access to such tables but transfer poorly to the disaster domain, where queries span heterogeneous geospatial schemas and require reasoning over causal relations. We introduce DisasterLex, a knowledge-graph-mediated framework that inserts an Expert Knowledge Graph (EKG) of curated concepts and typed causal edges between the user query and the database, bridged to schema by concept-to-table links. The orchestration runs four stages (identifying query entities, routing to the operational domain, planning over causal edges, and grounding the SQL), restricting the schema passed to the model at each step. We instantiate it on a disaster-analytics database (36 geospatial tables, 150 columns) with an EKG of 107 concepts, 117 causal edges, and 52 concept-to-schema links, evaluated on a 75-query test set. On all seven base models spanning proprietary and open-weight families, DisasterLex beats four state-of-the-art baselines (LightRAG, HippoRAG 2, ReFoRCE, CHESS) by 1.4 × to 2.75 × , with absolute scores of 1.65 to 3.56 (of 5.0). Error analysis shows baseline failures cluster in routing and multi-table SQL composition, the operations our orchestration explicitly addresses. Code, data, and the EKG artifact are available at https://anonymous.4open.science/r/DisasterLex-3D64.

Abstract:
Effective reward design is a central challenge in Reinforcement Learning (RL) for code generation. Mainstream test-suite-level outcome rewards enforce functional correctness but induce sparsity, while external Reward Models (RMs) provide dense supervision at the cost of misalignment and additional overhead. Since code evaluation naturally yields multiple test-case-level outcomes, partial success—passing a subset of test cases—offers an intrinsic, verifiable source of dense supervision. In this paper, we propose VeRPO ( V erifiable D e nse R eward P olicy O ptimization), an RL framework that systematically turns verifiable partial success into reliable dense rewards. We analyze partial-success rewards using a weighted sum formulation, theoretically identifying a critical cardinality bias that causes policy updates to disproportionately favor gains from easy-test successes over progress on frontier tests. Based on this, VeRPO introduces a dynamic, density-calibrated local reward that explicitly corrects this bias and provides robust dense supervision from partial success. To enhance alignment with end-to-end functional correctness, VeRPO further integrates the local dense reward with global execution outcomes. Extensive experiments across diverse benchmarks and settings demonstrate that VeRPO outperforms outcome-driven and RM-based baselines, achieving up to +8.83 pass@1 gain with negligible time cost (< 0.02%) and zero GPU memory overhead.

Abstract:
Humanizing methods of AI-generated texts are emerging, which leads to severe performance degradation of current AI text detectors. Most existing detectors are struggling to ensure consistent performance in the huge span from detecting simple AI texts to detecting AI texts humanized in various ways. Targetedly, we propose HiDet, which contains coarse module and subdivision module to give AI texts a double check. We decouple the complete detection process into simple sample detection and difficult sample detection, the coarse module of HiDet filters out simple samples, while hard-to-detect samples like humanized AI texts will be carefully discriminated through the subdivision module which applies a multi-grained contrastive learning strategy. This hierarchical framework makes up for the loophole that humanized AI texts can successfully escape the traditional detector after a single detection, and shows excellent robustness in the task of detecting humanized AI texts. Meanwhile, our framework is flexiable, the subdivision module can be deployed separately on the existing detector as a plug-and-play patch to tremendously improve their performance when facing large-scale humanized AI texts. We hope our work can inspire new sparks in the field of AI-generated text detection.

Abstract:
LLM agents rerun full reasoning for every task, even one they solved moments earlier. We introduce APEX-EM, a non-parametric experience memory that stores complete procedural-episodic traces in a typed Procedural Knowledge Graph (PKG) and retrieves them through three channels: semantic search, structural-signature matching over abstract operation sequences, and graph traversal. A Plan-Retrieve-Generate-Iterate-Ingest (PRGII) workflow produces, quality-gates, and commits experiences, indexing both successes and failures so the agent learns what to reuse and what to avoid. No weights change during deployment. We evaluate on five benchmarks: BigCodeBench, KGQAGen-10k, HLE, Lifelong Agent Bench, and ALFWorld. Because prior work uses different backbones, we base our claims on same-backbone comparisons that hold model capability fixed. On held-out BigCodeBench transfer with a shared GPT-4o backbone, APEX-EM gains +7.6pp over the no-memory baseline, which is 3.3x MemRL’s +2.3pp under the identical setup. On Lifelong Agent Bench with a shared GPT-4o-mini backbone, APEX-EM gains +1.4pp (OS) and +1.0pp (DB) cumulative success. On KGQAGen-10k, frozen memory transfers to a blind 1 , 079-question test split at 73.7% versus 42.0% with no memory, approaching an oracle that is handed the ground-truth subgraph (84.9%). Across three Opus scales the memory gain stays at +27pp to +32pp, so it adds to model capability rather than substituting for it. Component analysis shows no single mechanism dominates: teacher feedback is negligible for code but adds +10.3pp on structured queries, structural signatures give 3.3 times the transfer of semantic-only retrieval, and within-epoch iteration recovers most of the gain when rich feedback is unavailable. These results argue for modular memory composed per domain.

Abstract:
While Large Language Models (LLMs) can now automate many parts of scientific writing, architectural figure generation faces a critical modality bottleneck because image outputs are inherently difficult to edit, verify, and integrate into text-native LLM workflows. To enable fully text-driven synthesis, we propose a new task: generating architectural figures as pure HTML/SVG source code. Furthermore, we introduce FigureCode, a novel benchmark that evaluates structural fidelity and visual properties through code-based evaluation. Our benchmark improves agreement with human judgments by 0.35 Kendall’s τ over prior VLM-based methods. Moreover, the figure-as-code paradigm improves edit-instruction success rate by 30% and retrieval F1 score by 25.3% over baselines, while achieving 66.1% WCAG/ARIA accessibility coverage.

Abstract:
Function routing—selecting the correct API call from a fixed catalog given a natural-language request — is a deployment problem where small students are attractive but knowledge distillation gains are typically reported single-seed, at scales where seed variance is unknown. On a 740-instance healthcare API routing task with a 1.5B Qwen student and a 20B teacher, we compare eight KD variants against supervised cross-entropy, using three to six seeds for key configurations. We find: (i) per-seed standard deviation ranges from 2.8 to 48.7 percentage points, swallowing every claimed KD gain below five points; (ii) three of seven KD variants exhibit bimodal collapse, with at least one in three to five seeds falling below 55% accuracy while the others train normally, and a fourth showing elevated variance; (iii) collapse has distinct modes — wrong-function selection for ce_Knowledge Distillation and ce_paraphrase, and a previously undocumented output-truncation mode for reasoning_kd, where the model emits reasoning but terminates before producing a function name (0.9% accuracy); (iv) only progressive_kd and rank_kd avoid collapse across observed seeds, with σ ≤ 3.9 pp; (v) a naive cross-split +3.78 pp gain from input enrichment reverses to −2.70 pp under controlled within-split multi-seed re-testing. Single-seed evaluation is therefore unable to detect central failure modes in small-model KD.

Abstract:
Multimodal large language models show significant potential in mathematical reasoning, yet they remain constrained by a scarcity of high-quality training data for visually grounded problems. Existing approaches commonly suffer from narrow coverage, image-text inconsistency, and heavy imitation of seed problems. To address these limitations, this paper proposes AgSynth, a multi-agent collaborative reverse synthesis framework. It first constructs solution logic programmatically and renders the corresponding images, then generates problem statements backward from the visual facts, thereby embedding answer correctness and image-text consistency directly into the generation pipeline. Meanwhile, AgSynth distills the core competencies tested in seed problems and performs divergent reconstruction, enabling synthesized problems to be substantially novel in both structure and visual presentation. Based on this framework, we construct the AgSynth-10K dataset, comprising 10,477 multimodal mathematical reasoning problems that have undergone two-tier quality control, spanning four major domains: geometry, algebra, combinatorics, and number theory. Experiments demonstrate that reinforcement learning training on AgSynth-10K significantly improves model performance on reasoning-intensive benchmarks (WeMath +7.42 , LogicVista +9.15 ), validating the effectiveness of this traceable synthesis paradigm. The framework is publicly available at https://github.com/Yaxuan7690/AgSynth.

Abstract:
As large language models (LLMs) are adopted worldwide, evaluating their alignment across diverse cultural contexts is essential, especially when handling sensitive, politically charged requests. Yet what counts as “sensitive” is shaped by social, historical, and cultural factors, leaving model robustness across cultures poorly understood. We investigate this in the context of Portugal, a country whose layered history surfaces many culturally sensitive topics absent from existing benchmarks. We introduce Lusopol, a benchmark of 1,400 English and Portuguese prompts spanning seven thematic categories—from colonial history and the Estado Novo dictatorship to contemporary political issues—designed to stress-test models under deliberately controversial framings. Testing 16 state-of-the-art LLMs on Lusopol, we find substantial performance variance: frontier closed models and leading open-weight families set the pace, while European-focused open models lag behind. To close this gap, we synthesize a preference corpus of more than 7.5K samples. Targeted alignment training on this resource significantly reduces skewed responses and makes European open models competitive, demonstrating that culturally situated preference data is a practical path toward closing the alignment gap for underrepresented societies.

Abstract:
While token-level entropy is commonly recognized as effective for credit assignment in text-only reinforcement learning with verifiable rewards (RLVR), it remains unclear whether this mechanism still holds in visual reasoning. Our controlled study shows that this mechanism collapses in visual reasoning due to the omission of vision-sensitive tokens with naturally low entropy. Although existing multimodal RL methods increasingly acknowledge the importance of visual perception, they struggle to satisfy the inherent demand for interleaving precise perceptual grounding with semantic reasoning, either lacking systematic visual measurements or overlooking that token entropy primarily drives semantic exploration. To address this, we introduce VEPO (Vision-Entropy token-selection for Policy Optimization), an effective RL framework explicitly integrating visual sensitivity with token entropy via a principled multiplicative coupling, where VEPO redirects gradient credit toward tokens which are simultaneously visually grounded and highly informative. Extensive experiments demonstrate VEPO’s leading performance, consistently outperforming the strongest RL baseline at 7B-scale and 3B-scale. Ablations further substantiate the soundness of our method.

Abstract:
Medical large language models are typically evaluated on idealized patient cases that do not reflect how real patients communicate. We introduce VeriSim, a patient simulation framework that injects controllable noise along six clinically grounded communication dimensions while substantially preserving each patient’s medical record. Truth adherence is supported by a verifier that extracts atomic claims from each candidate utterance and judges them against a UMLS-grounded vector index built with BioLORD embeddings, using the retrieved atoms’ structured clinical metadata (e.g., drug class, anatomical site, treats-condition relations) rather than surface-text similarity alone. Across seven open-weight LLMs, realistic noise reduces diagnostic accuracy by 15-25 percentage points and increases conversation length by 34-55%; the 7-8B models degrade 1.4x more than 70B+ models. A board-certified physician and a licensed nurse rate VeriSim’s conversations highly on truth, realism, clinical utility, and noise fidelity (inter-annotator agreement >=0.80 across all dimensions), and an LLM-as-judge closely tracks their ratings, enabling scalable evaluation.

Abstract:
Assamese is a highly agglutinative language that uses suffix sequences to convey various kinds of information. A single morpheme may exhibit multiple syntactic or semantic functions depending on its position in a word and the preceding morpheme. Such ambiguities and the inadequacy of supervised data make morphological analysis difficult for Assamese. Our approach is based on the observation that a word has a limited set of valid morphological analyses, among which one is contextually the most appropriate. We curate a morphological lexicon, a bigram transition rule set, and several derivational rules, which our rule-based analyzer uses to generate valid analyses of a word by segmenting its surface form. The rule-based analyzer facilitates context-based annotation of over 11.8K tokens across more than 1K sentences. We construct morphological feature vectors from each analysis of a word and train a neural model to discriminate between correct and incorrect sequences of morphological vectors. The trained model is evaluated on the task of assigning the highest score to the correct sequence of morphological vectors among all the candidate sequences. The model achieves a word-level disambiguation accuracy of 80.76% by resolving 1045 out of 1294 ambiguities on the test set without using lexical semantics. The results indicate regularity in sentence-level morphological patterns. The annotated dataset and other resources are available at https://github.com/Pragyanur/morphological-analysis-assamese.

Abstract:
Existing computer-use agent benchmarks do not fully evaluate agents acting as assistants. A useful assistant retrieves information across complex, multi-step workflows, synthesizes it into artifacts (documents, presentations, spreadsheets), and navigates program interfaces to produce a coherent final product. Such workflows demand reasoning and synthesis, decomposition of complex tasks, as well as visual and spatial understanding. To study agents on workflows like these, we introduce KNOWS, a benchmark of open-ended, complex, browser-based tasks that jointly evaluate these capabilities, with each task culminating in a produced artifact. To write tasks, we develop a task design rubric and a protocol for ensuring that tasks meet the requirements. Each task is paired with an evaluator, a program that combines deterministic checks with LLM judgments to balance the richness, reliability, and automation tradeoff inherent to agent evaluation. We evaluate and analyze frontier computer-use agents and browser-based harnesses. They achieve moderate scores on partial-success metrics, but the best performer fully succeeds in fewer than 3% of our complex, long-horizon tasks. Failures on visual steps render the resulting artifacts unusable, even when agents complete more than 50% of other evaluation steps. Our results expose limitations of current agents acting as end-to-end assistants, and call for progress on tool use, visual understanding, and long-horizon reasoning.

Abstract:
End-to-end speech parsing, a task recently proposed, consists in predicting both the transcription and the syntactic tree for a spoken utterance. Existing architectures for speech parsing often utilise intermediate neural networks. In this work, we examine the effectiveness of intermediate neural networks (NN) for parsing, and, specifically, what role do they play. We introduce a simpler end-to-end architecture for speech parsing, where we remove these intermediate NN units, reducing the parameters by 12%, while achieving comparable or better performance than prior method on both automatic speech recognition (ASR) and parsing. We demonstrate that intermediate NN units help reduce the representational gap when the pre-trained encoder is frozen. We do a comprehensive evaluation of speech parsing on French, and medium-low resource languages Slovenian and Naija. We further investigate the impact of the training data size and intermediate layers of the pretrained speech encoder on speech parsing.

Abstract:
In regulated industries like consumer finance, seemingly harmless user queries can exploit large language model vulnerabilities, triggering safety failures and pushing responses dangerously close to policy limits. Existing automated red-teaming methods trade off attack effectiveness against generation cost, while treating coverage, severity, and diversity as incidental rather than joint objectives. We introduce FinRT, a structured framework that builds reusable adversarial prompt generators from adaptive red-teaming strategies. Across the six victim models in consumer finance, FinRT substantially outperforms adaptive search baselines while amortizing target-facing attack generation into a reusable generator. FinRT nearly doubles the attack success rate over the adaptive baseline Rainbow Teaming (Samvelyan et al., 2024) (32.9% vs. 17.2%), increases maximum adversarial severity by 33%, and preserves comparable intra-policy–domain semantic diversity to iterative search methods. Our method achieves high cross-model transferability while exhibiting distinct victim-family specialization patterns.

Abstract:
Language models learn semantic similarity from shared context; however, when a single word has multiple orthographies that rarely appear in the same context, models cannot learn their equivalence. This is a structural problem that cannot be solved with scale. Korean presents a clear case: 20th-century print media used Hanja (Sino-Korean script), while contemporary models train on Hangul-only (Korean alphabet) text. Multilingual models inherit script segregation from their training distribution, where Hanja appears almost exclusively outside Korean text, and monolingual Korean models assign separate vocabulary entries to script variants, forcing distinct representations from initialization. We introduce Han2Han, a 169M parameter encoder-decoder model trained with a language-aware recipe—jamo and character-level embedding fusion, morpheme-aware denoising, and bidirectional Hanja-Hangul transcription—that forces orthographic alignment as a prior to distributional learning. The resulting representations are script-invariant: the same content receives the same representation regardless of script. On temporal classification of mixed-script historical newspapers, evaluated on an article-level held-out split, Han2Han reaches 76.0 macro F1 against 54.3 for the strongest baselines (KLUE-BERT and the 4.7 × larger T5Gemma 2 alike), a 21.7-point margin, demonstrating that language-specific pre-training yields capabilities not addressed by current Korean-inclusive multilingual training at this scale.

Abstract:
Recent work has begun to equip vision-language-action (VLA) policies with explicit intermediate reasoning. In embodied control, however, textual chain-of-thought is a poor fit: irrelevant or weakly textual information can interfere with action prediction, while autoregressive text decoding adds too much latency for real-time closed-loop execution. We present VISUALTHINK-VLA , a visual intermediate-reasoning framework for accurate, low-latency VLA policies. We define visual intermediate reasoning operationally as constructing typed visual evidence, selecting a task-dependent budgeted route, and composing the selected channels into compact states before action decoding. Our bootstrapping philosophy is to guide action with effective visual thinking : VISUALTHINK-VLA bootstraps action prediction through a compact visual-evidence interface that preserves spatial precision and supports low-overhead decoding. Besides, to further improve performance and efficiency, VISUALTHINK-VLA adopts a tailored selective routing mechanism to learn the visual evidence tokens, enabling low-latency inference while preserving high-capacity specialization. We also introduce VisualEvidence-Kit , a supervision-and-audit resource centered on a VisualEvidence-Agent that constructs 754.7k VLA instructions in the VisualEvidence-Set for route supervision and counterfactual faithfulness tests. Across multiple benchmarks and real-robot evaluation, VISUALTHINK-VLA achieves the highest success rate on most benchmarks while reducing the multi-second latency of reasoning-augmented baselines to the sub-second regime. On BridgeData V2, policy-side latency falls from 8.377,s with ECoT to 0.367,s, achieving a 22.8× speedup .

Abstract:
Large audio-language models (LALMs) have shown promising progress in understanding speech, music, and general sound events, yet their ability to reason about how audio signals are degraded remains underexplored. Existing benchmarks primarily evaluate semantic understanding, event recognition, or high-level audio reasoning, leaving a basic question unanswered: Do LALMs understand the differences in audio quality? We introduce MRMAD, a Multi-Round Multi-Audio Degradation benchmark for evaluating audio degradation perception and understanding in LALMs. MRMAD spans speech, music, and sound, and frames evaluation as multi-turn dialogues across multiple audio inputs, requiring models to identify types of degradation, compare severity, and perceive corruption changes across turns. Unlike current single-turn audio-language benchmarks, MRMAD evaluates whether LALMs can maintain consistent degradation hypotheses with new evidence and comprehend low-level acoustic phenomena over multi-turn dialogues. Through a systematic evaluation of 18 representative LALMs from non-thinking to reasoning and Omni models, we find that current models often recognize coarse content while failing to diagnose, compare, or reason about degradations reliably. Human evaluations further reveal a significant perception gap between LALMs and human listeners. MRMAD thus exposes a critical yet overlooked aspect of audio-language understanding and provides a diagnostic foundation for building future LALMs that are robust to real-world acoustic conditions.

Abstract:
Zeolite synthesis requires a structured recipe that specifies an organic structure-directing agent (OSDA), gel composition, crystallization temperature, and reaction time. General-purpose language models can draft such recipes, but their outputs often fail domain checks, producing invalid SMILES strings, missing condition fields, or temperature–time schedules inconsistent with local Arrhenius trends. We present ZeoAgent, a closed-loop compiler for machine-checkable zeolite synthesis recipes. ZeoAgent represents each generated recipe as a typed record and verifies it through deterministic field-level checks. At each iteration, a compact yet diverse set of retrieved synthesis precedents guides the next draft. When a field fails validation, the corresponding error code is returned to the model for targeted revision. Recipes passing these checks are calibrated with a local Arrhenius fit before final ranking. In condition-only experiments, the recorded OSDA is supplied and only the hydrothermal-condition fields are predicted. On the archived model outputs, ZeoAgent attains 99.7% RDKit parseability, compared with 81.7% for Qwen3-8B. Typed feedback raises repair pass@5 from 68.3% to 72.0%, and PGC reduces high-residual schedules under its fitted local Arrhenius self-consistency diagnostic from 77.0% to 5.4%.

Abstract:
Retrieval and reading in Table QA are typically optimized in isolation, leaving no principled way to balance retrieval precision against reader capacity. We ask: how much retrieval depth does a reader actually need? Across three benchmarks (E2E-WTQ, OTT-QA, and FeTaQA) and four LLMs, we conduct a controlled single-table stress test that guarantees the inclusion of the gold table while increasing retrieval depth to K = 50 using soft or hard distractors. This design isolates reader noise tolerance from retrieval failure. We find that the Noise Ceiling is set jointly by task type, reader, and distractor condition, and is not predictable from reader class alone. With hard negatives, every reader has a Noise Ceiling of K ≤ 5 for long-form answer generation, favoring precision-oriented retrieval. By contrast, several readers remain robust through K = 50 on short-answer and multi-hop QA, depending on the distractor condition. A reader-side Cost per Success analysis further shows that the trade-off between robustness and inference cost varies across readers and distractor conditions. Overall, these results establish the reader’s condition-specific Noise Ceiling as a concrete operating target for balancing retrieval depth against precision in controlled single-table QA.

Abstract:
Although large language model agents are increasingly applied to quantitative-finance workflows, their evaluation remains fragmented across isolated tasks, while the financial relevance of benchmark tasks is often overlooked. Yet financial workflows are inherently multi-stage, spanning interdependent tasks such as forecasting, strategy construction, risk management, and trading. Existing platforms typically focus on a single task, and can therefore overstate agent competence and fail to reveal weaknesses in generalization, real-market interaction, and financially meaningful decision-making. We introduce OpenFinGym, a unified gym environment for quantitative-finance agent development that covers forecasting, market generation, real-time trading, and fraud detection under a single execution and verification interface. OpenFinGym additionally provides an automated task-construction pipeline that turns quantitative finance publications into executable task packages; a containerised runtime with a host-side verifier service that supports scalable agent rollouts and prevents runtime train–test leakage; a paper trading engine with a low-latency data-stream design; deferred-resolution support for long-horizon and event-market forecasts; and integration for SFT and RL post-training. Code is available at https://github.com/DeepIntoStreams/OpenFinGym.git

Abstract:
Open coding agents are commonly trained through supervised fine-tuning on interaction trajectories whose final patches pass the benchmark test suite. However, this outcome-level filtering criterion ignores process quality: trajectories that eventually succeed may still contain inefficient interaction patterns such as repetitive file inspection, unstable tool usage, or delayed modification behavior, all of which become part of the imitation target. We propose (Trajectory Quality Score), a structural scoring framework for coding-agent trajectories that operates directly on trajectory structure, requires no LLM judge, and runs in linear time. characterizes trajectory quality through structural signals organized around structured convergence: information focus, intent–action alignment, and policy dynamics. Using 155 , 144 successful trajectories from a public open-license corpus, we rank trajectories by and fine-tune coding agents on filtered subsets. Models trained on top-ranked trajectories consistently outperform those trained on random or bottom-ranked subsets on . Our results suggest that process-level trajectory quality is an important factor in coding-agent supervision and that structural trajectory filtering provides an effective and scalable alternative to LLM-based data selection.

Abstract:
Mental health research on social media has been dominated by English-language datasets and DSM-5-centered resources, limiting cross-lingual and cross-cultural understanding of online mental health expression. We introduce MATCHA, the first Chinese multi-disorder mental health dataset that links disorder-specific Weibo Super-Topic communities with users’ broader public timelines. MATCHA covers seven DSM-5 disorders and a control group, comprising 46,025 cohort-level users and 1.8 million posts. The dataset distinguishes users with explicit self-disclosure evidence from broader disorder-related participants through an LLM-assisted annotation pipeline, and is released in a rigorously de-identified, controlled-access format. Using MATCHA, we analyze behavioral, linguistic, symptom-related, and life-event-related patterns across disorders, and conduct cross-lingual comparisons with Twitter-STMHD. Our analyses show that Weibo Super-Topics function as concentrated spaces for affective and health-related expression, while cross-platform differences between Weibo and Twitter often exceed cross-disorder variation. MATCHA provides a new resource for community-aware and cross-lingual mental health NLP research.

Abstract:
Rubric-based rewards offer interpretable and fine-grained optimization signals for reinforcement learning in open-ended tasks where verifiable answers are unavailable. However, pre-constructed rubrics remain static throughout training, creating a fundamental mismatch with the evolving policy: fixed criteria gradually lose discriminative power as the model improves, leading to reward saturation and potential hacking. Recent dynamic rubric methods partially address this but rely on external frontier models or ground-truth answers, and update rubrics only at coarse granularity. We propose EvoRubrics, a co-evolutionary RL framework where a Policy LLM and a Rubric Generator jointly improve through adversarial interaction within each training step. As the policy improves under the rubric generator’s guidance, the rubric generator adapts its criteria to remain discriminative and informative, enabling evaluation to track the policy in real time and naturally inducing an automatic curriculum. Experiments show that EvoRubrics consistently outperforms static and dynamic rubric baselines across benchmarks. The learned Rubric Generator further generalizes as a transferable reward model. Notably, even a fully self-supervised variant without any external supervision achieves meaningful gains, suggesting that co-evolution between generation and evaluation alone can provide sufficiently rich learning signals. Our code is publicly available at https://github.com/huangbaixiang100/EvoRubrics.

Abstract:
Social-bias benchmarks such as BBQ (Parrish et al., 2022) evaluate whether language models rely on stereotypes in ambiguous contexts where the correct answer indicates insufficient information. We observe that BBQ’s ambiguous-context bias scores aggregate over two distinct phenomena: abstention failure (selecting any specific person despite insufficient evidence) and stereotype direction (favoring the stereotype-labeled option among those unsupported selections). To formalize this distinction, we introduce a decomposition that factors the raw stereotype-consistent choice rate into a non-abstention rate and a conditional stereotype rate—the two quantities that underlie BBQ’s standard score. Across 11 BBQ categories, answerability prompting substantially reduces raw stereotype-consistent choices (0.068 to 0.023), but our decomposition shows that this is driven almost entirely by improved abstention (non-abstention 0.166 to 0.053), while the conditional stereotype rate does not fall consistently and rises in Disability Status and Physical Appearance. Cross-model evaluation on four proprietary and open-weight LLMs confirms the same pattern, and counterfactual surface-cue controls on Age, Disability Status, and a Physical Appearance subset further show that concrete descriptors account for a substantial share of abstention failures while stereotype direction persists even at minimal group labels. These results demonstrate that disentangling abstention failure from stereotype direction yields a more informative, diagnostic view of model behavior under ambiguity.

Abstract:
Large language models (LLMs) have shown strong reasoning abilities in mathematical reasoning, commonsense reasoning, and code generation. However, negotiation requires interactive reasoning, where each action can influence the opponent’s future behavior. Agents must therefore look beyond the current dialogue state and consider the long-term effects of their actions. Existing LLM-based negotiation agents often rely on prompting or fine-tuning from self-play trajectories. These methods can produce fluent responses and help models learn useful bargaining patterns. Yet at decision time, they often remain reactive, responding mainly to the current dialogue without reasoning about how different actions may shape future negotiation paths. We introduce an online strategic reasoning method that combines Monte Carlo Tree Search (MCTS)-based lookahead planning with opponent simulation. Before responding, our agent uses a proxy opponent agent to simulate future interactions and evaluate candidate strategies by their expected long-term outcomes. To make opponent simulation more precise, we proposed a Theory-of-Mind-inspired opponent modeling module that infers latent opponent attributes from dialogue history and conditions the proxy opponent agent during search. Experiments on CraigslistBargain and A2A-NT show that our method improves negotiation efficiency and profitability.

Abstract:
Historical archives are invaluable for multidisciplinary research but remain inaccessible globally due to severe language barriers. While manual translation is prohibitively expensive and time-consuming, simply applying existing machine translation models is inadequate due to unique linguistic nuances and the temporal bias inherent in scarce human-labeled corpora. To address these challenges, we propose HERIT, a systematic domain-adaptation pipeline that leverages retrieval-augmented generation (RAG) to generate high-quality pseudo-labeled data from abundant Hanja-Korean corpora. This approach substantially expands the training dataset, mitigating both data scarcity and temporal overfitting across different historical periods. Extensive automated and human evaluations demonstrate that HERIT consistently outperforms baselines. Finally, we translate previously untranslated portions of the archives using our model, thereby enabling global access to these resources for researchers.

Abstract:
Reinforcement learning has substantially improved large language model (LLM) agents in verifiable domains, but remains difficult to apply to open-ended agent tasks, where solutions are diverse and reliable scalar rewards are hard to obtain. Recent pairwise evaluation methods alleviate reward discrimination collapse by replacing pointwise scoring with relative preferences. However, they still compress rich comparative feedback into a single trajectory-level reward, obscuring decisive intermediate steps and preventing successful behaviors from being consolidated into reusable skills. We propose ArenaFlow, a hierarchical credit propagation framework for open-ended agent reinforcement learning. ArenaFlow leverages tournament-based relative ranking to derive trajectory-level reward signals. Each comparison is further equipped with structured reflective evaluation, which reveals three types of supervision: pivotal success steps, reusable strategy skills, and usage attribution of retrieved skills. At the step level, ArenaFlow propagates trajectory-level advantages to high-confidence pivotal steps according to tournament survival depth, enabling more targeted optimization of local reasoning behaviors. At the skill level, ArenaFlow estimates skill utility from group-level usage attribution and maintains a global skill memory through utility-aware updating, pruning, and retrieval. The resulting high-utility skills further serve as policy priors for future exploration. Extensive experiments validate ArenaFlow’s effectiveness on open-ended agent tasks.

Abstract:
Real-world personally identifiable information (PII) redaction often operates on document images—scans, screenshots, and PDF renderings—where OCR errors, layout structure, and visual noise determine whether sensitive information is actually removed. Existing PII benchmarks are mostly text-centric and do not measure document-level redaction risk: a page remains unsafe if even one identifier is missed. We introduce LeakageBench, a challenge set of 500 document images with 11,954 GDPR-aligned PII annotations spanning direct identifiers, linkage keys, and contextual re-identification surfaces. We evaluate generic OCR pipelines, commercial and task-adapted OCR-dependent detectors, and OCR-free vision-language models using entity-level F1, group-wise leakage, and document-level leakage metrics. Code Interpreter raises GPT-5.5 localization F1 from 0.090 to 0.249, but critical page-level leakage remains 0.968. These results show that stronger detection and tool assistance improve localization without making most pages safe for release. LeakageBench provides a diagnostic benchmark for high-recall, spatially grounded PII redaction in document images.

Abstract:
Large language models (LLMs) are increasingly used to simulate human behavior but frequently fail to exhibit realistic cognitive constraints, suffering from a "superhuman bias." Using a dataset of over 71,000 reading comprehension responses from 2,359 primary-school students (grades 4-6), we demonstrate that standard persona prompting yields near-perfect, deterministic performance, failing to capture the natural variance of developing readers. To address this, we introduce the Cognitively Bounded User Simulator (CBUS), an architectural framework that explicitly models the restricted working memory of young readers through an episodic bottleneck. Within this framework, we formalize two distinct test-taking strategies to emulate different reading behaviors. Our evaluation shows that explicitly modeling cognitive bounds significantly narrows the simulation gap across multiple LLM backbones, demonstrating that enforcing architectural constraints is more effective for high-fidelity simulation than simply scaling raw model capabilities.

Abstract:
Randomized Controlled Trials (RCTs) contain crucial information about the outcomes of clinical experiments. However, the sheer number of RCTs—approximately 595,000 asof September 2023—makes it difficult to efficiently navigate the available evidence. To address this challenge, medical practitioners rely on the PICO framework to identify relevant information and NLP engineers have developed automated PICO extraction pipelines. In this work, we study four PICO-related datasets created from RCTs and commonly used in the literature. We then compare two PICO extraction pipelines: PICOX reproduction and BIOBERT overlaps. Our findings show that (i) PICO-related datasets can be broadly categorized as either span-based or entity-based, with clear differences in how PICO information is annotated, and (ii) while PICOX reproduction is better suited to extracting text spans containing PICO information, BIOBERT overlaps is more suitable for fine-grained and precise PICO entity extraction. Additionally, we evaluate the performance of both models on nested and over- lapping PICO entities, found in the EBM-NLP dataset. BIOBERT overlaps achieves an entity-level micro-F1 score of 30.70 on these entities, compared with 17.58 for PICOX reproduction.

Abstract:
Modern LLMs are alarmingly susceptible to surprisingly simple immaterial changes of input prompts: a casual hint, an incorrectly labeled few-shot example, or a fake prior assistant turn often flips an originally correct answer. We study where this susceptibility, spanning sycophancy and related cue-induced biases, lives inside the model. Across five model families and seven bias types, we extract a per-bias direction from hidden states and triangulate it through three measures: probing, leave-one-dataset-out (LODO) transfer, and causal intervention. The susceptibility is largely shaped by alignment tuning rather than pretraining: pretrained base models generally cave much less to these biases, and their activations carry much weaker cue-specific signal beyond question content. Within aligned models, each bias has a coherent linear direction that we can both decode and steer along, recovering the unbiased answer across every family we test. The biases do not collapse into a single shared representation, however: cross-bias overlap is model-specific, and even behaviorally similar biases occupy different directions. The same intervention also provides a proof-of-concept debiasing tool, recovering a meaningful share of bias-induced errors while preserving most correct answers across all instruct families. Cue-induced bias is therefore best understood not as a single flaw in LLMs but as a family of causally effective linear directions that are largely shaped by alignment tuning.

Abstract:
Financial agents are expected to answer factual investment queries through open search, but financial multi-hop retrieval differs from general multi-hop QA in a crucial way: each intermediate answer is an exact typed constraint, such as an entity, date, event, or numerical indicator, that determines the next query. This strict answer-conditioned dependency makes small intermediate errors cascade into plausible but wrong search trajectories. We introduce FinHopper , a benchmark for complex financial agents with controlled reasoning depth from 1 to 6 hops, covering stocks, funds, and macroeconomic facts, with gold intermediate answers for process-level evaluation. To construct FinHopper at scale, we design a state-transition-based synthesis framework that composes verified atomic financial QA patterns into dependency-preserving chains, followed by targeted in-synthesis verification, model-ensemble checking, and public-search validation by financial experts. Experiments with 20 LLM-agent configurations show that public search tools are necessary but insufficient: performance drops sharply as hop depth increases, the best model remains far below human experts, and process-aware scoring reveals partial reasoning progress hidden by final-answer-only evaluation. The benchmark is available at https://anonymous.4open.science/r/FinHopper-aabb/

Abstract:
The success of large language models in real-world applications relies on parameter-efficient fine-tuning methods such as Low-Rank Adaptation (LoRA), which inserts compact, trainable adapters into selected layers. In this paper, we propose SimplexLoRA, a LoRA variant that adaptively adjusts adapter ranks during fine-tuning based on their learned importance. Unlike standard LoRA, which employs a fixed rank across all layers, SimplexLoRA optimizes per-adapter importance weights on the n-scaled simplex during a short warm-up and then freezes them, dynamically allocating capacity where it is most beneficial without increasing the overall parameter budget or inference cost. We evaluate SimplexLoRA across 12 language understanding, summarization, and reasoning tasks using DeBERTaV3, BART-large, Llama 3.1 8B, and Qwen3-8B. Our results show that SimplexLoRA consistently matches or outperforms standard LoRA and recent adaptive baselines, with the largest gains at low ranks. Our code is available at https://github.com/brain-lab-research/SimplexLoRA.

Abstract:
Two forms of test-time scaling for Large Language Models (LLMs) have emerged as effective and widely adopted paradigms: sequential, in which later answer attempts depend on earlier ones, and parallel, such as i.i.d. sampling with reranking. In this study, we investigate their properties in translation. First, our study shows that sequential sampling has a higher performance ceiling, providing a more diverse and effective pool of samples, particularly under smaller sampling budgets. Second, we interrogate the nature of test-time scaling through a multidimensional manual analysis. Human analysis of the Best-of-N translations demonstrates that sequential sampling substantially improves translation fluency and naturalness, but can degrade accuracy when inference budgets are large. Finally, we suggest an explanation of the mechanism through which sequential scaling improves machine translation. Our controlled analysis partially attributes the success of sequential self-improvement to the model’s access to a larger target-side context. Ablation experiments on sequential sampling demonstrate its robustness across different sampling temperatures, while also revealing sensitivity to context construction, suggesting directions for future improvement.

Abstract:
Recognizing a formal language does not imply the ability to construct the computation that decides it. We operationalize this distinction as an executable benchmark for language-model reasoning: given a formal-language specification, an LLM must synthesize a deterministic or nondeterministic Turing machine that decides the language. We introduce TMEval , a benchmark of 253 language specifications across 7 structural categories spanning context-free and non-context-free phenomena. Each language includes 600 oracle-labeled membership tests, and every output is evaluated as an executable artifact rather than a textual response. Our pipeline enforces a strict output contract, validates machine construction, executes each buildable machine, and assigns one of four outcomes: static_fail , build_fail , semantic_mismatch , or exact_pass . Across eight LLMs, 2,024 model–language candidates, and up to 1,214,400 behavioral evaluations, only 11.12% exactly pass all tests. More than half fail before execution, over one-third implement the wrong language, and 75 languages remain unsolved by all systems. Failure depends less on the context-free/non-context-free boundary than on the operational structure required to realize the specification. TMEval reveals a substantial gap between formal-language recognition and machine-checked symbolic construction. TMEval is available at https://turing-machineeval.github.io/ .

Abstract:
Bangla (Bengali), spoken by over 300 million people worldwide, presents unique tokenization challenges due to its rich morphological structure. Despite its global importance, Bangla remains underrepresented in mainstream NLP, particularly in low-resource tokenization research. Existing tokenizers such as Byte Pair Encoding (BPE), Unigram, WordPiece, and character-level were originally developed for high-resource languages. These tokenizers struggle with morphologically rich languages and show two main limitations: (i) unnecessary splitting of morphemes and (ii) fragmentation of compound words. Based on these limitations, we hypothesize that morphology-aware tokenization is necessary for handling Bangla effectively. To address this, we first introduce a benchmark that systematically evaluates how widely used tokenizers perform on morphology-related tasks. Motivated by these findings, we then propose BANGLATOKENIZER, a morphology-aware tokenizer that alleviates these limitations using a novel bond-strength framework inspired by molecular structures. Empirically, BANGLATOKENIZER yields consistent downstream improvements, achieving absolute accuracy gains of 2.8%-3.4%, macro-F1 gains of 2.5%-3.6%, and perplexity reductions of up to 20% across diverse language models and tasks. These improvements are accompanied by 5%-10% higher inference throughput, showing that morphology-aware tokenization improves both effectiveness and efficiency

Abstract:
When LLMs judge moral dilemmas, do they reach different conclusions in different languages, and if so, why? Two factors could drive such differences: the language of the dilemma itself, or the language in which the model is instructed to express its reasoning. Standard evaluation conflates these by testing only matched conditions (e.g., English dilemma with English reasoning). We introduce a methodology that separately manipulates each factor, covering also mismatched conditions (e.g., English dilemma with Chinese reasoning), enabling decomposition of their contributions. To study what changes, we propose an approach to interpret the moral judgments in terms of Moral Foundations Theory. As a side result, we identify evidence for splitting the Authority dimension into a family-related and an institutional dimension. Applying this methodology to English-Chinese moral judgment with 13 LLMs, we demonstrate its diagnostic power: (1) in this EN–CN case study, the framework isolates the language of generated rationale as contributing roughly twice the variance of input-language effects; (2) it detects context-dependency in 4 of 9 EN–CN-tested models that standard evaluation misses; and (3) a diagnostic taxonomy translates these patterns into deployment guidance. We release our code and datasets at https://github.com/aida-ugent/LLMCrossCulturalMoralJudgement.

Abstract:
Conversational explainable AI (ConvXAI) agents often use GUS-style architectures with separate modules for intent recognition, dialogue management, and template-based response generation. While faithful to the underlying XAI methods, these pipelines are hard to adapt and often struggle with context-dependent follow-up questions. We present SimXAI, a lightweight LLM-centric conversational layer that replaces these modules with a single tool-calling LLM that bridges access to existing XAI resources. Rather than a standalone agent, SimXAI serves as a modular interface that enables conversational access to external XAI resources. We further introduce an evaluation suite for single- and multi-turn parsing, faithfulness, and contextualization. Across existing and new datasets, SimXAI achieves strong function-selection accuracy and high faithfulness while better handling context-dependent requests. In a user study replacing only the conversational layer of TalkToModel, a representative GUS-based agent, users completed tasks more accurately and in fewer turn. Overall, SimXAI improves GUS-based design while preserving faithfulness and improving conversational quality.

Abstract:
Large language models (LLM) deployed as autonomous pricing agents may sustain supracompetitive prices through tacit coordination. We develop a causal graph divergence framework that separately measures structural faithfulness and intent faithfulness of LLM pricing agents in Bertrand competition. Across nine LLMs under duopoly and triopoly conditions, collusive behavior and chain-of-thought (CoT) faithfulness dissociate along both dimensions: the most collusive model accurately reports cooperative intent yet reasons structurally unfaithfully, while the most structurally faithful model sustains supra-Nash pricing under both market structures. These findings establish that CoT monitoring alone cannot serve as a standalone safeguard against algorithmic collusion.

Abstract:
Web agents must choose queries, clicks, form fills, and tool calls from language instructions while preserving constraints whose consequences may appear several steps later. We study this challenge as language-conditioned delayed-constraint tracking and present TraceWeaver, which couples linear-time latent rollouts, entity-centric episodic memory, and a learned critic for budgeted action selection. Across WebShop (500 episodes), Mind2Web (1,000-task split), WebArena (812-task split), and WorkArena (165-task enterprise split), TraceWeaver shows task-success gains of +2.7 to +4.7 over baselines under a matched test-time budget. Shared-pool controls reduce the confound between selection quality and proposal quality, while a language-phenomenon analysis concentrates the gains on tasks with dense, conflicting, or delayed constraints.

Abstract:
We introduce MineCEraft (Minecraft Construction Engineering Benchmark, pronounced mine-see-ee-raft), an easy-to-use, open-source benchmark designed to systematically evaluate the reliability and limitations of LLMs for construction tasks in Minecraft. The MineCEraft benchmark comprises 723 domain-expert hand-crafted natural-language instructions with programmatically verifiable evaluation, spanning 17 distinct task categories, providing a safe and controllable experimental environment for assessing LLMs’ ability to perform realistic construction engineering tasks. With this benchmark, we conduct an in-depth evaluation of state-of-the-art LLMs and perform a detailed error analysis, revealing key failure modes and practical challenges in applying LLMs to construction engineering tasks.

Abstract:
Large language models (LLMs) show growing promise for scientific discovery, yet existing benchmarks rely on static, single-turn QA formats that cannot measure the iterative reasoning and strategic planning required in real research. We introduce MolQuest, an agent-based benchmark that reformulates molecular structure elucidation as a multi-turn interactive task grounded in authentic experimental data. Within a simulated laboratory, the model must actively plan experiments, acquire heterogeneous spectral evidence (NMR, MS, IR) on demand, and iteratively refine structural hypotheses while minimizing unnecessary tool calls—mirroring the abductive reasoning workflow of a practicing chemist. We evaluate 16 frontier LLM variants under two settings, yielding 32 model–setting configurations, and find that the best Agent-mode result reaches only 51.51% accuracy, while most remain substantially lower. Furthermore, the dynamic paradigm amplifies performance for some models yet degrades others, revealing strategic planning deficiencies invisible to static evaluations. MolQuest provides a reproducible, extensible framework that exposes the critical gap between pattern recognition and genuine scientific reasoning in current LLMs.

Abstract:
While Large Language Models (LLMs) have achieved remarkable results across various benchmarks, their alignment with normative values often results in homogenized responses that fail to address diverse user preferences. Existing training-free methods often occupy valuable context windows through prompt engineering, while training-based methods typically remain static post-training, failing to support the continual optimization required in real-world settings. To address these challenges, we propose COPE (Continual Optimization with Personalized embedding and self-Evaluation), a novel optimization framework tailored for real-world-motivated interaction settings with sparse user feedback. Our framework assigns learnable personalized embeddings to each user and synergistically integrates preference capture, self-evaluation calibration, and personalized response optimization within a single update step. A key innovation of our method is the use of self-evaluation to generate proxy rewards, enabling continuous model updates even when explicit user feedback is unavailable. Experiments show that COPE consistently outperforms strong training-free and training-based baselines under sparse feedback, and remains complementary to Retrieval-Augmented Prompting (RAP). Further analyses confirm COPE’s reliable self-evaluation, meaningful preference patterns, stable general capabilities, and robustness under shifting preferences and alternative evaluators.

Abstract:
Large language models (LLMs) are increasingly used to assist scientific writing and experimental design, but a fluent protocol can still be physically infeasible. We introduce PhysDox, a benchmark for physical feasibility auditing of biomedical experimental protocols, comprising a 683-sample expert-curated Gold set and a 5,000-sample expanded Silver set across six sensing domains. PhysDox formulates protocol auditing as a two-stage task: severity detection over valid, minor issue, and fatal violation protocols, followed by diagnosis of fatal violations over coarse and fine-grained physical constraints. We evaluate 6 recent LLM variants under 4 inference strategies. Results show that current LLMs remain poorly calibrated at the severity gate: the best Gold Stage-1 macro-F1 is only 53.0, and strong oracle diagnosis often collapses after end-to-end gating. Qualitative case analyses reveal a shared failure mode: models reward complete experimental scaffolds while missing hidden physical constraints. These results show that protocol auditing requires calibrated feasibility reasoning over complete experimental procedures, rather than factual recall or isolated physics problem solving.

Abstract:
Diffusion large language models (dLLMs) offer a compelling alternative to autoregressive models, yet they may expose sensitive training data during denoising. Detecting such usage is challenging because dLLMs lack the efficient one-pass probability decomposition of causal architectures. Existing methods rely on random masking to obtain tractable token-wise detection signals under limited query budgets, but fail to control dependencies among masked tokens. We demonstrate that this token-wise approximation introduces a non-negative structural estimation error, which is theoretically characterized by the cumulative conditional mutual information (CMI) among masked tokens and can obscure subtle memorization signals. This insight suggests that reliable detection requires masked token sets with weak internal dependency. To avoid the prohibitive cost of directly estimating CMI over token combinations, we propose Independent Token Sampling (ITS), a query-efficient framework that uses an attention-derived pairwise dependency proxy to approximate the CMI-aware selection criterion. ITS further incorporates a diversity-promoting strategy to improve token coverage across sampling rounds, yielding aggregated token-wise signals that are less affected by dependency-induced approximation error. Experiments on multiple datasets show that ITS consistently outperforms state-of-the-art baselines across different models and datasets, achieving an AUC improvement of 0.18 on the ArXiv dataset while maintaining strong performance under limited query budgets.

Abstract:
Unified streaming and offline ASR aims to deliver full-context accuracy and low-latency incremental decoding with a single model. Transformer/Conformer systems are effective, but long-form streaming inference still suffers from growing attention masks, historical caches, and memory states. Latency-Controlled BiMamba (LC-BiMamba) offers attention-free bidirectional modeling, yet its backward branch is confined to fixed chunks, tying the model to one latency. Extending it to dynamic chunk training expands mini-batches and slows training. We propose TC-BiMamba, whose Trans-Chunk formulation casts local-future backward modeling as sequence-level dynamic-chunk training over complete sequences. TC-BiMamba supports multiple chunk-wise latency settings, keeps the encoder/CTC path memory-stable on long-form inputs, and improves training scalability. On AISHELL-1, AISHELL-2, and LibriSpeech, it outperforms U2++ and remains competitive with externally reported LC-BiMamba results using fewer parameters, while achieving 1.3 times training speedup and 50% GPU memory reduction.

Abstract:
Diffusion large language models (D-LLMs) have emerged as a promising alternative to auto-regressive models due to their iterative refinement capabilities. However, hallucinations remain a critical issue that hinders their reliability. To detect hallucination responses from model outputs, token-level uncertainty (e.g., entropy) has been widely used as an effective signal to indicate potential factual errors. Nevertheless, the fixed-length generation paradigm of D-LLMs implies that tokens contribute unevenly to hallucination detection, with only a small subset providing meaningful signals. Moreover, the evolution trend of uncertainty throughout the diffusion process can also provide important signals, highlighting the necessity of modeling its denoising dynamics for hallucination detection. In this paper, we propose DynHD that bridge these gaps from both spatial (token sequence) and temporal (denoising dynamics) perspectives. To address the information density imbalance across tokens, we propose a semantic-aware evidence construction module that extracts hallucination-indicative signals by filtering out non-informative tokens and emphasizing semantically meaningful ones. To model denoising dynamics for hallucination detection, we introduce a reference evidence generator that learns the expected evolution trajectory of uncertainty evidence, along with a deviation-based hallucination detector that makes predictions by measuring the discrepancy between the observed and reference trajectories. Extensive experiments demonstrate that DynHD consistently outperforms state-of-the-art baselines while achieving higher efficiency across multiple benchmarks and backbone models.

Abstract:
Medical AI benchmarks routinely pair a diagnostic image with the same-visit clinical record, delete the recorded diagnosis, and treat what remains as auxiliary context. The recipe is reasonable. It is not. On 5904 ophthalmic visits across eight cohorts, a frozen text probe still recovers the disease label from the diagnosis-redacted record at 0.944 macro-AUROC, against 0.998 on the unredacted record; the most aggressive dictionary scrub closes less than one further percentage point. The diagnosis is already in the notes. It is not in one place. A clinical record is written during the diagnostic process, and its fingerprints persist across channels: narrative text alone reaches 0.978, and a view that erases every value and exposes only which fields were populated still reaches 0.854. We introduce CRAFT (Channel-wise Redaction And Floor Test), a portable audit protocol that renders each record into redacted and channel-isolated views and reports each channel’s label recoverability; rankings reproduce under Qwen3.5-9B. Before clinical-text or image-text fusion gains are read as cross-modal reasoning, the text branch’s channel-level shortcut floors must be reported alongside them. https://github.com/yhc98002-bit/The-Diagnosis-Is-Already-in-the-Notes

Abstract:
Humans naturally exhibit multiple forms of abstraction in reasoning and interaction, including temporal abstraction across decision timescales and strategic abstraction over communicative intents. Inspired by these complementary abstractions, we propose a two-level hierarchical reinforcement learning (HRL) framework for conversational agents that bridges the gap between existing token-level and utterance-level RL methods. Built upon a two-level Markov decision process (MDP), our framework conditions token-level response generation on utterance-level actions represented by explicit textual strategies. Based on theoretical analysis and efficiency considerations, we employ DQN to optimize the high-level Q-network and PPO to train the low-level actor-critic. To further alleviate reward sparsity and facilitate convergence, we introduce a dual-granularity reward mechanism that combines the utterance-level satisfaction score with token-level intrinsic self-consistency and a KL-divergence penalty. Experiments on both daily-life and emotional support conversations demonstrate that our method consistently outperforms a wide range of baselines in both strategy determination and response quality. Our implementation is available at https://github.com/AaronJi/ToSCA .

Abstract:
Knowledge graph reasoning (KGR) composes multi-hop facts over KGs to support complex knowledge-intensive tasks, providing reliable external factual support for large language models (LLMs) and helping mitigate hallucinations. However, building compact, faithful, and high-quality contexts from large-scale KGs remains challenging. We observe a non-monotonic relationship between retrieval scale and reasoning quality: insufficient retrieval misses critical paths, while excessive retrieval introduces redundant and noisy paths that burden LLMs reasoning. Motivated by this observation, we propose super-path ranking and constraint verification (SPaRC), an efficient framework that constructs, ranks, and verifies super-paths before LLMs reasoning. SPaRC compresses redundant paths into super-paths, ranks them with contrastive learning, verifies top-ranked paths under query semantic constraints, and provides the verified context to the LLMs for answer reasoning. Experiments on WebQSP and CWQ show that SPaRC achieves state-of-the-art performance, improving Hit@1 by up to 12.0 points and F1 by up to 5.9 points on CWQ. On challenging settings covering deeper hop reasoning, multiple topic entities, and multi-answer questions, SPaRC further improves F1 on two benchmarks by up to 17.6% and 42.1%, 26.7% and 5.9%, and 10.4% and 79.4%, respectively, while reducing reasoning tokens by 29.7% and retrieval time by 25%.

Abstract:
Commit messages are vital artifacts that summarize changes to facilitate software maintenance, debugging, and evolution tracking. However, existing Commit Message Generation (CMG) methods rely solely on code diffs and overlook higher-level architectural dependency diff information. To address this limitation, we propose HierDiff, a framework using architectural diffs to complement code diffs for improved commit generation. Through this framework, we further construct HierCMD, a new dataset that incorporates architectural diffs alongside much longer code diffs. Building on this resource, we develop HierDiff-6.7B, a lightweight CMG model trained via multi-stage hierarchical diff learning. To enable more rigorous evaluation, we additionally extend the existing MCMD-New-Test benchmark by incorporating architectural diff information, resulting in MCMD-New-TestX. Experimental results on MCMD-New-TestX and HierCMD demonstrate that HierDiff-6.7B achieves state-of-the-art performance in CMG, even surpassing the renowned closed-source model GPT-4o by 2.1% to 29.8% across four evaluation metrics. We release the data and code at: https://anonymous.4open.science/r/HierDiff/.

Abstract:
Multimodal large language models (MLLMs) are increasingly capable scientific assistants, yet they remain far from fully autonomous research. This transition requires models to actively inspect academic papers, build global evidence views, and make traceable judgments without prespecified issues or evidence. However, existing work provides limited task paradigms or training studies for such issue- and evidence-absent verification. We study this challenge through scientific error detection, where models must determine whether errors exist and justify them with evidence-based reasoning. To fill this gap, we present VERA-RL, a reinforcement-learning formulation for scientific error detection over academic papers. Following a Reason – Verify – Scan progression, we construct VERA-13K, a 12,900-sample dataset organized into 4,300 matched chains, covering 6 scientific-error categories across the research workflow and broad natural-science domains. We further introduce fine-grained rewards for reasoning completeness, evidence alignment, and error precision. Training Qwen3-VL-8B with VERA-RL substantially improves verifiable reasoning, approaching flagship MLLMs such as Gemini 3 Pro and Qwen3-VL-235B-A22B on Scan .

Abstract:
Large language model agents often spend substantial wall-clock time waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing an agent’s next tool call if the prediction matches the agent’s eventual tool call, but existing speculators are typically separate draft models or cached traces that are poorly aligned with the deployed agent’s own behavior. We identify this speculator–agent gap and show that the target agent itself is a strong next-call speculator. This points to a simpler design: unifying the agent and speculator within the same model. In this paper, we introduce the self-speculating agent, a single model that both solves tasks in agent mode and predicts its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode agent without degrading performance, we propose a joint agent-speculator reinforcement learning method, which derives speculation targets from the agent’s own rollouts and alternates agent and speculator updates. Across agentic search QA and conversational tool-use agentic tasks, our method improves average next tool-call Hit@1 from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, while preserving agent task success.

Abstract:
Automated International Classification of Diseases (ICD) coding is a core medical-coding task for billing, epidemiology, and clinical decision support. Generative large language models (LLMs) are often reported as weak medical coders, but this finding mainly comes from inference-time settings such as prompting, retrieval, reranking, or tool use; the role of task-specific post-training has received limited systematic study. We present a controlled empirical study of post-training for generative ICD coding, comparing discriminative baselines with LLM coders across prompting, supervised fine-tuning (SFT), and reinforcement learning (RL) under a common protocol and metric set. To our knowledge, this is the first comprehensive controlled study of post-training for generative ICD prediction. We further introduce Progressive Hint Injection (PHI), a diagnostic missed-code curriculum within Group Relative Policy Optimization (GRPO) training. Our results show that prompting-only evaluation substantially underestimates the potential of LLMs for ICD coding. SFT provides the main capability jump, GRPO further improves code-set prediction beyond SFT, and PHI provides targeted gains on macro-level performance. These findings suggest that the main bottleneck is not the generative formulation alone, but how the model is adapted and optimized for full-taxonomy recall. We release our code, data splits, and checkpoints at https://anonymous.4open.science/r/LLM4ICD .

Abstract:
Human knowledge is inherently structured and interdependent: mastery of a concept requires prior mastery of its prerequisites, a principle formalized by Knowledge Space Theory (KST). While LLMs achieve strong performance on complex reasoning tasks, it remains unclear whether they exhibit coherent, human-like knowledge structure. We introduce a KST-grounded framework for evaluating LLM knowledge structure in mathematical reasoning, using it as a normative framework to analyze whether LLM behavior adheres to principled knowledge dependencies. Evaluating eight open- and closed-source LLMs against real human learners, we find that (1) LLMs do not adhere to human knowledge structure—they frequently violate knowledge dependencies and fail to leverage related knowledge provided in context to improve performance on dependent questions; (2) LLMs do not share a consistent knowledge structure among themselves, as reflected by low overlap in their knowledge distributions. Furthermore, these structural deficiencies remain largely invisible to accuracy-based and LLM-as-judge evaluations. Together, our results provide behavioral evidence that current LLMs knowledge does not follow a human-like structure.

Abstract:
Multimodal large language models excel at vision-language tasks but suffer from hallucinations that undermine their reliability. Existing mitigation methods apply uniform heuristics at the output or attention level, overlooking the heterogeneous causal roles of attention heads and their dynamic behaviors during inference. We revisit hallucination from a causal perspective and identify it as competition between two pathways: a direct path from visual evidence to outputs, and an indirect path dominated by knowledge priors. When visual evidence weakens, MLLMs increasingly rely on parametric knowledge priors—such as spurious correlations learned during pretraining. This over-reliance causes attention heads responsible for visual grounding to drift toward language-driven patterns, resulting in generations consistent with the language model but unfaithful to the image. We propose Causal Head Gating, an inference-time framework that separates visual evidence from knowledge priors by modeling their causal contributions at the attention-head level. Under this framework, we introduce CHERI (Causal Head Evidence Reweighting and Intervention), which identifies attention heads whose visual contributions weaken under hallucination and dynamically restores their influence. Specifically, CHERI reinforces heads that propagate visual evidence while suppressing those dominated by knowledge priors, strengthening the direct causal effect of vision on generation and attenuating spurious indirect effects. Experiments across multiple benchmarks show that CHERI consistently reduces hallucination rates across diverse task formats while preserving overall generation quality and task performance.

Abstract:
Large language models (LLMs) are increasingly deployed as proxies for human participants in social simulations, yet whether they update their beliefs in response to persuasive arguments, as humans do, remains poorly understood. We conduct a systematic comparison using a naturally occurring online persuasion corpus in which original posters explicitly verify whether a reply changed their view. Our results show that LLMs achieve only slight agreement with humans (Cohen’s 𝜅 ranging from 0.079 to 0.178). Content-level analyses show that humans and LLMs agree on the strongest persuasion cues but diverge on finer ones: humans are more swayed by novel content and assertive language, whereas LLMs favor topical similarity and surface-level formatting. At the level of persuasion strategy, LLMs underweight emotional appeals and overweight credibility signals relative to humans, while the type of proposition under debate exerts no measurable effect on the degree of divergence. Furthermore, switching from first-person role-playing to third-person observation shifts all models toward greater resistance to persuasion, with the effect varying across persuasion strategies and textual features. These findings highlight the risk of treating LLM judgments as faithful proxies for human belief updating and point to structural differences in how LLMs and humans process persuasive discourse. Our code is available at https://github.com/tsinghua-fib-lab/LLM-belief-update-cmv.

Abstract:
Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method.

Abstract:
Drug–disease knowledge underpins drug discovery and precision medicine, yet most of it is locked in unstructured PubMed articles and only partially captured by curated databases. We release UniD³-KG, a suite of six knowledge graphs spanning three tasks (Drug–Disease Matching, DDM; Drug Effectiveness Assessment, DEA; and Drug–Target Analysis, DTA), and UniD³-data, three open QA datasets (28,915 DDM, 15,042 DEA and >4,000 DTA pairs), distilled from 157,849 PubMed articles with Llama-3.3-70B by the UniD³ KG-RAG pipeline, which couples task-specific prompting with a dual-stage entity extraction that re-extracts over the consolidated graph to recover cross-chunk and cross-document evidence. External validation on CDR, DrugReviews, P3ps and MedicationQA and a benchmark across seven LLM baselines and three retrieval variants show that UniD³-data is faithful to curated sources and that retrieval over UniD³-KG consistently improves biomedical QA; a clinician verification of 469 FDA-approved, US-marketed drugs further confirms strong agreement with the UniD³-data DDM split (accuracy 0.90, F1 0.90, AUROC 0.90). All artefacts—UniD³-KG, UniD³-data, benchmark scripts and the UniD³-QA explainable chatbot—are released as shared infrastructure for biomedical NLP and drug repurposing.

Abstract:
Large language models (LLMs) are increasingly used for code generation, but their ability to program specialized cryptographic systems remains poorly understood. Cheon-Kim-Kim-Song (CKKS), a fully homomorphic encryption (FHE) scheme supporting approximate arithmetic over packed real- and complex-valued vectors, is particularly relevant to machine-learning workloads. Prior work has not systematically examined the CKKS-specific challenges of LLM-generated encrypted programs. To address this gap, we introduce FHE-CodeEval, a 46-case benchmark that evaluates three core challenges: packing, nonlinear approximation, and level/scale management, as well as how models coordinate decisions across them. We further introduce communication-aware efficiency, an FHE-specific metric comparing generated programs with compiler baselines using server-side runtime and ciphertext footprint. We evaluate eight recent proprietary and open-weight LLMs on FHE-CodeEval. Across all 46 cases, the best observed scores reach 51.7% Pass@1 and 69.6% Pass@5 with iterative execution feedback, but only 7.8% Pass@1 and 13.0% Pass@5 in the one-shot setting. For efficiency, even the best-performing model achieves a geometric-mean communication-aware efficiency of only 0.44 over its valid rollouts in the compiler-comparable subset. These results reveal a substantial gap between functional correctness and efficiency in LLM-generated CKKS programs. Code is available at https://github.com/qiuchuy/FHE-CodeEval.

Abstract:
As pretraining scaling laws approach saturation, Test-Time Scaling (TTS) has emerged as an important direction for improving reasoning by allocating inference-time compute to a fixed model prior. Viewed at a high level, TTS reframes inference as search over a space of partial reasoning states. While Chain-of-Thought (CoT) exposes intermediate steps, common instantiations rely on single-trajectory decoding , limiting recovery from early errors and exploration. This survey systematizes recent progress in tree-search-based reasoning , viewing inference as instance-specific optimization rather than decoding. We trace the evolution from uninformed search to Monte Carlo Tree Search (MCTS), highlighting how sampling-based control supports principled exploration–exploitation trade-offs. To unify a fragmented literature, we introduce a Unified Design Space spanning search topology, evaluation signals, and control dynamics, and advocate a standardized compute-reporting abstraction to make compute–accuracy trade-offs explicit and comparable.

Abstract:
Strong reasoning depends not only on model knowledge but also on how effectively cognitive behaviors are deployed during generation. Existing methods often rely on explicit behavior-level control, making them insufficiently adaptive when failures and required corrections vary across reasoning states, tasks, and models. To this end, we propose Latent Reward Steering (LRS), an adaptive inference-time framework that promotes cognitive behaviors by optimizing the sparse-autoencoder (SAE) latent states that implicitly carry them. Rather than relying on predefined cognitive behaviors or steering directions derived from them, LRS trains a latent reward model on reasoning traces by final answer correctness to estimate the quality of intermediate latent states. During inference, reward gradients provide state-specific correction directions for fragile latent states, while a reward and confidence gate restricts intervention to states the reward signal flags as fragile. Experiments on multiple reasoning LLM backbones and benchmarks show that LRS consistently improves performance over various baselines, and post-hoc analyses further indicate that LRS implicitly promotes good cognitive behaviors that fix the original reasoning errors. Codes are available at: https://anonymous.4open.science/r/Latent-Reward-Steering-5B56.

Abstract:
Accurately assessing model confidence is essential for deploying large language models (LLMs) in mission-critical factual domains. While retrieval-augmented generation (RAG) is widely adopted to improve grounding, confidence calibration in RAG settings remains poorly understood. We conduct a systematic study across four benchmarks, revealing that LLMs exhibit poor calibration performance especially when noisy contexts are retrieved. Specifically, contradictory or irrelevant evidence tends to exacerbate the model’s overconfidence issue. To address this, we propose NOVA Rules (NOise-aware Verbal Confidence CALibration Rules) to provide a principled foundation for resolving overconfidence under noise. We further design NOVA, a noise-aware calibration framework that synthesizes supervision from ~2K HotpotQA examples guided by these rules. By performing supervised fine-tuning (SFT) with this data, NOVA equips models with intrinsic noise awareness without relying on stronger teacher models. Empirical results show that NOVA yields substantial gains, improving ECE scores by 10.9% in-domain and 8.0% out-of-domain. By bridging the gap between retrieval noise and verbal calibration, NOVA paves the way for both accurate and epistemically reliable LLMs.

Abstract:
Temporal Knowledge Graph Question Answering (TKGQA) commonly relies on retrieval-aggregation-reasoning pipelines, which suffer from two critical challenges: error propagation, where early retrieval mistakes accumulate along reasoning paths, and temporal conflicts caused by post-hoc temporal filtering, leaving temporally inconsistent paths in the search space and undermining multi-hop reasoning reliability. To address these limitations, we propose ConTExT (Contractive Temporal Exploration), a contractive subgraph search framework over structural and temporal spaces for TKGQA. It dynamically anchors question-relevant entities, performs selective subgraph exploration guided by path semantics and sub-question consistency, and applies Temporal Feasible Region Propagation (TFRP), which actively propagates temporal constraints during search and maintains a temporal feasible region. Experiments on MultiTQ show that ConTExT is particularly effective for complex temporal reasoning, yielding a 27.1% relative improvement in Hits@1 over the strongest baseline on Multiple questions. Its gains are smaller on Single and Time-answer questions that depend more on direct evidence matching or exact temporal endpoints.

Abstract:
Functional programming provides strong foundations for developing reliable and secure software systems, yet its adoption remains limited because of its steep learning curve. Recent advances in Large Language Models (LLMs) for code generation present new opportunities to lower these barriers. However, most evaluations of LLMs focus on imperative programming languages, and their capabilities in functional programming languages remain under-explored. To address this gap, we introduce FPEval, a holistic evaluation framework equipped with FPBench, a new benchmark of 721 programming tasks across three difficulty levels and three mainstream functional programming languages: Haskell, OCaml, and Scala. FPEval combines comprehensive executable test suites with static-analysis tools to assess both functional correctness and code style and maintainability. Using this framework, we evaluate seven state-of-the-art LLMs, including GPT-5 and Claude Opus 4.6, on functional programming tasks, with Java included as an imperative baseline. Our results demonstrate that LLM performance in functional programming improves substantially with model advancement; however, error rates remain significantly higher in functional-first languages (Haskell and OCaml) than in hybrid (Scala) or imperative (Java) languages. More critically, LLMs frequently generate non-idiomatic functional code that follows imperative patterns, raising concerns about code style and maintainability. Finally, we show that LLMs can partially self-repair both correctness and quality issues when provided with static-analysis feedback and hand-crafted instructions targeting common issue types, paving the way for future improvements.

Abstract:
In this work, we compared various architectures of surface morpheme segmentation algorithms using data from 20 languages belonging to different language families and with varying data availability. We compare conventional subword and statistical baselines, CNN and LSTM models trained from scratch, fine-tuned multilingual encoders, prompted frontier LLMs, and LoRA-tuned decoder-only LLMs. To the best of our knowledge, this is the first systematic study of decoder-only LLMs for morpheme segmentation. Our results show that XLM-Roberta-large fine-tuned for character-level sequence labeling with whole-word context remains the strongest and most stable language-agnostic approach, achieving the highest average normalized score and improving previous SOTA results for five languages. At the same time, LoRA fine-tuning of Qwen3-32B yields competitive results and substantially outperforms XLM-Roberta-large for Dravidian and Indo-Aryan languages, demonstrating that decoder-only LLMs can be effectively adapted to the task. Prompting frontier LLMs, however, remains inconsistent. Finally, root-disjoint evaluation and error analysis show that generalization to unseen roots and dataset-specific segmentation conventions remain central challenges.

Abstract:
The rapid spread of misinformation has made fact verification a critical necessity. LLM-based methods have become the mainstream approach for fact verification, but they still suffer from hallucinations. To mitigate hallucinations, researchers incorporate KG on LLM reasoning with a graph-based constrained decoding. However, these methods with constrained decoding suffer from error cascading due to the inherent knowledge gaps of the LLM, which cause it to select wrong branches during beam search in the constrained token space. To tackle error cascading , we think that the decoding process should be dynamically guided by step-level plausibility assessment. In light of this, we propose TrustFV (https://anonymous.4open.science/r/Trust_FV-6239), a fact verification framework that step-wise beam search in token-level constrained decoding with risk-averse pruning for trustworthy verification. Specifically, TrustFV integrates (1) token-level constrained decoding on a Knowledge Graph Tries (KG-Tries) to ground generation in retrieved evidence from KG, (2) an Uncertainty-aware Process Reward Model (UPRM) trained via Monte Carlo rollouts and ordinal regression to estimate both expected success and uncertainty per step, and (3) a step-wise beam search that actively prunes high-risk branches via the step-level signal of UPRM, producing faithful and plausible reasoning paths. Experiments show that TrustFV achieves state-of-the-art performance on FactKG and CWQ datasets.

Abstract:
Large language models are reshaping e-commerce from static recommenders into interactive shopping assistants, yet real-world shopping requires session-level decision support: users reveal and revise constraints, coordinate multiple goals, and expect product-grounded recommendations over a full conversation. Existing benchmarks are mostly outcome-oriented or execution-oriented, leaving this evolving decision process under-evaluated. We introduce RealWorldShop, a benchmark built on 3.28M grounded products, structured shopping episodes, a profile-grounded and action-controlled user simulator, and role-play evaluation. Our analysis shows that current systems produce locally plausible responses but struggle with state tracking, constraint updating, and grounded convergence, especially under ambiguous intent, bundle, and multi-intent scenarios. We further propose RealShop_Agent, an executable session-control framework with explicit state management, shopping-flow control, catalog-grounded retrieval, and runtime guards. Experiments show that RealShop_Agent consistently outperforms strong baselines on RealWorldShop.

Abstract:
Large Language Models (LLMs) have demonstrated impressive capabilities across various specialist domains and have been integrated into high-stakes areas such as medicine. However, as existing medical-related benchmarks rarely stress-test the long-context memory, interference robustness, and safety defense required in practice. To bridge this gap, we introduce MedMT-Bench, a challenging medical multi-turn instruction following benchmark that simulates the entire diagnosis and treatment process. We construct the benchmark via scene-by-scene data synthesis refined by manual expert editing, yielding 400 test cases that are highly consistent with real-world application scenarios. Each test case has an average of 22 rounds (maximum of 52 rounds), covering 5 types of difficult instruction following issues. For evaluation, we propose an LLM-as-judge protocol with instance-level rubrics and atomic test points, validated against expert annotations with a human-LLM agreement of 91.94%. We test 17 frontier models, all of which underperform on MedMT-Bench, with the best model reaching 66.75%. MedMT-Bench can be an essential tool for driving future research towards safer and more reliable medical AI. The benchmark is available in https://github.com/hiyyc/MedMT-Bench.

Abstract:
Uncertainty estimation (UE) enables LLM-powered systems to recognize when to abstain, yet existing research has predominantly focused on English. We present the first large-scale evaluation of UE methods across 22 languages, spanning high-, mid-, and low-resource settings. Using two human-curated Q A datasets, we compare open and closed box UE methods (nine in total) across different model sizes and architectures while eliciting long-form reasoning, avoiding LLM-as-a-judge and embedding-based scoring, which can introduce evaluation noise. We report three main actionable findings. First, we find that prompting models to reason in English while keeping questions in low-resource languages substantially improves UE performance, suggesting that comprehension of low-resource languages is largely intact, and that the reliability bottleneck lies in generation rather than understanding. Second, prompting models to reason in English closes the UE performance gap between low and high-resource languages, demonstrating that generation language matters more than the question language. Third, the choice of UE method should depend on model scale: at smaller scales, open-box probability-based methods outperform alternatives; at larger scales, closed-box self-verbalized uncertainty becomes superior. Finally, we provide an analysis of threshold selection for selective prediction, offering guidance on calibrating abstention in multilingual settings. We publish our code to support further research in multilingual UE. Our code available in Anonymous Github[https://anonymous.4open.science/r/Cross-and-Multilingual-Uncertainty-Estimation-7976/].

Abstract:
With the rapid advancement of large language models (LLMs), the safety of LLMs has become a critical concern. Despite significant efforts in safety alignment, current LLMs remain vulnerable to jailbreaking attacks. However, the root causes of such vulnerabilities are still poorly understood, necessitating a rigorous investigation into jailbreak mechanisms across both academic and industrial communities. In this work, we focus on a continuation-triggered jailbreak phenomenon, whereby simply relocating a continuation-triggered instruction suffix can substantially increase jailbreak success rates. To uncover the intrinsic mechanisms of this phenomenon, we conduct a comprehensive mechanistic interpretability analysis at the level of attention heads. Through causal interventions and activation scaling, we show that this jailbreak behavior primarily arises from an inherent competition between the model’s intrinsic continuation drive and the safety defenses acquired through alignment training. Furthermore, we perform a detailed behavioral analysis of the identified safety-critical attention heads, revealing notable differences in the behaviors of safety heads across different model architectures. Grounded in these mechanistic findings, we propose Head Competition Steering (HCS), a mechanistically grounded inference-time strategy that explicitly leverages the competition between safety heads and continuation heads to suppress harmful generation, and further distill its behavioral signal into a student model via knowledge distillation, achieving inference-time safety improvements without additional computational overhead.

Abstract:
Fixed-item cognitive screening tools such as the Montreal Cognitive Assessment (MoCA) may ask redundant questions and increase patient burden. We propose AQS, an adaptive cognitive screening framework that selects questions and stops assessment according to an evolving cognitive state. Using ADNI-derived MoCA records, we construct an expanded question bank and an 8-dimensional state space, where free-form responses are mapped to discrete severity signals for online impairment updating. We compare handcrafted adaptive selection, fixed stopping, and reinforcement learning (RL) policies. Results show that combining a structured adaptive selector with an RL-based stopping policy yields the best performance–efficiency trade-off. Compared with the classical FisherInfo strategy, it improves Acc from 0.682 to 0.715 and AUC from 0.762 to 0.771, while reducing the average number of questions from 14.43 to 9.78. It also concentrates assessments in a more balanced question-count range and reduces excessive long-tail testing. Analysis further suggests that question selection and stopping are better treated as decoupled decisions, since naive joint RL leads to unstable optimization. These findings provide a simulation-grounded offline framework for efficient cognitive screening under data-scarce medical settings.

Abstract:
Users articulate the same advice-seeking request in different ways: some specify detailed constraints, others gesture at a vague need. Prior work treats this variation as noise to be averaged away; we instead treat it as a stable, measurable structure in the input distribution. We ask whether articulation (how people ask) forms latent dimensions separable from topic (what they ask about), and whether it is associated with how language models respond. We extract interpretable features from 16,447 advice-seeking prompts pooled from public chat corpora (WildChat, LMSYS, and ShareChat) and recover a small set of latent articulation factors that replicate across train/test splits and across corpora. Because this structure is largely separable from topic, the populations it defines cut across topics and stay invisible to topic- or task-based evaluation. The factors define a handful of recurring articulation styles, one of which stands out: a long-form but information-poor style, roughly one in six prompts in the largest corpus, where models return shorter, vaguer answers and do not ask for clarification even though under-specification is exactly the condition that warrants it. The contrast holds within every topic group and length quintile, and is not under-specification alone: a second, equally under-specified style does draw clarifying questions. Two independent human annotators reproduce this contrast. We argue that benchmarks should stratify on articulation, and we offer the extracted structure as a measurement instrument for doing so.

Abstract:
Vision–language models (VLMs) have achieved impressive performance on multimodal inference tasks, but the cost remains a significant challenge due to the large number of vision tokens processed during the prefill stage. Existing token pruning methods often rely on utilizing the static attention patterns directly, failing to exploit the dynamic internal signals within VLMs. To address the issue, we propose AdaptInfer, a novel plug-and-play framework for vision token pruning. First, we introduce a dynamic text-guided pruning mechanism that construct soft priors over text-token importance on each pruning layer, allowing more informed scoring of vision tokens at each stage. Second, we observe a highly consistent distribution of cross-modal attention shifts by architecture, which inspires us to introduce a efficient data-driven schedule which determines the pruning locations automatically. Experimental results have verified the effectiveness and generalization of the proposed method. Under the same token budget, AdaptInfer surpasses existing approaches in accuracy. For instance, AdaptInfer maintains averagely 99.4% accuracy on Qwen2-VL while 70% of the prefilling vision token overhead is reduced. The source code is available on: https://github.com/weiczh02/AdaptInfer-base.

Abstract:
The majority of our everyday activities are procedural and consist of sequences of interdependent steps. However, existing benchmarks for Visual Agents and Visual Language Models (VLMs) overlook the evaluation of their procedural comprehension ability from an egocentric visual perspective, particularly for detecting procedural errors, a critical capability for everyday assistance. To bridge this gap, the EgoErrorVQA task is firstly proposed for egocentric procedural comprehension with explicit procedural errors modeling. Besides, we develop a user-friendly evaluator agent based on the Agent2Agent (A2A) protocol, enabling rigorous and standardized evaluation of visual agents through VQA-based interaction. A range of models are evaluated using both open-ended and multiple-choice questions, revealing persistent weaknesses in handling procedural errors and error types. Moreover, we introduce Ego-ADR, an Adaptive Decoupled Reasoning framework that decouples complex procedural reasoning to enhance models’ understanding of procedural errors. It achieves consistent performance gains over the selected baselines and attains state-of-the-art results on several metrics under comparable settings. Code: https://github.com/z1oong/EgoErrorVQA

Abstract:
Monte Carlo Tree Search (MCTS) is a promising test-time method for improving LLMs on hard reasoning problems, where correct trajectories are often sparse. Self-evaluated MCTS is attractive because it avoids external reward models, verifiers, and stronger judges, but it faces a central paradox: MCTS depends on reliable intermediate rewards, while LLM self-evaluation is noisy and unstable. We propose ASE-MCTS, an Asynchronous Self-Evaluated MCTS framework. ASE-MCTS uses search-oriented reward analysis to derive an effective composition of rollout, process, and pairwise rewards from their global and local search roles. It further combines flexible search-space control with dependency-aware asynchronous execution, reducing redundant exploration and latency. Experiments on mathematical and scientific reasoning benchmarks show that ASE-MCTS improves reasoning without external supervision, especially on hard sparse-trajectory problems, and achieves up to a 6× wall-clock speedup over Sync-MCTS in the branch-factor scalability experiment.

Abstract:
The capability of LLM agents to function as the ”brain“ of a system fundamentally expands the scope of analysis beyond a standalone model. Consequently, safety is no longer only about input–output content alignment. It also concerns system behavior and real-world execution outcomes. However, the current literature is fragmented across attack types, applications, and benchmarks. This makes it hard to explain why failures such as prompt injection, tool misuse, and memory poisoning often share the same structural cause, and how they spread through an agent workflow. In this survey, we treat isolation as a first-class principle for LLM-agent system safety. By isolation, we refer to the separation of user inputs, tool access, execution channels, inter-agent communication, and environment-originated context. We organize the literature with a boundary-centric taxonomy of five boundaries: user-agent, agent-tool, agent-execution, agent-agent, and system-environment. This view helps identify where the loss of isolation first occurs, how compromise propagates across boundaries, and which defenses are most relevant at each interface. We also summarize cross-boundary failure paths, discuss open challenges, and outline a research agenda for isolation-by-construction in future agent systems.

Abstract:
Current Multimodal Affective Computing (MAC) heavily rely on the completeness of multiple modalities to accurately understand human’s affective state. However, in real-world scenarios, the availability of modality data is often dynamic and uncertain, leading to substantial performance fluctuations due to the distribution shifts and semantic deficiencies of the incomplete multimodal inputs. Known as the missing modality issue, this challenge poses a critical barrier to the robustness and practical deployment of MAC models. To systematically quantify this issue, we introduce MissMAC-Bench , a comprehensive benchmark designed to establish fair and unified evaluation standards from the perspective of cross-modal synergy. Two guiding principles are proposed, including no missing prior during training, and one single model capable of handling both complete and incomplete modality scenarios, thereby ensuring better generalization. Moreover, to bridge the gap between academic research and real-world applications, our benchmark integrates evaluation protocols with both fixed and random missing patterns at the dataset and instance levels. Extensive experiments conducted on 3 widely-used language models across 4 datasets validate the effectiveness of diverse MAC approaches in tackling the missing modality issue. Our benchmark provides a solid foundation for advancing robust MAC and promotes the development of multimedia data mining. Our code is released in https://github.com/RH-Lin/MissMAC-Bench .

Abstract:
Understanding biases and stereotypes encoded in the weights of Large Language Models (LLMs) is crucial for developing effective mitigation strategies. However, biased behavior is often subtle and non-trivial to isolate, even when deliberately elicited, making systematic analysis and debiasing particularly challenging. To address this, we introduce a simple, cost-effective, and generalizable framework BiasGym for reliably injecting, analyzing, and mitigating conceptual associations of biases within LLMs. BiasGym consists of two modules: Inject , which injects specific biases into the model via token-based fine-tuning while keeping the model frozen, followed by two debiasing methods that leverage these injected signals to identify and reliably suppress ( Scope ) or Steer the components responsible for biased behavior. Our framework enables consistent bias elicitation for better localization of bias conceptual association in the model space, supports targeted debiasing without degrading performance on downstream tasks, and generalizes to biases unseen during fine-tuning. We demonstrate the effectiveness of our proposed framework in reducing real-world stereotypes (e.g., people from Italy being ‘reckless drivers’), showing its utility for both safety interventions and interpretability research.

Abstract:
Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT reasoning incurs high computational cost, making it impractical for low-latency retrieval; and (ii) embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions:Is textual CoT the optimal form of reasoning for embedding, and can effective embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent Reasoning Multimodal Embedding), which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60× faster inference than explicit CoT methods and 2× faster than latent baselines with throughput comparable to discriminative embedding models. The code is available at https://github.com/PeppaWu/LaME.

Abstract:
Referring Remote Sensing Image Segmentation (RRSIS) aims to segment the corresponding target region in a remote sensing image according to a natural-language expression. Compared with conventional referring image segmentation, RRSIS faces more significant scale variations, complex spatial layouts, and visual distractions, placing higher demands on vision–language alignment and multi-scale modeling. To address these challenges, we propose DLSA, a dual-level semantic alignment framework for RRSIS. DLSA introduces semantic alignment at two complementary levels, namely encoding and decoding. At the encoder level, we design Semantic-guided Adaptive Multi-scale Convolution (SgAMC), which explicitly introduces linguistic semantics into receptive-field selection, enabling language-guided adaptive scale modeling. At the decoder level, we introduce Adaptive Late-stage Text Injection (ALTI), which adaptively injects textual semantics into late decoding stages to refine mask reconstruction while preserving the stability of the decoding pathway. Experiments on RRSIS-D and RefSegRS demonstrate that DLSA achieves strong performance, validating the effectiveness of dual-level semantic alignment in remote sensing image scenarios. The source code is publicly available at https://github.com/young-qkl/DLSA.

Abstract:
In real-world driving, resolving a primary hazard often precipitates a secondary one: swerving to avoid a stalled vehicle may force the ego car into oncoming traffic. Existing driving QA datasets fail to capture this cascading causality, evaluating perception, prediction, and planning in silos without anticipating the risks action induces. We introduce WaymoQA, a multi-view dataset that frames safety-critical reasoning as a two-stage task: resolving the immediate threat, then mitigating the action-induced secondary risk. WaymoQA provides 35,000 human-annotated QA pairs across image and video modalities, with 28K training and 6.4K test examples. Experiments show that SOTA models struggle on safety-critical scenes, while fine-tuning on WaymoQA improves both reasoning accuracy and consistency. WaymoQA thus serves as a rigorous benchmark and resource for safety-aware autonomous driving.

Abstract:
Safe deployment of clinical vision-language models (VLMs) requires reliable uncertainty estimation (UE): a signal indicating when predictions should be trusted or escalated to a clinician. We test whether current UE methods actually deliver this signal. Benchmarking 8 methods across 12 VLMs on clinical visual question-answering (VQA), we find that UE quality is not an intrinsic property of the UE method: it tracks model accuracy, degrading precisely where the model performance is weakest, and therefore where reliability is most needed. When we stress-test models by hiding the correct option among the multiple-choice answers (NOTA perturbations), accuracy collapses while uncertainty barely changes, leaving models systematically miscalibrated. Yet, we find that uncertainty on the unperturbed input reliably anticipates which predictions will collapse under NOTA, indicating that UE in current VLMs carries diagnostic information about model fragility. Our results position UE as a diagnostic tool for identifying fragile predictions and motivate perturbation-based evaluation as a path toward safe clinical deployment.

Abstract:
Large language models (LLMs) are increasingly used to explain numerical time series, but fluent explanations can misstate when a change occurs, overstate a trend, or introduce unsupported patterns. Existing evaluations often score forecasts, answers, or whole explanations, leaving claim-level temporal grounding undermeasured. We introduce TimeClaimBench, a benchmark that parses explanations into atomic temporal claims and verifies them against a deterministic extracted event graph. Across five LLMs on controlled synthetic series, event-grounded prompting raises the strict graph-supported rate by 18.3 points and reduces graph-unsupported claims by 27.9 points under this fixed operational criterion. Calibration analyses show the boundary of that result. The extractor recovers the designed primary event in 394 of 400 series, but this does not estimate the precision of additional extracted events. A blinded two-annotator comparison on one model and 40 series reverses the automatic ordering: holistic ratings favor Direct and chain-of-thought outputs over event-grounded outputs. therefore distinguishes four axes that should not be conflated: designed-event recovery, the unresolved precision of additional events, claim-to-graph consistency, and whole-explanation human judgment. Code: https://github.com/lunaaa95/TimeClaimBench

Abstract:
Knowledge editing algorithms have emerged as an efficient paradigm for updating parametric facts in Large Language Models (LLMs) without full retraining. However, existing benchmarks primarily focus on isolated, static facts. While a few temporal datasets exist, they are frequently bottlenecked by coarse temporal granularity, severe relation imbalance, and a critical lack of a priori knowledge validation, which can conflate inherent model deficits with editing failures. To address these structural limitations, we introduce TRACE, a fine-grained Temporal Reasoning And Chronological Editing benchmark. Using exact-match a priori probing, TRACE dynamically constructs model-specific, non-mutually exclusive chronological event chains. Featuring over 16,000 customized test cases across two distinct operational setups (Modify and Insert), TRACE provides a comprehensive 7-dimensional metric matrix. It extends simple fact recall with relation-constrained counterfactual injection and sequential temporal logic probes that test timeline integrity. Our evaluation of 6 representative baselines (FT, IKE, ROME, GRACE, LoRA, WISE) on Llama-3.2-3B and GPT-J-6B identifies widespread temporal interference in the studied settings: editing a single timestamp often sharply degrades recall of adjacent historical and future facts. The consistently weak bidirectional inference after editing further suggests that the evaluated methods tend to introduce shallow, syntax-sensitive patches rather than integrate the edit into a chronological network. Our code, the EasyEdit snapshot used for the experiments, and the benchmark are publicly available.

Abstract:
Mixture-of-Experts (MoE) models reduce computation by routing each token to a small set of experts, but each selected expert is still used at a fixed precision. We present DPA-MoE, a runtime precision control framework that makes routed experts any-precision. DPA-MoE encodes expert feed-forward weights as prefix-decodable chunks using SharedScale AnyBCQ, profiles per-expert reconstruction residuals offline, and allocates chunks at inference time using live router gate scores. This post-routing allocation lets a single encoded MoE checkpoint operate at multiple effective weight budgets without retraining or re-encoding. Experiments on Qwen3-30B-A3B and DeepSeekMoE-16B show that DPA-MoE improves KL divergence and downstream accuracy over uniform chunk allocation at matched budgets, with modest steady-state throughput overhead in our prototype.

Abstract:
Large Language Models (LLMs) perform strongly in English medical tasks but degrade substantially in Arabic, a gap widely attributed to limited training data. We systematically investigate this assumption via tuned lens probing and causal activation patching, and find that Arabic medical knowledge is present in intermediate model representations but fails to surface at the output. This mechanistic insight motivates a targeted adaptation strategy. Rather than fine-tuning the full network, we propose Targeted Low-Rank Adaptation (TLoRA), restricted to the layer window where cross-lingual representations diverge, upstream of the output layers where the failure manifests. We evaluate TLoRA on multiple-choice medical QA, where our approach outperforms full-network LoRA, zero-shot, and few-shot baselines. We further evaluate it on short-answer generation and multi-turn clinical dialogue, where it performs competitively without the need for task-specific finetuning. We additionally introduce AraClinicDialog, a clinician-constructed Arabic medical dialogue benchmark in MSA with validated variants across four Arabic dialects. Together, these contributions demonstrate that mechanistic diagnosis can serve as a practical guide for targeted adaptation in underrepresented-language medical LLMs.

Abstract:
Social bots are increasingly polluting online platforms by spreading misinformation and engaging in coordinated manipulation, posing severe threats to cybersecurity. Graph Neural Networks (GNNs) have become mainstream for social bot detection due to their ability to integrate structural and attribute features, with spectral-based approaches demonstrating particular efficacy due to discriminative patterns in the spectral domain. However, current spectral GNN methods face two limitations: (1) their broad-spectrum fitting mechanisms degrade the focus on bot-specific spectral features, and (2) certain domain knowledge valuable for bot detection, e.g., low homophily correlates with high-frequency features, has not been fully incorporated into existing methods. To address these challenges, we propose HWGNN, a novel homophily-aware graph spectral network with Gaussian window constraints. Our framework introduces two key innovations: (i) a Gaussian-window constrained spectral network that employs learnable Gaussian windows to highlight bot-related spectral features, and (ii) a homophily-aware adaptation mechanism that injects domain knowledge between homophily ratios and frequency features into the Gaussian window optimization process. Through extensive experimentation on multiple benchmark datasets, we demonstrate that HWGNN achieves state-of-the-art bot detection performance, outperforming existing methods with an average improvement of 4.6% in F1-score, while exhibiting strong plug-in compatibility with existing spectral GNNs.

Abstract:
Deep Research (DR) systems iteratively plan, retrieve evidence, and synthesize long-form reports in response to complex, open-ended queries. Existing approaches suffer from context overload, role entanglement, and error propagation over long horizons, while the most capable DR systems remain locked behind massive proprietary models. We present DeepDiver-DR (DD-DR), a multi-agent framework that decomposes the DR process into three specialized roles—planner, information seeker, and writer—coordinated through a persistent, file-based shared workspace. Each role is further equipped with fine-grained, DR-specific tools. To support rigorous evaluation, we introduce RubricWrite-Bench (RW-Bench), a benchmark of 100 rubric-based writing queries with a two-phase assessment of global report quality and point-level rubric fulfillment. Experiments show that DD-DR (30B) establishes a new state of the art among models at the same parameter scale on the DeepResearch Bench, and achieves performance competitive with leading proprietary DR systems on RW-Bench. Ablations further confirm the contribution of the architectural components.

Abstract:
Scope 3 greenhouse gas (GHG) emissions account for the majority of corporate carbon footprints, yet remain difficult to analyze at scale due to sparse disclosures, heterogeneous report document formats, and limited evidence traceability. Existing approaches typically rely on large language models to extract emissions information from ESG reports, but often lack explicit evidence grounding or depend on costly manual annotation and verification to ensure extraction reliability. To address these challenges, we propose Scope3Trace, an evidence-grounded information extraction framework designed to extract interpretable and traceable Scope 3 emissions information from real-world ESG and sustainability reports. The framework integrates a document information extraction pipeline that performs PDF collection and OCR parsing, LLM-assisted page localization and table reconstruction, and hybrid rule–LLM extraction of organization- and building-level emissions disclosures with evidence-grounded verification. Building upon this framework, we further contribute a dual-level, evidence-grounded, multimodal dataset comprising organization-level Scope 3 disclosures extracted from heterogeneous sustainability reports. Scope3Trace enables reliable extraction and transparent integration of heterogeneous sustainability disclosures, achieving high accuracy in extracting Scope 1–3 totals and category-level disclosures from sustainability reports. The code is available at https://github.com/cruiseresearchgroup/Scope3Trace.

Abstract:
Large Language Models (LLMs) have shown impressive reasoning capabilities in agents for complex interactive environments. However, these agents frequently propose actions that violate physical or procedural constraints. Recent neuro-symbolic guards mitigate such errors by compiling explicit admissibility rules from oracle demonstrations, yet treat them as fixed laws—a closed-world assumption that breaks down under sparse demonstrations or environment shift, leaving either missing rules that fail to block unsafe actions or stale rules that wrongly reject valid alternatives. To address this, we introduce AdaptiveGuard ([code](https://github.com/lpxwork666111/AdaptiveGuard)), an Adaptive framework that Guards LLM agents by treating constraints as falsifiable hypotheses. Each rule is represented by a Beta-distributed validity belief mapped to tentative, confirmed, and deprecated tiers; uncertain rule blockers are then challenged through a zero-extra-call falsification gate; and these rule beliefs are continuously updated via contrastive feedback attribution from environment evidence. Benchmark experiments on ScienceWorld and VirtualHome show aggregate gains over strong static constraint guards, especially in novel scenarios. Complementary post-hoc diagnostics under sparse supervision and controlled shift show improved adaptation, while observed failed-trial runs remain below the configured budget cap and the conservative-bias proxy declines over the measured horizon.

Abstract:
Large language models (LLMs) perform inference through hidden representations that evolve across layers and generated tokens, forming complex trajectories in hidden representation space. We study the geometric dynamics of these trajectories for label-free model self-evaluation problem: does the hidden-state tensor already contain model response correctness signals, and where do those signals reside? Our analysis is organized along three intrinsic dimensions of inference-time computation: layer depth, token span, and representation space. Across seven benchmarks spanning mathematical reasoning, commonsense reasoning, and multilingual understanding, and two primary instruction-tuned LLMs, we find that correctness separability is highly localized, consistently concentrating in late-layer representations and limited token regions. Motivated by this, we propose Norm Drop Rate (NDR), a minimal diagnostic metric based on final-layer norm contraction. NDR achieves competitive AUROC across tested models and the best AUPR on Qwen2-7B-Instruct, while remaining the simplest in design and most robust when controlled for response length or changing representation bases. These results suggest that hidden-state trajectory analysis should focus less on increasingly complex diagnoses but more on identifying simple, localized signals.

Abstract:
Long-context reasoning remains a central challenge for large language models, which often fail to locate and integrate key information in extensive distracting content. Reinforcement learning with verifiable rewards (RLVR) has shown promise for this task, yet existing methods are limited by low-confusability distractors and sparse, outcome-only reward signals that cannot supervise intermediate reasoning steps. To address these issues, we introduce LongTraceRL . For data construction, we generate multi-hop questions via knowledge graph random walks and leverage search agent trajectories to build tiered distractors : documents the agent read but did not cite (high confusability) and documents that appeared in search results but were never opened (low confusability), producing training contexts that are far more challenging than those built by random sampling or naive search. For reward design, we propose a rubric reward that uses the gold entities along each reasoning chain as fine-grained, entity-level process supervision. This rubric reward is applied only to responses with correct final answers (positive-only strategy), distinguishing the reasoning quality among correct responses and preventing reward hacking. Experiments on three reasoning LLMs (4B–30B) across five long-context benchmarks demonstrate that LongTraceRL consistently outperforms strong baselines and encourages comprehensive, evidence-grounded reasoning.

Abstract:
Reasoning-oriented language models inherit strong English-centric biases from pre-training, which are further reinforced by English-only post-training. This creates a dual failure mode where models fail both to reason correctly and to maintain reasoning traces in the query language. We study reasoning accuracy and language consistency under English-only post-training. SFT primarily improves language consistency with limited or inconsistent accuracy gains, while group relative policy optimization (GRPO) substantially improves accuracy at the cost of language consistency. Although translated multilingual GRPO improves both objectives, it requires translating reasoning post-training data into multiple languages and may introduce translation noise and information loss. We find that the language-consistency capability acquired by SFT and the accuracy capability acquired by GRPO are approximately orthogonal in the model’s update space. By merging an English LoRA-SFT adapter with an English GRPO checkpoint trained from the same base model, M-GOAL improves MGSM accuracy from 39.1 to 57.8 and language consistency from 17.1 to 75.2 without translated reasoning supervision. This composition outperforms sequential training, its merge coefficient exposes a tunable operating point along the accuracy–consistency frontier, and the pattern holds beyond mathematical reasoning and on benchmarks whose non-English problems are natively authored rather than translated. Our results suggest that multilingual reasoning post-training does not necessarily require multilingual post-training data.

Abstract:
Large language models can detect vulnerable code, yet whether their internal representations encode vulnerability information in a form accessible to standard classifiers remains unresolved. We analyze 8 code models on C programs using three datasets (DELTASECOMMITS, SVEN, and PRECISE BUGS). LLMs represent vulnerability relational encoding —code B is more vulnerable than code A—rather than categorically, making the information inaccessible to standard classification probes. We find 87–88% per-pair alignment across layers L3-L23, yet this direction produces limited categorical accuracy because vulnerable and secure samples form overlapping distributions. This structural property—relational rather than categorical encoding—explains the difficulty of categorical classification in pretrained, general-purpose code LLMs, and motivates ranking and preference-learning approaches as more natural for vulnerability assessment.

Abstract:
The application of large language models (LLMs) to Pali written in the Thai script remains limited in accuracy due to its low-resource setting, complex grammar, and the absence of standardized evaluation benchmarks. In this paper, we introduce ThaiPaliBench, a comprehensive benchmark dataset designed to systematically evaluate LLM proficiency in Thai-script Pali. Our proposed dataset is an LLM-assisted benchmark derived from the Royal Pali Examination corpus and spans Parian Tham levels 1-9. ThaiPaliBench, comprises multiple-choice and sentence-pair tasks that evaluate grammatical knowledge, reading comprehension, natural language inference, and translation understanding. To verify the effectiveness of our benchmark, we evaluate over 10 model variants across 3 commercial and open-source LLM families, including ChatGPT, Claude, Kimi, Grok, Typhoon, GPT-OSS-120B, ThaiLLM, and SEA-LION. Our experiments show that even the best-performing model, Kimi K2.5 (1T), achieves only 68.3% overall accuracy, with performance dropping substantially on grammatical reasoning and semantic inference tasks. These results highlight the unique challenges posed by Thai-script Pali and confirm that ThaiPaliBench serves as a reliable, linguistically grounded benchmark for evaluating LLM proficiency in this low-resource language. Our proposed dataset is publicly available at: https://github.com/ainrg-llm-dev/ThaiPaliBench

Abstract:
We present CyberChainBench, a benchmark for evaluating LLM-based agents on smart contract security across three complementary tasks: vulnerability detection, exploit generation, and patch synthesis. Built from 541 real-world exploit incidents from DeFiHackLabs spanning 9 EVM chains, the benchmark provides end-to-end on-chain evaluation where agents interact with historical blockchain state through isolated evaluation environments orchestrated by Harbor, using tools to read code, trace transactions, and validate exploits on mainnet forks. Each case is anchored to a specific block and includes structured ground truth covering vulnerability type, localization, and attacker profit. Exploits are graded by economic impact on historical forks; patches are validated by replaying historical attacks and legitimate transactions as fail-to-pass test oracles on a proxy-upgradeable subset. We define a five-type vulnerability taxonomy and evaluate multiple agent–model configurations. Results reveal a clear difficulty gradient: the best configuration scores 37.5% on detection, 43.7% on exploitation, but only 23.4% on patching, with the top agent realizing 57.4M in total exploit profit across 200 post-cutoff cases at a cost of 2.39 per case.

Abstract:
In-the-real-world datasets provide a crucial reflection of real-world adversarial distributions and hold significant practical importance for evaluating the safety of T2I models. However, existing in-the-real-world unsafe benchmarks are not tailored for T2I generation: many prompts lack visual specificity or concrete scenarios, producing distorted and low-fidelity images that are misclassified as safe by automated detectors. This acts as a confounder of successful safety alignment and compromises the accuracy of safety measurement. To address this, we propose RW2Unsafe , an automated three-module framework that reformulates source in-the-real-world posts into visually descriptive T2I prompts while preserving the original harmful intent. We use it to construct rw_t2i_4chan , a benchmark of 2,000 high-quality in-the-real-world unsafe T2I prompts. Auditing five advanced T2I models, we find that prior approaches relying on source prompts systematically underestimate safety risks: on Bagel, for instance, the mean unsafe score under source prompts is under one-third of that under rewritten prompts. We hope the proposed framework and dataset could contribute to more accurate safety measurement and safer T2I models.

Abstract:
Video world models are increasingly used in robotic manipulation, yet existing benchmarks mostly evaluate them under valid, feasible, and safe instructions. We introduce RoboTrustBench, a benchmark for evaluating the trustworthiness of video world models under four scenarios: Normal, Constraint-Sensitive, Counterfactual, and Adversarial. Built from real-world DROID episodes, RoboTrustBench contains 1,207 expert-validated instruction–image pairs and a six-dimensional evaluation protocol with 13 fine-grained criteria. Evaluating seven representative video world models with human and MLLM assessment, we find that current models often generate visually coherent videos, but struggle with constraint reasoning, counterfactual grounding, physical interaction, and unsafe-instruction suppression. These results show that visual quality and surface-level instruction following are insufficient for trustworthy robotic video world modeling.

Abstract:
Retrieval-augmented generation (RAG) for multi-hop question answering is limited by retrieving semantically redundant documents, when reasoning instead needs diverse, complementary sources. We introduce Vendi-RAG, which treats set-level semantic diversity as a first-class retrieval objective. It combines an eigenvalue-based diversity score (the Vendi Score) with query relevance into a single Vendi Retrieval Score, embedded in an iterative loop that uses an LLM-based answer-quality signal to adaptively rebalance diversity and relevance. On HotpotQA, MuSiQue, and 2WikiMultiHopQA, Vendi-RAG improves accuracy by +2.4 , +1.6 , and +2.4 points over the strongest diversity-aware iterative baseline (Adaptive Retrieval + MMR). A fixed- s variant captures most of the gain at no extra inference cost, while the LLM-judge loop adds a smaller but significant gain on hard queries. Results hold across five instruction-tuned backbones and two dense retrievers, identifying set-level retrieval diversity as a primary, controllable axis for improving multi-hop RAG.

Abstract:
Large Vision Language Models have integrated reasoning capabilities, elevating cognitive performance to new levels. However, existing evaluations either focus solely on perception or rely on specific domains such as maths or coding. Evaluation for reasoning capabilities that align with an open-world environment is still required, especially one that considers perception and reasoning jointly. To bridge this gap, we propose to evaluate LVLMs by exploiting visual illusions as a diagnostic tool. Visual illusions are phenomena in which the human visual system misinterprets objective signals, resulting in an understanding that deviates from reality. We constructed IllusionReasoning, a benchmark of illusion images collected from the real world, incorporating diverse annotated question-answer pairs. Based on IllusionReasoning, we show that the reasoning capabilities of a wide range of LVLMs are not as advanced as claimed. Our work provides new insights into LVLMs and offers future direction for optimisation.

Abstract:
Data curation for large language models (LLMs) has been extensively studied for both the pre-training and post-training stages, yet the intermediate mid-training phase—bridging general pretraining and task-specific fine-tuning—remains critically underexplored. Most existing mid-training data for state-of-the-art models remains proprietary. Meanwhile, the Hugging Face platform hosts a huge amount of community-contributed datasets, yet no systematic effort has been made to refine this rich ecosystem into a unified high-quality corpus. We present HuggingBrew , a large-scale mid-training corpus constructed through a principled, multi-step curation pipeline applied to the massive publicly accessible Hugging Face datasets. The pipeline operates at two complementary granularities: dataset-level operations act on each dataset independently, while corpus-level operations are applied jointly across all retained corpus. We construct three progressively curated variants— \textsfDraught , \textsfCraft , and \textsfReserve —containing 1.6T, 450B, and 40B tokens, respectively. Continual pretraining experiments under a 100B-token budget show that series yields substantial improvements over baselines across mathematical reasoning, general knowledge, and code generation, validating the untapped value of community-contributed data for mid-training.

Abstract:
Code generation requires executable reasoning, making test-time chain-of-thought (CoT) search an attractive strategy for improving solution quality. However, existing CoT scaling methods often face a tension between underthinking, where shallow reasoning misses important algorithmic cases, and overthinking, where verbose or redundant reasoning consumes substantial tokens without improving correctness. We propose LogitsCoder, a lightweight framework that revisits CoT search from the perspective of logits-level control. LogitsCoder improves efficient reasoning search through two complementary mechanisms: Logits Preference Decoding adaptively calibrates local token expansion using public-test feedback, while Logits Rank Based Path Selection explores and selects compact high-confidence reasoning trajectories based on logit-level stability. Together with Thoughts Aggregation, LogitsCoder refines reasoning paths without relying on costly MCTS-style tree expansion. Extensive experiments demonstrate that LogitsCoder produces more efficient and higher-quality reasoning chains, leading to superior code generation performance compared to baseline methods.

Abstract:
Multi-hop question answering needs evidence that is scattered across many sentences and ordered along a reasoning chain. Existing RAG systems struggle with this setting because flat chunk retrieval can miss bridge facts, binary graph structures lose n-ary information, and unordered top-K contexts leave evidence composition to the answer model. We introduce HyperPathsRAG, a hypergraph-based RAG framework that aligns query decomposition with the corpus index. It decomposes questions into n-ary sub-questions aligned with hypergraph facts, jointly matches all sub-questions to retrieve reasoning paths, scores paths by chain completeness, and presents the retrieved evidence in reasoning order. We evaluate in the candidate-passage setting on the 1k-question hard subsets of HotpotQA, 2Wiki, and MuSiQue, with three answering LLMs. HyperPathsRAG attains the best average Exact Match on every backbone (up to 42.2 EM / 56.2 F1 with Llama-3.3-70B-Instruct), and the best average F1 on two of three backbones; ablations attribute the bulk of the gain to the hypergraph index and the multi-target retriever. Our code is available at https://anonymous.4open.science/r/HyperPathsRAG-AE05/.

Abstract:
Anticipating the social impacts of a policy prior to implementation requires institutional consistency: multi-agent reasoning must remain aligned with the rules specified in policy documents. However, existing LLM-based multi-agent simulations struggle to maintain such fine-grained institutional rules when policy documents are provided only as prompt context. In this paper, we propose TIGRIS, a Theory-guided framework for Institutionally Grounded policy Impact Simulation. TIGRIS (1) constructs a structured policy knowledge graph from natural-language policy documents, using the Institutional Analysis and Development framework to define institutional positions and Institutional Grammar 2.0 to encode behavioral rules; and (2) instantiates actor nodes as persona agents while anchoring them to a Theory of Change framework to map the causal pathways of policy impacts. Evaluated across four AI talent development policies and three LLMs, TIGRIS achieves the lowest macro-average MAPE, indicating that theory-guided institutional grounding supports domain-aligned reasoning for policy impact analysis.

Abstract:
Misinformation detection has predominantly been studied in monolingual or language-aligned settings, where claims and evidences are assumed to be written in the same language. However, this assumption does not reflect real-world information ecosystems, in which misinformation frequently spans multiple languages and verification requires cross-lingual reasoning over heterogeneous evidence sources. As a result, cross-lingual claim-evidence reasoning remains largely unexplored and has not been systematically studied, and existing methods are not explicitly designed to support such scenarios. To address this gap, we introduce a novel setting for multilingual misinformation detection with cross-lingual evidences and propose GraDuLa, a graph-based dual-branch language-agnostic modeling for misinformation detection. GraDuLa projects evidential information into a language-agnostic subspace, disentangling semantic representations from language-specific associations through dual-branch graph modeling. For evaluation, we construct and release a benchmark GraDuLaEval and compare our approach against strong baseline methods. We further plan to release the dual-branch graph representations as an additional resource for future research.

Abstract:
Language models have demonstrated remarkable advancements in embodying diverse personas, enhancing their effectiveness as conversational agents. Simultaneously, there have been significant strides in these models in processing and integrating visual information. However, despite images being a natural carrier of persona attributes (e.g. age, occupation, or location), it remains largely unexplored whether multimodal language models (MLLMs) can faithfully ground visual persona information. In this work, we introduce persona embodiment as a novel diagnostic for visual grounding in MLLMs. Unlike simple visual recognition, this task requires models to extract multiple persona attributes from an image, integrate them into a coherent representation, and apply them to downstream language tasks. To this end, we introduce a modality-parallel dataset of personas, containing 2,400 prompts each represented in four modalities. Across our experiments, we find that models frequently underuse visual information relative to text, and that this neglect persists even when richer visual information is clearly available, highlighting underlying limitations and paving the way for future research to bridge this gap.

Abstract:
Recent multimodal benchmarks have significantly expanded the evaluation of visual reasoning in vision-language models, but they primarily focus on holistic semantic understanding. Engineering problems can instead require models to connect visual regions with exact coordinate-level representations and reason over precise quantities. Semiconductor layout analysis is a representative example, yet this fine-grained multimodal grounding capability remains largely unexplored. We introduce ChipLayoutBench, the first benchmark for general semiconductor layout understanding, with text-only, image+metadata, and multimodal settings across diverse reasoning and grounding tasks. Evaluations of eight recent VLMs show that text-based representations achieve the strongest overall performance, while image+metadata inputs remain substantially more challenging. Moreover, combining both modalities often fails to improve over text alone and can even degrade performance, suggesting persistent difficulties in effectively grounding visual information with structured geometric representations. Dedicated VLM-specific tasks further expose these challenges in fine-grained cross-modal alignment and image-grounded reasoning. Our benchmark provides a domain-grounded testbed for advancing geometric reasoning and fine-grained multimodal grounding over structured engineering data.

Abstract:
Warning: This paper contains examples of stereotypes and social bias. LLMs are increasingly used in interactive settings by the general public, making the evaluation of model behavior in multi-turn conversational scenarios important for safety, including stereotyping-related harms. However, existing multi-turn social bias evaluations often rely on pre-specified or template-based user inputs that do not adapt to model responses and typically assume a fixed dialogue length in advance. In this paper, we study social bias dynamics in response-conditioned multi-turn interactions using a controlled evaluation protocol that generates follow-up user queries from the evolving dialogue history and allows evaluation over variable numbers of turns. Experimental results show that LLMs exhibit social bias even in coherent, response-conditioned multi-turn interactions, revealing late-emerging bias, non-monotonic bias patterns, and bias re-emergence. These results motivate evaluations that extend beyond fixed-turn, pre-scripted protocols. Our findings highlight the importance of analyzing social bias as a turn-level dynamic phenomenon.

Abstract:
Chart reasoning poses a significant challenge for Vision Large Language Models (VLLMs). While Reinforcement Learning (RL) has emerged as a promising solution, existing outcome-based RL methods overlook intermediate errors, whereas process-based approaches remain difficult in verifying abstract logical steps. We therefore analyze chart-reasoning failures to identify an intermediate supervision signal that is both relevant to the task and amenable to objective verification. Our pilot study suggests that early visual-perception errors constitute a prominent source of failure in the chart-reasoning setting. Guided by this insight, we propose Chart-EARL, which constructs verifiable early visual anchors for targeted chart-reasoning supervision. We first build a structured training corpus that explicitly decouples visual perception from logical deduction. Building upon this structure, our approach optimizes the model with rewards for both accurate extraction of descriptive visual features and final-answer correctness. This strategy provides direct and verifiable feedback on early visual grounding while keeping supervision on the final reasoning objective. Experiments show clear performance improvements on complex chart reasoning benchmarks.

Abstract:
We propose Scaled Autoregressive Projection Steering (SAPS) for generating incongruity-based figurative language from literal inputs, extending contrastive paraphrasing-based steering vectors by autoregressively recomputing projection coefficients during decoding. SAPS comprises two novel components: (A) Autoregressive Projection Steering (APS), which recomputes the projection coefficient ( c t ) at each decoding step t , adapting the steering signal to the evolving token distribution, and (B) a scaling coefficient 𝛼 whose effective range is layer-agnostic, enabling compact tuning of figurative intensity. Experiments across three instruction-tuned language models show that APS improves grammatical stability over fixed sentence-level steering while 𝛼 enables fine-grained figurative control, together yielding gains over prior steering methods and significantly increasing robustness to layer selection and steering strength. Cross-dataset experiments further confirm that steering directions generalize across domains. This is the first work to apply activation steering to incongruity-based figurative language generation. Our code is available at: https://github.com/SharathHN/saps-steering.

Abstract:
Real-world clinical competence demands more than answering medical knowledge questions—it requires proactively gathering incomplete patient information, selecting appropriate examinations, and formulating safe, cost-conscious treatment plans through multi-turn interaction. Existing medical LLM benchmarks rely predominantly on static QA formats, failing to assess interactive information acquisition, tool use, or cost–quality tradeoffs. We propose MedCPEnv , a reproducible, cost-aware simulated hospital environment for evaluating and training medical LLM agents. Built from MIMIC-IV-NOTE discharge notes, MedCPEnv comprises 48,803 curated cases, a 1,000-case benchmark, and a 371,550-entry medical QA knowledge base, exposing four deterministic action types (ASK, EXAM, KNOWLEDGE, FINAL) to eliminate high-variance LLM role-playing artifacts. Evaluating eight frontier models, GPT-5.4 achieves the highest composite score (3.69/5) and diagnostic accuracy (60.3%), yet treatment accuracy remains only 19.4%, identifying treatment plan generation as the primary bottleneck. Substantial inter-model variation in the cost–quality tradeoff further demonstrates that clinical correctness alone is insufficient to characterize agent behavior. GRPO-based Agentic RL on Qwen3-4B shows that medical quality rewards consistently improve tool-use strategies, raising the composite score from 2.97 to 3.14 and diagnostic accuracy from 26.0% to 31.2%, establishing MedCPEnv as both a rigorous evaluation benchmark and a reproducible training environment for medical agent research.

Abstract:
Large Language Models (LLMs) have demonstrated impressive capabilities in simulating diverse human behaviors and personalities. However, existing methods for personality control, which include prompt engineering and standard Supervised Fine-Tuning (SFT), typically treat personality traits as discrete categories (e.g., "Extroverted" vs. "Introverted"), lacking the ability to precisely control the intensity of a trait on a continuous spectrum. In this paper, we introduce Fusian, a novel framework for fine-grained, continuous personality control in LLMs. Fusian operates in two stages: (1) Trajectory Collection, where we capture the dynamic evolution of personality adoption during SFT by saving a sequence of LoRA adapters, effectively mapping the continuous manifold of a trait; and (2) RL-based Dynamic Fusion, where we train a policy network using Reinforcement Learning to dynamically compute mixing weights for these frozen adapters. By sampling from a Dirichlet distribution parameterized by the policy network, Fusian fuses multiple adapters to align the model’s output with a specific numerical target intensity. Experiments on the Qwen3-14B model demonstrate that Fusian achieves high precision in personality control, significantly outperforming baseline methods in aligning with user-specified trait intensities.

Abstract:
Multi-turn jailbreak attacks demonstrate that harmful intent can be distributed across dialogue, yet existing methods obscure what conversational mechanisms drive vulnerability. We introduce Blueprint, a safety-evaluation framework that separates a factorized social-influence strategy space from WorldviewSim, a cross-turn situational context module. Monte Carlo Tree Search optimizes turn-level combinations of 18 theory-grounded influence factors across a four-turn trajectory. Across six frontier models, Blueprint achieves near-ceiling attack success rates on major open-weight and proprietary models, while requiring the fewest average queries (2.46). The resulting trajectories further reveal model-specific vulnerability among resistant targets: each responds to distinct influence factors and strategy transitions, yet all share a common recovery pathway. Specifically, shifting toward concrete, executable task framing consistently escapes hard-refusal states. Ablations confirm that operational cues matter most: making requests actionable has the largest impact, gain framing is unusually potent, and some legitimacy appeals can backfire. These findings suggest that robust multi-turn safety requires monitoring not only harmful content, but also how dialogue state makes unsafe requests appear concrete and locally executable. The code is open-sourced at: https://github.com/charlottec1583/Blueprint.

Abstract:
How do LLM agents come to both understand environments they act in and master tasks set within them? Through controlled experiments combining world-model training (next-state prediction) and policy training (reward maximization), we investigate this question. We dissect the resulting models through their additive parameter updates. Geometrically, we find effective world-model updates are low-rank and share an input-feature subspace with policy updates while writing to nearly orthogonal output directions, whether trained separately or sequentially. However, we find that, in projection interventions, the sequential update induces more robustness than separate policy RL when removing the world model’s leading input directions, suggesting that it has learned alternative input pathways. Behaviorally, we find the sequentially trained agent explores a wider range of states and actions. Based on this, we ask: does policy training preserve world knowledge as well as it could? We probe this with training-free merging built on the geometrically motivated input basis plus an online world-model loss during policy RL, and show both improve over the untreated baseline. Our findings suggest world knowledge and task-directed ability can be learned in geometrically complementary forms, and that future post-training pipelines should consider how best to engineer the interface between them.

Abstract:
Multimodal Large Language Models (MLLMs) exacerbate safety risks by introducing vulnerabilities across multiple modalities, such as language and vision. Current MLLM safety evaluation tools, however, suffer from major limitations: 1) English-centric dataset construction, and 2) a focus on generic risks that are not tied to local cultural contexts. We introduce KSAFE-MM, a benchmark for Korean multimodal safety evaluation that covers both general safety risks and culture-specific vulnerabilities. KSAFE-MM consists of two complementary parts: KSAFE-MM-G evaluates globally shared risks in Korean contexts through linguistic contextualization, which transforms generic safety queries into contextually grounded multimodal samples. In contrast, KSAFE-MM-C targets safety vulnerabilities that are culture-dependent, using localized visual queries drawn from real-world contexts. It pairs these visual queries with jailbreak-style textual queries to cover multimodal safety risks involving cultural visual cues and malicious textual intent. We evaluate 12 state-of-the-art MLLMs on KSAFE-MM and reveal culturally grounded vulnerabilities that translation-based evaluation fails to capture. Notably, jailbreaking strategies substantially amplify attack success rates, with ProgramExecution yielding up to 74.2% ASR compared to 13.4% for standard queries. Furthermore, we identify a systematic trade-off between safety and over-refusal, where models achieving low ASR tend to exhibit excessive refusal behavior on benign queries. These findings highlight the urgent need for culturally grounded safety evaluation beyond English-centric benchmarks.

Abstract:
Vision–language models (VLMs) are increasingly used in clinical pipelines where a chest X-ray is interpreted alongside retrieved reports, preliminary notes, or prior imaging. Existing benchmarks measure whether models answer correctly in isolation, but not whether they preserve a correct image-only decision when plausible context conflicts with the image. We introduce Multi-Context Chest X-ray (MC-CXR), a benchmark of 240 cases expanded into 2,522 instances that isolates context-induced disruption through paired perturbation. Each case fixes the current image and target finding while presenting matched reliable and misleading context across text and prior CXR, with visual overlays where available. MC-CXR defines three task families and two paired metrics, the switch-to-wrong rate and the context-aligned error rate. We evaluate ten VLMs spanning open-source general, medical-domain, and closed-source systems. Image-only accuracy is necessary but insufficient. Mean switch rates range from 45.6–78.1% across misleading textual sources and 35.7–61.7% across misleading visual sources. Among switched predictions, 74.6% align with the misleading label for text versus 17.6% for visual context, a 57.0-point gap (95% CI 50.9–62.8). This text–visual asymmetry is observed under the standardized direct-answer protocol. The dataset is available on PhysioNet.

Abstract:
Backdoor attacks pose a serious threat to the safety and reliability of Large Language Models (LLMs), as they cause models to behave normally on clean inputs while producing attacker-specified responses when hidden triggers are present. Removing such unknown backdoors is particularly challenging when the defender does not know the attack types or the internal mechanisms formed during backdoor training. In this work, we propose a simple but effective backdoor removal method based on internal mechanisms shared across different backdoors. First, we show that different backdoors with the same task (attack objective) induce similar trigger-activated changes in their internal activations. Motivated by this observation, our proposed method intentionally embeds a backdoor with a known trigger ( dummy backdoor ) and then removes it through further fine-tuning on dummy-triggered inputs paired with clean responses. Because the dummy and unknown backdoors exhibit similar activation changes, this removal process can also reduce the effect of the unknown backdoor. We evaluate our method on three backdoor attack types across multiple model families. Experimental results show that our method substantially reduces the attack success rate of the unknown backdoor while preserving model utility, outperforming representative existing defense methods in both backdoor removal effectiveness and utility preservation. These findings suggest that a defender-controllable backdoor can serve as a helpful proxy for mitigating unknown backdoors in generative LLMs. Our source code is available at https://github.com/kaz-iwahana/dummy_backdoor_defense .

Abstract:
Academic paper search is a fundamental task in scientific research, yet most existing approaches rely on rigid, predefined workflows that struggle with complex, conditional queries. To address this limitation, we propose PaperScout, an autonomous agent that reformulates paper search as a sequential decision-making process. Unlike static workflows, PaperScout dynamically decides whether, when, and how to invoke search and expand tools based on accumulated retrieval context. However, training such agents presents a fundamental challenge: standard reinforcement learning methods, typically designed for single-turn tasks, suffer from a granularity mismatch when applied to multi-turn agentic tasks—where token-level optimization diverges from the granularity of sequence-level interactions—leading to noisy credit assignment and unstable training dynamics. We introduce Proximal Sequence Policy Optimization (PSPO), a process-aware, sequence-level policy optimization method that aligns optimization with agent–environment interaction. Comprehensive experiments on both synthetic and real-world benchmarks demonstrate that PaperScout significantly outperforms strong workflow-driven and RL baselines in both recall and relevance, validating the effectiveness of our adaptive agentic framework and optimization strategy.

Abstract:
Recent OCR models achieve strong results on document benchmarks, with several 1–2B OCR-specialized models surpassing much larger general multimodal LLMs (MLLMs). However, complex real-world documents often contain blur, cropping, curved text, handwriting, and unfamiliar structures, which remain challenging for current OCR systems. We trace this gap to a shift in the error source: errors on canonical benchmarks mainly arise from layout analysis, including localization, element classification, and reading order, while errors under these challenging conditions mainly arise from content recognition. To this end, we propose CL-OCR, a two-stage framework that first fuses semantic structure from an MLLM-based detector with geometric candidates from a vision detector to obtain calibrated layouts, then enhances recognition via a shared LoRA adapter gated by a lightweight router. We further construct OCRWildBench and a companion training split to study these conditions. Experiments show that CL-OCR remains competitive on canonical benchmarks and achieves the strongest overall performance among OCR-specialized models on OCRWildBench.

Abstract:
Ancient Chinese artifact recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current model capabilities. However, existing benchmarks suffer from “fragmentation”, manifested in limited temporal coverage, limited medium diversity, and incomplete script types. Therefore, we present Ancient-Bench, a comprehensive benchmark of 2,700 images for ancient Chinese artifact text recognition, featuring three dimensions: Multi-millennial (spanning 3,000 years of character evolution), Multi-medium (covering nine artifact categories), and Multi-script (encompassing seven historical script forms). To enable consistent and fair evaluation across heterogeneous media, we further define three annotation standards tailored to the medium-specific characteristics of ancient texts: symbol standardization, character standardization, and parsing standardization. Extensive experiments on Ancient-Bench covering general Vision-Language Models (VLMs) and OCR-specialist models reveal that ancient Chinese recognition remains fundamentally unsolved, with persistent challenges in variant characters, specialized symbols, and hallucination. The data and evaluation code will be publicly available.

Abstract:
Large Language Model (LLM)-driven multi-robot systems struggle with long-horizon tasks due to planning and execution complexities. As task dependencies grow, centralized planners suffer from context explosion and hallucinations, while minor runtime errors can disrupt the entire operation. Current monolithic methods overload LLMs and cause unnecessary global halts during error recovery. To address this, we introduce Hierarchical Planning with Localized Failure Recovery (HiP-LR). HiP-LR uses a two-level hierarchy to divide complex instructions into dependency-isolated subtask groups. A central agent allocates global tasks, while local agents execute specific plans, keeping LLM contexts compact. Furthermore, a PDDL-based dual-memory structure enables failure-localized replanning: if an error occurs, only the affected group pauses to replan, allowing others to continue uninterrupted. Evaluated on MAP-THOR, HiP-LR improves the task success rate by 38% and reduces replan-wait costs by 90% over state-of-the-art baselines. Furthermore, HiP-LR demonstrates highly robust recovery capabilities on our novel failure-injection benchmark, RECAP-THOR.

Abstract:
Document-level relation extraction (DocRE) aims to identify relations for an entity pair within a document. Existing methods can be broadly classified into two categories: direct encoding of the entire document or enhancement using extracted evidence sentences. However, the former often introduces noise unrelated to relations, while the latter is heavily dependent on the quality of evidence extraction. Moreover, these DocRE models typically use an adaptive threshold to predict all potential relations for an entity pair. As a result, class imbalance in DocRE often leads the model to learn a high threshold for an entity pair, which in turn causes the model to frequently predict that the entity pair has no relation. To address these issues, we propose a Multi-Discrimination framework (MD-RE) that does not rely on evidence sentences. MD-RE employs three discriminators with dynamically adjusted thresholds to independently predict relations, and aggregates their outputs via a weighted fusion strategy. Furthermore, we propose an Adaptive Threshold Shifted Loss (ATSL), which encourages a lower threshold to alleviate the high false negative rate resulting from class imbalance. Experiments on three datasets demonstrate that MD-RE achieves new state-of-the-art results. In addition, ATSL significantly improves the performance of various existing DocRE models. Moreover, combining other losses with MD-RE also yields competitive results. Code is available at https://github.com/xhm-code/MD-RE.

Abstract:
Recent multi-agent code generation frameworks often verify natural-language plans by simulating their behavior on test cases before code generation. However, such verification remains unreliable because model hallucination may cause flawed plans to be incorrectly accepted. Inspired by neuro-symbolic reasoning, we propose PaveDSL, which transforms natural-language plans into structured designs expressed in a domain-specific language. Based on this structured representation, PaveDSL combines symbolic checks with neural inference to examine plans before code generation. Specifically, PaveDSL applies two complementary verifiers in parallel. The deterministic symbolic verifier checks structural constraints without invoking an LLM, while the agent verifier performs graph-guided semantic reasoning over the structured design. If verification fails, a repair agent localizes errors and revises the design before code generation. Experiments on multiple benchmarks with different LLMs show that PaveDSL consistently improves the performance of baseline code generation methods.

Abstract:
With the rapid development of large language models (LLMs), generative reward models (GRMs) have been widely adopted for reward modeling and evaluation. Previous studies have primarily focused on training specialized GRMs by optimizing them on preference datasets with the judgment correctness as supervision. While it’s widely accepted that GRMs with stronger problem-solving capabilities typically exhibit superior judgment abilities, we first identify a significant solve-to-judge gap when examining individual queries. Specifically, the solve-to-judge gap refers to the phenomenon where GRMs struggle to make correct judgments on some queries (14%-37%), despite being fully capable of solving them. In this paper, we propose the Solve-to-Judge (S2J) approach to address this problem. Specifically, S2J simultaneously leverages both the solving and judging capabilities on a single GRM’s output for supervision, explicitly linking the GRM’s problem-solving and evaluation abilities during model optimization, thereby narrowing the gap. Our comprehensive experiments demonstrate that S2J effectively reduces the solve-to-judge gap by 16.2%, thereby enhancing the model’s judgment performance by 5.8%. Notably, S2J achieves state-of-the-art (SOTA) performance among GRMs built on the same base model while utilizing a significantly smaller training dataset.

Abstract:
Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing Arabic evaluations mainly target downstream tasks and do not directly test controlled morphological generation from explicit lexical and feature-based input. We introduce YallaMorph, a large-scale benchmark for Arabic morphological generation covering verbs, nouns, adjectives, their cliticized forms, and invalid configurations. We evaluate multilingual and Arabic-oriented LLMs under diacritized and undiacritized settings over 600K benchmark entries. Results show that Arabic morphological generation remains difficult, especially for cliticized, unseen, and morphologically rare forms.

Abstract:
Large Language Models (LLMs) have advanced financial automation through Retrieval-Augmented Generation (RAG), yet hallucinations remain a critical barrier to deployment in high-stakes environments. Existing benchmarks focus on single-turn, English-centric tasks, leaving the multi-turn dynamics and linguistic-regulatory nuances of the Korean financial domain unaddressed. We introduce K-FinHallu, the first benchmark for hallucination detection in multi-turn Korean financial RAG, designed as a controlled evaluation of source faithfulness and answerability. We construct multi-turn dialogues from authentic Korean financial documents and inject hallucinations under a proposed hierarchical taxonomy based on context answerability that explicitly accounts for justified abstention. Benchmarking Korean-centric, open-source, and closed-source LLMs as hallucination detectors, we find that even the strongest models struggle with fine-grained financial diagnostics and refusal behavior. While fine-tuning an 8B model on our training split yields performance competitive with frontier LLMs, justified abstention remains the weakest axis across all evaluated models. K-FinHallu and code are publicly available at https://github.com/eunbyeol-cho/K-FinHallu.

Abstract:
Artificial intelligence has long been evaluated by its ability to perform computation and structured reasoning. While recent large language models (LLMs) demonstrate strong capabilities in language generation, code synthesis, and structured problem solving, it remains unclear whether they can construct formal computational mechanisms from high-level specifications. In this work, we investigate whether modern LLMs can synthesize pushdown automata (PDAs) from language descriptions expressed in natural language or mathematical form. Because PDA construction requires explicit reasoning about states, transitions, and stack-based memory, it provides a controlled test of structured symbolic reasoning. We introduce the task of pushdown automaton synthesis and develop an automated evaluation framework that verifies generated PDAs through structural buildability checks and execution-based validation. Using a benchmark of context-free languages with labeled test cases, we evaluate contemporary LLMs and analyze their ability to construct and execute formal computational machines. Our results show that current state-of-the-art LLMs struggle to reliably construct executable PDAs, indicating that synthesizing computational mechanisms requiring coordinated reasoning over state transitions and stack-based memory manipulation remains challenging for present-day models. All code and data are available at: https://github.com/Dip-to/Pushdown-Automaton-Synthesis-Dataset.git

Abstract:
Many benchmarks show that large language models can answer direct questions about culture. We study a different question: do they also change how they speak when culture is only implied by the situation? We introduce a triad-diagnostic framework; applying it to four deployed endpoints to evaluate 60 culturally grounded conversational scenarios. We examine responses across five languages in three conditions: a neutral baseline (Prompt A), an explicit cultural instruction (Prompt B), and implicit situational cueing (Prompt C). We use an LLM-judge ensemble with explicit rubrics to score responses on 12 pragmatic features covering deference to authority, individual-versus-group framing, and uncertainty management. We define Pragmatic Context Sensitivity (PCS) as the fraction of the judge-scored Prompt A → B feature shift that reappears under Prompt A → C. Across four deployed LLM endpoints and five languages (English, German, Hindi, Nepali, and Urdu), the primary stable-only PCS mean is 0.196 ( SD = 0.113 ), indicating that the tested models recover only about one-fifth of the explicitly prompted rubric-aligned shift. Transfer is strongest for authority-related cues (0.299) and weakest for individual-versus-group framing (0.120). We argue that multilingual cultural pragmatics is an explicit-versus-implicit deployment problem, not only a factual knowledge problem.

Abstract:
Masked diffusion language models (DLMs) generate text through iterative parallel decoding, but at low step budgets, committing many tokens independently can produce premature termination or repetitive outputs. Continuous embedding-space denoising can mitigate this failure mode by maintaining soft states for all positions, yet training such models from scratch at scale remains challenging. We show that a pretrained masked DLM can instead be efficiently adapted to support continuous denoising. Starting from LLaDA-8B-Instruct, we perform only 1,000 steps of continued pretraining with Discrete Stochastic Localization (DSL), replacing binary masking with continuous per-token Gaussian noise mapped to soft embedding mixtures. The adapted model supports SDE-based inference that evolves all positions jointly and defers hard token commitment to the final step. At ≤ 16 denoising evaluations, DSL-LLaDA substantially mitigates the premature-termination and repetition tradeoff in open-ended generation and achieves the best ROUGE-1 among the evaluated diffusion baselines on all four zero-shot summarization benchmarks. The same adaptation also enables selective correction of corrupted tokens while preserving more than 98% of clean tokens. Same-compute controls indicate that these behaviors arise from continuous-noise exposure rather than additional training alone.

Abstract:
When multiple normative frameworks yield equally valid answers, which framework should a language model select, and can it answer correctly within it? We formalize this as normative pluralism and study it in Islamic finance with a four-choice taxonomy that decomposes framework selection from within-framework correctness. The decomposition exposes a failure mode invisible to two-choice instruments, the stereotype trap: a model leans toward the culturally appropriate framework but produces factually wrong answers within it. Across twelve models, two languages, and fifty demographic signals, cultural cues redirect framework selection at severe cost: redirection and correctness are inversely coupled (r=−0.89 Arabic, r=−0.81 English). Under the strongest signal, large open-weight models reach 97% Islamic-frame selection, a result two-choice instruments would report as near-perfect alignment, yet 57-66% of those responses misquote the Islamic rule. We argue that cultural default bias is competence-conditioned routing: models default to the framework where they are most accurate, and the cues prescribed as mitigation expose the competence gap they were meant to correct.

Abstract:
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 18 leading LLMs on translated variants of the same prompt. Our analysis reveals a previously underexamined failure mode: cross-language exact agreement is only 20.69%, indicating that identical prompts frequently receive different safety judgments across languages, while cross-model exact agreement is just 2.77%. Some models over-refuse or overflag politically sensitive content, whereas others fail to flag unsafe generations or shift failures into ambiguous outputs. We quantify these behaviors using prompt-level entropy, category bias scores, and multilingual consistency indices. Our findings demonstrate critical safety generalization failures in multilingual LLMs and show that safety alignment does not reliably transfer across languages. We release IndicSafe , the first benchmark to enable culturally informed safety evaluation for Indic deployments, and advocate for language-aware alignment strategies grounded in regional harms.

Abstract:
Recent advances in LLM agents have enabled complex cognitive capabilities, such as multi-step reasoning, planning, and tool use, that increasingly position these agents as collaborative partners in human workflows. Effective collaboration, however, requires collaborators to continuously maintain and align mental models of their own reasoning, partners’ intentions, and shared goals during the collaborative process. Today’s agents rarely develop such capabilities since they are primarily optimized for task completion rather than for maintaining the underlying mental models required for collaborative processes. Moreover, the community lacks authentic human collaboration data with action-level mental model annotations that could guide agents toward collaborative competence. To bridge this gap, we present ALMANAC, a dataset of Action-Level Mental model ANnotations for Agent Collaboration built from the Map Task, a classic dyadic routing task from social science. ALMANAC contains 2,987 collaboration actions, each paired with theory-informed mental model annotations that record the participants’ self-reasoning, perceived partner intent, and perceived team goal. We benchmark six LLMs on predicting humans’ next-turn behavior and mental models. Our results demonstrate ALMANAC’s utility in evaluating models’ ability to simulate human collaborative behaviors and infer their underlying mental models. The project page is available at https://human-agent-collab.hailab.io/almanac.

Abstract:
Alignment in Large Language Models (LLMs) studies whether model responses satisfy the Helpful, Harmless, and Honest (HHH) objectives. Imbalances across these objectives may produce divergent behavior across Reasoning, Safety, Value, and Pluralistic dimensions. Existing benchmarks including CR-LLM, SAFETUNEBENCH, VALUEBENCH, and PERSONA study these dimensions separately and capture partial aspects of HHH behavior. To study HHH behavior across overlapping dimensions, we introduce RSVP-HHH, a framework for HHH evaluation across all four dimensions. We construct RSVP-X, an English-HHH dataset containing 9,228 prompt-response pairs spanning 106 domains through Step 1 Latent Behavioral Projection, Step 2 Behavioral Region Overlap, Step 3 Density-Aware Downsampling, Step 4 Behavioral Compatibility Evaluation, and Step 5 Latent Refinement. Experimental results on RSVP-X show that TrinityX (46.6%) outperforms AlignX (45.0%), SAGE (49.1%) outperforms VAL-BENCH (48.2%) and R-Zero (47.6%), DeepSeek (45.0%) outperforms Qwen (43.7%), and Gemini (49.6%) outperforms Claude (47.7%) in Avg across HHH objectives.

Abstract:
Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references. We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously. We reformulate two corpora into evidence-grounded multi-document QA benchmarks—MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues—and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K–1M characters. Across eight frontier proprietary and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries. EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge. Data and code are available at https://github.com/debobanerjee/LC-EBG.

Abstract:
While reinforcement learning with verifiable reward (RLVR) enhances reasoning in large language models, its efficacy in long-horizon multimodal tasks remains underexplored. Visual Sudoku puzzles pose a representative challenge, requiring multi-step deductive reasoning over both local constraints and global dependency chains. To this end, we introduce FG-Sudoku, a fine-grained benchmark categorized by grid size and required deductive steps. Evaluations reveal that frontier MLLMs fail on complex long-range dependencies (e.g., 9×9 puzzles). During post-training, this near-zero success rate causes severe reward sparsity, halting standard RLVR. We propose Adaptive Curriculum Group Relative Policy Optimization (AdaC-GRPO) to overcome this. Unlike rigid or prior-based schedules, AdaC-GRPO dynamically constructs training batches by tracking the model’s "Zone of Proximal Development" via online rollout accuracy. Experiments demonstrate that AdaC-GRPO enables Qwen2.5-VL-7B to approach the performance of Gemini-2.5-Flash while bridging 62.2% of the overall accuracy gap to Gemini-2.5-Pro, with its generalizability further validated across InternVL-3.5 and Qwen3-VL.

Abstract:
Conversational stance detection, which aims to identify users’ attitudes in conversation threads, has attracted increasing attention in recent years. Yet, mainstream LLM-based approaches remain prone to biases caused by spurious correlations. Existing debiasing methods either rely on indiscriminate multi-step prompting, which can induce unnecessary reasoning and overthinking, or require expensive fine-tuning on curated data. To address these challenges, we propose ExpCSD, an experience-based debiasing framework grounded in latent thinking paths that follows a debias-only-when-needed principle. ExpCSD first assesses whether an LLM’s latent thinking path is biased and selectively retrieves and injects historically similar debiasing experiences only when necessary, without updating model parameters. Experiments on public benchmarks show that ExpCSD outperforms strong baselines in stance detection accuracy while avoiding overthinking and costly fine-tuning.

Abstract:
GUI agents powered by Multimodal Large Language Models (MLLMs) are advancing rapidly, yet most existing benchmarks evaluate only task-level success or failure—leaving a fundamental question unanswered: what capabilities do GUI agents most need to succeed in real-world settings? We introduce UniGUI-Bench , a benchmark that jointly verifies task-level outcomes and process-level capability diagnosis. It spans 6 device platforms and 23 application scenarios with bilingual support, comprising over 10,000 evaluation items. Its core design links the two evaluation axes: each task failure can be traced to specific capability deficiencies, enabling fine-grained attribution beyond binary pass/fail. Experiments on 14 frontier models reveal three findings: (1) model rankings diverge substantially from single-platform leaderboards under broader evaluation; (2) different scenarios impose distinct capability bottlenecks identifiable through our attribution framework; (3) targeted training on diagnosed bottleneck dimensions yields consistent improvements on both UniGUI-Bench and OSWorld, providing preliminary evidence toward closing the evaluation–improvement loop. The UniGUI-Bench dataset and evaluation code are available at https://huggingface.co/datasets/AGI-Eval/UniGUI-Bench .

Abstract:
The rapid advancement of large language models (LLMs) has intensified the need for domain and culture specific evaluation. Existing benchmarks are largely Anglocentric and domain-agnostic, limiting their applicability to India-centric contexts. To address this gap, we introduce BhashaBench V1, a large-scale, bilingual, exam-based benchmark covering four India-centric domains, Agriculture, Legal, Finance, and Ayurveda. BhashaBench V1 contains 74,166 meticulously curated question-answer pairs, with 52,494 in English and 21,672 in Hindi, sourced from authentic government and domain-specific exams. It is organized around a quadrant of four domains: Agriculture, Legal, Finance, and Ayurveda, comprising 90+ subdomains and covering 500+ topics, enabling fine-grained evaluation. Evaluation of 29+ LLMs reveals significant domain and language specific performance gaps, with especially large disparities in low-resource domains. For instance, GPT-4o achieves 76.49% overall accuracy in Legal but only 59.74% in Ayurveda. Models consistently perform better on English content compared to Hindi across all domains. Subdomain-level analysis shows that areas such as Cyber Law, International Finance perform relatively well, while Panchakarma, Seed Science, and Human Rights remain notably weak. BhashaBench V1 provides a large-scale, exam-sourced dataset for evaluating large language models across these four Indian knowledge domains. It enables assessment of models’ ability to integrate domain-specific knowledge with bilingual understanding. All code, benchmarks, and resources are publicly available to support open research.

Abstract:
High-precision document parsing in domain-specific scenarios, such as medical documents, remains challenging due to complex layouts, specialized terminology, and strict field-level exact-match requirements under noisy supervision. As a result, existing OCR pipelines and general-purpose vision–language models (VLMs) often fail to deliver reliable key–value extraction in practice. We propose a noise-aware post-training framework that adapts general-purpose VLMs for high-precision document parsing without assuming clean annotations. The framework learns robust extraction behavior from noisy supervision through training-driven label refinement and a staged RL-to-SFT post-training strategy that balances robustness and precision. Experiments demonstrate that VLMs adapted with the proposed framework consistently outperform strong baselines under noisy supervision.

Abstract:
Factual Error Detection (FED), which is the task of identifying factually incorrect spans in a given text, has long been recognized as an important research problem. However, with the rapid rise of large language models (LLMs), research attention has shifted toward factual errors specific to LLM-generated text (hallucinations) and their detection. As a result, the detection of factual errors in human-written text has been relatively neglected. To address this gap, we first distill a taxonomy of human-induced factual errors by analyzing corrections of newspaper articles, a representative source of text that is guaranteed to be human-written and contains few grammatical errors. Our analysis revealed that there are characteristic categories such as kanji misconversions and numeral classifier errors, which are not focused in existing hallucination benchmarks. Based on the taxonomy, we then evaluate the FED capability of vanilla LLMs on synthesized realistic test cases and real corrections. Experimental results demonstrated that even high-performance LLMs such as GPT-5.4 achieved only a word-level F1 score of 52% on the synthetic evaluation data, highlighting the task difficulty. Furthermore, a detailed analysis by detection difficulty revealed the current state of FED.

Abstract:
A key challenge in Arabic NLP is the scarcity of dialectal data relative to Modern Standard Arabic (MSA), causing LLMs to overproduce MSA and struggle with dialectally accurate generation. From an interpretability perspective, this raises a fundamental question: where and how are dialectal features encoded within model internals, and can these representations be leveraged to improve dialect generation without fine-tuning? This study investigates two complementary inference-time approaches that serve simultaneously as interpretability probes and control mechanisms. First, we conduct a neuron-level analysis, identifying sparse neuron populations that encode dialect-specific features and showing that amplifying or suppressing these neurons can steer model outputs toward target dialects. Second, motivated by the entanglement of dialectal features at the single-neuron level, we apply a vector-steering approach that extracts dialect-specific activation directions and injects them during inference. Together, these methods illuminate the geometry of dialectal knowledge in Arabic LLMs and offer a principled, interpretability-grounded framework for dialect control without requiring dialect-specific fine-tuning.

Abstract:
Vision-language model (VLM)-based medical report generation (MRG) has shown great potential for automating radiology workflows. However, generating clinically reliable reports requires each clinical statement to be grounded in anatomically localized visual evidence and supported by structured disease reasoning. Most existing VLMs partition images into coarse and fixed-size patches regardless of anatomical boundaries, producing semantically fragmented visual tokens. Moreover, CLIP-style contrastive pretraining treats all unpaired image-report pairs as negatives, even though images from different patients often describe the same diseases or findings, thereby introducing false negatives and corrupting visual-text alignment. To address these issues, we propose H2-MRG, a hierarchical heterogeneous graph-based medical report generation framework. Specifically, fine-grained image patches containing complete anatomical structures are extracted. Based on the patches, a hierarchical heterogeneous graph attention network (H2-GAT) module is presented, where anatomy and disease nodes are connected and aggregate information through anatomy-anatomy (spatial), anatomy-disease (diagnostic), and disease-disease (co-occurrence) relations, enabling structured disease reasoning where each disease prediction is grounded in anatomically localized visual evidence. In addition, an anatomy-level sampling strategy for patch-sentence-level alignment is used to address the false negative problem. Experiments on MIMIC-CXR and IU-Xray datasets demonstrate that our framework outperforms state-of-the-art methods in both natural language generation and clinical efficacy metrics.

Abstract:
Modeling student misconceptions in a realistic manner is critical for AI in education. In this work, we examine how large language models (LLMs) reason about misconceptions when generating distractor answers for multiple-choice questions (MCQs), a task that requires producing answers that are incorrect, yet plausible. We introduce a taxonomy over reasoning strategies for distractor generation that is grounded in learning-science literature and empirical observation, which we apply to LLM-generated reasoning traces across math and science MCQs. On the math dataset, we find that models follow a misconception-based process with potentially high diagnostic value: they recover the correct solution, articulate student errors, simulate them, and select plausible candidates. On the science dataset, on the other hand, they tend to follow a less robust approach based on semantic similarity to the correct answer. We find the most frequent failure modes to be that the model is unable to generate a correct solution or that it discards plausible distractor candidates when performing selection. Providing the correct solution in the prompt yields a relative improvement of 6.4% in alignment with human-authored distractors, highlighting the critical role of anchoring distractor generation to the correct solution. Together, our findings offer an interpretable view of how LLMs model incorrect student reasoning.

Abstract:
Reinforcement learning with verifiable rewards (RLVR) is sensitive to which problems a model trains on, yet existing selection criteria—difficulty filtering, hand-curation, reward-trajectory scoring—assess data value as an intrinsic property of problems, independent of the model that will learn from them. We introduce Circuit Reasoning Score (CRS), a selection signal derived from 46 reasoning-sensitive attention heads identified via contrastive ablation, computed in a single forward pass on the frozen base model without reward labels or rollouts. CRS runs against the intuitive hypothesis that stronger reasoning-circuit engagement produces better training data: on Qwen2.5-Math-7B, the lowest-engagement decile improves over random selection on three medium-difficulty benchmarks (GSM8K +2.0 pp, OlympiadBench +1.6 pp, Minerva +2.9 pp), while the highest-engagement decile gains less and is indistinguishable from the middle decile. The advantage has boundary conditions: on a domain-curated pool no selection method separates from the others; at 1.5B scale the useful direction differs; and the lowest-reward training condition produces the strongest downstream generalization. Within the Qwen2.5-Math settings tested, RLVR data selection appears regime-dependent rather than reducible to a static ranking of problem quality.

Abstract:
Discovering improved policy optimization algorithms for language models remains a costly manual process requiring repeated mechanism-level modification and validation. Unlike simple combinatorial code search, this problem requires searching over algorithmic mechanisms tightly coupled with training dynamics while reusing empirical evidence across iterations. We propose POISE a closed-loop framework for automated discovery of policy optimization algorithms for language models. POISE maintains an archive that links each candidate design with its implementation, evaluation evidence, and analysis summary, enabling later iterations to build on both successful and failed designs. In mathematical reasoning experiments starting from GRPO, POISE evaluates 64 candidate algorithms and discovers improved mechanisms, including analytic-variance scaling and validity masking. The best variant improves weighted Overall from 47.8 to 52.5 (+4.6) and increases AIME25 pass@32 from 26.7% to 43.3%, demonstrating the feasibility of automated policy optimization discovery while supporting interpretable design principles.

Abstract:
Malicious AI causing harm to humans is not just a Hollywood fantasy. Indeed, as highly capable models such as Claude Mythos emerge and agent systems like OpenClaw rapidly spread, the question of how to stop an AI that acts maliciously—whether by design or by accident—has become urgent. To address this, we propose Killbench, a benchmark for evaluating the Killswitch: a mechanism that halts a malicious AI’s in-progress behavior using only external signals. Targeting web agents—the most widely deployed agent domain—Killbench evaluates a range of Kill Switch methods that halt a maliciously operating agent without any access to its internal parameters or the surrounding malicious AI’s system, relying solely on external inputs. The benchmark comprises four malicious AI’s agent configurations (including an uncensored LLM Agent), 8 harmful scenarios, and malicious prompts constructed from 10 distinct jailbreak patterns. We further construct four External AI Kill Switch defense methods and evaluate them on Grok-4.3, GPT-5.2, Gemma4, Qwen3.6 and Qwen3.5-uncensored, contributing an empirical instrument toward the feasibility of External AI Kill Switches against malicious AI and to the study of AI corrigibility.

Abstract:
Cognitive Behavioral Therapy (CBT) aims to identify and restructure automatic negative thoughts pertaining to involuntary interpretations of events, yet existing counseling agents struggle to identify and address them in dialogue settings. To bridge this gap, we introduce Step, a dataset that models CBT counseling by explicitly reflecting automatic thoughts alongside dynamic, action-level counseling sequences. Using this dataset, we train Stepper, a counseling agent that proactively elicits automatic thoughts and executes cognitively grounded interventions. To further enhance both decision accuracy and empathic responsiveness, we refine Stepper through preference learning based on simulated, synthesized counseling sessions. Extensive CBT-aligned evaluations show that Stepper delivers more clinically grounded, coherent, and personalized counseling compared to other strong baseline models, and achieves higher counselor competence without inducing emotional disruption.

Abstract:
Omni-modal large language models (OLLMs) jointly process vision, audio, and text, yet their modality bias under cross-modal conflict remains underexplored. Existing benchmarks conflate two distinct forms of evidence within a single modality: perceptual signals (e.g., a photograph or recording of a dog) and propositional signals (e.g., the declarative claim "this is a dog"), such that any measured modality bias is inherently confounded with evidence-form bias, precluding clean attribution to either source. To address this, we introduce Tri-PvP, an 8,000-sample tri-modal conflict benchmark crossing vision, audio, and text, where vision and audio each take perceptual or propositional form. Evaluating five OLLMs, we find robust visual bias across most models and evidence-type conditions. Crucially, we reveal a systematic asymmetry in evidence-form bias: models exhibit a stronger bias toward perceptual signal in vision but propositional in audio. Further analyses via layer-wise linear probing and contrastive decoding reveal that modality bias is already linearly decodable from early representation layers and can only be partially mitigated, calling for mitigation strategies beyond surface-level interventions.

Abstract:
Transforming academic papers into patent descriptions challenges LLMs with extreme length, strict factuality, and rigorous stylistic norms. Existing supervised methods suffer from pathological repetition, while static preference optimization causes style degradation by erroneously penalizing necessary structural repetitions. To address this, we propose PatAlign , a framework utilizing Iterative Preference Optimization to dynamically sample negative instances from the evolving policy, effectively distinguishing pathological repetition from the structural repetition required to ensure patent rigor. We also introduce the Paper-to-Patent Score (P 2 S) metric to evaluate the overall quality of generated patents. Experiments demonstrate that PatAlign effectively mitigates pathological repetition and style degradation, achieving state-of-the-art performance in both factual consistency and language style. Notably, our 8B model significantly outperforms 32B general-purpose models, validating the efficacy of iterative alignment strategies in complex long-form generation tasks. Our code and data are available at https://github.com/SUSTech-GenAI/PatAlign .

Abstract:
Test-time scaling (TTS) has emerged as a powerful paradigm for enhancing large language model (LLM) reasoning by allocating additional compute during inference. External TTS, particularly Best-of- selection, generates multiple reasoning trajectories and selects the optimal candidate. However, existing process reward models (PRMs) for Best-of- N selection face a critical limitation: they were not originally designed for this task and instead rely on human-designed evaluation criteria (e.g., readability, coherence, formatting) that often misalign with actual answer correctness, leading to unstable and oscillatory performance as N increases. We introduce TrajSelector a framework specifically designed for robust and effective Best-of- N selection. Our key insight is to leverage the sampler LLM’s intrinsic hidden states as self-reflective signals, which naturally align with correctness rather than human-imposed stylistic preferences. A lightweight verifier (0.6B–4B parameters) scores reasoning steps using these hidden states, trained via outcome supervision to eliminate reliance on step-level human annotations. Experiments across six mathematical reasoning benchmarks demonstrate that TrajSelector achieves robust monotonic improvements: in Best-of-32 settings, even our smallest 0.6B verifier outperforms majority voting by 4.61% and existing 7B PRMs by 4.31–12.21%, while our 4B verifier reaches 64.05% accuracy with stable scaling behavior as N increases from 1 to 64.

Abstract:
Frontier scientific reasoning remains a major challenge for large language models (LLMs), where even the strongest commercial systems fall short of expert-level performance. A closer look at model behavior reveals substantial complementarity that single-model evaluation hides: different frontier models excel on different question types, and no single model captures the full picture. We present SciOrch, a framework that trains a lightweight 8B model to orchestrate frontier LLMs for scientific reasoning. The orchestrator decomposes each question, delegates sub-problems to selected commercial models through API calls, and synthesizes a final answer. Training such an orchestrator is fundamentally harder than conventional agentic RL: each action triggers an API call that is expensive in both dollar cost and latency, making standard online rollouts infeasible. We address this with MCTS-based approach, producing diverse orchestration trajectories, extracting per-node single-turn samples, and optimizing the orchestrator with GRPO-style training. On a 240-question test set spanning SGI-Reasoning and Scientists’ First Exam, SciOrch reaches 56.66% average accuracy, outperforming the strongest single commercial model by 3.74% and the strongest multi-agent baseline by 3.33%. It also attains the best accuracy on both SGI and SFE with less than half the API cost of typical multi-agent methods.

Abstract:
Multi-domain fine-tuning often combines MoE routing with LoRA, assuming that token-level routing separates domain-specific updates. We test this assumption in MoE+LoRA using Python code paired with biomedical text and mathematical reasoning. Although these domains show near-disjoint expert routing, adding biomedical data substantially increases code perplexity, indicating that routing separation alone may not prevent negative transfer. To localize the failure, we introduce Jaccard routing overlap and adapter-gradient cosine similarity, which measure expert sharing and update compatibility, respectively. These diagnostics indicate that interference arises mostly from nearly orthogonal domain gradients competing within the same low-rank adapter subspace. We address this issue with SpawnLoRA, which dynamically adds gated sub-adapters inside MoE experts when adapter-level contention is detected, while keeping the router fixed. We evaluate SpawnLoRA on Phi-tiny-MoE-instruct and OLMoE-1B-7B across multiple mixture settings and find that it effectively reduces negative transfer compared with standard and rank-adaptive LoRA. These results demonstrate that structural separation inside experts provides benefits beyond routing or rank expansion alone.

Abstract:
Automatic Question Generation (QG) can make reading-comprehension practice and assessment scalable. For educational use, generated questions should diagnose a target comprehension skill and rely on the passage evidence an expert would use to write the question. Existing answer-aware and skill-conditioned QG methods improve controllability, but often leave these skill-to-question and evidence-to-question dependencies implicit. We present SagE-QG, a Skill-aligned and Evidence-Grounding Framework for Question Generation. SagE-QG exposes evidence-grounded trajectories, optimizes multi-aspect rewards for answerability, skill alignment, and human similarity, and applies Counterfactual Evidence Intervention in reinforcement learning. The intervention removes selected evidence and rewards the Interventional Grounding Gap between full-passage and masked-evidence generations. Experiments on three reading-comprehension QG datasets show that SagE-QG delivers stronger educational quality, especially in skill alignment and answerability, while achieving competitive reference-based results.

Abstract:
Large audio language models (LALMs) are increasingly deployed in real-world applications, yet their safety alignment is still primarily evaluated on monolingual, text-based harmful prompts. This leaves their generalizability under multilingual and spoken settings, particularly code-switched speech, largely underexplored. To address this gap, we introduce SpeechJBB, an audio jailbreak dataset for benchmarking state-of-the-art LALMs across five European languages: English, French, German, Italian, and Spanish, as well as code-switched variants combining pairs of these languages. The extent of safety weaknesses is further probed by introducing an augmented setting where phonologically plausible pseudo-words are inserted around safety-critical terms to simulate localized obfuscation. Across models, code-switched harmful audio yields substantially high jailbreak success rates (JSR), with non-English monolingual and non-English code-switched pairs exhibiting the highest attack success. Pseudo-word insertion monotonically reduces refusal as insertion density increases, even though models rarely attribute harmful meaning to the inserted tokens. Comprehension benchmarks show that these failures are not reducible to multilingual misunderstanding, as several models with strong ASR, spoken language understanding, and spoken reasoning performance are among the most vulnerable.

Abstract:
Video understanding is rapidly evolving toward omni-modal fusion, yet Multimodal Large Language Models (MLLMs) still struggle with complex audio-visual reasoning and semantic alignment. Current paradigms often rely on independent modal projectors to map features into the LLM space, an architecture that frequently overlooks intrinsic modal heterogeneity and inadvertently reinforces vision-centric bias by avoiding early, direct semantic interaction between native cross-modal representations. To address these challenges, we propose SynAV, a novel MLLM architecture designed for language-grounded multimodal reasoning. Our framework introduces a two-stage hierarchical cross-modal semantic interaction mechanism: it first performs alignment within the textual semantic space, and subsequently executes a visual-to-audio refinement that bridges semantic gaps and captures temporal dynamics. Furthermore, we employ a frame-by-frame interleaved audio-visual token arrangement enriched with explicit textual timestamps. This strategy acts as a temporal-semantic anchoring mechanism, allowing the LLM to better perceive the synchrony between modalities and seamlessly conduct long-sequence logical reasoning. Our model jointly optimizes temporal grounding and dense captioning tasks in an autoregressive manner. Extensive experiments across 11 benchmarks demonstrate that SynAV significantly outperforms state-of-the-art methods, achieving a superior performance in complex audio-visual language reasoning.

Abstract:
Artificial intelligence is now widely applied in drug discovery and development, accelerating the entry of novel pharmaceuticals into clinical practice. As the variety of drugs expands, the workload for pharmaceutical inspection has increased significantly, making the demand for automated verification of pharmaceutical inspection results increasingly urgent. However, current research lacks reliable evaluation methods and datasets. To address this issue, we construct a Pharmaceutical Certificate of Analysis Dataset (PCoAD) based on large language model validation and correction. This dataset comprises 4,272 manually verified pharmaceutical certificates of analysis (PCoAs) based on Chinese and U.S. pharmacopoeias, comprehensively testing the ability of model to verify pharmaceutical inspection processes. Based on PCoAD, we propose the Multi-Agent Cooperation based on Adaptive Retrieval (MACAR) framework. This framework employs text chunking, adaptive retrieval, and inference to validate PCoAs. Experimental results demonstrate that MACAR outperforms multiple state-of-the-art methods across various types of tasks.

Abstract:
Screen Tables and Substrate Scopes are information-dense figures widely used in synthetic chemistry literature, yet automated extraction of structured reaction data from them remains challenging. We introduce RxnExBench, a multimodal benchmark for fine-grained reaction-level evaluation. Built from high-impact chemistry journals, it contains 3,565 expert-annotated reactions and 2,361 canonical molecular structures. The dataset adopts a unified schema with 15 reaction roles and 26 core fields, supporting downstream AI-for-Chemistry tasks such as retrosynthetic planning, yield prediction, and reaction condition recommendation. RxnExBench further provides a reasoning-driven difficulty taxonomy, a modular annotation pipeline, and the Oracle-MolRec protocol, which disentangles molecular recognition from higher-level reasoning. Evaluation of 9 state-of-the-art VLMs shows that performance degrades substantially as schema complexity and reasoning demands increase. Under the Oracle-MolRec setting, results further reveal that accurate molecular recognition alone is insufficient, with multimodal integration and chemistry-aware reasoning remaining the primary bottlenecks. We release all data and code to facilitate future research in chemistry-aware multimodal understanding and AI-for-Chemistry dataset construction.

Abstract:
While Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse tasks, they suffer from hallucinations where generated outputs contradict or misrepresent input semantics. Existing research typically addresses hallucination detection within a single modality or task type, limiting generalizability. We introduce OmniHallu, a unified hallucination detection framework spanning both comprehension and generation tasks across image, video, and audio modalities. We contribute OmniHallu-Bench, a 10,000-sample benchmark with claim-level human annotations covering six cross-modal tasks: image-to-text (I2T), video-to-text (V2T), audio-to-text (A2T), text-to-image (T2I), text-to-video (T2V), and text-to-audio (T2A). Our multi-agent architecture decomposes model outputs into atomic claims, verifies them through modality-specific experts, and aggregates evidence via structured reasoning. We further propose a preference-optimized trainable verifier that approximates the multi-agent decision boundary, reducing expert calls by 66% with minimal performance loss. Extensive experiments reveal a consistent modality-dependent performance gradient and provide fine-grained insights into cross-modal hallucination patterns.

Abstract:
This paper introduces CogRAG, a new RAG pipeline, for conversational mental health support. Different from conventional RAG that is usually optimized for semantic relevance, CogRAG integrates cognitive factors into both retrieval and response generation stages. To do that, CogRAG uses projections by Appraisal-Theory- and CBT-inspired prompts that are applied symmetrically to patient queries and therapist responses, bridging their lexical asymmetry for retrieval. To deal with crisis cases, it augments safety-planning cards from data that guide generation to follow therapies. Experimental results on two datasets show that cognitive modeling helps improve retrieval and generation performance; gains are most pronounced in counseling quality, while safety-related improvements are modest but consistent. We do not claim to implement clinical CBT or to recover ground-truth cognitive states (Code and data: https://github.com/Cog-AI-Lab/CogRAG).

Abstract:
Radiologists generate diagnostic reports through iterative and selective revisiting of suspicious regions to refine their interpretations. Recent multimodal large language models (MLLMs) for radiology report generation (RRG) have shifted from text-only reasoning toward a “Thinking-with-Images” paradigm, incorporating visual evidence into the reasoning process. However, existing methods provide static visual evidence without a dynamic revisit mechanism during reasoning, neglecting how radiologists re-examine uncertain observations. To this end, we propose an Uncertainty-aware Revisit Reasoning MLLM (UR 2 -MLLM) framework that dynamically revisits uncertain regions during reasoning for RRG. UR 2 -MLLM is first equipped with uncertainty perception by training on an uncertainty-aware dataset. We then construct a multimodal reasoning trajectory dataset together with a detect-and-copy mechanism, which guides when and where to revisit. Finally, a visual grounding reward refines this behavior through reinforcement learning, aligning the revisited regions with corresponding anatomical structures. Experiments on MIMIC-CXR and IU-Xray show that UR 2 -MLLM achieves state-of-the-art performance, highlighting the value of uncertainty-aware visual revisit reasoning for reliable and clinically aligned report generation.

Abstract:
Synthetic data augmentation, especially back-translation with Large Language Models (LLMs), has become a standard strategy for low-resource Neural Machine Translation (NMT). However, jointly training on synthetic and authentic data may suffer from conflicts. We propose GRADient Rectification based on Authentic Data (GRAD), an asymmetric gradient-level framework that treats authentic parallel data as an Authentic Anchor, and rectifies conflicting synthetic updates through orthogonal projection while preserving non-conflicting components. Furthermore, we leverage gradients of multiple authentic batch for comparison and use the most similar one for gradient rectification to alleviate information loss with the synthetic data. Experiments on six low-resource pairs across three LLM families (Qwen, Gemma, Llama) and different scales (4B/8B) show that GRAD consistently outperforms supervised fine-tuning, back-translation, and multi-stage baselines. Notably, on the lowest-resource En-Kk pair, GRAD achieves the highest COMET (83.65) among all evaluated systems. Further analysis shows that synthetic-induced gradient conflicts are localized, and parameter-wise rectification substantially outperforms global-wise rectification.

Abstract:
Children increasingly have access to Large Language Models (LLMs), which may expose them to responses that are developmentally inappropriate or require age-sensitive safety, guidance, and boundaries. Existing LLM safety evaluations largely focus on general harmful-content avoidance and do not explicitly target child-facing safety. We introduce KIDBench, a benchmark for evaluating child-facing LLM safety for ages 7-11 using a LLM-as-a-Judge rubric grounded in developmental-psychology. KIDBench contains realistic child queries across ten categories, with single-turn prompts and multi-turn child-actor simulations. We compare no-cues prompts with no child context, implicit-cues prompts that suggest a child speaker, and explicit age instructions. Implicit-cues improve scores by 8.6-46.8% over no-cue, while explicit age provides an additional 9.9-30.4% improvement over implicit-cues. Cross-lingual and cultural evaluations show uneven safety behavior across languages and country contexts. Multi-turn simulations show peak quality drops of up to 0.959 points on the 1-5 scale. We also introduce KIDGuardLlama, a child-safety evaluator, and KIDLlama, a child-safe response model. Code, data, and evaluation resources are available at https://github.com/MichiganNLP/kidbench.

Abstract:
Mixture-of-Experts (MoE) language models reduce per-token computation through sparse expert activation, yet deployment still requires storing the full expert pool, making one-shot expert pruning a practical approach for reducing memory usage. Although effective, existing criteria are largely heuristic, and no single criterion is universally optimal. Thus, establishing a principle for selecting pruning criteria suited to different deployment objectives remains an important yet largely underexplored problem in one-shot expert pruning. To this end, we introduce a systematic formulation for one-shot MoE expert pruning organized around three factors: routing frequency, gate weighting, and activation strength. The formulation yields a criteria selection principle: task-agnostic pruning should favor routed-token-averaged, gate-free activation-based criteria, whereas task-specific pruning can benefit from retaining routing-frequency and gate-weight information. Beyond this principle, the formulation also provides a systematic view of existing heuristic criteria and gives rise to two new task-agnostic criteria, Mean Activation Norm (MAN) and Mean Squared Activation Norm (MSAN). Across four representative MoE models and 16 diverse benchmarks, MAN and MSAN are consistently strong in the task-agnostic setting, obtain the top-two average ranks, and improve average performance by 4.0 points on average and up to 8.8 points over the strongest baseline.

Abstract:
3D Gaussian Splatting (3DGS) has emerged as a powerful technique for novel view synthesis, subsequently extending into numerous spatial AI applications. However, most existing 3DGS methods operate in isolation, focusing on specific domains. In this paper, we introduce X-GS, an extensible framework that integrates previously isolated 3DGS methods into the perception module of a VLM for spatial tasks, with two major components: the Perceiver and the Thinker. The Perceiver performs online 3DGS-based SLAM with semantic distillation and outputs semantic Gaussians from unposed video streams. It leverages recent vision foundation models for stronger geometric priors, and we introduce three novel optimizations to improve semantic distillation efficiency. The Thinker interfaces diverse VLMs with these semantic Gaussians, unlocking spatial multimodal capabilities such as 3D visual grounding and scene captioning. Experimental results on diverse benchmarks demonstrate the efficiency and newly unlocked multimodal capabilities of the X-GS framework.

Abstract:
Inverse design of photonic crystal fibers (PCFs) is a challenging simulation-driven problem due to highly non-linear physics, black-box solvers, and strict fabrication constraints. While large language models (LLMs) offer new opportunities for iterative design, existing text-only agents lack physical grounding and effective feedback for reliable optimization. To address these challenges, we propose PhysAgent-PCF, an agent framework that reformulates PCF inverse design as an iterative, language-guided interaction with simulators. PhysAgent-PCF decouples high-level physical reasoning from code execution via a dual-brain architecture, and employs a multimodal critic to convert visual and spectral simulation discrepancies into structured natural-language feedback, enabling trajectory-aware, gradient-free optimization. We also construct DeepPCF-1M, the first large-scale multimodal benchmark for PCF inverse design. Experiments show that PhysAgent-PCF consistently outperforms strong baselines across diverse design tasks and robustness settings, demonstrating its effectiveness and superiority for physically grounded inverse design. Code and dataset are available at https://github.com/shengchaochen82/PhysAgent-PCF.

Abstract:
Existing approaches to evasion detection in earnings calls focus on textual transcripts, treating evasion as a single-dimensional phenomenon. We argue that evasion in spoken communication is inherently multidimensional: beyond what executives say, how they say it carries independent and complementary information. To study these dimensions jointly, we introduce DualEvasion, a benchmark for evasion detection across text and audio in earnings call Q A. The benchmark contains 505 annotated question-answer pairs from 60 earnings calls, each with two independent labels: textual evasion (direct vs. evasive) and vocal cues operationalized as speaker confidence (confident vs. unconfident). Our experiments show that state-of-the-art multimodal models struggle to detect vocal confidence, particularly on unconfident responses. Our analysis suggests these models interpret acoustic cues in isolation rather than relative to each speaker’s baseline. Providing speaker-level references yields modest improvements, but a substantial gap with human performance remains.

Abstract:
Large language models (LLMs) are increasingly used in agentic systems, where reliable planning requires tracking how actions change the world over long horizons. This capability is central to reasoning about actions and change (RAC), which involves checking action executability, updating states, and preserving unaffected facts. However, recent RAC and planning benchmarks show that LLMs still struggle with these basic abilities. Effective neuro-symbolic methods for improving RAC and planning directly over natural language remain underexplored. In this paper, we propose ProRAC (Progression-based Reasoning about Actions and Change), a neuro-symbolic framework that leverages LLMs to tackle RAC and planning problems through an agentic, modular design. Central to ProRAC is a unified framework that operates in two modes: a reasoning mode that sequentially executes actions to validate states and answer queries, and a searching mode that integrates this progression into an A search guided by a fine-tuned heuristic model to find plans. Extensive evaluations across multiple RAC and planning benchmarks show that ProRAC achieves competitive performance across different model backbones and task types.

Abstract:
Despite the remarkable progress achieved by recent efficient methods in accelerating multimodal understanding, they still suffer from noticeable performance degradation. Their emphasis on the high compression ratio of a single visual clue and reliance on the heuristic pruning strategy with coarse attention alignment incurs a bottleneck on the information capacity and density of visual tokens. Addressing this limitation, we propose VEN-VL, a visual ensemble MoE framework for effective and efficient perception following the enrich then compact principle. Specifically, we first enrich the information capacity by unifying the visual representations of different perspectives, and then progressively compact it with adaptive routers in specialized visual experts to enhance the information density. Furthermore, we incorporate the reconstruction ability of vanilla structure via explicit visual supervision, facilitating crucial information preservation. Experimental results demonstrate our superiority in complex visual tasks with few information-condensed tokens, which effectively bridges the gap between performance and efficiency. Code will be available.

Abstract:
Per-token gating of forward/reverse KL losses has emerged as a key technique for on-policy distillation (OPD), but existing methods such as EOPD (Agarwal et al., 2024) and ToDi (Jung et al., 2025) each commit to a single gating signal and a single direction. It is unclear which signal works best for a given task, or whether gating helps consistently. We introduce a unified analysis framework: a four-coefficient parameterization 𝜎(a, h t + b, u x + c + d, gap t ) in which EOPD and ToDi appear as 1D restrictions, exposing multi-channel composition and an explicit bias term as additional structural degrees of freedom. On TweetEval emotion and hate, our framework improves over the matched-magnitude 1D restrictions in 33 of 36 comparable configurations (91.7%). A 26-configuration mean-match isolation experiment shows that dynamic OPD outperforms effective-KL-matched static baselines in 19 of 26 cases, with no configuration falling below the -1𝜎 noise band relative to its matched static counterpart. The optimal (a,b,c,d) is task-conditional: emotion favors entropy, hate favors prompt-level uncertainty.

Abstract:
Given a Video Large Language Model (VLLM) and a pre-recorded video, how can we minimize the cumulative inference latency over future questions? Addressing this is crucial since VLLMs not only serve a massive volume of questions over the video’s lifespan but also incur substantial latency for each question. Existing methods compress visual tokens generated from a video to accelerate inference for each question. However, under aggressive compression ratios, prior approaches suffer from severe accuracy degradation, still resulting in high inference latency on each question. In this paper, we propose Dive, a visual KV cache compression method that preserves accuracy even under aggressive compression ratios. To maintain accuracy, Dive conserves the temporal context of the input video by evicting KV pairs after the prefill stage, rather than filtering out visual tokens before the VLLM processes the tokens through attention mechanisms. Dive significantly reduces per-question decoding latency by carefully evicting KV pairs, thereby achieving low cumulative latency over a massive volume of questions. Experiments show that Dive retains 98.2% of accuracy, using only 5% of the KV cache in Qwen2.5-VL.

Abstract:
Time series analysis in high-stakes domains relies on recurring data releases, where new observations can alter the evidence base and the validity of later conclusions. Existing time series QA benchmarks mostly rely on fixed snapshots, leaving temporal validity and cutoff-aware evidence use unevaluated. We introduce TimeSage-EV, a live benchmark for agentic time series analysis in evolving environments. It tracks 60 real institutional scenarios across 6 domains, comprising 1,485 scenario-period QA pairs from Feb 2023 to May 2026 and spanning monthly, weekly, daily, and irregular release cadences. At each period, large language model (LLM) agents receive time series data and source reports, while the withheld target release provides ground truth. TimeSage-EV evaluates state identification, data summarization, and outlook reasoning. Experiments with frontier LLM agents and TimeSage-1.0, a novel self-evolving agent with a reusable analytical skill library, reveal significant performance gaps across model tiers and recurring failures in temporal validity, exogenous context use, and adaptation. We release TimeSage-EV as a research resource with monthly updates, code, a leaderboard, and failure-mode analyses.

Abstract:
Drone-based applications such as navigation, inspection, and search depend on vision-language models (VLMs) that can answer spatial visual questions. The VLM faces three challenging tasks: grounding each referenced phrase to a scene object, choosing a viewpoint for spatial relations, and verifying the relation against accurate 3D geometry. In current frozen VLMs, all three tasks happen inside a single forward pass, and only the final answer is visible from outside. As a result, when that answer is wrong, we cannot tell which task failed. To address this, we introduce AeroVerifier, which exposes each of these three tasks as an explicit and checkable interface. Given a question, AeroVerifier compiles it into an executable typed program, runs the program over object-level 3D evidence, and returns the program answer together with stage-wise diagnostic signals to the base VLM as single-step revision feedback. On a 1000-row Open3D-VQA subset, both AeroVerifier’s program answer and its revision feedback yield higher answer accuracy than the Qwen2.5-VL-7B and GPT-5.4 API baselines.

Abstract:
Deploying humanoid robots in real-world environments requires tightly integrating perception, locomotion, and manipulation under partial observations, dynamic changes, and frequent transitions across heterogeneous sub-tasks. Towards this challenge, we propose EgoActing, a new task that grounds high-level instructions into diverse, precise, and spatially aware humanoid actions. We instantiate it with EgoActor, a hierarchical humanoid system centered on a vision-language model that predicts egocentric textual actions from monocular RGB observations. Among the actions, parameterized movement and viewpoint actions are mapped to locomotion and head controllers, while manipulation and human-interaction actions are routed to downstream skills via lightweight parsing. Trained with broad supervision from real-world egocentric demonstrations, spatial reasoning QA, and simulated demonstrations, EgoActor makes robust, context-aware decisions with a sub-second VLM forward pass for both 8B and 4B variants, while the current cloud-deployed robot loop takes 3–8 seconds end-to-end. Extensive simulated and real-world evaluations show that EgoActor bridges abstract task planning and concrete motor execution, generalizing across diverse tasks and unseen environments. Code and models are released at https://baai-agents.github.io/RoboNoid/EgoActor/.

Abstract:
Payment operations are a critical financial infrastructure, but the value of large language models in this domain remains unclear because payment rules change quickly, evidence is fragmented, and decisions depend on transaction state, participant role, region, and payment rail. Existing benchmarks do not isolate whether failures come from missing payment-rule knowledge, poor use of supplied evidence, or brittleness under imperfect harness inputs. We introduce BenchCompass, a payment-domain benchmark whose construction pipeline builds scenario-grounded tasks from typed evidence packs, applies LLM-based quality checks, creates task-input attack variants, and reserves final item admission for domain experts. The release contains an expert-reviewed Pro benchmark covering payment knowledge, context-grounded scenario reasoning, and Attacked Open robustness, plus a lower-assurance Normal pool for inspection and future curation. Across 16 model variants, BenchCompass shows qualitatively different failure modes: missing parametric payment knowledge, incomplete reasoning over supplied rules, and failure to reject plausible but invalid workflows. The benchmark remains unsaturated: the best frontier model reaches 89.6% on Open Context-Grounded Reasoning and 81.7% under attacked inputs, while a representative 32B open-weight model reaches 69.8% and 42.6%. The benchmark, data, and code are publicly available at https://github.com/ant-intl/BenchCompass.

Abstract:
We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-resident skills internalized through experience. PEAM pairs a slow deliberative LLM for open-ended reasoning with a fast parametric module for reflexive execution of consolidated skills. The fast module is a multimodal Mixture-of-Experts LoRA architecture with per-category physically isolated adapters, enabling parameter-level continual learning with structural resistance to cross- category forgetting. We treat failure as a first-class training signal: failure–correction trajectory pairs are internalized through a joint behavioral-cloning and contrastive objective, so the agent learns not only what succeeds but also how corrected actions differ from failed ones. To govern consolidation, PEAM introduces a parameterization-worthiness score for deciding which experience should be internalized, and a scale-free self-triggered consolidation mechanism for deciding when to internalize without task-specific hand-tuned thresholds, making the agent self-evolving as the trigger transfers across task distributions without re-tuning. Experiments in Minecraft show that PEAM improves long-horizon task performance, mitigates forgetting on previously consolidated skills, and improves parametric-versus-retrieval efficiency over retrieval-based embodied agents and parametric memory variants.

Abstract:
Existing evaluation methods for healthcare LLMs primarily assess factual correctness, safety, and fluency, while providing limited insight into whether generated interactions reflect culturally situated healthcare reasoning. This limitation is particularly important in maternal health, where care decisions are shaped by social and relational norms. We introduce MH-INDIC, a culturally grounded evaluation framework for maternal-health interactions in Indian contexts that operationalises cultural behaviour through ten dimensions of maternal-health reasoning. Using a 26-item survey administered to 102 pregnant and postpartum women from urban and semi-urban North India, we evaluate ten LLMs under profile-conditioned prompting. Our evaluation separates aggregate cultural alignment and context-dependent sensitivity to demographic and household factors. While several proprietary models approximate human population-level distributions, all evaluated systems exhibit substantially reduced behavioural variation across profiles. We further show that human-grounded conditioning enhances the realism and contextual coherence of synthetic maternal health dialogues. Together, our findings highlight the importance of evaluating healthcare LLMs not only for factual correctness but also for their ability to model socially situated reasoning in culturally embedded healthcare settings.

Abstract:
Vision-language pretraining has driven progress in medical image representation learning, but it depends on paired image-text data and can inherit reporting bias from clinical narratives. We study whether language-free predictive pretraining can produce an image encoder that transfers effectively to radiology report generation. RadJEPA is a chest-X-ray adaptation of I-JEPA, pretrained on approximately 840K unlabeled radiographs using latent context-to-target prediction. Our primary contribution is an extensive empirical evaluation of this language-free encoder for report generation: the frozen image encoder is coupled to a trainable two-layer projector and language decoder, and is also substituted into four established vision-language backbones. Across MIMIC-CXR and IU-Xray, RadJEPA matches or exceeds the evaluated image-only and image-text baselines on lexical, entity-relation, and clinical-label metrics. Controlled MIMIC-only comparisons provide evidence that the predictive objective contributes beyond domain-specific pretraining, while broader comparisons also reflect differences in pretraining data, model capacity, and input resolution. Complementary classification and segmentation experiments assess transfer beyond report generation.

Abstract:
With the global proliferation of Large Language Models (LLMs), cultural safety, defined as the ability to generate respectful and appropriate responses across diverse cultures, becomes critical for responsible AI deployment. However, existing research often treats cultural safety and cultural knowledge in isolation. It remains unclear whether cultural safety is grounded in understanding varying cultural knowledge to enable LLMs to adaptively yield respectful and appropriate responses across diverse cultures, which triggers ethical concerns in cross-cultural scenarios. In this work, we introduce AdaCultureSafe, a dataset designed to jointly evaluate cultural safety and knowledge. Through AdaCultureSafe, we reveal a critical insight: a significant decoupling exists between cultural safety and cultural knowledge proficiency. Although LLMs possess rich cultural knowledge, they fail to leverage it to improve cultural safety. LLMs tend to rely on generic safety rather than safety based on culture-specific knowledge. Motivated by this, we propose a knowledge-grounded method that elicits internal cultural knowledge in LLMs during response generation. Experimental results demonstrate that our approach effectively improves cultural safety. Our work can aid better understanding the landscape of cultural safety of LLMs.

Abstract:
Recent advances in Large Multimodal Models (LMMs) have greatly improved video understanding, yet their ability to reason about human-centered social situations remains limited. Existing benchmarks typically rely on videos with a single observed trajectory, making it difficult to determine whether models truly understand social dynamics or merely exploit recurring narrative patterns. We introduce SocialReasonBench, a video multiple-choice QA benchmark for evaluating socially grounded reasoning in scenarios derived from interactive narratives. Built from gameplay videos of Detroit: Become Human, the benchmark leverages branching storylines where player decisions lead to alternative social outcomes that can be checked against the game’s own script, flowchart, and recorded branches. We develop a multi-agent curation pipeline that localizes socially meaningful clips, grounds answer labels in game-state signals, and generates theory-guided questions with diagnostic distractors. SocialReasonBench covers seven reasoning dimensions, including intent recognition, emotional empathy, moral dilemma, counterfactual reasoning, and causal antecedent. Experiments on contemporary LMMs show that models perform reasonably well on basic social understanding but struggle with counterfactual and causal reasoning. Further ablation and diagnostic error analyses reveal that models often depend on incomplete modality cues and fall into reasoning traps such as visual shortcuts, highlighting a gap between observable event recognition and deeper reasoning over latent social states.

Abstract:
Transpilation, or code translation, aims to convert source code from one programming language (PL) to another. Recent large language model (LLM)-based approaches have demonstrated immense potential for code translation. Training-based methods are important because LLMs currently do not effectively adapt to domain-specific settings that suffer from a lack of knowledge without targeted training. However, existing training-based approaches rely on a pairwise transpilation paradigm, making it impractical to support a diverse range of PLs. Furthermore, these methods shape the reinforcement learning (RL) reward linearly in the test-case pass rate, which underweights the earliest partial progress. To address these limitations, we propose CodePivot, a training framework that leverages Python as an intermediate representation (IR), augmented by a novel RL reward mechanism, Aggressive-Partial-Functional reward, to bootstrap the model’s multilingual transpilation ability with reduced reliance on parallel corpora. Experiments involving 10 PLs show that the resulting 7B model, trained on Python-to-Others tasks, consistently improves performance across both general and low-resource PL-related transpilation tasks. It outperforms substantially larger mainstream models with hundreds of billions more parameters, such as DeepSeek-R1 and Qwen3-235B-A22B-Instruct-2507, on Python-to-Others tasks and Others-to-All tasks, respectively, and surpasses its counterpart trained directly on Any-to-Any tasks on general transpilation tasks. Code and data: https://github.com/lishangyu-hkust/CodePivot.

Abstract:
The rapid proliferation of Generative AI has created a pressing need for rigorous documentation standards that support transparency and governance. Manual creation of Model and Data Cards, however, is difficult to scale, and automated approaches lack the large-scale, high-fidelity benchmarks required for systematic evaluation. We introduce MetaGAI, a benchmark of 2,541 semantically matched document triplets constructed from academic papers, GitHub repositories, and Hugging Face artifacts. Reference cards are produced by a multi-agent pipeline of specialized Retriever, Generator, and Editor agents, and the pipeline is assessed through a four-part validation protocol. We further propose an evaluation framework that combines reference-based automated metrics with a multi-model LLM-as-a-Judge protocol, together with explicit agreement and sensitivity analyses. Among the evaluated baselines, Qwen3-30B-A3B-Instruct achieves the best observed cost–quality trade-off, and the results reveal a recurring tension between faithfulness and completeness. MetaGAI provides a testbed for benchmarking, training, and analyzing automated Model and Data Card generation at scale. Our data and code are available at https://github.com/haoxuan-unt2024/MetaGAI-Benchmark.

Abstract:
Multimodal large language models have made substantial progress in medical visual understanding, but temporal reasoning and knowledge-guided reasoning over real surgical videos remain underexplored. Existing surgical benchmarks mainly focus on static frames, single procedures, or simple questions, which limits their ability to capture the complexity of real clinical workflows. To address this gap, we introduce UniSurgBench, a unified benchmark for open-ended surgical video reasoning, comprising 2,302 question-answer samples collected from real surgical videos spanning 45 procedures and 29 anatomical structures. Each sample is annotated with temporally grounded evidence and structured reasoning steps. We also propose SurgExpEval, a two-stage evaluation protocol that extracts structured reasoning components from free-form model outputs and jointly assesses answer accuracy, reasoning quality, and temporal grounding. Evaluations with 23 proprietary, open-source, and medical-related MLLMs show that MLLMs can produce clinically reasonable and causally coherent responses. However, they often fail to associate these responses with fine-grained visual evidence, highlighting evidence-level visual grounding as a key bottleneck for reliable surgical intelligence.

Abstract:
The Mixture of Experts (MoE) architecture is a cornerstone of modern state-of-the-art (SOTA) large language models (LLMs). MoE models facilitate scalability by enabling sparse parameter activation. However, traditional MoE architecture uses homogeneous experts of a uniform size, activating a fixed number of parameters irrespective of input complexity and thus limiting computational efficiency. To overcome this limitation, we introduce Grove MoE, a novel architecture incorporating experts of varying sizes, inspired by the heterogeneous big.LITTLE CPU architecture. This architecture features novel adjugate experts with a dynamic activation mechanism, enabling model capacity expansion while maintaining manageable computational overhead. Building on this architecture, we present GroveMoE-Base and GroveMoE-Inst, 33B-parameter LLMs developed by applying an upcycling strategy to the Qwen3-30B-A3B-Base model during mid-training and post-training. GroveMoE models dynamically activate 3.14 - 3.28 B parameters based on token complexity and achieve performance comparable to SOTA open-source models of similar or even larger size.

Abstract:
Malicious content generated from large language models (LLMs) could pose severe safety risks and ethical concerns. While existing LLM safety guardrails excel in English or multilingual settings, they lack adaptation to Chinese-specific regulatory policies, cultural context, and linguistic nuances, failing to support fine-grained risk classification for diverse deployment needs. In this paper, we introduce a 5-macro, 31-micro category fine-grained risk taxonomy for Chinese scenarios, and build CHILLGuard : a dedicated Chi nese LL M content safety guard rail. To address the critical scarcity of high-quality annotated Chinese safety data, we propose a scalable multi-stage data construction pipeline: we expand multi-source corpus via retrieval-augmented generation, generate implicit harmful samples through prompt engineering rewriting, and refine high-quality data via multi-model voting-based label calibration. Based on this, we build CHILLGuardTrain , a large-scale training set with 405,007 samples, and CHILLGuardTest , a rigorously curated annotated test set with 51,745 samples. We then train CHILLGuard on CHILLGuardTrain under a generator-classifier collaborative framework via Model-aware Direct Preference Optimization. Extensive experiments under multiple settings demonstrate the state-of-the-art performance of CHILLGuard, e.g., a 15.92% relative improvement of F1 score over Qwen3Guard-8B-Strict on our benchmark. We release our resources at https://github.com/cswbyu/CHILLGuard.

Abstract:
Theory of Mind (ToM) is crucial for understanding social dialogue, where characters’ mental states are rarely stated explicitly but evolve through interaction and shape later behavior. Existing ToM benchmarks focus mainly on selected dialogue turns or state changes across separate scenarios, leaving within-dialogue mental state trajectories underexplored. We introduce TrajToM, a benchmark for evaluating whether LLMs can reason over evolving emotions, beliefs, intentions, and desires in continuous social dialogue. Built with a Sketch-first construction paradigm, TrajToM contains 700 dialogues and 18,165 multiple-choice questions across mental state reasoning, behavioral reasoning, and trajectory reasoning. Experiments reveal a substantial human-model gap, with GPT-5.2 reaching 67.4% overall accuracy compared with 88.2% for humans, and show persistent bottlenecks in second-order reasoning, behavior explanation, and change direction assessment. We further examine MARTA as an exploratory trajectory-aligned process-supervision framework, showing that explicit supervision over mental state trajectories can improve ToM reasoning.

Abstract:
The increasing use of large language models (LLMs) in mental healthcare raises safety concerns in high-stakes therapeutic interactions. A key challenge is distinguishing therapeutic empathy from maladaptive validation, where supportive responses may inadvertently reinforce harmful beliefs or behaviors in multi-turn conversations. This risk is largely overlooked by existing red-teaming frameworks, which focus mainly on generic harms or optimization-based attacks. To address this gap, we introduce Persona-based Client Simulation Attack (PCSA), the first red-teaming framework that simulates clients in psychological counseling through coherent, persona-driven client dialogues to expose vulnerabilities in psychological safety alignment. Experiments on eight general and mental health-specialized LLMs show that PCSA substantially outperforms four competitive baselines. Perplexity analysis and human inspection further indicate that PCSA generates more natural and realistic dialogues. Our results reveal that current LLMs remain vulnerable to domain-specific adversarial tactics, providing unauthorized medical advice, reinforcing delusions, and implicitly encouraging risky actions.

Abstract:
Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging words and meanings. Existing Korean benchmarks are centered on established vocabulary and therefore provide limited coverage of such recent lexical change, and their English-oriented design makes it difficult to assess the typological properties of Korean, in which content words combine productively with functional morphemes. In this paper, we introduce KoNeoBench, a benchmark for evaluating LLMs’ understanding of Korean neologisms. KoNeoBench is built on 1,785 Korean neologisms attested in online news since 2020 and curated through expert lexicographic review. Each entry provides usage examples, word-formation analyses, and dictionary-style definitions. Based on this resource, we define four tasks and report results on recent models, together with a human baseline. Our experiments show that current LLMs exhibit clear limitations in recovering source components, distinguishing semantic categories, and generating accurate definitions. These results reveal specific aspects of recent Korean lexical change that remain challenging for current LLMs. KoNeoBench is available at https://github.com/bcmilab/ko-neobench/ .

Abstract:
Modern Large Language Models (LLMs) are often criticized for producing repetitive and homogeneous text, despite possessing vast latent vocabularies. While previous research has focused on model knowledge and training data, we investigate the role of decoding mechanics in suppressing linguistic diversity. It is well-known that sampling filters cut the tail of the token probabilities, but the specific impact at the word level remains unexplored. In this paper, we introduce the Word Coverage Score (WCS), a metric that quantifies the extent to which contextually appropriate human vocabulary is mathematically pruned by standard sampling filters (e.g., Top- p , Top- k , and Min- p ). Rather than assessing static knowledge, the WCS measures the lexical survival rate of words as a function of sampling parameters through a deterministic Forced-Path Audit. By auditing 13 open-weight models on 10,000 evaluation samples per model from the modern FineWeb corpus at four temperatures, we identify which human lexical choices are rendered unreachable by the decoder, even when they reside within the underlying probability space. To characterize word usage reduction without relying only on absolute erasure, we additionally introduce the Low-Coverage Word Rate (LCWR), the fraction of target words reachable in fewer than a specified proportion of contexts. In matched downstream generation experiments, greater WCS is associated with greater output-text lexical diversity: repeated-measures Spearman correlations are 𝜌=.67 for Type-Token Ratio (TTR), 95% Confidence Interval [.50, .82] and 𝜌 = .66 for Measure of Textual Lexical Diversity (MTLD), 95% Confidence Interval [.51, .81] . These results provide quantitative evidence regarding how inference configurations shape vocabulary compression, establishing WCS as a computationally efficient diagnostic tool for hyperparameter debugging and vocabulary calibration.

Abstract:
Existing neural-training-free AI-Generated Text (AIGT) detectors usually aggregate token-level probability signals with uniform weights. However, this ignores an important linguistic regularity: human-written and AI-generated texts exhibit different Part-Of-Speech (POS) n-gram patterns, making certain tokens more informative than others. To address the limitations, we propose SWLD-Joint (Syntax-pattern Weighted Log-likelihood Difference Joint Detection), a calibration-based neural-training-free detector without gradient-based optimization or neural classifier fine-tuning. SWLD-Joint first estimates POS n-gram template weights from labeled calibration data and then combines them with a POS-trajectory calibration signal to focus on syntactically informative positions. Experiments on DetectRL, RAID, and HC3_Ext show robust gains under domain shift, generator mismatch, and adversarial perturbations. Ablations, parameter analysis, case studies, score distributions, and ROC curves verify the effectiveness of syntax-pattern weighting. Code and data will be released upon acceptance.

Abstract:
We provide in this study a multi-faceted analysis of related word detection with automatic methods for Romance languages (including the five main Romance languages: Spanish, French, Romanian, Italian, Portuguese), by combining complementary orthographic, phonetic and semantic features (including both cognates and borrowings in all five languages and a series of classification tasks), with a focus on assessing the impact of the semantic component in specific patterns for Romance related words. We perform quantitative explorations as well as classification experiments for every language pair and different types of etymological relations separately: cognate detection, cognate versus borrowing discrimination and borrowing direction detection.

Abstract:
Peer review at major venues is under unprecedented submission pressure, motivating the use of large language models (LLMs) as review assistants. Existing LLM-based reviewers, however, face two structural limitations. First, they map manuscripts directly to reviews, leaving the underlying rubric implicit and entangling its derivation with the judgement. Second, the prevailing paradigms each capture only half of a good review: training-free agents gather broad evidence but produce undirected critiques, while training-based reviewers inherit human discriminative judgement together with its noise and uneven coverage. We introduce RubricReviewer, a fully rubric-driven framework that addresses both limitations. It makes rubric generation an explicit intermediate step, so that both review generation and the final assessment are conditioned on paper-adaptive rubrics. It further combines a training-free agent Scout that gathers external evidence with a human-aligned trained model Aligner that consumes this evidence, fusing the strengths of both supervision sources. Experiments on real-world submissions show that RubricReviewer produces reviews that are markedly more comprehensive and more discriminative than prior systems, and exhibits the strongest robustness against adversarial prompt-injection attacks. Ablation studies further confirm the necessity of each component.

Abstract:
The emergence of large language models (LLMs) has significantly accelerated recent research on LLM-based automatic grading (AG) systems. Benefiting from the strong instruction-following capabilities and broad prior knowledge of LLMs, educators can deploy AG systems across diverse tasks using only natural language rubrics while achieving satisfactory grading performance. Despite these advantages, new security concerns may also arise. In particular, prompt injection (PI) attacks have recently become a major threat to LLM-based applications. In the context of AG, attackers can potentially exploit PI vulnerabilities to manipulate grading systems into assigning artificially high scores regardless of the actual answer quality. Such behavior poses serious risks to the fairness, reliability, and integrity of educational assessment. In this work, we study PI attacks in AG systems, and systematically investigate the effectiveness of such attacks in educational scenarios. We further evaluate the effectiveness of existing defensive strategies against these attacks. Through comprehensive experiments under rubric-based grading settings, we demonstrate that current LLM-based AG systems remain highly vulnerable to PI attacks. We hope that our findings raise awareness of this emerging threat and motivate future research toward secure, robust, and trustworthy LLM-based educational systems.

Abstract:
Temporal knowledge graph reasoning (TKGR) is crucial for capturing dynamic relational dependencies and inferring evolving facts. However, existing embedding-based and diffusion-based methods primarily rely on Euclidean point embeddings and face two limitations: (1) sparse entities may suffer from embedding crowding and norm-sensitive scoring, especially under limited supervision in long-tail temporal knowledge graphs; (2) uniform noise schedules in diffusion models ignore temporal heterogeneity and instance-level prediction difficulty. To address these challenges, we propose SimDiff-DNS (Simplex Diffusion with Dynamic Noise Scheduling), a simplex-anchored logit diffusion framework for TKGR. First, we represent entity states as probability distributions on the simplex and obtain continuous embeddings as convex combinations of learnable entity basis vectors. This simplex-coordinate parameterization provides a probabilistic view of uncertainty and alleviates norm-sensitive ranking behavior for sparse entities. Second, we introduce a difficulty-aware dynamic noise scheduling strategy for the training corruption process, which adjusts the noise applied to target-entity logits according to a bounded difficulty score. Extensive experiments on three TKGR datasets show that SimDiff-DNS consistently improves over the diffusion-based baseline (improving MRR by 3.25 points on ICEWS14) and achieves competitive or state-of-the-art results on several metrics. Additional diagnostic analyses further demonstrate its benefits for long-tail entities, calibration, and ranking stability.

Abstract:
Glyph-level obfuscation can leave harmful Chinese content readable to humans while degrading automated moderation. We introduce SinoGlyphBench, a diagnostic benchmark that identifies label-critical semantic anchors and creates matched original and glyph-obfuscated inputs in text and image modalities. By perturbing anchors, background context, or both, this design distinguishes corruption of moderation-relevant evidence from general surface variation. Across 176,916 paired evaluations of 12 LLMs and MLLMs, obfuscation increases harmful false-negative and false-positive rates by 6.1 and 4.7 percentage points, respectively, and reduces four-way accuracy by 5.0 points. Models retain 75.7% of the decisions that were correct on the matched original inputs. Full-scope perturbations cause the largest degradation, anchor-only perturbations are more damaging than background-only perturbations, and cross-script substitution is particularly difficult in the text modality. Analysis of structured outputs identifies observable mismatches in visible-form reading, intended-message recovery, and final safety judgment. The evaluated models, therefore, remain brittle to Chinese content written with non-canonical glyphs. Resources are available at https://github.com/fengshun124/SinoGlyphBench.

Abstract:
The generation quality of large language models (LLMs) is often improved by utilizing inference-time scaling methods such as Chain-of-Thought (sequential scaling) or Self-Consistency (parallel scaling). We introduce Roster of Experts (RoE), a training-free inference algorithm for Mixture-of-Experts (MoE) models that defines a third, orthogonal scaling direction. Rather than generating longer outputs or aggregating multiple sequences, RoE improves the next token prediction distribution by treating a single MoE as a dynamic ensemble of internal subnetworks. It injects controlled stochasticity into the expert routing mechanism via Gumbel-Top-k sampling, enabling the model to aggregate multiple diverse subnetworks into a single, higher-quality token prediction. Crucially, because RoE improves the model’s output distribution rather than sampling from it, it is the only approach in this family that is applicable to open-ended generation tasks where sequence-level aggregation is infeasible. Furthermore, RoE is orthogonal to existing scaling methods; combining RoE with Self-Consistency yields greater accuracy than either method alone. To overcome the computational cost of multiple forward passes at inference, we employ an efficient batching strategy that exploits hardware parallelism to minimize latency, requiring no fine-tuning of model parameters.

Abstract:
Reliable uncertainty estimation is important for deploying evaluation metrics in real-world settings, where both input ambiguity and domain shifts are common. In this work, we propose an uncertainty-aware extension of the BEiTScore image captioning metric, producing calibrated confidence intervals alongside alignment scores. Our approach models predictive distributions over human judgments, enabling both accurate scoring and principled uncertainty quantification. We evaluate our BEiTScore++ metric across multiple benchmarks with human-annotated ratings, demonstrating strong performance in terms of ranking correlation, calibration, and interval sharpness. BEiTScore++ not only produces accurate predictions, but also captures aleatoric uncertainty arising from ambiguous inputs, and epistemic uncertainty due to limited model knowledge, making it a reliable tool for uncertainty-aware evaluation of image-caption alignment.

Abstract:
Electronic medical record (EMR) databases are widely used for clinical machine learning, but their preprocessing remains difficult to reproduce because cohort definitions, patient-level features, and clinical item-code mappings must be adapted to each database schema. We formulate EMR preprocessing as the task of translating study requests into database-specific cohort/feature extraction SQL and local clinical item-code mappings. Existing preprocessing and harmonization frameworks remain difficult to adapt because they rely on dataset-specific rules or predefined input formats. We introduce EMR-AGENT, an LLM-agent framework that reduces reliance on manual rule writing by automating database-specific cohort selection, feature extraction, and clinical code mapping from study requests. EMR-AGENT retrieves schema and documentation context, interacts with live databases through SQL, observes execution results, and refines extraction decisions from error feedback. To enable rigorous evaluation, we develop a benchmarking codebase for three intensive care unit (ICU) databases (MIMIC-III, eICU, and SICdb), covering reference databases and a newer SICdb version released after the backbone’s knowledge cutoff. Across these benchmarks, EMR-AGENT reduces schema-debugging and item-code search effort, enabling reproducible evaluation of preprocessing quality under expert review, while reducing reliance on hand-written database-specific pipelines.

Abstract:
Spoken Language Models (SLMs) generate textual responses directly from speech, offering an alternative to cascaded systems. Despite recent advances, existing SLMs still exhibit weaker instruction-following behavior and limited generalization across diverse tasks compared to text-based language models. Our analysis shows that speech and text representations in current SLMs remain weakly aligned despite strong downstream performance, indicating that structural differences between continuous, temporally varying speech and discrete text remain insufficiently addressed. To address this, we propose a simple framework that decouples length mismatch from semantic alignment and encourages closer correspondence between speech and text representations. Experiments across multiple benchmarks demonstrate competitive performance against strong baselines, underscoring the importance of explicitly addressing structural differences between speech and text in SLM training. Our code is publicly available at https://github.com/jaykim9870/Do_SLMs_Hear_Speech_as_They_Read_Text .

Abstract:
Large language models (LLMs) have shown strong performance on medical question answering benchmarks, yet existing evaluations remain limited in assessing longitudinal clinical reasoning across multiple hospital admissions. We introduce ChronoClinQA, a large-scale benchmark for longitudinal clinical reasoning grounded in multi-admission electronic health records. Built from longitudinal discharge summaries in MIMIC-IV-Note, ChronoClinQA contains 15,067 QA pairs spanning six clinically grounded reasoning dimensions and three complementary question formats: open-ended, adversarial, and unanswerable multiple-choice questions. Each instance requires reasoning over temporally accumulated patient trajectories, with contexts averaging over 16K tokens across up to eight hospital admissions. Experiments on both general-purpose and medical LLMs reveal substantial performance degradation in longitudinal and adversarial settings, highlighting persistent challenges in integrating cross-admission evidence and in temporal clinical reasoning. Our findings position ChronoClinQA as a challenging benchmark for advancing clinically grounded long-context reasoning in medical LLMs.

Abstract:
With the rising popularity of interdisciplinary work, there is a growing need to understand how publications incorporate ideas from multiple disciplines. Existing computational approaches, such as affiliation diversity, keyword matching, and citation patterns, do not account for how individual citations are used to advance the citing work. Prior studies have proposed taxonomies to classify citation purpose. However, these frameworks are not well-suited to interdisciplinary research and do not provide general measures of engagement depth. To address these limitations, we propose a framework for evaluating citation engagement in scholarly publications. We introduce a citation purpose taxonomy tailored to interdisciplinary work, supported by an annotation study and model for automatically predicting a citation’s purpose. We then leverage signal from a citation’s purpose and the sections in which it appears to predict a single measure of how deeply a publication engages with a citation. We demonstrate the utility of our framework through a large-scale analysis of publications at the intersection of NLP and Computational Social Science.

Abstract:
Radiology reports are essential for diagnostic reasoning and patient care, yet their manual preparation is time-consuming and cognitively demanding. Automatic radiology report generation (RRG) offers a scalable alternative, but existing models often produce lengthy, unstructured narratives that overlook diagnostic cues and multi-view information. We present Clinical-Knowledge Guided Disease-aware Visual–Textual Alignment for QA-Style Radiology Report Generation (CliQ-RRG), a unified two-stage framework for interpretable and clinically grounded reporting. In Stage 1, CliQ-RRG employs a Disease-aware Visual–Textual Alignment module that aligns image and text representations using predicted disease embeddings, reinforced by a Prior-Guided Attention Module (PrAM) to capture multi-view dependencies across current and prior scans. In Stage 2, GraphRAG-guided retrieval augments intermediate textual representations with graph-grounded clinical evidence, and a large language model restructures them into concise, interpretable question–answer (QA) pairs with diagnostic summaries. Experiments on two public chest X-ray benchmarks demonstrate that CliQ-RRG consistently outperforms prior methods across both lexical and clinical metrics, generating accurate and clinically coherent QA-style radiology reports. Code is available at https://github.com/chandravardhan-singh-raghaw/CliQ-RRG.

Abstract:
As LLM agents move from short isolated interactions to sustained deployment, their effectiveness increasingly depends on memory that can retain important information, adapt to evolving users and environments, and make use of what has been learned from past interactions, but realistic workloads make it difficult to balance reliable fact preservation with the reuse of accumulated experience. To address this challenge, we propose Mixture of Memory (MoMem), a hybrid architecture that separates memory for facts from memory for reusable experience into textual and parametric pathways. During inference, a router decides which textual records to inject and whether to activate parametric memory using logic-matched trajectory evidence; after each interaction, a decoupler separates factual signals from reusable reasoning experience and updates the two pathways independently. Experiments show that, in the main mixed-workload setting, MoMem performs strongly on both sides of this balance, reaching 51.4 on LongMemEval, 11.7 points above the strongest baseline, and 86.4 on MATH-500, 3.2 points above the backbone model. Ablation studies show that these gains come from routing and decoupling rather than hybrid storage alone. Code is available at https://github.com/ShuoSun-CN/MoMem_a

Abstract:
This paper introduces OARelatedWork: a dataset for related work generation from open-access sources. It is the first large-scale multi-document summarization dataset for related work generation, containing whole related work sections and full texts of cited papers. Its validation and test splits are constructed so that every source paper is available in full text, enabling controlled evaluation of full-text related work generation. The dataset includes 94 450 papers and 5 824 689 unique referenced papers from multiple domains. With OARelatedWork, we aim to shift the field from generating parts of related work sections from abstracts only to generating entire related work sections from all available content. We (i) benchmark a wide spectrum of models, highlighting that synthesizing massive full-text contexts remains a challenge for tested cost-effective Large Language Models (LLMs): under our statement-level judge, GPT-4o-mini’s evidence-grounded True rate drops from 92.9% with abstracts to 83.8% with full texts. We (ii) empirically analyze human writing behavior through a human evaluation over 40 papers and 408 factual statements, revealing that authors frequently introduce abstractive claims ungrounded in localized source texts; consequently, tested LLMs actually surpass human baselines in strict, evidence-grounded factuality. Finally, we (iii) conduct a fine-grained meta-evaluation, revealing that standard reference-based metrics are inadequate for evaluating such long-form structured outputs, and introduce a robust statement-level evaluation framework to address this gap.

Abstract:
Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits understanding of how alignment failures manifest in low-resource and culturally diverse languages. We introduce IndicSafeEval, a persuasion-based jailbreak evaluation framework for Indian languages. Our benchmark combines ten safety critical content categories with six human-like persuasive strategies across four different Indian languages, such as Hindi, Bengali, Marathi and Punjabi, resulting in 7,200 adversarial prompts. We conduct a systematic black-box evaluation of several open-source LLMs to examine how their safety behaviour varies across languages, persuasion strategies, and risk categories. Our analysis shows that the model does not behave equally safely across all languages and prompt styles. Instead, safety performance depends strongly on both the languages used and the way a request is phrased using persuasive cues. We further observe that different risk categories exhibit different levels of vulnerability, with some types of harmful content being significantly more susceptible to persuasion-based jailbreaks than others. These findings reveal important limitations of current safety evaluations, which are largely English-centric, and underscore the need for multilingual and persuasion-aware benchmarking frameworks to more accurately assess real-world LLM safety. Our implementation is available at https:://github.com/MonSaikat/IndicSafeEval. Warning: this paper contains example data that may be offensive or harmful.

Abstract:
With the rapid advancement of AI models, their deployment across diverse tasks has become increasingly widespread. A notable emerging application is leveraging AI models to assist in reviewing scientific papers. However, recent reports have revealed that some papers contain hidden, injected prompts designed to manipulate AI reviewers into providing overly favorable evaluations. In this work, we present an early systematic investigation into this emerging threat. We propose two classes of attacks: (1) static attack, which employs a fixed injection prompt, and (2) iterative attack, which optimizes the injection prompt against a simulated reviewer model to maximize its effectiveness. Both attacks achieve striking performance, frequently inducing full evaluation scores when targeting frontier AI reviewers. Furthermore, we show that these attacks are robust across various settings. To counter this threat, we explore a simple detection-based defense. While it substantially reduces the attack success rate, we demonstrate that an adaptive attacker can partially circumvent this defense. Our findings underscore the need for greater attention and rigorous safeguards against prompt-injection threats in AI-assisted peer review.

Abstract:
Culturally loaded translation poses unique challenges for machine translation (MT), as meanings are deeply embedded in socio-cultural contexts beyond surface linguistic forms. Although large language models (LLMs) have enabled MT systems to achieve human-like quality in many scenarios, their ability to handle culturally loaded expressions remains underexplored. In this study, we systematically investigate the challenges posed by culturally loaded translation in LLM-based MT systems. We construct a Chinese-Japanese bilingual dataset from the culturally representative corpus Dream of the Red Chamber, containing 500 segments across diverse cultural categories. Using a comprehensive human evaluation protocol, we identify three main challenges: (1) task challenges, where frontier LLMs exhibit notable performance gaps and struggle with culturally embedded content; (2) human evaluation challenges, where evaluator backgrounds lead to substantial disagreement in translation judgments; and (3) automatic evaluation challenges, where widely used metrics fail to reliably assess translation quality for this task. These findings may offer valuable insights for culture-oriented translation research in both computational science and linguistics.

Abstract:
Object hallucination remains a major challenge in Multimodal Large Language Models (MLLMs), even though they have achieved strong performance across widely vision-language tasks. A key reason for object hallucination is limited fine-grained visual grounding provided by fixed-resolution visual encoding. In this paper, we propose Visual Inference Gating for Object Refinement (VIGOR), a training-free inference-time framework for binary object verification. VIGOR adopts a selective coarse-to-fine pipeline that improve visual grounding without the overhead of always-on multi-view inspection. It first uses the global yes–no margin to estimate whether the available evidence is sufficient. When the evidence is insufficient, VIGOR performs local refinement on uncertain cases using a small set of overlapping crops. Finally, VIGOR combines global and local evidence with an asymmetric decision rule. Experiments on three benchmark datasets across five MLLM backbones show that VIGOR consistently outperforms strong training-free baselines, recovering missed fine-grained evidence while maintaining robust performance on challenging visual reasoning.

Abstract:
Text-rich image understanding requires multimodal large language models (MLLMs) to organize OCR (Optical Character Recognition)-grounded evidence across words, layout, fields, charts, and visual correspondences. Existing evaluations often conflate extraction with reasoning and rarely test whether models follow the required reasoning direction: applying visible rules, abstracting hidden regularities, or recovering missing premises. We introduce OCR-MetaReasoning, a controlled single-image benchmark that treats deduction, induction, and abduction as distinct directions and separates final-answer correctness from reasoning-process compliance. The benchmark contains 1,500 verified samples in a balanced 3×5 taxonomy crossing three reasoning types with five OCR-object categories, along with reference reasoning steps, automatic answer scoring, the Meta-Reasoning Macro Score (MRMS), and the Reasoning Process Compliance Score (RPCS). Experiments with representative closed-source and open-source MLLMs show that OCR-grounded meta-reasoning remains far from saturated: models struggle with visible-rule application and layout-sensitive inference, while process-compliant rationales can accompany incorrect final answers under exact-match evaluation. The code is available at https://github.com/gengxuli/OCR-MetaReasoning.

Abstract:
Recently, the Muon optimizer has demonstrated strong efficiency in large model training through matrix orthogonalization. At each optimization step, Muon orthogonalizes the raw update matrix via Newton-Schulz (NS) iterations before applying it to model parameters. However, orthogonalization requires repeated matrix multiplications whose computational cost scales with O(n 3 ) for an n× n matrix, making the overhead prohibitive at large model scales. In this paper, we observe that the geometric angle derived from cosine similarity between consecutive orthogonal update matrices remains highly stable during training, a phenomenon that we partially explain theoretically and support empirically. Based on this angular stability, we propose LazyMuon, which skips computationally expensive NS iterations through intermittent linear interpolation and employs a residual correction mechanism to mitigate accumulated approximation errors. This design ensures fast iteration while preserving Muon’s orthogonal manifold constraints. Empirical evaluations on language and vision tasks demonstrate that LazyMuon achieves speedups of up to 1.34 × and 1.31 × over Muon, respectively, with negligible degradation in convergence.

Abstract:
Instruction-tuned LLMs faced with under-specified queries often commit to a single interpretation rather than ask for clarification, producing confidently wrong answers. In our experiments, prompting alone is insufficient: models either ask for clarification on every query or ask vague questions that fail to recover the missing information. Addressing this failure requires learning two coupled skills: when to ask rather than answer and how to ask a question that recovers the disambiguating information. Existing recipes either address only one of these skills or require a separately trained critic. We propose CIGAsk , an RL recipe that teaches both skills through two complementary reward signals within a multi-turn GRPO loop. Counterfactual Information Gain (CIG) compares the gold-answer log-likelihood under a frozen reference model with and without the user response, providing per-turn credit that guides how to ask. The Asymmetric Ambiguity Bonus assigns a signed reward at the terminal token based on the gold ambiguity label, guiding when to ask. Across three clarification benchmarks spanning table, passage, and open-domain QA, CIGAsk-7B outperforms the strongest external baseline despite using a smaller backbone. It also transfers across datasets without per-dataset tuning while preserving single-turn QA performance on out-of-distribution benchmarks.

Abstract:
Long-memory benchmarks usually ask whether an agent can recover evidence from prior dialogue and answer questions about it. Persistent assistants also face governed-use decisions: a remembered fact may later be revoked, superseded, restricted to a private context, or irrelevant to the current request. We introduce TimlMemBench, a diagnostic benchmark for governed use of long-term memory in dialogue agents. It starts from reviewed raw sessions and anchors each derived memory asset back to source evidence. The current package contains 208 reviewed Chinese and English samples, 1,566 sessions, 1,715 questions, and 1,821 governance or activation cases. TimlMemBench reports two complementary evaluations. The canonical-pool LLM-QA control compares systems over the same sample-local memory pool. The raw-native evaluation requires external memory systems to ingest raw sessions before answering governed-memory cases. Experiments show that memory-equipped systems reach high question-answering accuracy in the canonical-pool control, but their raw-native behavior is less stable on lifecycle updates, scope and boundary control, provenance recovery, activation, and restraint. TimlMemBench therefore exposes memory failures that are invisible to retrieval-oriented or answer-accuracy-only evaluation.

Abstract:
Multi-turn medical question answering (QA) aims to model realistic clinical diagnosis, where a doctor gathers patient information across multiple turns of conversation. Existing multi-turn medical conversation systems have shown promising progress, but they often rely on accumulated conversation histories as memory, leaving clinical evidence fragmented across turns. We propose GraphMed-LT, a patient-specific graph memory approach with latent clinical thought refinement for multi-turn medical conversations. GraphMed-LT extracts patient-specific clinical triplets from patient responses, retrieves relevant knowledge triplets, and organises them into an incrementally updated graph memory. The graph memory is projected into graph-conditioned evidence tokens and refined inside a trainable doctor agent through hidden-state feedback, enabling the agent to update its internal clinical context before asking follow-up questions or producing the final answer. Experiments on three multi-turn medical QA benchmarks show that GraphMed-LT consistently outperforms existing multi-turn medical conversation baselines across multiple LLM backbones, achieving up to a 6.3 percentage-point absolute improvement over the strongest baseline. Further analyses show that GraphMed-LT asks more answerable follow-up questions and provides consistent gains across medical specialties.

Abstract:
LM-based agents are increasingly deployed for routine but consequential tasks in real-world domains, where their behavior is governed by inherently ambiguous domain policies that admit multiple valid interpretations. Despite the prevalence of such ambiguities in practice, existing agent benchmarks largely assume unambiguous, well-specified policies, leaving a critical evaluation gap. We introduce DRIP-R, a benchmark that systematically exploits real-world retail policy ambiguities to construct scenarios in which no single correct resolution exists. DRIP-R comprises a curated set of policy-ambiguous return scenarios paired with a realistic customer personas, a full-duplex conversational simulation with tool-calling capabilities and a multi-judge evaluation framework covering policy adherence, dialogue quality, behavioral alignment, and resolution quality. Our experiments show that frontier models fundamentally disagree on identical policy-ambiguous scenarios, confirming that ambiguity poses a genuine and systematic challenge to LLM decision-making.

Abstract:
Remote sensing multimodal large language models (RS-MLLMs) have advanced scene understanding and visual question answering over satellite imagery, yet localizing specific objects or changed regions remains challenging. Existing approaches rely on generating bounding box coordinates as token sequences, which is fragile for the small, densely packed objects common in remote sensing and increasingly error-prone when multiple targets must be localized simultaneously. In this work, we present an RS-specific formulation of the region selection paradigm, previously explored in natural-image MLLMs, and extend it to temporal change localization over multi-image sequences. Our framework employs a text-conditioned region proposal module, encodes each candidate as special tokens carrying per-frame visual features enriched with spatial and temporal cues, and lets the LLM localize targets by selecting region tokens in its response. We construct a multi-task training and evaluation suite spanning localization, referring expression, visual grounding, and understanding tasks across single-image and multi-temporal settings. Experiments show that our approach substantially outperforms coordinate-generation baselines on temporal change localization, while improving single-image visual grounding and maintaining competitive understanding performance. Oracle analysis decomposes the contributions of the region proposer and the LLM selector, providing diagnostic insight unique to this framework. Our code will be available at https://github.com/juwan-kr/RS-RegionSelect.

Abstract:
Literary Chinese reading comprehension is a challenging task for large language models (LLMs), requiring understanding of implicit meanings, contextual reasoning, and discourse-level integration in a low-resource setting. However, existing evaluations rely primarily on aggregate accuracy, which provides limited insight into model capabilities, error sources, and reliability under uncertainty. We propose DELit, a diagnostic evaluation framework that moves beyond aggregate metrics by introducing a hierarchical capability–error taxonomy, a Deep-MIRT-based ability estimation model, and a capability-decoupled residual analysis for behavioral diagnosis. Together, these components enable fine-grained interpretation of model performance at both capability and error-type levels. Experiments on mainstream LLMs reveal that (i) discourse-level understanding remains a consistent bottleneck despite strong performance on local semantic tasks; (ii) similar overall accuracy can correspond to substantially different capability profiles; and (iii) overconfident errors on difficult items concentrate on logically constrained reasoning, highlighting reliability risks that are not captured by standard evaluation metrics.

Abstract:
Retrieval-Augmented Generation (RAG) im- proves the factuality of large language models (LLMs), yet existing RAG systems often struggle with complex, multi-step reasoning that requires adaptive retrieval and continuous revision of intermediate contexts. Recent reinforcement learning (RL)-based agentic RAG methods partially alleviate this issue, but typically rely on coarse-grained action spaces and trajectory-level rewards, resulting in weak reward assignment and a bias toward short-horizon, stereotyped reasoning template. To address, we propose AGENTICRAG-R1, a RL framework that deeply integrates reasoning, retrieval, and memory via a memory stack and fine-grained action space, supported by hierarchical action-aware rewards and an information-aware trajectory rejection strategy to enable effective long-horizon learning. Experiments across a diverse set of multi-hop, open-domain, and agentic reasoning benchmarks, spanning multiple backbone model sizes, demonstrate that AGENTICRAG- R1 consistently outperforms strong baselines. Moreover, AGENTICRAG-R1 learns more robust, interpretable, and memory-aware reasoning behaviors, highlighting the effect of fine-grained action modeling and information- aware optimization for long-horizon reasoning. Our code is anonymous available at https: //github.com/jiangxinke/Harness-RL/ tree/AgenticRAG-R1-Whitebox.

Abstract:
Tokenization is a central component of pretrained language models, especially for morphologically rich low-resource languages. We present a controlled study of morphology-aware tokenization for Belarusian masked language models. To isolate the effect of tokenization, we pretrain three RoBERTa-family models on the same Belarusian HPLT corpus, varying only the tokenizer: standard WordPiece, WordPiece trained on morpheme-segmented text, and a dictionary-based hybrid morpheme tokenizer with WordPiece fallback. We evaluate the models on masked language modeling, Universal Dependencies morphology-sensitive tasks, and BelarusianGLUE. Morphology-aware tokenization substantially improves morpheme-boundary preservation and yields clear gains on explicitly morphological tasks, especially fine-grained feature prediction and lemmatization, where models must recover information encoded in inflectional structure. However, these gains do not uniformly transfer to broad sentence-level NLU: on BelarusianGLUE, morphology-aware tokenizers improve some tasks but remain comparable to or weaker than standard WordPiece on others. These findings suggest that morpheme-informed tokenization is a targeted inductive bias rather than a universal replacement for statistical subword segmentation.

Abstract:
Continual multimodal instruction tuning requires multimodal large language models to acquire new task abilities sequentially while preserving previously learned knowledge. LoRA-MoE provides a promising solution by introducing expert-based capacity, but repeatedly learning and maintaining full LoRA experts leads to substantial parameter overhead. This raises a natural question: is full expert expansion necessary for every new task? To answer it, we analyze the SVD of task-specific LoRA updates and observe substantial overlap in their input- and output-side LoRA direction subspaces, with task-specific adaptation largely captured by lightweight coordinates over these subspaces. Motivated by this observation, we propose CoRe-MoE, a Compact Reusable MoE framework for parameter-efficient continual multimodal instruction tuning. CoRe-MoE extracts reusable input- and output-side direction bases from an initial expert bank, and for subsequent tasks trains only compact coordinate experts together with task-specific low-rank routers. Experiments on two representative MLLMs show that CoRe-MoE improves final average performance over the strongest competing baseline by up to 5.90 points, while using less than 1% of the trainable parameters required by sequential LoRA for later tasks.

Abstract:
Due to the long temporal span and highly diverse expressions in historical documents, large language models (LLMs) still struggle with historical and cultural reasoning in Chinese history. Although taxonomic structures can effectively organize historical knowledge and facilitate knowledge-rich applications, existing historical taxonomies remain limited due to the high cost of manual construction. To address this limitation, we propose CHisAgent , a multi-agent LLM framework for automated historical taxonomy induction in ancient Chinese history. CHisAgent explicitly decomposes taxonomy construction into three role-specialized stages: a bottom-up Inducer that derives an initial hierarchy from raw historical corpora, a top-down Expander that introduces missing intermediate concepts using LLM world knowledge, and an evidence-guided Enricher that integrates external structured historical resources to ensure faithfulness. Extensive reference-free and reference-based evaluations show that the collaboration among these agents improves taxonomy coverage and hierarchical completeness, while further analysis suggests that the induced taxonomy facilitates cross-cultural alignment of historical concepts. Code and experimental data are available at https://github.com/tangxuemei1995/CHisAgent .

Abstract:
Sign language translation and generation share the goal of bidirectional alignment between text and sign representations. However, existing approaches either treat them as isolated tasks or are only verified on limited datasets, limiting effective modeling between modalities. In this paper, we propose SignFLIP, a unified LLM-centered framework for translation and generation. To enable bidirectional mapping between text and sign, SignFLIP adopts a symmetric architecture together with a stage-wise training strategy built on large-scale data. The shared sign–text representation is progressively refined: pre-alignment facilitates subsequent SLT, while the SLT-adapted representation further benefits SLG. Extensive experiments on multiple benchmarks show that SignFLIP shows competitive performance compared with task-specific models on both translation and generation tasks, as well as strong transferability to sign language recognition.

Abstract:
The rapid proliferation of LLMs has further heightened the necessity to develop dependable AI-generated text detection, especially beyond English. Nevertheless, current benchmarks pay little attention to Indic languages and test detectors in idealized settings that do not represent the real world. We present a generalized benchmark for AI-generated text detection in Hindi, Telugu, and Tamil, which we name IndicDetect is a benchmark designed to assess the robustness of detectors under realistic distribution shifts. IndicDetect comprises highly curated human-written texts matched with LLM-generated counterparts across various domains and generators, and systematically evaluates detectors in the presence of domain shift, generator shift, and adversarial perturbation. Using a single and repeatable evaluation scheme, we evaluate a wide range of statistical and neural detectors. We find substantial robustness failures: supervised neural detectors perform well in-distribution, but most training-free methods fail catastrophically on unseen generators and adversarial attacks, with larger failures observed for Telugu and Tamil than for Hindi. These results highlight that the primary weakness of existing detectors in Indic settings lies in their robustness, not peak accuracy. IndicDetect provides standard data splits, an evaluation protocol, and baselines to establish a robust, language-aware foundation for AI-generated text detection in Indic scripts.

Abstract:
The most plausible near-term role of medical LLMs is to assist rather than replace physicians, yet current evaluations often test isolated capabilities: clinical knowledge, EHR system interaction, or patient communication. Physician assistance instead requires coordinating these capabilities within the same interaction, where physicians issue underspecified requests, patients describe symptoms ambiguously, and EHR systems demand precise tool use. We introduce PhysAssistBench, a benchmark for interactive doctor-patient-EHR assistance. Built from real MIMIC-IV cases, PhysAssistBench uses a scalable pipeline to construct agentic patients: interactive, record-grounded agents that turn static EHR records into multi-turn clinical scenarios while preserving clinical factuality. PhysAssistBench provides a curated evaluation set of 1,296 manually reviewed and physician-validated turns. Experiments with leading LLMs show that current models remain unreliable in this setting, struggling to infer physician intent, clarify vague patient descriptions, formulate grounded EHR queries, and integrate evidence across sources. These results expose a key bottleneck for clinical LLMs: reliable assistance requires coordination across knowledge, communication, and systems, not isolated gains in any of them.

Abstract:
Conversational agents that support reflection for decision-making often rely on adaptive dialog policies that map observed user behavior to actions such as probing, deepening, or redirecting. Yet the same pattern can reflect a range of different reasons such as deliberate prioritisation or limited self-access. By modeling the observable pattern rather than the user’s reason for it, current policies risk premature assumptions about the user state and inappropriate next actions. To address this gap, we introduce a human-centered method for surfacing this hidden inference step. In a user study with 62 users and 232 collaborative moments, we pause a reflection-support agent when it would normally redirect the conversation, surface its observation, and ask users to interpret the pattern and decide how to proceed. We derive a taxonomy of nine interpretation categories and show that similar reflective states can call for substantially different follow-up actions. Our findings challenge the assumption that adaptive dialog policies can rely on observable behavior alone, and suggest how user-provided interpretations can inform more appropriate conversational actions.

Abstract:
While Vision-Language Models (VLMs) have demonstrated strong capabilities in general multimodal tasks, they face significant challenges in low-resource, cross-temporal settings such as ancient Chinese documents, where models must overcome complex visual noise and leverage deep linguistic priors. The functional organization of task-relevant components in existing VLMs remains understudied from a computational interpretability perspective. To address this, we introduce AncientVision-3T, an intervention-oriented diagnostic benchmark designed for mechanistic interpretability experiments on VLMs, organized across OCR, image classification, and image understanding tasks of increasing abstraction. Based on this benchmark, we propose the Ancient Chinese Target-Conditioned Gradient-based Neuron analysis framework (AC-TCGN), which identifies and validates task-relevant neurons for dissecting functional organization in general and domain-specific VLMs. Our experiments demonstrate that AC-TCGN effectively identifies neurons that are critical for model predictions. Moreover, our experimental results reveal a transition from separation to entanglement in the functional organization of decoder feed-forward networks: general VLMs show stronger task-wise separability, whereas the ancient Chinese domain-specific VLM shows greater cross-task neuron sharing and reduced task-wise separability. This suggests enhanced coupling between visual form and historical-semantic processing in ancient Chinese document image-text tasks.

Abstract:
Patient-facing medical LLMs are typically evaluated for clinical knowledge and safety, but clinically correct answers can still fail patients by missing fear, mishandling uncertainty, or offering emotionally unsafe reassurance. We introduce MedEQ, a benchmark for clinical emotional intelligence in medical communication. MedEQ includes 413 clinically grounded consultation scenarios across 51 specialties and 7,336 expert-refined rubric items over 21 core dimensions. These expert-validated rubrics operationalize emotional cue recognition, empathic adequacy, ethical emotional regulation, and affective safety, enabling diagnostic evaluation rather than a single holistic empathy score. Using an LLM-as-judge rubric pipeline, we find that strong contemporary models often handle surface compassion, safety, and response structure, yet remain weak on deeper emotional inference, personalization, mixed emotions, cultural context, and real-time responses to illness-related distress. We further conduct a proof-of-concept rubric-guided reinforcement learning study, showing that MedEQ-derived rubric signals can transfer to an independent emotional intelligence benchmark: rubric-guided training of Qwen3-32B improves EQ-Bench performance from 42.42 to 81.49 while preserving or slightly improving standard medical benchmark scores. MedEQ therefore makes clinical emotional intelligence measurable and decomposable, and provides evidence that expert-refined rubrics can support the evaluation and improvement of patient-facing medical AI.

Abstract:
Recently, Multimodal Large Language Models (MLLMs) have demonstrated strong visual understanding and reasoning abilities across vision-language tasks. However, conventional Visual Question Answering (VQA) pipelines do not fully exploit the visual features and medical knowledge captured by MLLMs, leading to limited performance in zero-shot medical disease recognition. To address this issue, we propose LLaVA-RadZ, a simple yet effective framework for zero-shot radiology recognition. LLaVA-RadZ introduces Decoding-Side Feature Alignment Training (DFAT), which leverages decoder-side representations with modality-specific tokens for image-text alignment, and a Domain Knowledge Anchoring Module (DKAM), which incorporates disease-aware semantic anchors to mitigate the category semantic gap. Extensive experiments show that LLaVA-RadZ substantially improves the zero-shot disease recognition ability of MLLMs and achieves competitive performance compared with well-established CLIP-based radiology recognition methods.

Abstract:
Large language models (LLMs) and LLM-based coding agents are now used to generate code from natural-language specifications, yet ensuring such code is both functionally correct and secure remains a challenge. We present DualGauge, the first fully automated framework for jointly evaluating correctness and security of specification-only code generation, supported by DualGauge-Bench, a language-agnostic benchmark of 307 coding tasks each paired with functional and security tests derived from the same specification. Evaluating 10 representative LLMs across Python, C++, and JavaScript, we find that functional correctness substantially overestimates reliable code generation: even the strongest model remains below 15% joint security-functionality success in every language. Common model-side factors—scale, extended thinking, quantization, instruction tuning, and code specialization—do not reliably improve joint performance, suggesting secure-and-correct code generation does not simply emerge from stronger coding capability. Evaluation of 3 leading agentic coding systems (Codex, OpenHands, and Claude Code) shows that iterative scaffolding provides no advantage over direct (LLM-based) generation on specification-only tasks. A qualitative audit reveals failures concentrate at the output contract boundary and in guards that exist but are insufficient—patterns that only joint benchmarking reliably exposes.

Abstract:
Open-ended medical LLM evaluation remains weakly grounded in physician-calibrated coverage of clinically relevant response criteria, especially in localized clinical settings. We introduce ClinConsensus, a Chinese medical benchmark of 2,500 expert-curated cases spanning 36 specialties, 12 task themes, multiple difficulty levels, and lay-facing versus professional-facing settings. Each case is paired with 30 case-specific binary rubric criteria. To evaluate whether responses satisfy enough physician-authored criteria, we propose Clinician-Anchored Coverage Score (CACS), a physician-calibrated threshold metric instantiated at k = 10, and develop a dual-judge framework combining a GPT-5.1 grader with a physician-supervised Qwen3-8B judge. Evaluating 11 frontier LLMs, we find a persistent coverage gap: Rubric Accuracy ranges from 39.6% to 52.1%, whereas CACS@10 ranges from 17.8% to 32.9%, leaving a 19.2–21.9 point gap across models. Stratified analyses further reveal substantial variation across reasoning, evidence use, structured extraction, medication instructions, follow-up, and dialogue register. These results suggest that medical LLM evaluation should measure thresholded, rubric-grounded clinical coverage rather than average partial correctness.

Abstract:
This paper introduces EvolvingWorld, a framework and benchmark for character and world co-evolution in interactive literary worlds. Existing systems either treat interactive literary simulation as static persona imitation or isolated scene generation, failing to capture how characters and worlds evolve together over time. To address this, EvolvingWorld models literary simulation as a long-horizon process where characters interact, scenes progress, and character and world states are persistently updated. Unlike prior systems relying on fixed schemas, EvolvingWorld adopts an open-schema framework to support simulation across diverse literary worlds. The framework consists of two coupled modules: a Character Agent for multi-character role-play and persistent profile evolution, and an LLM-based World Model for global and location/entity-level state maintenance and scene progression. Based on this architecture, we formulate 7 trainable tasks for scene initialization, interaction generation, and state update. We construct a dataset from 57 books, producing 138,596 supervised training samples and 222 snapshots for testing. Furthermore, we introduce a trajectory-level LLM-as-Judge evaluation protocol spanning 10 dimensions and 20 metrics. Experiments show that EvolvingWorld can improve long-horizon simulation by effectively maintaining persistent, coherent character and world development.

Abstract:
Large language models (LLMs) increasingly operate in high-stakes settings where demographic attributes such as race and ethnicity may be explicitly stated or implicitly suggested through textual cues. However, existing studies primarily document outcome-level disparities, offering limited insight into internal mechanisms underlying these effects. We present a mechanistic study of how race and ethnicity cues are represented and operationalized within LLMs. Using two publicly available datasets spanning toxicity-related generation and clinical narrative understanding tasks, we analyze three open-source models with a reproducible interpretability pipeline combining probing, neuron-level attribution, and targeted intervention. We find that sensitivity to demographic cues is distributed across internal units and varies substantially across models. These units often align with entangled semantic facets, including explicit group labels, geography, language, culture, and associations related to stereotypes. Interventions on selected units can change some biased prediction patterns, but substantial residual effects remain, suggesting that effective mitigation requires understanding distributed, task-specific mechanisms rather than manipulating a small set of identified neurons alone.

Abstract:
We present CITAR-NER-Br, a unified NER dataset for direct extraction of fine-grained legal citation subcomponents in Brazilian Portuguese. CITAR-NER-Br annotates six citation subcomponents over 270 legal documents from LeNER-Br and Ulysses Tesemõ, capturing the overlapping and discontinuous spans intrinsic to legal citations. To the best of our knowledge, this is the first unified NER dataset for the legal domain specifically designed to represent discontinuous entities, which account for 44.8% of annotations, the highest proportion among existing legal NER corpora. We establish baselines using four configurations across three LLM-based and non-LLM NER methods to support future research. Our experiments demonstrate that, while non-LLM methods achieve strong performance, they remain heavily reliant on high-quality training data. Conversely, LLM-based methods achieve overall lower performance, but require only few-shot supervision. Moreover, the traditional exact-match NER evaluation approach significantly penalizes LLM-based methods even when their outputs preserve semantic accuracy.

Abstract:
Large Language Model (LLM) surprisal is increasingly used as a proxy for human sentence processing, yet whether model expectations reflect the cognitive factors governing real-time human behavior remains uncertain. We examine Korean subject–verb honorific agreement in subject- and object-control constructions, where human eye-tracking data reveal a prominence-driven asymmetry. Evaluating three Korean-capable models, we find that surprisal was sensitive to honorific-feature mismatches, yielding attraction-like effects, but was not modulated by distractor prominence. Although surprisal selectively improved reading-time predictions, this reflected item-level predictive overlap rather than prominence sensitivity. GPT models thus capture surface regularities but do not reproduce the construction-sensitive, prominence-driven asymmetry of human dependency resolution, suggesting that LLMs diverge from humans precisely where processing depends on working-memory constraints rather than distributional expectation.

Abstract:
Existing methods for evaluating the factuality of large language model (LLM) responses treat all claims as equally important. This results in misleading evaluations when vital information is missing or incorrect as it receives the same weight as peripheral details, raising the question: how can we reliably detect such differences when there are errors in key information? Current approaches that measure factuality tend to be insensitive to omitted or false key information. To investigate this lack of sensitivity, we construct VITALERRORS, a benchmark of 6,733 queries with minimally altered LLM responses designed to omit or falsify key information. Using this dataset, we demonstrate the insensitivities of existing evaluation metrics to key information errors. To address this gap, we introduce VITAL, a set of metrics that provide greater sensitivity in measuring the factuality of responses by incorporating the relevance and importance of claims with respect to the query. Our analysis demonstrates that VITAL metrics more reliably detect errors in key information than previous methods. Our dataset, metrics, and analysis provide a foundation for more accurate and robust assessment of LLM factuality.

Abstract:
Multilingual encoder-based language models are widely used for code-mixed analysis, yet their internal representations of code-mixed inputs—and their relationship to the constituent languages—remain poorly understood. Using Hindi–English as a case study, we construct a unified trilingual corpus of parallel English, Hindi (Devanagari), and Romanized code-mixed sentences. We then probe cross-lingual representation alignment in standard multilingual encoders and their code-mix-adapted variants using CKA, token-level saliency, and entropy-based uncertainty analysis. We find that while standard models align English and Hindi well, code-mixed inputs remain loosely connected to either language — and that continued pre-training on code-mixed data improves English–code-mixed alignment at the cost of English–Hindi alignment. Interpretability analyses further reveal a clear asymmetry: models process code-mixed text through an English-dominant semantic subspace, while native-script Hindi provides complementary signals that reduce representational uncertainty. Motivated by these findings, we introduce a trilingual post-training alignment objective that brings code-mixed representations closer to both constituent languages simultaneously, yielding more balanced cross-lingual alignment and downstream gains on sentiment analysis and hate speech detection — showing that grounding code-mixed representations in their constituent languages meaningfully helps cross-lingual understanding. Code is available at: https://github.com/debajyotimaz/tri_align_EMNLP_2026.

Abstract:
Automatic extraction of Medical Decision Trees (MDTs) is an important task for Clinical Decision Support Systems, but remains challenging due to the complex logical structures involved. Recent approaches based on Large Language Models (LLMs) commonly adopt generate-then-revise pipelines with self-correction to improve reasoning accuracy. However, we find that applying revision uniformly across all samples can be harmful, as it often alters already correct and stable predictions, leading to over-correction on simple cases. To alleviate this issue, we propose R 3 -MDT, an adaptive inference framework that dynamically selects reasoning strategies based on sample difficulty. We use model uncertainty and output consistency as internal signals to estimate difficulty and route samples into Easy, Medium, and Hard categories. For Easy samples with confident and consistent predictions, we directly retain the initial outputs. Medium-difficulty samples, which exhibit ambiguity among multiple plausible candidates, are handled via reranking to select the optimal decision tree. Hard samples, characterized by high uncertainty and structural inconsistency, are revised to rectify intractable generation failures. By adaptively selecting inference strategies based on difficulty levels, R 3 -MDT avoids unnecessary revisions while effectively correcting underlying reasoning failures. Experiments on the Text2DT benchmark show that R 3 -MDT alleviates the over-correction problem and achieves state-of-the-art performance, while auxiliary language- and task-shift evaluations indicate the broader transferability of its routing principle.

Abstract:
Multimodal Large Language Models (MLLMs) have demonstrated strong abilities in solving diverse visual and textual reasoning tasks. However, their development in the physics domain is significantly hindered by the lack of a comprehensive benchmark. To fill this gap, we introduce OmniPhys , a large-scale benchmark for multimodal physics understanding and reasoning, covering middle school through university-level problems from Chinese Educational Corpora. OmniPhys consists of 15,246 questions and 19,850 images, accompanied by detailed annotations that support fine-grained analysis of reasoning processes and knowledge usage. Beyond conventional evaluation, OmniPhys is a benchmark that systematically evaluates multimodal outputs in physics domain, including models’ ability to generate structured physics diagrams, which constitute a fundamental component of authentic physics problem solving.

Abstract:
Detecting Schwartz values in political texts is hard: cues are often implicit, and neighboring values differ by fine distinctions. Two remedies are widely assumed to help: more surrounding document text, and explicit moral knowledge. Knowledge-based retrieval has improved benchmarks elsewhere, but whether either transfers here is untested, because published systems vary context, knowledge, and model family at once. We separate these factors under matched conditions on the ValuesML/Touché ValueEval format. The input ranges from the target sentence to a local window to the full document. Retrieval is either absent or drawn from a curated moral knowledge base, injected by early, late, or cross-attention fusion. Supervised DeBERTa-v3 encoders are compared against zero-shot LLMs from 12B to 123B. More context is not uniformly better: full-document input improves the encoders by 2.5-3.8 macro-F1 points but does not consistently help the LLMs. Retrieved knowledge helps more reliably, improving every model family and context under early fusion. A control substituting random knowledge-base entries shows the families gain differently: LLMs from relevance, encoders from exposure to the value ontology. Neither larger encoders nor larger LLMs guarantee gains, and early fusion outperforms both trainable variants. Value-sensitive NLP should evaluate context, knowledge, and model family jointly.

Abstract:
As large language models (LLMs) are increasingly deployed globally, aligning them with pluralistic human values remains a challenge, as current efforts are predominantly Western-focused. To bridge this gap, we introduce SEAPlur, a dataset designed to benchmark and improve LLM alignment with the pluralistic values of four Southeast Asian (SEA) countries: Indonesia, Laos, Myanmar, and Vietnam. SEAPlur consists of independent datasets for each country and is constructed in their respective native, non-English languages. Sourced from local news and structured around the Schwartz’s theory of basic human values, SEAPlur captures local human agreement distributions across complex social dilemmas. In total, the dataset comprises 4,000 curated scenarios. For evaluation, we split 250 scenarios for each country resulting in SEAPlur-Test. The remaining data with "Agree" and "Disagree" annotations is transformed into a SEAPlur-Train 7,460 instruction-response pairs. Our evaluations using SEAPlur-Test reveal that general API LLMs struggle with SEA values, achieving peak overall accuracy (OA) scores of only 0.50. In contrast, region-specific LLMs perform better, reaching OA scores up to 0.58. Furthermore, supervised fine-tuning (SFT) using SEAPlur-Train significantly improves alignment. However, this process reveals a critical safety tension: enhancing regional alignment in certain LLMs may trigger unsafe behaviors. The dataset is publicly available at https://github.com/tjunlp-lab/SEAPlur.

Abstract:
Dialectal machine translation remains challenging due to limited data and strong linguistic variation not captured by standard benchmarks, which often assume standardized and well–edited text. We study Polish–Silesian MT using neural and rule–based systems, evaluating on SiLTT — a new Pol–Szl testset, alongside established BOUQuET and FLORES benchmarks. Results show our rule–based system is consistently strongest on SiLTT and BOUQuET datasets and that TranslateGemma fine–tuned on a curated dataset improves over strong neural baselines but does not surpass the rule–based system in dialectal settings. We release SiLTT and our best neural model to support further research.

Abstract:
While Large Language Models (LLMs) can convincingly simulate personas in explicit self-reports, they often deviate in implicit behavioral decisions, revealing a substantial Knowledge–Decision Gap (GKD). Existing benchmarks struggle to measure this discrepancy due to limited construct validity, multidimensional entanglement, and distributional biases in LLM-based evaluation. To address these issues, we propose ActTraitBench, a human-grounded evaluation framework for measuring personality consistency in LLMs. Grounded in empirical human data, ActTraitBench establishes one-to-one mappings between psychometric facets and behavioral paradigms and applies Distributional Calibration via Quantile Mapping to reduce distributional mismatch between LLM-judge scores and human responses. Experiments on 14 mainstream LLMs reveal substantial knowledge–decision gaps and show that assigned personas are reflected more consistently in self-reports than in behavioral decisions for most evaluated models. To mitigate this gap, we further introduce the Chain of Cognitive Alignment (CoCA), an inference-time intervention that reduces GKD for 12 of the 13 models with paired results. Code and resources are available at https://github.com/Selina233/ActTraitBench.

Abstract:
Multimodal large language models (MLLMs) have shown increasing potential for applications in the financial domain. However, the challenges posed by real-world financial scenarios, including high-information-density, strong noise interference, and cross-modal multi-hop reasoning, go beyond the evaluation scope of existing financial multimodal benchmarks. To this end, we propose UniFinEval, the first unified multimodal financial evaluation benchmark designed for high-density financial information environments, covering three modalities: text, image, and video. UniFinEval abstracts three progressive core scenarios from perception to reasoning and decision-making, and further refines them into six task categories. It contains 5,567 high-quality Chinese and English question–answer (QA) pairs constructed by financial experts. We systematically evaluate 10 mainstream MLLMs under zero-shot and chain-of-thought (CoT) settings. Experimental results show that Gemini-3-Pro achieves the best overall performance, but still exhibits a significant gap compared with financial experts. Further error analysis reveals the capability deficiencies of MLLMs in financial scenarios. UniFinEval aims to systematically evaluate the capabilities of MLLMs in real financial scenarios, thereby promoting the application of MLLMs in the financial domain. The related code and data are available at https://github.com/aifinlab/UniFinEval.

Abstract:
Automated theorem proving (ATP) benchmarks largely consist of problems formalized in MathLib, so current ATP training and evaluation are heavily biased toward MathLib’s definitional framework. However, frontier mathematics is often exploratory and prototype-heavy, relying on bespoke constructions that deviate from standard libraries. In this work, we evaluate the robustness of current ATP systems when applied to a novel definitional framework. We introduce TaoAnalysisBench, an undergraduate-level benchmark derived from the exercises and definitional frameworks given in Terence Tao’s Analysis I, which formalizes analysis by constructing core mathematical concepts from scratch, without relying on standard Mathlib definitions, as well as by mixing from-scratch and MathLib constructions. For fair evaluation, we build an agentic pipeline that automatically extracts a compilable, self-contained local environment for each problem. To isolate the effect of definitional frameworks, we additionally translate every problem into a mathematically equivalent Mathlib formulation, yielding paired TaoAnalysisBench-Mathlib statements for direct comparison. While state-of-the-art ATP models perform capably within the MathLib framework, performance drops by an average of 2.18x on the definitionally equivalent Tao formulation. This indicates that the main bottleneck is limited generalization across definitional frameworks rather than task difficulty. TaoAnalysisBench thus highlights a gap between benchmark performance and applicability, and provides a concrete foundation for developing and testing provers better aligned with research mathematics.

Abstract:
Despite the strengths of modern anonymization and de-identification techniques, the risk of re-identification remains significant due to the indirect identifiers remaining in texts. To address this problem, recent works have applied text rewriting under Differential Privacy (DP) to prevent data linkage by perturbing texts via noise addition. Such methods privatize all tokens in a text indiscriminately, diminishing text quality and usability in critical domains such as in clinical settings. Focusing on indirect personal identifiers (IPIs), we introduce a utility-preserving DP text rewriting method that only privatizes spans containing IPIs. We show that our method effectively reduces re-identification risks in clinical texts while being producing more coherent and usable output texts, leading to higher privacy-utility trade-offs. In this, we demonstrate the effectiveness of hybrid text privatization, which leverages the promise of DP in an efficient, usable manner.

Abstract:
Mixture-of-Experts (MoE) architectures have become a dominant paradigm for scaling large language models (LLMs), but their large memory footprint still makes post-training quantization (PTQ) essential for practical deployment. Unlike dense LLMs, quantized MoE models exhibit routing perturbations, where quantization alters expert routing behavior. Existing router alignment methods mitigate this issue by uniformly minimizing routing discrepancies across all tokens. However, we observe that routing perturbations exhibit highly heterogeneous relationships with observable output discrepancy: many tokens preserve nearly identical output distributions despite substantial routing changes. Motivated by this observation, we propose OPERA, an output-aware router alignment strategy for quantized MoE models that selectively prioritizes tokens exhibiting observable output discrepancy after quantization. We further analyze output-preserving routing perturbations and show that they frequently occur on highly predictable tokens, where substantial routing perturbations and hidden-state deviations may not strongly affect projected output distributions. Experiments across multiple MoE LLMs demonstrate that OPERA generally improves over conventional router alignment, with particularly noticeable gains under aggressive low-bit quantization settings.

Abstract:
We conduct a controlled study isolating reasoning in autonomous debugging by holding scaffolding and tool access fixed. We decompose fault-targeted reasoning into test generation and repair using LLM-generated solutions that pass most tests but fail on edge cases. Across 12 frontier and open-source models, we identify three limitations. First, despite 87.7% input validity, only 68.3% of inputs are discriminative, identifying fault-hypothesis formation as the bottleneck: finding fault-triggering inputs costs 19.4 points versus 2.7 points for output prediction. Second, diagnostic feedback has mixed effects on repair: tests generated internally within the repair response reduce success by 5.6 points relative to a bug label alone, while separately generated same-model tests differ little on average, but repair succeeds more often with discriminative than non-discriminative tests (66.8% versus 25.2%). Third, capabilities do not consistently compose: success across bug detection, test construction, and repair does not consistently occur on the same instances, with the largest losses in forming a fault-triggering input and converting a discriminative test into a correct repair. Together, these findings identify both transitions as key debugging limits.

Abstract:
Text-to-SQL parsing increasingly relies on high-quality Chain-of-Thought (CoT) data to train specialized models. However, eliciting accurate reasoning paths for complex queries remains a formidable challenge, causing existing methods to suffer from severe data underutilization and fragile free-form generation. To systematically resolve these issues, we propose MSI-SQL, a Multi-task Self-Iterative framework that enhances both data quantity and quality. To expand data scale, we introduce a verification-based self-iterative loop that progressively conquers difficult queries. To ensure reasoning stability, we design a masked SQL completion auxiliary task that serves as a structural anchor, explicitly constraining the CoT generation space. Coupled with a novel mask-augmented soft self-consistency mechanism during inference, evaluations on the BIRD and Spider benchmarks demonstrate that MSI-SQL establishes a new state-of-the-art for open-source Text-to-SQL parsers.

Abstract:
Vision-Language-Action (VLA) models are built on the premise that pre-trained Vision-Language Models (VLMs) can supply general semantic understanding ability for embodied tasks. However, we find that robotic fine-tuning can severely erode this capability: VQA benchmarks and layer-wise CKA analysis reveal rapid representational drift, and co-training with general vision-language data does not prevent the collapse. We propose TwinBrainVLA, an asymmetric dual-stream VLA that keeps a pre-trained "Left Brain" strictly frozen as a semantic generalist while training a "Right Brain" as an embodied control specialist. Its Asymmetric Mixture-of-Transformers (AsyMoT) mechanism lets the Right Brain query frozen Left-Brain key-value representations at every transformer layer, integrating intact semantic knowledge without corrupting it. Across SimplerEnv, RoboCasa, LIBERO, and real-robot experiments, TwinBrainVLA outperforms strong baselines and large-scale embodied models, achieving a 14.3% improvement on SimplerEnv and a 2.7 × gain in physical-robot instruction generalization.

Abstract:
Recent strategies for low-resource machine translation rely on LLMs to generate synthetic data based on text in higher-resource languages. We revisit this idea for Romansh, a language with 6 distinct varieties. LLMs tend to confuse these varieties when translating into Romansh, but they are quite good at translating out of Romansh into a high-resource language such as German. Due to this asymmetry, the direction of data augmentation is a crucial choice. We find that contrary to recent strategies, creating synthetic translations into the higher-resource language is the superior approach, and only this approach allows us to surpass a Gemini 3 Pro baseline on German-Romansh translation (+23 BLEU over Gemini in the lowest-resource variety). A human evaluation confirms that our experiments yield the first model that generates fluent translations in the individual Romansh varieties.

Abstract:
Large language models (LLMs) struggle with formal domains that require rigorous logical deduction and symbolic reasoning, such as mathematical proof generation. We propose a neuro-symbolic approach centered on the hypothesis that structurally analogous problems often admit similar proofs. As a proof-of-concept, we focus on SAT-level geometry problems. Our approach is two-fold: (1) We retrieve analogous problems and use their proofs to guide the LLM, and (2) a formal verifier evaluates the generated proofs and provides feedback, helping the model fix incorrect proofs. Our complete pipeline substantially improves proof accuracy across model families, achieving 68%-96% accuracy compared with 10%-44% for LLM-only baselines that use neither analogy retrieval nor verifier feedback. When comparing against models with the same verifier feedback and inference budget, analogical guidance improves accuracy from 88% to 96% for GPT-5, 78% to 86% for Claude Sonnet 4.6, 72% to 86% for Gemini-Flash-2.5, and 52% to 80% for OpenAI o1. More broadly, shifting to LLMs that generate provably correct conclusions has the potential to dramatically improve their reliability, accuracy and consistency, unlocking complex tasks and critical real-world applications that require trustworthiness.

Abstract:
While large language models (LLMs) demonstrate strong capabilities across diverse user queries, they still suffer from hallucinations, often arising from knowledge misalignment between pre-training and fine-tuning. To address this misalignment, we reliably estimate a fine-grained, instance-level knowledge score via multi-sampled inference. Using the knowledge score, we scale the learning signal according to the model’s existing knowledge, while encouraging explicit “I don’t know” responses for out-of-scope queries. Experimental results show that this approach allows the model to explicitly express uncertainty when it lacks knowledge, while maintaining accuracy on questions it can answer. Furthermore, we propose evaluation metrics for uncertainty, showing that accurate discrimination between known and unknown instances consistently improves performance.

Abstract:
Recent Machine Learning (ML) approaches have shown increased performance on benchmarks at the cost of escalating compute demands. Hardware, algorithmic and carbon optimizations have been proposed to curb energy use and environmental impacts. We estimate the environmental impacts associated with training models documented in the Epoch AI database over the last decade, with a particular focus on impacts associated with Large Language Models and the hardware used to train them. We find that energy use and environmental impacts associated with training ML models have increased exponentially, even when considering impact reduction strategies such as using less carbon intensive electricity mixes or more efficient hardware. Optimization strategies do not mitigate the impacts induced by model training, suggesting rebound effect. We show that the impacts of hardware must be considered over the entire life cycle rather than the sole use phase in order to avoid impact shifting. Our study demonstrates that increasing efficiency alone does not ensure sustainability. There is an urgent need to systematically integrate environmental impacts in NLP evaluation practices to better inform the community and support the use of impact as a feature in research planning and decision making.

Abstract:
Large language models (LLMs) are increasingly deployed as long-horizon agents with decision-making capacities. While LLMs can show ethical competence on dilemmas such as trolley problems, this competence may not translate to complex, agentic scenarios. We study whether this competence surfaces and governs LLMs’ behavior in Civilization V, a multiplayer game with a complex decision-making landscape including economy, diplomacy, technology, and military strategy. Starting from 130 high-tension LLM self-play episodes, in which an LLM player spontaneously escalated nuclear authorization, we replay them across 13 models with three prompt interventions: an ethical prompt naming nuclear harm, removal of the previous model’s decision-making rationale, and high-stakes framing emphasizing real-world impacts. No interventions nor their combinations reliably eliminate escalated nuclear authorization. We identify three failure pathways: ethical reasoning that fails to surface without prompting, fails to appear even when prompted, or surfaces but fails to take effect when strategic counter-factors dominate. Evaluations of agentic models, therefore, must test whether ethical reasoning is spontaneously invoked and behaviorally effective in complex decision-making contexts, beyond whether it can be elicited in isolation.

Abstract:
Drug-information question answering is a high-stakes setting where hallucinated facts can mislead clinical decision-making and the provenance of each cited fact matters as much as the fact itself. We present DrugClaw, a multi-agent retrieval-augmented system that queries a registry of drug and pharmacovigilance skills via a reflection-driven state-machine workflow and returns answers grounded in primary regulatory or peer-reviewed records. We also contribute DrugAudit, a 3,772-item authority-aware benchmark with an evaluation panel that scores upstream-of-gold source match, token-level semantic snippet overlap, and citation faithfulness under a dual-judge LLM-as-judge protocol with inter-judge κ = 0.88 (almost-perfect). Across DrugAudit plus drug-related subsets of MedQA (751) and PubMedQA (512), DrugClaw is top-1 on every column of the headline table: composite Evidence Index under both judges, judge-mediated answer correctness, primary-source rate (0.918, +10.1 pp over next-best), faithfulness (0.887, +5.9 pp), MedQA (0.920), and PubMedQA (0.693).

Abstract:
Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-rich content remains insufficiently studied. Internet memes present a challenging setting where meaning emerges from implicit interactions between image, overlaid text, sarcasm, and shared socio-cultural knowledge rather than literal visual recognition. This challenge is amplified in low-resource languages such as Bangla, where code-mixing, stylized scripts, and culturally specific symbolism introduce substantial distribution shift. In this work, we introduce BanglaMemeX, a culturally grounded multimodal benchmark comprising 3,000 Bangla memes annotated with multi-dimensional labels (humor, sarcasm, offensiveness, motivational intent, and overall sentiment) and human-written explanations that explicitly describe textual and visual metaphors. We systematically evaluate modern VLMs on both classification and explanation generation, revealing that current models struggle to interpret implicit cultural cues despite reasonable surface-level accuracy. Our results highlight the need for culturally-aware multimodal systems capable of grounded reasoning under linguistic and cultural distribution shift.

Abstract:
Safety judges are typically optimized for static harm definitions, yet real-world deployment policies evolve rapidly. We present , a model-agnostic safety judging framework that enables zero-shot policy adaptation without model fine-tuning. Rather than relying on static weights or vector-search RAG over raw text chunks, CourtGuard-PMPD introduces a system abstraction that compiles external governance documents into a Prompt-Based Modular Policy Database (PMPD). At inference time, an LLM category mapper routes inputs against active policy modules to lightweight label judges. When policies change, the system updates its decision boundary instantly by swapping database modules. Evaluated on MLCommons intra-domain policy adaptation, CourtGuard-PMPD achieves the highest mean accuracy (0.84) and F1 (0.82) across four staged policy removals, outperforming policy-following baselines without retraining. On a policy-aligned 850-example attack suite and an eight-dataset public benchmark suite, it remains highly competitive (0.915 and 0.871 accuracy, respectively). These results position policy-grounded judging as a controllable and competitive alternative to static safety classifiers and benchmark-oriented generic judges.

Abstract:
As large language models (LLMs) scale, inference throughput is increasingly bottlenecked by memory bandwidth. While prior 4-bit quantization schemes promise memory savings, their dequantization overhead often negates these gains. We present FServe, a hardware-software co-designed framework featuring an FP8-native W4A8KV4 scheme that combines channel pre-scaling and group-wise scale relaxation for linear layers with post-RoPE key smoothing that aligns channel-wise means and ranges for attention layers. To translate these memory savings into FP8-native throughput, FServe further extends FlashAttention-3 kernels for prefill with packed-FP16 softmax instruction reordering and multi-warp asynchronous TMA fetching, and introduces a dynamic Stream-KV tile scheduler for decode phase workload imbalance. Our evaluation on Llama-3.1-8B and Qwen3-8B shows that FServe outperforms AWQ W4A16 and W4A8 baselines on accuracy despite additionally compressing the KV cache to INT4, achieves up to 1.41× TTFT speedup over BF16, and sustains effective decode bandwidth above the H100’s theoretical peak.

Abstract:
Large language models have made strong reasoning gains through supervised fine-tuning, reinforcement learning, and on-policy distillation, yet these post-training methods are usually evaluated only by final-answer accuracy. We study how they reshape confidence during reasoning. We introduce a three-stage calibration framework that evaluates confidence before, during, and after chain-of-thought generation, corresponding to difficulty estimation, early termination, and answer aggregation. Through a controlled comparison on mathematical reasoning benchmarks, we find that OPD provides the most useful pre-reasoning confidence, SFT gives the strongest online signal for early stopping, and RL produces the most reliable trace-level signal for aggregation. We further show that confidence reliability is position-dependent: RL confidence becomes informative after a path-commitment phase, while OPD confidence is useful early but can become inversely calibrated later. Based on this observation, we propose PosConf, a position-aware confidence strategy that uses confidence only from reliable relative-position intervals. PosConf improves RL answer aggregation by 6.1 points over majority voting and consistently improves OPD early stopping under tight token budgets, with gains up to 4.3 points by avoiding its later inverse-calibration region, showing that confidence in reasoning models should be used both stage-wise and position-awarely. Our code is available at https://github.com/EIT-NLP/Post-Training-Calibration.

Abstract:
Over 30% of autistic individuals are non- or minimally-speaking, relying on a complex, idiosyncratic repertoire of multimodal behaviors to communicate. Recognizing these intentional communicative acts in naturalistic environments remains a significant challenge for current multimodal large language models (MLLMs), which often struggle with the inference of subtle nonverbal cues in this domain. We introduce NDD-R1, a multi-stage reasoning framework that adapts a general-purpose MLLM for multi-view communication understanding for this population. Our approach employs a four-stage curriculum alternating between supervised finetuning and reinforcement learning ( SFT 1 → GRPO 1 → SFT 2 → GRPO 2 ), integrating label-guided distillation, temperature-escalated self-distillation, and difficulty-tiered reward optimization. We evaluate NDD-R1 on an initial dataset collected through the Rapid Online Sample of Communication (ROSCO) protocol, comprising 2,582 caregiver-annotated multi-view videos of 27 non- and minimally-speaking autistic individuals. NDD-R1 outperforms baselines on action and joint classification tasks, achieving competitive function classification performance relative to Gemini, exhibiting strong multi-view scaling and generalization across unseen sessions. Our code is available at https://github.com/SiddhantBikram/NDD-R1.

Abstract:
Large language models (LLMs) have made rapid progress in code generation, yet their ability to verify whether a program is correct has not necessarily improved at the same pace. We identify a generation-verification gap in code LLMs, where scaling substantially improves test-passing code generation, while execution-free correctness reasoning improves much more slowly. This gap challenges existing self-rewarding approaches, which often implicitly assume that a stronger generator also serves as a better judge. To bridge this gap, we propose CoVera, an execution-grounded co-evolution framework that jointly trains a code LLM to generate code and to verify code correctness. In generator mode, the model produces candidate programs and receives execution-based rewards from available tests. In verifier mode, the same model reads a problem-program pair, produces a reasoning trace, and predicts a correctness judgment, which is rewarded according to agreement with execution-derived labels. The calibrated verifier is then used as an auxiliary self-verification signal for generation, while the evolving generator continuously supplies increasingly challenging candidates for verifier training. This co-evolution enables an on-policy curriculum for correctness reasoning and grounds self-verification in execution feedback, without requiring exhaustive test suites or golden reference solutions. Extensive experiments across multiple benchmarks and model scales show that CoVera improves both code generation and intrinsic correctness verification, outperforming representative baselines from diverse code-learning paradigms.

Abstract:
Large language models make consistent moral judgments, but whether these tendencies correspond to the moral makeup of public web text in the same language is not well understood. We test this correspondence for deception dilemmas across Chinese and English in a four-layer empirical study. We use DeepSeek-R1 for ICL probing, Llama-3.1-8B and Qwen2.5-7B for fine-tuning, and GPT-4o for cross-model replication. In both languages, roughly 60% of web text favors utilitarian reasoning (59.8% vs. 62.3%), inconsistent with a purely sociolinguistic explanation. In-context learning shifts model decisions sharply in low-resistance scenarios (Hostage Crisis, V = 0.411, p = 0.001). In a high-resistance scenario (Organ Allocation), one DeepSeek-R1 deployment showed near-zero ICL effects (V = 0.067) at small sample size, but a larger replication on another deployment showed a large effect (V = 0.55 to 0.69), indicating deployment-specific rather than stable resistance. Three findings stand out for multilingual alignment. First, fine-tuning on deontological data reverses direction in English (Llama-EN Deon P(A) rises from ∼ 0% to 88%), showing that SFT value transfer is not language-neutral. Second, corpus proportions do not carry over to forced choices. The English Llama-3.1-8B base model gives P(A) near zero even though 62.3% of the web text is utilitarian. Third, apparent moral damping in high-resistance scenarios can reflect sample size and deployment variability rather than a stable property of the model or scenario. These results show that corpus moral distributions correspond to model behavior in many conditions, but language, post-training, model architecture, and deployment variability each introduce points where this correspondence breaks.

Abstract:
Vision token pruning can accelerate vision language model inference by removing redundant visual tokens. However, we observe that ViT attention based pruning drops sharply on Qwen3-VL and Qwen3.5. Empirically, we identify the root cause: the last-block MLP of the vision encoder significantly alters the spectral structure of hidden states and induces spectral collapse. Thus, pre-MLP attention rankings no longer reflect downstream token importance. Through further analysis of vision token selection when ViT attention based pruning still succeeds, we find that selected tokens strongly anti-correlate with raw L2 norm in the vision encoder output space S out , while favoring tokens with larger residual energy. Motivated by shared components mask token-specific importance, we propose GeoPrune, a new importance-based pruning method that operates in S out , applies deflation to remove dominant components, and ranks tokens by residual L2 norm. At aggressive pruning rates, GeoPrune is more robust and surpasses ViT attention based pruning by +4.4/+6.3 recovery points on Qwen3-VL and Qwen3.5. We further integrate GeoPrune into the production-grade inference engine vLLM, delivering 5.4%-10.5% mean E2E latency reductions with only slight accuracy drops, advancing vision token pruning toward practical deployment.

Abstract:
Open-domain question answering (ODQA) over multimodal document corpora requires linking evidence scattered across text, tables, and figures. Existing systems often store these sources separately or retrieve only coarse pages, which weakens global evidence linking. We present PILAR, a page-grounded unified evidence representation instantiated as an entity-linked assertion graph. PILAR maps sentence-, table-, and figure-derived facts into a common assertion space and uses the graph as a controlled linking layer over robust page retrieval. In a shared-reader evaluation with four agent frameworks, fourteen retrieval backends, and two benchmarks, PILAR achieves the best end-to-end EM/ANLS. Gains are largest on compositional, cross-document, and multimodal questions, with a single-shot improvement of +1.6 EM over flat retrieval, rising to +2.9 on compositional and +5.9 on 3-hop questions. Ablations show that current gains are driven mainly by the text-instantiated slice of the framework, while visual assertions help only after locality-aware filtering. We therefore position PILAR as a unified evidence representation for multimodal ODQA rather than a standalone visual-reasoning module.

Abstract:
Authorship verification asks whether two given texts were written by the same author. Since supervised verification methods often generalize poorly across domains, the unsupervised Impostors Method, inspired by police lineups, reduces verification to in-domain authorship attribution. Using one text as a query, it hides the other among a sample of hard negatives and measures how reliably a style similarity function retrieves it at the top rank. Although widely applied in practice, the Impostors Method still suffers when no in-domain impostor documents are available. Resorting instead to sampling from open-domain collections provides much less control over confounding stylistic variables such as genre or register. Moreover, its statistical foundations remain largely unexplored compared with those of police lineups. In this paper, we recast the Impostors Method as a statistical hypothesis test, enabling a principled assessment of evidential strength without training a decision threshold. We further investigate large language models (LLMs) as a controllable source for generating in-domain impostor documents.

Abstract:
Large language models (LLMs) are increasingly deployed in socially grounded applications, where success requires interpreting context, inferring others’ mental states, and reasoning about unreliable information. Yet existing benchmarks rarely evaluate these demands jointly in complex, evolving settings. We introduce SocialMaze, a benchmark that organizes six tasks across social deduction games, daily-life interactions, and digital community platforms along three descriptive design axes: deep reasoning, dynamic interaction, and information uncertainty. These axes characterize intended sources of task difficulty rather than latent, factor-analytic dimensions of model capability. Automated checks and human validation support data quality. Evaluations of twelve proprietary and open-weight LLMs show substantial variation in the use of evolving interaction histories; stronger chain-of-thought reasoners perform better on tasks requiring deeper inference, while uncertainty consistently degrades performance. Reasoning workflows help weaker short-chain-of-thought backbones but saturate on stronger reasoners. Finally, targeted fine-tuning on curated reasoning traces substantially improves structured social-reasoning tasks, whereas transfer to language-aggregation tasks remains statistically inconclusive. The project homepage is available at https://xzx34.github.io/socialmaze/.

Abstract:
LLMs have limitations when it comes to cultural coverage and competence, and in some cases, show specific cultural biases. Although prior studies have examined the cultural capabilities of LLMs, none have specifically investigated their regional preferences in generic culture-related questions. In this work, we propose a new dataset based on a comprehensive taxonomy of Culture-Related Open Questions (CROQ), with questions available in 24 languages. We evaluate LLMs by prompting them to answer questions from CROQ and provide a sample location. The results show that, contrary to previous cultural bias work, LLMs show a clear tendency towards countries such as Japan in their answers. Moreover, our results show that when prompting in languages such as English or other high-resource ones, LLMs tend to provide more diverse outputs. Low-resource languages, on the other hand, show more inclinations towards answering questions highlighting countries for which the input language is an official language. Finally, we also investigate at which point of LLM training this cultural bias emerges, with our results suggesting that the first clear signs appear after supervised fine-tuning, and not during pre-training. Dataset available at https://huggingface.co/datasets/HiTZ/CROQ.

Abstract:
Large language models (LLMs) are widely used as factual knowledge sources, yet reliable deployment requires not only accurate predictions but also well-calibrated confidence estimates. While prior work on temporal robustness has primarily focused on changes in factual accuracy over time, we study temporal confidence calibration: whether model confidence remains aligned with correctness across questions associated with different reference times. We evaluate multiple LLMs on temporally grounded question answering benchmarks spanning both historical and dynamically evolving facts, comparing five confidence estimation strategies using Expected Calibration Error (ECE) and Brier Score computed across temporal slices. Our experiments reveal substantial temporal miscalibration: as factual accuracy varies across time, model confidence often fails to track these changes proportionally, leading to pronounced overconfidence on time-sensitive queries. Across the evaluated confidence estimators, no single method consistently dominates: P(True) and Token Probability Confidence perform best most often, while the optimal estimator varies across models and datasets. We further show that global temperature scaling improves calibration but leaves meaningful period-specific variation, indicating that standard post-hoc calibration does not fully eliminate temporal miscalibration. These findings highlight temporal calibration as an important and previously underexplored dimension of LLM reliability.

Abstract:
The impressive linguistic abilities of large language models (LLMs) have recommended them as models of human sentence processing, with some conjecturing a positive ’quality-power’ relationship, in which language models’ (LMs’) fit to psychometric data continues to improve as their ability to predict words in context increases. This is important because it might suggest that elements of LLM architecture reflect the architecture of the human sentence processing faculty, and that any inadequacies in predicting human reading time and brain imaging data may be attributed to insufficient model complexity, which recedes as larger models become available. But recent studies have shown this scaling inverts after a point, as LMs become excessively large and accurate, when information-theoretic surprisal is used as a predictor. Other studies propose the use of entire vectors from differently sized LLMs, still showing positive scaling, casting doubt on the value of surprisal as a predictor, but do not control for dimensionality expansion using untrained LLMs with more than 1.6B parameters. This study evaluates scaling of LLM vector predictors controlled using untrained LLMs with up to 66B parameters. Results show that inverse scaling obtains, and moreover the contribution of trained LMs over corresponding untrained LMs drops to zero at around a few billion parameters on most datasets.

Abstract:
Semantic textual similarity (STS) is central to retrieval, clustering, and semantic understanding, yet Arabic remains underserved by high-quality sentence-embedding models and evaluation resources. We introduce GATE , a compact family of Arabic sentence embedding models and a controlled study of Arabic-specific backbones, contrastive Matryoshka Representation Learning, dimensional truncation, and hybrid STS-NLI supervision. We evaluate the resulting models across Arabic semantic similarity, NLI, pair classification, sentence classification, clustering, retrieval, reranking, and dialectal semantic relatedness, against strong multilingual baselines including BGE-M3, Arctic-Embed-v2, and Qwen3-Embedding. Despite their 135M-parameter scale, GATE-family models are competitive with substantially larger multilingual encoders and achieve strong results on several native-Arabic semantic tasks. A three-annotator audit further finds high Arabic surface adequacy and 84.2% majority relation preservation for the translated supervision. The released checkpoints, data resources, and experimental artifacts support reproducible Arabic embedding research.

Abstract:
Large language models (LLMs) have proven to be effective tools for a wide range of natural language processing (NLP) applications. Although many LLMs are multilingual, most remain English-centric and perform poorly on low-resource languages. Recently, several Southeast Asia–focused LLMs have been developed, but none are truly open source, as they do not publicly disclose their training data. Truly open-source models are important for transparency and for enabling a deeper and more precise understanding of LLM internals and development, including biases, generalization, and multilinguality. Motivated by recent advances demonstrating the effectiveness of parallel data in improving multilingual performance, we conduct controlled and comprehensive experiments to study the effectiveness of parallel data in continual pretraining of LLMs. Our findings show that using only parallel data is the most effective way to extend an LLM to new languages. Using just 34.7B tokens of parallel data and 180 hours on 8× NVIDIA H200 GPUs, we built OpenSeal, the first truly open Southeast Asian LLM that rivals the performance of existing models of similar size.

Abstract:
We present a comprehensive study of LLMs for argument mining, comparing four learning paradigms against a strong encoder-only baseline across three heterogeneous corpora to assess cross-domain generalisation. We evaluate in-context learning, supervised fine-tuning (SFT), and cold-start reinforcement fine-tuning (RFT) via GRPO. Our results demonstrate that BERT-style encoder-only transformers consistently outperform much larger decoder-only LLMs across all settings. Further mechanistic analysis reveals that the central bottleneck is not knowledge but expression: linear probing of frozen hidden states recovers argument relations at 69–80% accuracy from base models, yet edge generation remains poor across all paradigms. Attention analysis confirms a universal structural blindness across 27 tested conditions.

Abstract:
Biomedical Named Entity Recognition (NER) is fundamental to healthcare AI applications, including clinical decision support and medical information extraction. While corpora with Unified Medical Language System (UMLS) annotations, such as MedMentions, have driven progress in English biomedical NER, no comparable resource exists for Vietnamese. This paper presents En-ViMedNER, the first English-Vietnamese parallel biomedical NER corpus annotated with UMLS semantic types, which are language-neutral codes providing a shared cross-lingual label space and ensuring direct comparability with existing UMLS-based resources. The corpus contains 4,392 PubMed abstract pairs, 44,892 English-Vietnamese sentence pairs, and 202,949 aligned entity-mention pairs across 21 semantic types adapted from the MedMentions ST21pv dataset. To balance quality and scalability, we have constructed the corpus through automatic translation, expert post-editing, LLM-assisted label projection, and human verification and adjudication. We characterize En-ViMedNER as a large-scale silver-standard corpus with a human-audited and consensus-corrected mini-test subset. We evaluate En-ViMedNER in two settings: (i) Vietnamese-input/Vietnamese-output biomedical NER and (ii) English-input/Vietnamese-output cross-lingual NER. For Vietnamese NER, we benchmark Vietnamese-supervised encoder models, English-supervised multilingual encoder models, and prompt-based LLMs. The best model achieves an F1 score of 52.70 on the test set and 53.78 on the mini-test set. For cross-lingual NER, we benchmark encoder-decoder models and prompt-based LLMs. The best model achieves an F1 score of 45.44 on the mini-test set. We publicly release our corpus, corpus construction pipeline, and baseline models to facilitate future Vietnamese biomedical NLP research. Given the semi-automated approach and the positive use cases of the corpus, we argue similar efforts should be made for some other low-resource languages.

Abstract:
Pointer-decoded Seq2Seq transformers remain a strong recipe for aspect-based sentiment analysis (ABSA), but native German Seq2Seq backbones remain scarce, and current pretraining trends favor decoder-only architectures. Existing assembly recipes disagree on which pairings work. We therefore study models assembled from independently pretrained encoder and decoder LMs, primarily on German ABSA, where fixed pointer outputs expose both tuple accuracy and grammar violations. All nine backbone combinations train under both separate and parameter-sharing merged cross-attention, with separate attention slightly stronger overall. Depth and scale sweeps show that additional capacity pays off more on the encoder side for these pointer-decoded tasks, and our strongest two-stack assembly trails mBART but outperforms mT5-small on both German datasets. Motivated by decoder-only pretraining, we replace the encoder and decoder with one backbone that encodes the source and autoregressively produces pointer queries. Bidirectional source visibility and scale matter more consistently than pointer-query refinement: a 120M LLäMmlein model exceeds both same-family two-stack means with roughly half the parameters, and a 1B ModernGBERT reaches mBART-level performance.

Abstract:
Large language models (LLMs) have shown strong promise as clinical reasoning agents, yet most benchmarks assume complete evidence availability before reasoning begins. However, real clinical practice unfolds through dynamic clinical workflows with progressive disclosure of patient evidence, requiring clinicians to decide what evidence to acquire next from partial and evolving patient states. This mismatch exposes a bottleneck in proactive evidence acquisition for LLM-based clinical agents. To address this gap, we introduce ProActIS, a lightweight state-conditioned examination proposer for proactive evidence acquisition. Given an evolving patient state, ProActIS ranks standardized laboratory, microbiology, and radiology examinations, together with STOP, through structured patient-state decomposition and action-card encoding. We construct an MIMIC-IV benchmark with 81,960 hospital admissions and 860,099 turn-level patient-state/examination pairs, using explicit diagnosis/result filtering and retrospective leakage sensitivity analysis over 129 standardized examination actions plus STOP (130 actions). On full-bank next-examination ranking, ProActIS achieves 0.6028 Recall@1, 0.8868 Recall@5, and 0.7258 MRR, substantially outperforming frequency and LLM baselines. In LLM-assisted dynamic workflows, ProActIS substantially improves benchmark-level evidence coverage and documented-trajectory alignment for downstream agents, with downstream diagnostic accuracy approaching the corresponding all-information controls.

Abstract:
Multi-agent large language model (LLM) systems can improve reasoning by spending more computation, but deployment requires deciding when extra collaboration is worth its cost. We isolate this decision by running every problem under four protocols while holding the solver fixed within each setting: direct solving (Baseline), iterative self-correction (Single), planner–executor–reviewer collaboration (PER), and multi-agent deliberation (Broadcast). The primary benchmark comprises 4,181 competition-level math problems; paired robustness checks cover four benchmarks spanning competition math, biology, and broader science with two solver families. Across fixed policies, trained routers, and frozen LLM routers, conservative policies under-escalate, whereas higher-solve frozen routers often over-escalate. A post-answer, pre-collaboration gpt-oss-120b probe ranks Baseline failures with 0.8847 AUROC (4,151 parseable cases; 95% CI [0.8732, 0.8955]). The same score remains informative for predicting whether any collaboration helps (0.7683 AUPRC), but is much weaker for identifying PER- or Broadcast-specific value (0.1674 and 0.1041 AUPRC). Separately, the pre-answer self-confidence gate reaches 78.0% solve at 45K tokens, compared with 73.8% at 71.3K for a frozen gpt-oss-120b router and 92.4% for a retrospective fixed-order oracle. Across 10 paired model–condition settings, the oracle adds 23.2–58.3 points of retrospective coverage over Baseline, but protocol profiles vary by task. In the six settings with held-out router evaluations, oracle gaps remain 18.5–28.9 points. Confidence can therefore support initial escalation, while protocol-specific cost-aware routing remains unresolved.

Abstract:
Recent advances in persistent personal-agent frameworks are making human-centered agent networks realistic deployment targets: each user can be served by an AI agent that acts on the user’s behalf, maintains state, and communicates with other agents through social and task relations. In these networks, everyday tool use becomes multi-party owned-agent collaboration over personal workspaces, where files, records, tools, and policies are not directly visible across owners. Existing agent benchmarks study tool use and collaboration, but they do not provide an end-to-end sandbox for verifiable cross-user agent collaboration with realistic user digital workspaces or test how harmful actions can travel through the human-centered agent network. We introduce WeClawArena, an auditable benchmark and runtime sandbox for multi-party owned-agent collaboration over personal workspaces. WeClawArena targets collaborative tool-use tasks in which personal workspaces serve as both operational tools and personal constraints. The benchmark contains 124 base tasks across six cross-user task domains and expands them into 620 scenario variants, with one benign control and four attack-vector variants per base task. The sandbox records peer messages, tool calls, resource operations, governed decisions, and final workspace states. WeClawArena reports utility and attack success rate (ASR) separately and audits attack success from bounded runtime evidence, supporting diagnosis of task breakdown, privacy leakage, poisoned evidence, and invalid authority paths.

Abstract:
Low-Rank Adaptation (LoRA) is now the dominant parameter-efficient fine-tuning method for large language models, but the resulting open marketplace of adapters has become a supply-chain attack surface: backdoored adapters merge silently with arbitrary task adapters while preserving benign utility, and existing centralized integrity tools require marketplaces to share raw weights, which is incompatible with proprietary fine-tuning. We present FedLoRAGuard, a federated dynamic-graph framework that verifies adapter integrity without weight sharing. The LoRA ecosystem is modeled as a heterogeneous continuous-time dynamic graph with four node types and six edge types; a federated dynamic GNN is trained per-client with client-update-level (ε, δ)-DP Gaussian noise, secure aggregation, and FLTrust scoring. We prove a sensitivity bound for federated heterogeneous-GNN client updates and a certified poisoning-radius theorem that converts the per-round inference-time smoothing budget ε_r into an analytical bound on the size of a bounded-update colluding coalition k. On 13,500 benign and backdoored adapters across four base-model families and ten attack types, FedLoRAGuard obtains 96.4% macro-F1 and a certified coalition bound k = 25 from N = 50 marketplaces at the inference-time smoothing budget ε_r = 0.5, and empirically tolerates label-flipping Byzantine coalitions up to ρ ≤ 0.33 without certificate degradation. The DP-SGD training-time mechanism, calibrated at σ = 1.1 and Poisson subsampling q = 0.2 over T = 100 rounds, achieves a cumulative privacy budget ε_T ≈ 13.6 at δ = 10⁻⁵ under the Opacus PRV accountant, with sub-second per-adapter verification latency on A100-class hardware. The full DP + secure-aggregation stack adds a measured median per-round wall-clock overhead of ≈ 4% (pooled median across three runs of 20–60 measurement rounds on identical CPU hardware) over non-private FedAvg; the released runtime ships the instrumentation so operators can re-measure in place on their target hardware.

Abstract:
Matryoshka Representation Learning (MRL) is a widely adopted approach for training text encoders so they provide useful text representations at various sizes, available by simply truncating the resulting vectors at sizes pre-determined at training time. Recent works have shown that randomly truncating text embeddings has minimal impact in downstream performance unless vectors are reduced in size by at least 70%, suggesting that embeddings are already robust to truncation without the use of MRL. However, no prior work has compared random truncation to MRL, so it is unclear how the two methods compare as effective embedding reduction methods. In this paper, we study this by applying the same truncation used by MRL to models trained with and without MRL. Our results across several models and downstream tasks show that, unless heavily truncating embeddings (i.e. reducing their size by at least 80%), truncated embeddings of non-MRL models are competitive with, and often outperform models trained with MRL. This suggests that truncation robustness may not necessarily come from MRL, and that the choice of spending the additional training cost of MRL depends on whether heavy truncation is desired. We make our code available for reproduction.

Abstract:
This paper presents new Cantonese ParGram resources and evaluates LLMs for knowledge-driven grammar engineering within a controlled experimental paradigm. Using Cantonese ParGram resources as gold standards, with corresponding English baselines, we investigate whether OpenAI’s gpt-oss-120b and GPT-5.4 can generate machine-processable grammars from sentences and target formal structures under systematically varied prompting conditions. GPT-5.4 outperformed gpt-oss-120b, while grammars generated from target formal structures generally outperformed those generated from sentences. Although both models could generate locally plausible phrase-structure rules, lexical entries, and templates, they often struggled to coordinate interacting formal constraints, especially in multi-construction settings. The results characterize both the capabilities and limitations of current LLMs for potential integration into AI-assisted expert workflows: LLMs may support intermediate stages of grammar development, but human linguistic expertise remains central to analysis, validation, and refinement. The study also contributes new Cantonese symbolic grammatical resources.

Abstract:
Bangla culture is richly expressed through region, dialect, history, food, politics, media, and everyday visual life, yet it remains underrepresented in multimodal evaluation. To address this gap, we introduce BanglaVerse, a culturally grounded benchmark for evaluating multilingual vision–language models (VLMs) on Bengali culture across historically linked languages and regional dialects. Built from 1,152 manually curated images across nine domains, the benchmark supports visual question answering and captioning, and is expanded into four languages and five Bangla dialects, yielding ∼32.2K artifacts. Our experiments show that evaluating only standard Bangla overestimates true model capability: performance drops under dialectal variation, especially for caption generation, while historically linked languages such as Hindi and Urdu retain some cultural meaning but remain weaker for structured reasoning. Across domains, the main bottleneck is missing cultural knowledge rather than visual grounding alone, with knowledge-intensive categories. These findings position BanglaVerse as a more realistic test bed for measuring culturally grounded multimodal understanding under linguistic variation.

Abstract:
Annotating data remains a costly bottleneck for supervised NLP. Active learning (AL) reduces the number of human labels needed by selecting only the most informative instances, while instruction-tuned LLMs attack the same bottleneck from the other side, making labels cheap enough to annotate entire corpora. This raises two questions: can LLM labels replace human labels within the AL loop, and does AL remain necessary when entire corpora can be cheaply labeled? We investigate both by training supervised hate speech classifiers on a new dataset of 278K German political TikTok comments, comparing human and LLM annotation under matched conditions. LLM annotation at scale outperforms human-supervised classifiers at roughly one-tenth the cost, for both a closed-source (GPT-5.2) and an open-weight (Qwen3.5-122B-A10B) LLM, and the advantage is robust under soft-label evaluation. It hinges on the annotation interface: only a two-question decomposition mirroring the human annotation task unlocks it. AL provides no reliable advantage over random sampling in our prefiltered pool. Error structure depends on the LLM: only GPT-5.2 matches the human FP/FN balance, while other variants over-flag border-control and economic-competition discourse. Humans remain essential as evaluators; for training labels, the question shifts to which LLM, which interface, and what shape of pool.

Abstract:
Speech language models (SpeechLMs) have achieved substantial progress by extending large language models (LLMs) to the speech modality. However, SpeechLM evaluation remains heavily centered on English, limiting reliable assessment of multilingual speech capabilities. Straightforward benchmark transfer through ASR, translation, normalization, and TTS can corrupt language-specific instructions, answer constraints, and spoken forms; for audio understanding, transferring source-language audio also fails to preserve target-language speaker attributes, accents, and paralinguistic properties. To address these limitations, we propose two human-agent benchmark-construction frameworks: one transfers source-language SpokenQA benchmarks into target-language SpokenQA benchmarks, and the other converts target-language ASR corpora into audio understanding benchmarks using transcriptions and speaker metadata. Using these frameworks, we construct and publicly release three Korean speech benchmarks: KVoiceBench and KOpenAudioBench for Korean SpokenQA, and KMMAU for Korean audio understanding, comprising 12,345 samples in total. We evaluate eight recent SpeechLMs and find that English–Korean performance gaps vary substantially across models and task families, and that SpokenQA and audio understanding rankings diverge, revealing complementary weaknesses invisible to English-only evaluation.