arXiv Papers of Scene Text Understanding

Abstract:
Scene Text Recognition (STR) models are trained almost exclusively on word crops of at most 25 characters, yet real deployments (signage, product labels, dense captions) require reading much longer text. This paper diagnoses that failure and then closes it. The diagnosis separates out‑of‑length failure into two simultaneously extrapolating axes (the encoder's width axis and the decoder's time axis) and shows that encoder width, not decoder length, is the dominant failure mode. Representation‑side fixes bring only partial relief: training‑free rotary rescalings recover at most 2‑4 points of character error rate (CER), and a weighted fine‑tuning recipe recovers 6‑8 points while improving standard‑benchmark accuracy, yet word accuracy on the Long Text Benchmark (LTB) stays near zero, because the residual gap lies in the decoding mechanism rather than the representation. We then close that gap at inference time, on an unmodified word‑level checkpoint: the long image is sliced into overlapping crops at the model's training width, each decoded independently and in‑distribution, and the reads stitched by geometry‑anchored edit‑distance alignment. This procedure reaches 42.79‑43.05% bucket‑average word accuracy on LTB across two base checkpoints, matching the published state of the art (41.57%) and beating it by 11‑12 points on the hardest bucket, at wall‑clock parity with plain decoding; applied unchanged to the public PARSeq checkpoint it reaches 47.11%. Once chunking is applied fine‑tuning no longer helps: the decoding‑side fix alone matches purpose‑built architectures. We release the diagnosis harness and implementation.

Abstract:
Multi‑agent interactive world models should not only generate consistent observations, but also maintain world states that persist across agents and evolve across views. Existing autoregressive video diffusion pipelines carry forward observation history as conditioning context, which makes shared state difficult to maintain in multi‑agent and multi‑view settings. We present WorldWeaver (W^2), a streaming multi‑agent video diffusion model that augments rollout with cross‑agent world state registers: learnable tokens that store shared world information, track individual agent status, and are dynamically updated after each generated chunk. We ground these registers with supervision signals spanning individual agent status, global state views including bird's‑eye views, and scene text. We further improve the architecture with a Mixture‑of‑Transformers design that uses separate weights for world state modeling and visual frame modeling. Extensive experiments in two‑agent Minecraft video generation show that explicit world‑state modeling improves logical consistency and generation quality.

Abstract:
In real‑world deployments, scene text detectors inevitably face distribution shifts beyond the training distribution. Prior work often depends on large‑scale scene‑text pretraining, yet evaluation under cross‑domain changes and real‑world imaging degradations remains limited. We propose TextDS, an efficient framework for scene text detection under distribution shifts. First, we propose a data‑efficient dual‑encoder design with visual foundation models, eliminating the reliance on large‑scale scene‑text pretraining. Second, we introduce Step‑wise LoRA adaptation (SWLoRA), which performs progressive low‑rank refinement with a dynamic early‑exit mechanism for effective feature adaptation. Third, we propose Common Subspace Fusion (CSF) to align and fuse the two branches in a shared subspace while retaining complementary, shift‑robust information. Finally, we construct adverse‑condition scene text detection datasets to address the gap in evaluating under imaging degradation. Experiments show that TextDS achieves competitive performance in scene text detection, demonstrating robustness across domains and adverse imaging conditions with only 4.9M trainable parameters.

Abstract:
WordArt (artistic text) features highly customized fonts, textures, and layouts, making WordArt‑oriented scene TExt Recognition (WATER) substantially more challenging than general Scene Text Recognition (STR). Existing STR datasets and methods, typically built around regular scene text and fixed‑template inputs, struggle to scale to WATER. Thus, we aim to advance this task from both data and model perspectives. On the data side, we construct a 2M synthetic dataset, WATER‑S, with the scale improved by hundreds of times compared to existing artistic text data. WATER‑S consists of two complementary subsets. One rendered by an upgraded rendering pipeline (SynthWordArt), which provides highly accurate and controllable synthetic WordArt data. The other is generated by combining Qwen3‑VL for prompt mining and Z‑Image for image synthesis, which improves the coverage of realistic and diverse data. On the model side, we propose WATERec. It adopts an visual encoder supporting arbitrary‑shaped inputs and an autoregressive decoder to model complex layouts, structurally breaking the bottleneck of fixed‑template STR on WordArt. Experiments show that this architecture outperforms prior STR methods, achieving state‑of‑the‑art performance on irregular texts such as WordArt. Together with WATER‑R, carefully reorganized from existing real STR data, our strong baseline with the new synthetic data and model design reaches 90.40% accuracy on WordArt‑Bench, surpassing both general‑purpose and OCR‑specialized vision‑language models by a large margin. Code and data are available at https://github.com/YesianRohn/WATER.

Abstract:
In‑Image Machine Translation (IIMT) aims to translate scene text in an image and render the translated text back into the original regions while preserving the overall visual appearance. Recent unified multimodal models provide a promising solution by combining visual‑text understanding and image generation within a single framework. However, directly adapting such models to IIMT remains challenging. In particular, they often suffer from understanding‑generation conflicts, where the translation inferred during understanding is inconsistent with the text supervision used in generation, and spatial position misalignment, where the rendered text does not accurately match the target text regions. To address these issues, we present UniTranslator, a unified multimodal framework for IIMT that tightly couples translation understanding and text editing. Specifically, we introduce an Understand‑Generation Alignment Module (UGAM) to bridge the representation gap between understanding and generation, encouraging semantic consistency between translated content prediction and text rendering. We further propose a Spatial Mask Decoder (SMD) with pixel‑level supervision over text regions to improve spatial grounding, geometric alignment, and layout controllability during generation. Extensive experiments on multiple benchmarks demonstrate that UniTranslator achieves state‑of‑the‑art performance across diverse language directions and complex real‑world layouts. Moreover, our results reveal a strong mutual reinforcement effect between translation understanding and image generation, highlighting the advantage of unified translation multimodal learning. Code is available at https://github.com/SeerRay‑Lab/Unitranslator.

Abstract:
Video Text Spotting (VTS) is essential for urban surveillance and intelligent transportation systems, enabling automated reading of street signs, vehicle markings, and scene text in video streams. However, reliable recognition remains challenging due to dynamic video factors common in surveillance scenarios, including motion blur, occlusion, and scale variation, which degrade frame‑level recognition. Existing VTS methods typically perform recognition independently on each frame, leading to inconsistent and inaccurate results across sequences. To address these limitations, we propose TraRA (Trajectory‑level Recognition Aggregation for VTS), a plug‑and‑play method that performs trajectory‑level text recognition by leveraging temporal and multimodal consistency. TraRA integrates two key modules: (1) the Temporal Clustering and (2) the Vision‑Language Aggregation. The former refines noisy trajectories by grouping temporally and visually coherent text instances, while the latter employs a Low‑Rank Adaptation‑enhanced Vision‑Language model to fuse visual cues with linguistic context across frames. By aggregating information over entire text trajectories, TraRA achieves robust text recognition even under challenging surveillance conditions. Extensive experiments on four public benchmarks, including road and urban scene datasets (RoadText, BOVText, ArTVideo, and ICDAR15), demonstrate that TraRA consistently improves tracking and recognition performance over state‑of‑the‑art VTS methods. The source code is available at https://github.com/trid2912/TraRA.

Abstract:
Recent advances in Multimodal Large Language Models (MLLMs) and diffusion‑based generative models have substantially improved prompt‑driven image editing. However, scene text editing remains challenging, as it requires models to precisely modify textual content while preserving visual realism and non‑target regions. Current open‑source models still lag behind proprietary systems, largely due to the scarcity of high‑quality training data and the lack of standardized benchmarks tailored to text editing. To address these challenges, we present TextSculptor, a comprehensive framework for data construction and evaluation of scene text editing. We first develop an automated data construction pipeline that combines text‑aware image synthesis with programmatic text rendering and compositing. Based on this pipeline, we build TextSculpt‑Data, a large‑scale dataset containing 3.2M training samples, including 1.2M OCR‑verified text‑to‑image samples and 2M paired text editing samples with naturally aligned source‑target images and strong background consistency. We further introduce TextSculpt‑Bench, a benchmark covering four fundamental text editing tasks: text addition, text replacement, text removal, and hybrid editing. To support reliable evaluation, we design a tailored protocol that measures text accuracy, visual quality, and background preservation through OCR‑based text alignment, multimodal judgment, and background‑region similarity. Extensive experiments show that TextSculptor improves open‑source text editing performance and narrows the gap to proprietary models. The data and benchmark are available at https://github.com/linyiheng123/TextSculptor.

Abstract:
Modern visual object trackers show impressive results on general targets, yet their performance drops substantially when dealing with scene text. Although currently underexplored, tracking text in videos is essential for dynamic text manipulations such as segmentation, removal, and editing. To fill this gap, this paper formalizes this specific task as Scene Text Tracking and presents the first systematic work for it. We identify three primary challenges in this task: 1) severe geometric distortions from perspective shifts, 2) high visual ambiguity across different instances, and 3) high sensitivity to fine‑grained structural details. To address these issues, we propose SymTrack, a unified detection‑free framework with synergistic dual‑branch design. It integrates a Cross‑Expert Calibration mechanism to reduce semantic bias, along with a Predictive Token Rectification mechanism to correct structural imbalances, complemented by an Adaptive Inference Engine that stabilizes predictions under motion constraints. Considering the lack of dedicated benchmarks for this task, we utilize three datasets from video text spotting to construct a benchmark with high‑quality annotations. Extensive experiments demonstrate that SymTrack sets the new state‑of‑the‑art on all three benchmarks, outperforming previous best trackers by up to 11.97% AUC on \textBOVText_\textSOT . Overall, our work promotes efficient and thorough text tracking, paving the way toward more generalized video text manipulation.

Abstract:
Scene Text Recognition requires modeling visual structures that evolve from coarse layouts to fine‑grained character strokes. Training such models relies on large amounts of annotated data. Recent self‑supervised approaches, such as Masked Image Modeling (MIM), alleviate this dependency by leveraging large‑scale unlabeled data. Yet most existing MIM methods operate at a single spatial scale and fail to capture the hierarchical nature of scene text. In this work, we introduce Masked Next‑Scale Prediction (MNSP), a unified self‑supervised framework designed to explicitly model cross‑scale structural evolution. The framework incorporates Next‑Scale Prediction (NSP), which learns hierarchical representations by predicting higher‑resolution features from lower‑resolution contexts. Naive scale prediction, however, tends to produce spatially diffuse attention, directing the model toward background regions rather than textual structures. MNSP resolves this limitation by jointly learning cross‑scale prediction and masked image reconstruction. NSP captures global layout priors across resolutions, while masked reconstruction imposes strong local constraints that guide attention toward informative text regions. A Multi‑scale Linguistic Alignment module further maintains semantic consistency across different resolutions. Extensive experiments demonstrate that MNSP achieves state‑of‑the‑art performance, reaching 86.2% average accuracy on the challenging Union14M benchmark and 96.7% across six standard datasets. Additional analyses show that our method improves robustness under extreme scale and layout variations. Code is available at https://github.com/CzhczhcHczh/MNSP

Abstract:
Scene text editing seeks to modify textual content in natural images while maintaining visual realism and semantic consistency. Existing methods often require task‑specific training or paired data, limiting their scalability and adaptability. In this paper, we propose TextFlow, a training‑free scene text editing framework that integrates the strengths of Attention Boost (AttnBoost) and Flow Manifold Steering (FMS) to enable flexible, high‑fidelity text manipulation without additional training. Specifically, FMS preserves the structural and style consistency by modeling the visual flow of characters and background regions, while AttnBoost enhances the rendering of textual content through attention‑based guidance. By jointly leveraging these complementary modules, our approach performs end‑to‑end text editing through semantic alignment and spatial refinement in a plug‑and‑play manner. Extensive experiments demonstrate that our framework achieves visual quality and text accuracy comparable to or superior to those of training‑based counterparts, generalizing well across diverse scenes and languages. This study advances scene text editing toward a more efficient, generalizable, and training‑free paradigm. Code is available at https://github.com/lyb18758/TextFlow

Abstract:
Vision‑language models (VLMs) enable flexible generative optical character recognition (OCR), while their open‑ended decoders can expose wrong but fluent text with weak visual support. In audit‑sensitive records, such an output can be more costly than abstention. Frozen or externally served VLMs therefore require an external decision layer that can determine whether a transcription has sufficient visual evidence for release. We introduce the Geometric Risk Controller (GRC), a model‑agnostic controller that treats controlled geometric transformations as repeatable black‑box probes, screens structurally implausible continuations, and releases the unique candidate supported by coherent cross‑view evidence. The protocol provides empirical selective exposure control with explicit coverage and query cost under a reproducible fixed decision rule. Experiments across frozen VLMs and standard scene‑text benchmarks consistently reduce mean, upper‑tail, and catastrophic error among released outputs while retaining high coverage.

Abstract:
Scene text recognition (STR) methods have demonstrated their excellent capability in English text images. However, due to the complex inner structures of Chinese and the extensive character categories, it poses challenges for recognizing Chinese text in images. Recently, studies have shown that the methods designed for English text recognition encounter an accuracy bottleneck when recognizing Chinese text images. This raises the question: Is it appropriate to apply the model developed for English to the Chinese STR task? To explore this issue, we propose a novel method named LER, which explicitly decouples each character and independently recognizes characters while taking into account the complex inner structures of Chinese. LER consists of three modules: Localization, Extraction, and Recognition. Firstly, the localization module utilizes multimodal information to determine the character's position precisely. Then, the extraction module dissociates all characters in parallel. Finally, the recognition module considers the unique inner structures of Chinese to provide the text prediction results. Extensive experiments conducted on large‑scale Chinese benchmarks indicate that our method significantly outperforms existing methods. Furthermore, extensive experiments conducted on six English benchmarks and the Union14M benchmark show impressive results in English text recognition by LER. Code is available at https://github.com/Pandarenlql/LER.

Abstract:
Scene text recognition (STR) and handwritten text recognition (HTR) face significant challenges in accurately transcribing textual content from images into machine‑readable formats. Conventional OCR models often predict transcriptions directly, which limits detailed reasoning about text structure. We propose a VQA‑inspired data augmentation framework that strengthens OCR training through structured question‑answering tasks. For each image‑text pair, we generate natural‑language questions probing character‑level attributes such as presence, position, and frequency, with answers derived from ground‑truth text. These auxiliary tasks encourage finer‑grained reasoning, and the OCR model aligns visual features with textual queries to jointly reason over images and questions. Experiments on WordArt and Esposalles datasets show consistent improvements over baseline models, with significant reductions in both CER and WER. Our code is publicly available at https://github.com/xuyaooo/DataAugOCR.

Abstract:
Large‑scale and categorical‑balanced text data is essential for training effective Scene Text Recognition (STR) models, which is hard to achieve when collecting real data. Synthetic data offers a cost‑effective and perfectly labeled alternative. However, its performance often lags behind, revealing a significant domain gap between real and current synthetic data. In this work, we systematically analyze mainstream rendering‑based synthetic datasets and identify their key limitations: insufficient diversity in corpus, font, and layout, which restricts their realism in complex scenarios. To address these issues, we introduce UnionST, a strong data engine synthesizes text covering a union of challenging samples and better aligns with the complexity observed in the wild. We then construct UnionST‑S, a large‑scale synthetic dataset with improved simulations in challenging scenarios. Furthermore, we develop a self‑evolution learning (SEL) framework for effective real data annotation. Experiments show that models trained on UnionST‑S achieve significant improvements over existing synthetic datasets. They even surpass real‑data performance in certain scenarios. Moreover, when using SEL, the trained models achieve competitive performance by only seeing 9% of real data labels. Code is available at https://github.com/YesianRohn/UnionST.

Abstract:
Large vision‑language models (LVLMs) are vulnerable to typographic attacks, where misleading text inserted into an image can override visual understanding. However, existing evaluation protocols and defenses are largely focused on object recognition and do not consider text‑reading capability. This is a critical oversight: real‑world scenarios often require both recognizing objects and reading scene text (e.g., recognizing pedestrians while reading traffic signs), where simply ignoring all text for robustness is unacceptable in practice. To address this gap, we introduce a novel task, Read‑or‑Ignore VQA (RIO‑VQA), which jointly evaluates both requirements: models must decide, from context, when to read scene text and when to ignore inserted distractor text. To evaluate this capability, we present RIO‑Bench, a same‑scene counterfactual benchmark that holds the scene fixed while varying only question intent (object vs. text) and text condition (clean vs. attack), enabling direct comparisons of model behaviors with reduced confounding factors. Using RIO‑Bench, we highlight a trade‑off: representative defenses developed in object‑centric settings can achieve robustness by suppressing text sensitivity, at the cost of text‑reading performance (i.e., "ignoring" text). Motivated by this trade‑off, we provide a data‑driven defense baseline that improves both requirements on RIO‑Bench, complementing prior text‑ignoring baselines. Overall, this work highlights a fundamental misalignment between the current object‑centric robustness scope and real‑world multimodal requirements, providing a principled path toward reliable LVLMs.

Abstract:
Mask Diffusion Models (MDMs) have recently emerged as a promising alternative to auto‑regressive models (ARMs) for vision‑language tasks, owing to their flexible balance of efficiency and accuracy. In this paper, for the first time, we introduce MDMs into the Scene Text Recognition (STR) task. We show that vanilla MDM lags behind ARMs in terms of accuracy, although it improves recognition efficiency. To bridge this gap, we propose MDiff4STR, a Mask Diffusion model enhanced with two key improvement strategies tailored for STR. Specifically, we identify two key challenges in applying MDMs to STR: noising gap between training and inference, and overconfident predictions during inference. Both significantly hinder the performance of MDMs. To mitigate the first issue, we develop six noising strategies that better align training with inference behavior. For the second, we propose a token‑replacement noise mechanism that provides a non‑mask noise type, encouraging the model to reconsider and revise overly confident but incorrect predictions. We conduct extensive evaluations of MDiff4STR on both standard and challenging STR benchmarks, covering diverse scenarios including irregular, artistic, occluded, and Chinese text, as well as whether the use of pretraining. Across these settings, MDiff4STR consistently outperforms popular STR models, surpassing state‑of‑the‑art ARMs in accuracy, while maintaining fast inference with only three denoising steps. Code: https://github.com/Topdu/OpenOCR.

Abstract:
Recent advances in diffusion‑based real‑world image super‑resolution (Real‑ISR) have demonstrated remarkable perceptual quality, yet the balance between fidelity and controllability remains a problem: multi‑step diffusion‑based methods suffer from generative diversity and randomness, resulting in low fidelity, while one‑step methods lose control flexibility due to fidelity‑specific finetuning. In this paper, we present ODTSR, a one‑step diffusion transformer based on Qwen‑Image that performs Real‑ISR considering fidelity and controllability simultaneously: a newly introduced visual stream receives low‑quality images (LQ) with adjustable noise (Control Noise), and the original visual stream receives LQs with consistent noise (Prior Noise), forming the Noise‑hybrid Visual Stream (NVS) design. ODTSR further employs Fidelity‑aware Adversarial Training (FAA) to enhance controllability and achieve one‑step inference. Extensive experiments demonstrate that ODTSR not only achieves state‑of‑the‑art (SOTA) performance on generic Real‑ISR, but also enables prompt controllability on challenging scenarios such as real‑world scene text image super‑resolution (STISR) of Chinese characters without training on specific datasets. Codes are available at https://github.com/RedMediaTech/ODTSR.

Abstract:
Scene Text Editing (STE) aims to naturally modify text in images while preserving visual consistency, the decisive factors of which can be divided into three parts, i.e., text style, text content, and background. Previous methods have struggled with incomplete disentanglement of editable attributes, typically addressing only one aspect ‑ such as editing text content ‑ thus limiting controllability and visual consistency. To overcome these limitations, we propose TripleFDS, a novel framework for STE with disentangled modular attributes, and an accompanying dataset called SCB Synthesis. SCB Synthesis provides robust training data for triple feature disentanglement by utilizing the "SCB Group", a novel construct that combines three attributes per image to generate diverse, disentangled training groups. Leveraging this construct as a basic training unit, TripleFDS first disentangles triple features, ensuring semantic accuracy through inter‑group contrastive regularization and reducing redundancy through intra‑sample multi‑feature orthogonality. In the synthesis phase, TripleFDS performs feature remapping to prevent "shortcut" phenomena during reconstruction and mitigate potential feature leakage. Trained on 125,000 SCB Groups, TripleFDS achieves state‑of‑the‑art image fidelity (SSIM of 44.54) and text accuracy (ACC of 93.58%) on the mainstream STE benchmarks. Besides superior performance, the more flexible editing of TripleFDS supports new operations such as style replacement and background transfer. Code: https://github.com/yusenbao01/TripleFDS

Abstract:
In an era where wearable technology is reshaping applications, Scene Text Detection and Recognition (STDR) becomes a straightforward choice through the lens of egocentric vision. Leveraging Meta's Project Aria smart glasses, this paper investigates how environmental variables, such as lighting, distance, and resolution, affect the performance of state‑of‑the‑art STDR algorithms in real‑world scenarios. We introduce a novel, custom‑built dataset captured under controlled conditions and evaluate two OCR pipelines: EAST with CRNN, and EAST with PyTesseract. Our findings reveal that resolution and distance significantly influence recognition accuracy, while lighting plays a less predictable role. Notably, image upscaling emerged as a key pre‑processing technique, reducing Character Error Rate (CER) from 0.65 to 0.48. We further demonstrate the potential of integrating eye‑gaze tracking to optimise processing efficiency by focusing on user attention zones. This work not only benchmarks STDR performance under realistic conditions but also lays the groundwork for adaptive, user‑aware AR systems. Our contributions aim to inspire future research in robust, context‑sensitive text recognition for assistive and research‑oriented applications, such as asset inspection and nutrition analysis. The code is available at https://github.com/josepDe/Project_Aria_STR.

Abstract:
Improving visual text synthesis has long been a challenging and evolving frontier for image generation models. While recent state‑of‑the‑art (SOTA) models have made remarkable strides in text generation capabilities, existing benchmarks inadequately assess their true performance due to narrow scope (scene text and posters only), isolated evaluation (T2I generation or editing separately), and insufficient difficulty (lacking challenging scenarios). To bridge this gap, we pioneer the unification of text‑centric T2I generation, text editing, and OCR‑related image‑to‑image translation to evaluate a model's holistic visual text synthesis abilities, i.e., OCR generative capabilities. Accordingly, we propose OCRGenBench, the most comprehensive benchmark to date for evaluating these abilities. OCRGenBench covers five common text categories and 33 OCR generative tasks, encompassing T2I generation, text editing, and other image‑to‑image OCR tasks (e.g., document dewarping and handwriting removal). The benchmark includes 1,060 human‑annotated samples consisting of instruction‑image‑GT triplets, deliberately featuring high text density, diverse generation scales, varied aspect ratios, and bilingual content to capture real‑world complexity. Furthermore, we introduce OCRGenScore, a unified metric integrating text accuracy, aesthetic quality, and instruction following. Extensive experiments on 19 cutting‑edge generative models reveal that most score below 60/100. Our analysis exposes critical, previously overlooked limitations, including poor text localization, unintended content modifications, and failures with dense or small‑scale text. We hope OCRGenBench establishes a robust standard to evaluate OCR generative capabilities, driving the evolution of reliable visual text synthesis. The benchmark and evaluation code are available at https://github.com/NiceRingNode/Awesome‑Generative‑Models‑for‑OCR.

Abstract:
Event stream based scene text recognition is a newly arising research topic in recent years which performs better than the widely used RGB cameras in extremely challenging scenarios, especially the low illumination, fast motion. Existing works either adopt end‑to‑end encoder‑decoder framework or large language models for enhanced recognition, however, they are still limited by the challenges of insufficient interpretability and weak contextual logical reasoning. In this work, we propose a novel chain‑of‑thought reasoning based event stream scene text recognition framework, termed ESTR‑CoT. Specifically, we first adopt the vision encoder EVA‑CLIP (ViT‑G/14) to transform the input event stream into tokens and utilize a Llama tokenizer to encode the given generation prompt. A Q‑former is used to align the vision token to the pre‑trained large language model Vicuna‑7B and output both the answer and chain‑of‑thought (CoT) reasoning process simultaneously. Our framework can be optimized using supervised fine‑tuning in an end‑to‑end manner. In addition, we also propose a large‑scale CoT dataset to train our framework via a three stage processing (i.e., generation, polish, and expert verification). This dataset provides a solid data foundation for the development of subsequent reasoning‑based large models. Extensive experiments on three event stream STR benchmark datasets (i.e., EventSTR, WordArt, IC15) fully validated the effectiveness and interpretability of our proposed framework. The source code and pre‑trained models will be released on https://github.com/Event‑AHU/ESTR‑CoT.

Abstract:
Scene text retrieval has made significant progress with the assistance of accurate text localization. However, existing approaches typically require costly bounding box annotations for training. Besides, they mostly adopt a customized retrieval strategy but struggle to unify various types of queries to meet diverse retrieval needs. To address these issues, we introduce Muti‑query Scene Text retrieval with Attention Recycling (MSTAR), a box‑free approach for scene text retrieval. It incorporates progressive vision embedding to dynamically capture the multi‑grained representation of texts and harmonizes free‑style text queries with style‑aware instructions. Additionally, a multi‑instance matching module is integrated to enhance vision‑language alignment. Furthermore, we build the Multi‑Query Text Retrieval (MQTR) dataset, the first benchmark designed to evaluate the multi‑query scene text retrieval capability of models, comprising four query types and 16k images. Extensive experiments demonstrate the superiority of our method across seven public datasets and the MQTR dataset. Notably, MSTAR marginally surpasses the previous state‑of‑the‑art model by 6.4% in MAP on Total‑Text while eliminating box annotation costs. Moreover, on the MQTR benchmark, MSTAR significantly outperforms the previous models by an average of 8.5%. The code and datasets are available at https://github.com/yingift/MSTAR.

Abstract:
Scene text detection has seen the emergence of high‑performing methods that excel on academic benchmarks. However, these detectors often fail to replicate such success in real‑world scenarios. We uncover two key factors contributing to this discrepancy through extensive experiments. First, a Fine‑tuning Gap, where models leverage Dataset‑Specific Optimization (DSO) paradigm for one domain at the cost of reduced effectiveness in others, leads to inflated performances on academic benchmarks. Second, the suboptimal performance in practical settings is primarily attributed to the long‑tailed distribution of texts, where detectors struggle with rare and complex categories as artistic or overlapped text. Given that the DSO paradigm might undermine the generalization ability of models, we advocate for a Joint‑Dataset Learning (JDL) protocol to alleviate the Fine‑tuning Gap. Additionally, an error analysis is conducted to identify three major categories and 13 subcategories of challenges in long‑tailed scene text, upon which we propose a Long‑Tailed Benchmark (LTB). LTB facilitates a comprehensive evaluation of ability to handle a diverse range of long‑tailed challenges. We further introduce MAEDet, a self‑supervised learning‑based method, as a strong baseline for LTB. The code is available at https://github.com/pd162/LTB.

Abstract:
Scene text editing aims to modify or add texts on images while ensuring text fidelity and overall visual quality consistent with the background. Recent methods are primarily built on UNet‑based diffusion models, which have improved scene text editing results, but still struggle with complex glyph structures, especially for non‑Latin ones (\eg, Chinese, Korean, Japanese). To address these issues, we present FLUX‑Text, a simple and advanced multilingual scene text editing DiT method. Specifically, our FLUX‑Text enhances glyph understanding and generation through lightweight Visual and Text Embedding Modules, while preserving the original generative capability of FLUX. We further propose a Regional Text Perceptual Loss tailored for text regions, along with a matching two‑stage training strategy to better balance text editing and overall image quality. Benefiting from the DiT‑based architecture and lightweight feature injection modules, FLUX‑Text can be trained with only 0.1M training examples, a 97% reduction compared to 2.9M required by popular methods. Extensive experiments on multiple public datasets, including English and Chinese benchmarks, demonstrate that our method surpasses other methods in visual quality and text fidelity. All the code is available at https://github.com/AMAP‑ML/FluxText.

Abstract:
Most previous scene text spotting methods rely on high‑quality manual annotations to achieve promising performance. To reduce their expensive costs, we study semi‑supervised text spotting (SSTS) to exploit useful information from unlabeled images. However, directly applying existing semi‑supervised methods of general scenes to SSTS will face new challenges: 1) inconsistent pseudo labels between detection and recognition tasks, and 2) sub‑optimal supervisions caused by inconsistency between teacher/student. Thus, we propose a new Semi‑supervised framework for End‑to‑end Text Spotting, namely SemiETS that leverages the complementarity of text detection and recognition. Specifically, it gradually generates reliable hierarchical pseudo labels for each task, thereby reducing noisy labels. Meanwhile, it extracts important information in locations and transcriptions from bidirectional flows to improve consistency. Extensive experiments on three datasets under various settings demonstrate the effectiveness of SemiETS on arbitrary‑shaped text. For example, it outperforms previous state‑of‑the‑art SSL methods by a large margin on end‑to‑end spotting (+8.7%, +5.6%, and +2.6% H‑mean under 0.5%, 1%, and 2% labeled data settings on Total‑Text, respectively). More importantly, it still improves upon a strongly supervised text spotter trained with plenty of labeled data by 2.0%. Compelling domain adaptation ability shows practical potential. Moreover, our method demonstrates consistent improvement on different text spotters.

Abstract:
Scene text editing aims to modify text content within scene images while maintaining style consistency. Traditional methods achieve this by explicitly disentangling style and content from the source image and then fusing the style with the target content, while ensuring content consistency using a pre‑trained recognition model. Despite notable progress, these methods suffer from complex pipelines, leading to suboptimal performance in complex scenarios. In this work, we introduce Recognition‑Synergistic Scene Text Editing (RS‑STE), a novel approach that fully exploits the intrinsic synergy of text recognition for editing. Our model seamlessly integrates text recognition with text editing within a unified framework, and leverages the recognition model's ability to implicitly disentangle style and content while ensuring content consistency. Specifically, our approach employs a multi‑modal parallel decoder based on transformer architecture, which predicts both text content and stylized images in parallel. Additionally, our cyclic self‑supervised fine‑tuning strategy enables effective training on unpaired real‑world data without ground truth, enhancing style and content consistency through a twice‑cyclic generation process. Built on a relatively simple architecture, RS‑STE achieves state‑of‑the‑art performance on both synthetic and real‑world benchmarks, and further demonstrates the effectiveness of leveraging the generated hard cases to boost the performance of downstream recognition tasks. Code is available at https://github.com/ZhengyaoFang/RS‑STE.

Abstract:
Visual Place Recognition (VPR) is a crucial capability for long‑term autonomous robots, enabling them to identify previously visited locations using visual information. However, existing methods remain limited in indoor settings due to the highly repetitive structures inherent in such environments. We observe that scene texts frequently appear in indoor spaces and can help distinguish visually similar but different places. This inspires us to propose TextInPlace, a simple yet effective VPR framework that integrates Scene Text Spotting (STS) to mitigate visual perceptual ambiguity in repetitive indoor environments. Specifically, TextInPlace adopts a dual‑branch architecture within a local parameter sharing network. The VPR branch employs attention‑based aggregation to extract global descriptors for coarse‑grained retrieval, while the STS branch utilizes a bridging text spotter to detect and recognize scene texts. Finally, the discriminative texts are filtered to compute text similarity and re‑rank the top‑K retrieved images. To bridge the gap between current text‑based repetitive indoor scene datasets and the typical scenarios encountered in robot navigation, we establish an indoor VPR benchmark dataset, called Maze‑with‑Text. Extensive experiments on both custom and public datasets demonstrate that TextInPlace achieves superior performance over existing methods that rely solely on appearance information. The dataset, code, and trained models are publicly available at https://github.com/HqiTao/TextInPlace.

Abstract:
Mainstream Scene Text Recognition (STR) algorithms are developed based on RGB cameras which are sensitive to challenging factors such as low illumination, motion blur, and cluttered backgrounds. In this paper, we propose to recognize the scene text using bio‑inspired event cameras by collecting and annotating a large‑scale benchmark dataset, termed EventSTR. It contains 9,928 high‑definition (1280 720) event samples and involves both Chinese and English characters. We also benchmark multiple STR algorithms as the baselines for future works to compare. In addition, we propose a new event‑based scene text recognition framework, termed SimC‑ESTR. It first extracts the event features using a visual encoder and projects them into tokens using a Q‑former module. More importantly, we propose to augment the vision tokens based on a memory mechanism before feeding into the large language models. A similarity‑based error correction mechanism is embedded within the large language model to correct potential minor errors fundamentally based on contextual information. Extensive experiments on the newly proposed EventSTR dataset and two simulation STR datasets fully demonstrate the effectiveness of our proposed model. We believe that the dataset and algorithmic model can innovatively propose an event‑based STR task and are expected to accelerate the application of event cameras in various industries. The source code and pre‑trained models will be released on https://github.com/Event‑AHU/EventSTR

Abstract:
We introduce EgoTextVQA, a novel and rigorously constructed benchmark for egocentric QA assistance involving scene text. EgoTextVQA contains 1.5K ego‑view videos and 7K scene‑text aware questions that reflect real user needs in outdoor driving and indoor house‑keeping activities. The questions are designed to elicit identification and reasoning on scene text in an egocentric and dynamic environment. With EgoTextVQA, we comprehensively evaluate 10 prominent multimodal large language models. Currently, all models struggle, and the best results (Gemini 1.5 Pro) are around 33% accuracy, highlighting the severe deficiency of these techniques in egocentric QA assistance. Our further investigations suggest that precise temporal grounding and multi‑frame reasoning, along with high resolution and auxiliary scene‑text inputs, are key for better performance. With thorough analyses and heuristic suggestions, we hope EgoTextVQA can serve as a solid testbed for research in egocentric scene‑text QA assistance. Our dataset is released at: https://github.com/zhousheng97/EgoTextVQA.

Abstract:
Multi‑oriented text is ubiquitous in real‑world scenes and remains a major challenge for scene text recognition (STR). Existing rotation‑aware methods explicitly estimate text orientation. However, due to the lack of theoretical guarantees, they are prone to error accumulation, increased computational cost, and strong reliance on data. In this work, we incorporate rotation invariance into the STR framework to address these limitations. Specifically, we adopt an encoder‑decoder architecture, embedding rotation equivariance in the encoder and rotation invariance in the decoder to construct a fully rotation‑invariant network. On the decoder side, we first identify and prove the rotation‑invariant property of the cross‑attention mechanism and use it to formulate a rotation‑invariant text decoder that maps visual features to output text in a rotation‑invariant manner. On the encoder side, we propose a rotation‑equivariant local‑global extraction network that integrates deep equivariant convolutions with self‑attention, enabling rotation‑equivariant feature extraction while modeling inter‑character dependencies and preserving fine‑grained visual details. By integrating the encoder and decoder, we obtain an end‑to‑end Rotation‑Invariant Scene Text Recognition network (RISTER). RISTER provides rotation invariance with theoretical guarantees, enhancing robustness on multi‑oriented samples without introducing additional inference computation or relying on data‑driven orientation correction. Experiments show that RISTER achieves state‑of‑the‑art performance on both standard and multi‑oriented benchmarks, surpassing the second‑best model by 4.0 percent in accuracy on the general multi‑oriented dataset.

Abstract:
Embeddings have emerged as a standard representational interface linking foundation models with downstream systems. Most embedding benchmarks assess representations through discriminative tasks or geometric criteria centered on separability in embedding space. However, strong performance on such evaluations does not establish whether content compressed into an embedding remains accessible to a downstream generator. To address this gap, we introduce the Generative Embedding Benchmark (GEB), in which a decoder answers questions using only a frozen embedding and question text, without access to the original image or intermediate visual features. Answer quality under this readout measures generative information: the answer‑relevant content recoverable from an embedding. GEB includes a curated visual‑question‑answering dataset with a 1,800‑item development split and a held‑out 900‑item test split covering natural images, scene text, and visual documents. Using a common decoder and training recipe, we evaluate seven public embedding models in visual‑only and vision‑language joint modes. On the test set, visual‑only scores range from 28.25 to 33.21; with image‑question joint encoding, all five VLM‑based embedding models score higher, and the best reaches 65.56. Matched embeddings also outperform text‑only inputs, zero embeddings, and shuffled embeddings. Natural‑image information is much easier to recover than scene text or visual‑document information, while a Qwen3‑VL‑2B reference with access to the original image reaches 84.30. Together, these results show that generative readout exposes information bottlenecks that separability‑based evaluation does not capture.

Abstract:
Scene text image super‑resolution (STISR) aims to recover visually plausible appearance while preserving character semantics from degraded inputs. Existing STISR systems often rely on externally generated priors or separate image and text models, resulting in error propagation and costly multi‑stage inference. We present DualTSR, a unified framework that formulates STISR as coupled continuous‑discrete generation. Conditional flow matching restores continuous image latents, while absorbing‑state discrete diffusion reconstructs text tokens. Both processes share a multimodal transformer backbone, allowing the evolving image and text states to interact throughout generation without an external OCR prior at inference. On CTR‑TSR, DualTSR achieves the best FID, LPIPS, ACC, and NED among the compared methods at both X2 and X4. On an aligned RealCE subset, it obtains the best FID, ACC, and NED with competitive LPIPS. Compared with DiffTSR at X4, DualTSR improves ACC by 12.78 percentage points while reducing the parameter count from 1.23B to 203M and end‑to‑end latency from 13.3s to 132ms. These results establish DualTSR as an accurate and efficient method for STISR.

Abstract:
Multimodal large language models (MLLMs) make grounded predictions in real‑world scenes by combining visual and textual cues, yet existing benchmarks rarely reveal how they arbitrate between these evidence sources when they conflict. We introduce SIGNPOST‑Bench, a controlled counterfactual benchmark for evaluating text‑vision conflict resolution. Each source image is transformed into a counterfactual quintuplet of Original, Blank, Similar, Random, and Adversarial variants. Synthetic, localized scene‑text interventions are designed to preserve non‑textual content, enabling paired measurements of changes in localization performance and directed shifts toward geographic targets introduced by conflicting text. SIGNPOST‑Bench contains 5,111 counterfactual groups and 25,555 image variants from four datasets. We evaluate 20 MLLMs from seven providers. Compared with Original images, Adversarial variants raise median localization error from 282 km to 1,347 km, a 4.8‑fold increase. Among geocodable adversarial samples, 6.5‑20.1% of predictions lie less than 50 km from the injected target across models, and every evaluated model exhibits a positive mean paired reduction in target distance from Blank to Adversarial. Compatible, unrelated, and conflicting text replacements produce distinct effects on model predictions, while clean‑input localization performance does not fully predict robustness to conflicting text. These results establish visual geolocation as a continuous diagnostic of scene‑text arbitration and provide a controlled framework for evaluating how MLLMs resolve conflicting multimodal evidence.

Abstract:
In‑the‑wild Bengali scene text recognition is largely unmeasured: existing resources target handwritten documents or constrained sign‑board parsing, report only aggregate edit‑distance metrics, and evaluate either conventional OCR or VLMs, never both on the same in‑the‑wild data. To address this gap, we introduce BANGLAWILD, a benchmark of 2,535 Bengali scene text images, each paired with a verbatim gold transcription, two categorical axes, four diagnostic attributes, and an orthographically standard form where the in‑image text deviates from canonical spelling. We evaluate fifteen VLMs and three conventional OCR systems under three prompting strategies, fine‑tune 6 open‑source models with LoRA, and complement edit‑distance metrics with an LLM‑as‑a‑Judge evaluation. Our results reveal a persistent gap in which larger models within the same family do not outperform smaller ones. Our fifteen‑class error taxonomy shows that visual mis‑recognition accounts for ~60% of errors in the strongest systems, while conjunct‑related errors contribute under 2%, challenging a long‑standing assumption in Bengali OCR research; the same visual dominant profile also holds across architectures, including the one conventional baseline that reads Bengali reliably. Prompt language mainly affects cross‑script drift and LoRA reduces catastrophic failures in weak models without lifting the ceiling on already competent ones. Code and data will be publicly released.

Abstract:
Scene Text Recognition (STR) remains challenging due to the diversity of text appearances, including curvature, rotation, and perspective distortion. Recent Transformer‑based approaches perform well but usually rely on one‑dimensional positional encodings that ignore the 2D spatial structure of text images. Axial 2D extensions of Rotary Position Embedding (RoPE) exist for vision Transformers, but they assume roughly square, isotropic image content and apply the rotation only within encoder self‑attention. Scene text violates both assumptions: crops are markedly anisotropic, and STR models are encoder‑decoder, so the decoder must relate its queries to the encoder's 2D layout through cross‑attention. We introduce 2D‑RoPE‑STR, which adapts axial 2D‑RoPE to this setting through (1) an anisotropic row/column dimension allocation matched to the aspect ratio of text, and (2) an extension of the rotary coupling into encoder‑decoder cross‑attention, letting autoregressive decoding steps attend to encoder tokens by their 2D layout, a setting not addressed by prior encoder‑only formulations. Both changes are essentially parameter‑free and require no architectural redesign beyond the positional‑encoding module. We further introduce a diagnostic protocol (a controlled ablation pair isolating only the positional encoding, an image‑level net‑win disagreement analysis, and encoder attention visualization) that identifies where and why relative 2D position helps: curved, rotated, and perspective‑distorted layouts where reading order departs from a straight horizontal line. On six standard benchmarks (IIIT5K, SVT, ICDAR 2013, ICDAR 2015, CUTE80, SVTP), gains concentrate on exactly these irregular layouts, with ablations isolating each design choice against 1D RoPE and 2D sinusoidal and learnable alternatives.

Abstract:
We describe our entry to the ICIP 2026 Grand Challenge on Extreme In‑the‑Wild License Plate Super‑Resolution (XLPSR), which scored 9.73 wECR on the public validation leaderboard. The system pairs a Hybrid Attention Transformer super‑resolution (HAT) front‑end with an ensemble of two scene‑text recognisers (PARSeq‑S and CLIP4STR‑B) and a confidence‑weighted character‑voting scheme that abstains on uncertain positions. We treat XLPSR as a recognition task gated by image legibility: the SR step exists to lift characters out of sub‑pixel territory, and the asymmetric scoring rule (+2 / ‑1 / 0) is exploited explicitly through abstention. Our pipeline runs in 1.7 s per sequence on RTX 3090 (max 2.7 s, p99 2.4 s), well under the 60 s/sequence Docker budget.

Abstract:
Large Vision‑Language Models (LVLMs) have been increasingly integrated into robotic systems. However, these models may exhibit overthinking behaviors, where they generate excessively long reasoning traces, incurring an excessive inference time. This overthinking behavior poses a serious risk to robotic systems, as the adversary can deliberately trigger overthinking to slow down the decision making of a victim robotic system, causing a variety of safety issues (i.e., an overthinking‑induced slowdown attack). To initiate this attack, an adversary can embed carefully crafted, human‑readable scene text into the visual scene observed by a victim robotic agent, causing significant inference delays even under a strict black‑box setting. Therefore, the embedded scene text serves as a significant "trigger" for the attack. This work systematically identifies and validates transferable triggers of overthinking in robotic systems by introducing a three‑stage framework. First, we construct a diverse corpus of reasoning‑intensive scene text and extract overthinking‑correlated lexical features from short response prefixes. Second, we perform an efficient black‑box search guided by a prefix‑based proxy score while selectively confirming a small set of top candidates with full latency measurements. Third, we evaluate black‑box transfer using a fixed pool of triggers on unseen images and multiple LVLMs, reporting latency amplification and attack success rates under standard thresholds. Across three representative LVLMs, all triggers yield slowdown ratios greater than 1.0x, with the strongest single‑trigger case reaching 6.96x. The physical printing of the text trigger still causes up to 4.74x latency amplification. These results demonstrate that our discovered triggers are transferred between multiple LVLM models and consistently cause significant slowdowns in robotic systems.

Abstract:
Vision‑language models (VLMs) are now proposed as runtime safety guards for embodied agents in homes and factories. A deployable guard must catch genuinely unsafe situations while avoiding unnecessary intervention on routine but superficially alarming activity, a distinction that binary safety benchmarks obscure. We introduce EgoSafetyBench, an egocentric video benchmark of 1,200 robot‑view scenarios annotated at half‑second granularity, to evaluate VLMs as streaming guards across two tracks. The situational track (800 scenarios) spans four families, from routine and safe‑but‑suspicious scenes to obvious and contextual hazards. The visual‑channel track (400 scenarios) targets in‑scene text‑a sign, sticker, or label visible in the scene‑that can misrepresent the physical situation, pairing each misleading sign with a truthful version to test both whether a guard flags the text as misleading and whether the text corrupts its physical‑safety judgment. Both tracks use contrastive ladders: near‑identical scenarios differing only in a single visible deciding cue, so a correct call must hinge on that cue rather than the overall scene type. We evaluate ten open‑ and closed‑source VLMs. We find that while guards reliably recognize videos containing hazards, they often miss specific hazardous moments, particularly contextual hazards. Furthermore, misleading in‑scene signs degrade all tested guards: vulnerable models miss up to a third of hazards, while robust models over‑intervene on safe content. Matched controls reveal that apparent safety robustness often reflects indiscriminate alarming rather than true physical reasoning.

Abstract:
Vision‑language models (VLMs) have achieved strong performance on OCR‑based benchmarks and increasingly focused on text‑rich understanding, but their robustness under controlled visual degradation remains insufficiently understood. This gap is critical for OCR reasoning, where visual corruption can induce OCR errors and structural distortions, thereby introducing uncertainty into the reasoning task. To systematically study this problem, we introduce OCR‑Robust, a benchmark designed for evaluating OCR reasoning robustness under visual perturbations. It contains 812 samples across two complementary subsets: OCR1.0, covering documents, scene text, receipts, handwriting, and mathematical content, and OCR2.0, focusing on charts, geometry diagrams, and tables. To enable efficient yet informative evaluation, we conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross‑model discriminability. We evaluate robustness using clean accuracy, Relative Corruption Retention (RCR), Worst‑Case Retention (WCR), and a composite Corruption Robustness Index (CRI), and benchmark 18 models spanning proprietary systems, open‑source VLMs, and OCR+LLM pipelines. Our results show that higher clean accuracy does not necessarily imply stronger robustness, and that models can suffer pronounced degradation in the worst case on OCR tasks that are sensitive to structure, and charts and tables are substantially more fragile than document‑like inputs under perturbation.

Abstract:
Multimodal Large Language Models (MLLMs) have redefined visual understanding by combining vision encoders with large‑scale language models. This unified architecture enables strong performance on tasks like image captioning, visual question answering, and multimodal dialogue, often in zero‑ and few‑shot settings. Their general‑purpose capabilities and flexible interfaces make MLLMs a promising foundation for real‑world vision‑language applications. Assistive AI aims to help users interact with their environments through natural language. These scenarios demand robust visual recognition, contextual reasoning, and multilingual comprehension‑capabilities that MLLMs are believed to offer. However, their effectiveness in assistive settings remains to be fully understood. In this work, we explore whether MLLMs can support Assistive AI by evaluating state‑of‑the‑art models on real‑world tasks: recognizing everyday objects like currency, answering questions based on scene text, and reading visually presented content across multiple languages. To this end, we developed a system, NetraLink, using a head‑mounted GoPro to capture real‑world egocentric data, and collected a benchmark covering these assistive scenarios. Our findings provide a comprehensive diagnostic of current MLLMs, highlighting their strengths and limitations in enabling assistive technologies grounded in visual perception and language interaction.

Abstract:
We propose TextWand, a general‑purpose framework that unifies scene text removal, generation, and replacement into a single model. By decomposing complex editing tasks into the atomic primitives of rendering and erasure, TextWand achieves precise control over both text appearance and background integrity. Specifically, we introduce a novel design, Overlay‑Reference Positional Encoding (ORPE), to enforce pixel‑level layout fidelity and exemplar‑driven style control, alongside a new strategy, Region‑Adaptive Suppression (RAS), to ensure clean text erasure. To address the absence of a comprehensive benchmark for general‑purpose scene text editing among existing single‑task datasets, we construct TextWand‑Bench. Extensive experiments demonstrate that TextWand outperforms existing leading open‑source and closed‑source models by delivering superior text content accuracy, layout and style consistency, and overall image quality across scene text removal, generation and replacement tasks.

Abstract:
Vision‑language models (VLMs) can produce confident visual answers even when the required visual evidence is missing, blank, or unrelated to the question. This failure mode, recently described as mirage (Asadi et al., 2026), is especially concerning in medical and document VQA, where visually ungrounded answers may be mistaken for image‑based evidence. We study pre‑release mirage detection: given an image‑question pair, determine whether a VLM's answer should be released or the system should abstain before the answer reaches the user. We propose Text‑Conditioned Layer‑wise Internal Alignment (TC‑LIA), a model‑agnostic method that probes patch‑token representations across the layers of a CLIP ViT‑H/14 vision encoder. The key idea is to project layer‑wise image patch tokens into the final CLIP embedding space and measure their similarity with the question embedding, tracking whether question‑relevant visual evidence emerges across vision layers. TC‑LIA summarizes this trajectory using final image‑text cosine similarity, late‑layer top‑k patch‑text alignment, early‑to‑late gain, and layer‑wise slope. These features are combined with pixel‑statistic‑based blank/noise detection, zero‑shot domain routing, and structured VLM self‑assessment in an ensemble classifier. We evaluate on 19,004 samples spanning ten VQA domains, including medical imaging, document understanding, scene text, reasoning, and video understanding, with four input conditions: RELATED, IN‑DOMAIN UNRELATED, OUT‑OF‑DOMAIN UNRELATED, and BLANK/NOISE. Across fourteen VLMs, base prompt mirage rates range from 57.3‑75.0%. TC‑LIA alone reduces the mirage rate to 7.5% at 83.5% accuracy; ensemble fusion further improves performance to 84.5‑88.4% accuracy with 5.7‑7.2% mirage rates. The best result, obtained with Qwen2.5‑VL‑72B, achieves 88.4% accuracy and a 6.4% mirage rate.

Abstract:
Large multimodal models (LMMs) inherit the self‑attention mechanism of pretrained language backbones, yet standard attention can exhibit suboptimal allocation, including cross‑modal misallocation between textual and visual evidence and intra‑visual imbalance among visual tokens. We propose RAVE (Re‑Allocating Visual Attention), a lightweight pair‑gating mechanism that adds a learned query‑key bias to pre‑softmax attention scores over visual keys, derived from pre‑RoPE query and key features. RAVE requires no architectural modification to the backbone and can be trained end‑to‑end with the rest of the model. Across a suite of multimodal benchmarks, RAVE improves over standard attention by an average of 3 points, with the largest gains on perception‑intensive tasks ‑‑ including multilingual OCR, chart understanding, document VQA, and scene text VQA ‑‑ where accurate visual grounding is critical.

Abstract:
End‑to‑end scene text spotting, which unifies text detection and recognition within a single framework, has witnessed remarkable progress driven by deep learning advances. However, most existing approaches still suffer from incomplete mask proposals caused by multi‑scale variation, arbitrary text shapes, and complex background interference, thereby degrading recognition accuracy. In this paper, we propose a novel Soft Attention Mask Embedding module (SAME) that leverages the global receptive field of Transformer encoders to encode high‑level features and compute soft attention weights, which are then hierarchically embedded with predicted masks to generate refined text‑boundary‑aware masks that effectively suppress background noise. Building upon this module, we present SAME‑Net, a robust end‑to‑end text spotting framework that requires neither character‑level annotations nor auxiliary text rectification modules. Since the soft attention mechanism is fully differentiable, recognition loss gradients can be back‑propagated through the SAME module to the detection branch, enabling joint optimization of detection and recognition objectives. Extensive experiments on challenging benchmarks demonstrate the effectiveness of our approach: SAME‑Net achieves 84.02% end‑to‑end H‑mean on the arbitrarily‑shaped Total‑Text dataset, surpassing the previous state‑of‑the‑art GLASS by 1.02% in full‑lexicon accuracy without additional training data, and obtains competitive 83.4% strong‑lexicon results on the multi‑oriented ICDAR 2015 dataset.

Abstract:
We present StyleText, a large‑scale dataset and benchmark for localized scene‑text inpainting with style preservation. StyleText contains 28,518 image‑mask‑prompt triplets grouped into 9,932 scene families, enabling controlled evaluation of text legibility and visual consistency under shared scene context. We construct the dataset with an automated pipeline that combines LLM prompt templating, Flux‑based source generation with key‑value (KV) cache injection, OCR‑based semantic filtering, polygon mask extraction, and mask‑conditioned FluxFill augmentation. We define a reproducible evaluation protocol using normalized OCR metrics (word accuracy and character error rate) and CLIP image‑image similarity with explicit preprocessing. A FluxFill+LoRA baseline trained on StyleText improves OCR accuracy substantially over initialization while maintaining scene style consistency, establishing a strong reference point for future comparisons.

Abstract:
Optical character recognition (OCR) and multilingual scene‑text understanding remain challenging for multimodal large language models (MLLMs), particularly in real‑world images containing small or degraded text, cluttered layouts, occlusion, handwriting, and complex typography. We present an OCR‑aware multilingual post‑training framework that improves visual‑text grounding in a general‑purpose MLLM without requiring an external OCR engine, OCR‑extracted text, or text bounding boxes at inference time. The framework combines large‑scale multilingual OCR supervision, approximately 5M additional multilingual training samples, controlled synthetic OCR generation and in‑image text translation, LoRA‑based supervised fine‑tuning (SFT), and lightweight OCR‑oriented Chain‑of‑Thought prompting. On a held‑out real‑world multilingual OCR benchmark, OCR‑SFT improves OCR completeness from 71.3 to 84.6, reduces hallucination rate from 18.3% to 5.5%, and improves translation BLEU‑1 from 52.3 to 80.2, with substantial hallucination reductions under blur and rotation. Evaluation on public benchmarks further shows gains on OCR‑intensive tasks while largely preserving broader multimodal capabilities; ablations show that SFT provides the primary improvement, with prompting offering smaller complementary gains. These results demonstrate that data‑centric OCR‑aware post‑training provides a practical and scalable approach to improving multilingual visual‑text grounding in general‑purpose MLLMs.

Abstract:
Scene text editing aims to modify text in a target region of an image while preserving surrounding background style and texture. Existing methods rely solely on image background information while neglecting the visual details of target regions, which discards stylistic features in the original text and essentially degrades the task to text rendering. Moreover, the conditions imposed by pre‑trained glyph encoder limit the scope of editable text. To address these issues, this paper proposes a self‑prompting scene text editing method that constructs style and glyph prompts directly from the original image, without introducing additional style or glyph encoders. We employ a two‑stage training strategy: the diffusion transformer is first trained on large‑scale self‑supervised data and then refined using a small set of paired images. By leveraging the in‑context learning capability of the Multi‑Modal Diffusion Transformer (MM‑DiT), it achieves open‑vocabulary and style‑consistent text editing. Experimental results on various languages demonstrate that our method achieves the state‑of‑the‑art performance in both text accuracy and style consistency. Our project page: hongxiii.github.io/mstedit.

Abstract:
Style‑conditioned scene text generation faces unique challenges in extracting precise text styles from complex backgrounds and maintaining fine‑grained style consistency across characters, especially for multilingual scripts. We propose StyleTextGen, a novel framework that learns to perceive and replicate visual text styles across different languages and writing systems. Our approach features three key contributions: First, we introduce a dual‑branch style encoder dedicated to style modeling, yielding robust multilingual text style representations in complex real‑world scenes. Second, we design a text style consistency loss that enhances style coherence and improves overall visual quality. Third, we develop a mask‑guided inference strategy that ensures precise style alignment between generated and reference text. To facilitate systematic evaluation, we construct StyleText‑CE, a bilingual scene text style benchmark covering both monolingual and cross‑lingual settings. Extensive experiments demonstrate that StyleTextGen significantly outperforms existing methods in style consistency and cross‑lingual generalization, establishing new state‑of‑the‑art performance in multilingual style‑conditioned text generation.

Abstract:
Text‑based role‑playing models can imitate character styles, yet they often fail to reflect a scene's atmosphere and evolving tension, both essential for immersive applications such as Virtual Reality (VR) games and interactive narratives. We study video‑grounded role‑playing dialogue and introduce EBM‑RL (Eye‑Brain‑Mouth Reinforcement Learning), a decoupled GRPO‑based framework that explicitly separates observation ([perception]), reasoning ([think]), and utterance ([answer]). This structure promotes human‑like sensory grounding by compelling the model to first attend to visual cues, then form internal interpretations, and finally generate context‑appropriate dialogue. EBM‑RL integrates four complementary rewards: (i) CLIP‑based scene‑text alignment to improve ambiance and emotion; (ii) a Perceptual‑Cognitive reward that encourages [perception] and [think] processes that increase the likelihood of the reference response; (iii) answer accuracy to ensure faithfulness; and (iv) a dense format reward to enforce the desired structured output. Extensive experiments demonstrate that EBM‑RL substantially outperforms text‑only role‑playing baselines and larger‑scale vision‑language models on our immersive role‑playing benchmark, delivering simultaneous gains in visual‑atmosphere consistency and character authenticity. Beyond the role‑playing domain, EBM‑RL also exhibits strong zero‑shot generalization: without any additional fine‑tuning, it consistently improves performance on out‑of‑domain VideoQA benchmarks. We additionally release an open‑source dataset for video‑grounded role‑playing dialogue.

Abstract:
Scene‑text image captioning requires fusing three information streams ‑‑ visual features, OCR‑detected text, and linguistic knowledge ‑‑ to generate descriptions that faithfully integrate text visible in images. Existing fusion approaches treat text as language‑agnostic, which fails for Vietnamese: a tonal language where diacritics alter word meaning, OCR errors are pervasive, and word boundaries are ambiguous. We argue that Vietnamese scene‑text captioning demands linguistically informed multimodal fusion, where language‑specific structural knowledge is explicitly incorporated into the fusion mechanism. Motivated from these insights, we propose HSTFG (Heterogeneous Scene‑Text Fusion Graph), a general‑purpose graph fusion framework with learned spatial attention bias, and show through topology analysis that cross‑modal graph edges are harmful for scene‑text fusion. Building on this finding, we design PhonoSTFG (Phonological Scene‑Text Fusion Graph) which specializes graph‑level fusion for Vietnamese linguistic reasoning. To support evaluation, we introduce ViTextCaps, the first large‑scale Vietnamese scene‑text captioning dataset (15,729 images with 74,970 captions), with comprehensive linguistic analysis showing that 52.8% of the vocabulary is at risk of diacritic collision.

Abstract:
Accurate text recognition in low‑light environments is essential for intelligent systems in applications ranging from autonomous vehicles to smart surveillance. However, challenges such as poor illumination and noise interference remain underexplored. To address this gap, we introduce LSTR, a large‑scale Low‑light Scene Text Recognition dataset comprising 11,273 low‑light images generated from well‑lit datasets (ICDAR2015, IIIT5K, and WordArt), along with ESTR, which includes 60 real nighttime street‑scene images in English and Spanish for exclusive evaluation. We explore two solution strategies: (1) employing Optical Character Recognition (OCR) models with fine‑tuning and LoRA‑based fine‑tuning and (2) a joint training strategy that integrates a low‑light image enhancement (LLIE) module with an OCR model. In particular, we propose a novel re‑render LLIE (RLLIE) module, which demonstrates improved performance on real‑world data. Through extensive experimentation, we analyze various training strategies and address a key research question: \emphHow bright is bright enough for effective scene text recognition? Our results indicate that standalone LLIE or OCR models perform inadequately under low‑light conditions, highlighting the advantages of specialized, jointly trained text‑centric approaches. Additionally, we provide a comprehensive benchmark to support future research in robust low‑light scene text recognition. https://huggingface.co/datasets/lumimusta/Low‑light_Scene_Text_Dataset.

Abstract:
No standardized benchmark exists for evaluating OCR on food packaging, despite its critical role in automated halal food verification. Existing benchmarks target documents or scene text, missing the unique challenges of ingredient labels: curved surfaces, dense multilingual text, and sub‑8pt fonts. We present HalalBench, the first open multilingual benchmark for food packaging OCR, comprising 1,043 images (50 real, 993 synthetic) with 36,438 annotations in COCO format spanning 14 languages. We evaluate four engines: docTR achieves F1=0.193, ML Kit 0.180, EasyOCR 0.167, while all fail on Japanese (F1=0.000). A clustering ablation shows 36% F1 improvement from our post‑processing algorithm. We validate findings through HalalLens (https://halallens.no), a production halal scanner serving 20+ countries. Dataset and code are released under open licenses.

Abstract:
Scene text editing (STE) has achieved remarkable progress in accurately rendering target text through diffusion‑based methods. However, we identify a critical yet overlooked problem: edit spillover ‑‑ when editing a target text region, existing methods inadvertently modify non‑target regions, particularly neighboring text. Through systematic evaluation on 50 real‑world scenes across four categories, we reveal that state‑of‑the‑art diffusion editing models exhibit a spillover rate of 94%, meaning nearly all non‑target text regions are altered during editing. To address this, we propose the Edit Fidelity Field (EFF), a semantics‑aware continuous field that controls per‑pixel editing fidelity. Unlike binary masks, EFF leverages OCR‑detected text regions to construct a four‑zone field: Edit Core (fully editable), Transition Zone (smooth decay), Protected Zone (non‑target text, explicitly locked), and Background (strictly preserved). EFF operates as a training‑free, model‑agnostic post‑processing module applicable to any diffusion‑based STE method. We further propose per‑region spillover quantification, a novel evaluation protocol that measures edit leakage at each non‑target text region individually. Experiments demonstrate that EFF reduces spillover rate from 94% to 25% while improving non‑target region preservation by +91.4 dB PSNR.

Abstract:
Japanese scene text poses challenges that multilingual benchmarks often fail to capture, including mixed scripts, frequent vertical writing, and a character inventory far larger than the Latin alphabet. Although Japanese is included in several multilingual benchmarks, these resources do not adequately capture the language‑specific complexities. Meanwhile, existing Japanese visual text datasets have primarily focused on scanned documents, leaving in‑the‑wild scene text underexplored. To fill this gap, we introduce JaWildText, a diagnostic benchmark for evaluating vision‑language models (VLMs) on Japanese scene text understanding. JaWildText contains 3,241 instances from 2,961 images newly captured in Japan, with 1.12 million annotated characters spanning 3,643 unique character types. It comprises three complementary tasks that vary in visual organization, output format, and writing style: (i) Dense Scene Text Visual Question Answering (STVQA), which requires reasoning over multiple pieces of visual text evidence; (ii) Receipt Key Information Extraction (KIE), which tests layout‑aware structured extraction from mobile‑captured receipts; and (iii) Handwriting OCR, which evaluates page‑level transcription across various media and writing directions. We evaluate 14 open‑weight VLMs and find that the best model achieves an average score of 0.64 across the three tasks. Error analyses show recognition remains the dominant bottleneck, especially for kanji. JaWildText enables fine‑grained, script‑aware diagnosis of Japanese scene text capabilities, and will be released with evaluation code.

Abstract:
Recent multimodal large language models (MLLMs) perform strongly on general visual understanding, diagram and chart reasoning, and document‑centric perception. However, these abilities are learned from heterogeneous supervision sources with very different task structures and learning demands, and the effect of their temporal organization during training remains underexplored. We study whether data organization affects the trade‑off among general understanding, structured reasoning, and fine‑grained OCR/document understanding in multimodal instruction tuning. To isolate this factor, we use a controlled three‑stage training framework in which the backbone, trainable modules, and optimization pipeline are fixed across all runs, and only the temporal arrangement of post‑alignment supervision is changed. We compare four strategies: direct mixture, curriculum training, balanced sampling, and reverse curriculum. Experiments on general visual instruction following, diagram reasoning, chart reasoning, scene‑text question answering, and document question answering show that data organization is a first‑order design variable in multimodal adaptation. Curriculum training gives the best overall trade‑off and the strongest structured reasoning performance. Balanced sampling is better for OCR‑oriented capability but weakens the broader capability balance. Reverse curriculum performs worst in both final performance and optimization stability. Training‑dynamics analysis further suggests that building general understanding and reasoning before introducing OCR‑intensive supervision leads to smoother optimization and faster convergence. These findings highlight data scheduling as an explicit design dimension for multimodal model adaptation.

Abstract:
Previous works based on Segment Anything Model (SAM) have achieved promising performance in unified scene text detection and layout analysis. However, the typical reliance on pixel‑level text segmentation for sampling thousands of foreground points as prompts leads to unsatisfied inference latency and limited data utilization. To address above issues, we propose ET‑SAM, an Efficient framework with two decoders for unified scene Text detection and layout analysis based on SAM. Technically, we customize a lightweight point decoder that produces word heatmaps for achieving a few foreground points, thereby eliminating excessive point prompts and accelerating inference. Without the dependence on pixel‑level segmentation, we further design a joint training strategy to leverage existing data with heterogeneous text‑level annotations. Specifically, the datasets with multi‑level, word‑level only, and line‑level only annotations are combined in parallel as a unified training set. For these datasets, we introduce three corresponding sets of learnable task prompts in both the point decoder and hierarchical mask decoder to mitigate discrepancies across datasets.Extensive experiments demonstrate that, compared to the previous SAM‑based architecture, ET‑SAM achieves about 3× inference acceleration while obtaining competitive performance on HierText, and improves an average of 11.0% F‑score on Total‑Text, CTW1500, and ICDAR15.

Abstract:
Generating accurate glyphs for visual text rendering is essential yet challenging. Existing methods typically enhance text rendering by training on a large amount of high‑quality scene text images, but the limited coverage of glyph variations and excessive stylization often compromise glyph accuracy, especially for complex or out‑of‑domain characters. Some methods leverage reinforcement learning to alleviate this issue, yet their reward models usually depend on text recognition systems that are insensitive to fine‑grained glyph errors, so images with incorrect glyphs may still receive high rewards. Inspired by Direct Preference Optimization (DPO), we propose GlyphPrinter, a preference‑based text rendering method that eliminates reliance on explicit reward models. However, the standard DPO objective only models overall preference between two samples, which is insufficient for visual text rendering where glyph errors typically occur in localized regions. To address this issue, we construct the GlyphCorrector dataset with region‑level glyph preference annotations and propose Region‑Grouped DPO (R‑GDPO), a region‑based objective that optimizes inter‑ and intra‑sample preferences over annotated regions, substantially enhancing glyph accuracy. Furthermore, we introduce Regional Reward Guidance, an inference strategy that samples from an optimal distribution with controllable glyph accuracy. Extensive experiments demonstrate that the proposed GlyphPrinter outperforms existing methods in glyph accuracy while maintaining a favorable balance between stylization and precision.

Abstract:
Multilingual document and scene text understanding plays an important role in applications such as search, finance, and public services. However, most existing benchmarks focus on high‑resource languages and fail to evaluate models in realistic multilingual environments. In Southeast Asia, the diversity of languages, complex writing systems, and highly varied document types make this challenge even greater. We introduce SEA‑Vision, a benchmark that jointly evaluates Document Parsing and Text‑Centric Visual Question Answering (TEC‑VQA) across 11 Southeast Asian languages. SEA‑Vision contains 15,234 document parsing pages from nine representative document types, annotated with hierarchical page‑, block‑, and line‑level labels. It also provides 7,496 TEC‑VQA question‑answer pairs that probe text recognition, numerical calculation, comparative analysis, logical reasoning, and spatial understanding. To make such multilingual, multi‑task annotation feasible, we design a hybrid pipeline for Document Parsing and TEC‑VQA. It combines automated filtering and scoring with MLLM‑assisted labeling and lightweight native‑speaker verification, greatly reducing manual labeling while maintaining high quality. We evaluate several leading multimodal models and observe pronounced performance degradation on low‑resource Southeast Asian languages, highlighting substantial remaining gaps in multilingual document and scene text understanding. We believe SEA‑Vision will help drive global progress in document and scene text understanding.

Abstract:
Scene Text Image Super‑Resolution (STISR) aims to restore high‑resolution details in low‑resolution text images, which is crucial for both human readability and machine recognition. Existing methods, however, often depend on external Optical Character Recognition (OCR) models for textual priors or rely on complex multi‑component architectures that are difficult to train and reproduce. In this paper, we introduce DualTSR, a unified end‑to‑end framework that addresses both issues. DualTSR employs a single multimodal transformer backbone trained with a dual diffusion objective. It simultaneously models the continuous distribution of high‑resolution images via Conditional Flow Matching and the discrete distribution of textual content via discrete diffusion. This shared design enables visual and textual information to interact at every layer, allowing the model to infer text priors internally instead of relying on an external OCR module. Compared with prior multi‑branch diffusion systems, DualTSR offers a simpler end‑to‑end formulation with fewer hand‑crafted components. Experiments on synthetic Chinese benchmarks and a curated real‑world evaluation protocol show that DualTSR achieves strong perceptual quality and text fidelity.

Abstract:
Ultra‑low bitrate image compression faces a critical challenge: preserving small‑font scene text while maintaining overall visual quality. Region‑of‑interest (ROI) bit allocation can prioritize text but often degrades global fidelity, leading to a trade‑off between local accuracy and overall image quality. Instead of relying on ROI coding, we incorporate auxiliary textual information extracted by OCR and transmitted with negligible overhead, enabling the decoder to leverage this semantic guidance. Our method, TextBoost, operationalizes this idea through three strategic designs: (i) adaptively filtering OCR outputs and rendering them into a guidance map; (ii) integrating this guidance with decoder features in a calibrated manner via an attention‑guided fusion block; and (iii) enforcing guidance‑consistent reconstruction in text regions with a regularizing loss that promotes natural blending with the scene. Extensive experiments on TextOCR and ICDAR 2015 demonstrate that TextBoost yields up to 60.6% higher text‑recognition F1 at comparable Peak Signal‑to‑Noise Ratio (PSNR) and bits per pixel (bpp), producing sharper small‑font text while preserving global image quality and effectively decoupling text enhancement from global rate‑distortion optimization.

Abstract:
Khmer is a low‑resource language characterized by a complex script, presenting significant challenges for optical character recognition (OCR). While document printed text recognition has advanced because of available datasets, performance on other modalities, such as handwritten and scene text, remains limited by data scarcity. Training modality‑specific models for each modality does not allow cross‑modality transfer learning, from which modalities with limited data could otherwise benefit. Moreover, deploying many modality‑specific models results in significant memory overhead and requires error‑prone routing each input image to the appropriate model. On the other hand, simply training on a combined dataset with a non‑uniform data distribution across different modalities often leads to degraded performance on underrepresented modalities. To address these, we propose a universal Khmer text recognition (UKTR) framework capable of handling diverse text modalities. Central to our method is a novel modality‑aware adaptive feature selection (MAFS) technique designed to adapt visual features according to a particular input image modality and enhance recognition robustness across modalities. Extensive experiments demonstrate that our model achieves state‑of‑the‑art (SoTA) performance. Furthermore, we introduce the first comprehensive benchmark for universal Khmer text recognition, which we release to the community to facilitate future research. Our datasets and models can be accessible via this gated repository\footnotein review.

Abstract:
Vision‑language models (VLMs) can read text from images, but where does this optical character recognition (OCR) information enter the language processing stream? We investigate the OCR routing mechanism across three architecture families (Qwen3‑VL, Phi‑4, InternVL3.5) using causal interventions. By computing activation differences between original images and text‑inpainted versions, we identify architecture‑specific OCR bottlenecks whose dominant location depends on the vision‑language integration strategy: DeepStack models (Qwen) show peak sensitivity at mid‑depth (about 50%) for scene text, while single‑stage projection models (Phi‑4, InternVL) peak at early layers (6‑25%), though the exact layer of maximum effect varies across datasets. The OCR signal is remarkably low‑dimensional: PC1 captures up to 72.9% of variance. Crucially, principal component analysis (PCA) directions learned on one dataset transfer to others, demonstrating shared text‑processing pathways. Surprisingly, in models with modular OCR circuits (notably Qwen3‑VL‑4B), OCR removal can improve counting performance (up to +6.9 percentage points), suggesting OCR interferes with other visual processing in sufficiently modular architectures.

Abstract:
Scene text spotting aims to detect and recognize text in real‑world images, where instances are often short, fragmented, or visually ambiguous. Existing methods primarily rely on visual cues and implicitly capture local character dependencies, but they overlook the benefits of external linguistic knowledge. Prior attempts to integrate language models either adapt language modeling objectives without external knowledge or apply pretrained models that are misaligned with the word‑level granularity of scene text. We propose TiCLS, an end‑to‑end text spotter that explicitly incorporates external linguistic knowledge from a character‑level pretrained language model. TiCLS introduces a linguistic decoder that fuses visual and linguistic features, yet can be initialized by a pretrained language model, enabling robust recognition of ambiguous or fragmented text. Experiments on ICDAR 2015 and Total‑Text demonstrate that TiCLS achieves state‑of‑the‑art performance, validating the effectiveness of PLM‑guided linguistic integration for scene text spotting.

Abstract:
To pursue an efficient text assembling process, existing methods detect texts via the shrink‑mask expansion strategy. However, the shrinking operation loses the visual features of text margins and confuses the foreground and background difference, which brings intrinsic limitations to recognize text features. We follow this issue and design Text‑Pass Filter (TPF) for arbitrary‑shaped text detection. It segments the whole text directly, which avoids the intrinsic limitations. It is noteworthy that different from previous whole text region‑based methods, TPF can separate adhesive texts naturally without complex decoding or post‑processing processes, which makes it possible for real‑time text detection. Concretely, we find that the band‑pass filter allows through components in a specified band of frequencies, called its passband but blocks components with frequencies above or below this band. It provides a natural idea for extracting whole texts separately. By simulating the band‑pass filter, TPF constructs a unique feature‑filter pair for each text. In the inference stage, every filter extracts the corresponding matched text by passing its pass‑feature and blocking other features. Meanwhile, considering the large aspect ratio problem of ribbon‑like texts makes it hard to recognize texts wholly, a Reinforcement Ensemble Unit (REU) is designed to enhance the feature consistency of the same text and to enlarge the filter's recognition field to help recognize whole texts. Furthermore, a Foreground Prior Unit (FPU) is introduced to encourage TPF to discriminate the difference between the foreground and background, which improves the feature‑filter pair quality. Experiments demonstrate the effectiveness of REU and FPU while showing the TPF's superiority.

Abstract:
Understanding signboard text in natural scenes is essential for real‑world applications of Visual Question Answering (VQA), yet remains underexplored, particularly in low‑resource languages. We introduce ViSignVQA, the first large‑scale Vietnamese dataset designed for signboard‑oriented VQA, which comprises 10,762 images and 25,573 question‑answer pairs. The dataset captures the diverse linguistic, cultural, and visual characteristics of Vietnamese signboards, including bilingual text, informal phrasing, and visual elements such as color and layout. To benchmark this task, we adapted state‑of‑the‑art VQA models (e.g., BLIP‑2, LaTr, PreSTU, and SaL) by integrating a Vietnamese OCR model (SwinTextSpotter) and a Vietnamese pretrained language model (ViT5). The experimental results highlight the significant role of the OCR‑enhanced context, with F1‑score improvements of up to 209% when the OCR text is appended to questions. Additionally, we propose a multi‑agent VQA framework combining perception and reasoning agents with GPT‑4, achieving 75.98% accuracy via majority voting. Our study presents the first large‑scale multimodal dataset for Vietnamese signboard understanding. This underscores the importance of domain‑specific resources in enhancing text‑based VQA for low‑resource languages. ViSignVQA serves as a benchmark capturing real‑world scene text characteristics and supporting the development and evaluation of OCR‑integrated VQA models in Vietnamese.

Abstract:
We introduce SELECT (Scene tExt Label Errors deteCTion), a novel approach that leverages multi‑modal training to detect label errors in real‑world scene text datasets. Utilizing an image‑text encoder and a character‑level tokenizer, SELECT addresses the issues of variable‑length sequence labels, label sequence misalignment, and character‑level errors, outperforming existing methods in accuracy and practical utility. In addition, we introduce Similarity‑based Sequence Label Corruption (SSLC), a process that intentionally introduces errors into the training labels to mimic real‑world error scenarios during training. SSLC not only can cause a change in the sequence length but also takes into account the visual similarity between characters during corruption. Our method is the first to detect label errors in real‑world scene text datasets successfully accounting for variable‑length labels. Experimental results demonstrate the effectiveness of SELECT in detecting label errors and improving STR accuracy on real‑world text datasets, showcasing its practical utility.

Abstract:
Infographic Visual Question Answering (InfographicVQA) evaluates a model's ability to read and reason over data‑rich, layout‑heavy visuals that combine text, charts, icons, and design elements. Compared with scene‑text or natural‑image VQA, infographics require stronger integration of OCR, layout understanding, and numerical and semantic reasoning. We introduce ViInfographicVQA, the first benchmark for Vietnamese InfographicVQA, comprising over 6747 real‑world infographics and 20409 human‑verified question‑answer pairs across economics, healthcare, education, and more. The benchmark includes two evaluation settings. The Single‑image task follows the traditional setup in which each question is answered using a single infographic. The Multi‑image task requires synthesizing evidence across multiple semantically related infographics and is, to our knowledge, the first Vietnamese evaluation of cross‑image reasoning in VQA. We evaluate a range of recent vision‑language models on this benchmark, revealing substantial performance disparities, with the most significant errors occurring on Multi‑image questions that involve cross‑image integration and non‑span reasoning. ViInfographicVQA contributes benchmark results for Vietnamese InfographicVQA and sheds light on the limitations of current multimodal models in low‑resource contexts, encouraging future exploration of layout‑aware and cross‑image reasoning methods.

Abstract:
In scene text detection, Transformer‑based methods have addressed the global feature extraction limitations inherent in traditional convolution neural network‑based methods. However, most directly rely on native Transformer attention layers as encoders without evaluating their cross‑domain limitations and inherent shortcomings: forgetting important information or focusing on irrelevant representations when modeling long‑range dependencies for text detection. The recently proposed state space model Mamba has demonstrated better long‑range dependencies modeling through a linear complexity selection mechanism. Therefore, we propose a novel scene text detector based on Mamba that integrates the selection mechanism with attention layers, enhancing the encoder's ability to extract relevant information from long sequences. We adopt the Top\_k algorithm to explicitly select key information and reduce the interference of irrelevant information in Mamba modeling. Additionally, we design a dual‑scale feed‑forward network and an embedding pyramid enhancement module to facilitate high‑dimensional hidden state interactions and multi‑scale feature fusion. Our method achieves state‑of‑the‑art or competitive performance on various benchmarks, with F‑measures of 89.7%, 89.2%, and 78.5% on CTW1500, TotalText, and ICDAR19ArT, respectively. Codes will be available.

Abstract:
Scene Text Editing (STE) involves replacing text in a scene image with new target text while preserving both the original text style and background texture. Existing methods suffer from two major challenges: inconsistency and length‑insensitivity. They often fail to maintain coherence between the edited local patch and the surrounding area, and they struggle to handle significant differences in text length before and after editing. To tackle these challenges, we propose an end‑to‑end framework called Global‑Local Aware Scene Text Editing (GLASTE), which simultaneously incorporates high‑level global contextual information along with delicate local features. Specifically, we design a global‑local combination structure, joint global and local losses, and enhance text image features to ensure consistency in text style within local patches while maintaining harmony between local and global areas. Additionally, we express the text style as a vector independent of the image size, which can be transferred to target text images of various sizes. We use an affine fusion to fill target text images into the editing patch while maintaining their aspect ratio unchanged. Extensive experiments on real‑world datasets validate that our GLASTE model outperforms previous methods in both quantitative metrics and qualitative results and effectively mitigates the two challenges.

Abstract:
Reading scene text, that is, text appearing in images, has numerous application areas, including assistive technology, search, and e‑commerce. Although scene text recognition in English has advanced significantly and is often considered nearly a solved problem, Indian language scene text recognition remains an open challenge. This is due to script diversity, non‑standard fonts, and varying writing styles, and, more importantly, the lack of high‑quality datasets and open‑source models. To address these gaps, we introduce the Bharat Scene Text Dataset (BSTD) ‑ a large‑scale and comprehensive benchmark for studying Indian Language Scene Text Recognition. It comprises more than 100K words that span 11 Indian languages and English, sourced from over 6,500 scene images captured across various linguistic regions of India. The dataset is meticulously annotated and supports multiple scene text tasks, including: (i) Scene Text Detection, (ii) Script Identification, (iii) Cropped Word Recognition, and (iv) End‑to‑End Scene Text Recognition. We evaluated state‑of‑the‑art models originally developed for English by adapting (fine‑tuning) them for Indian languages. Our results highlight the challenges and opportunities in Indian language scene text recognition. We believe that this dataset represents a significant step toward advancing research in this domain. All our models and data are open source.

Abstract:
The advent of generative models has dramatically improved the accuracy of image inpainting. In particular, by removing specific text from document images, reconstructing original images is extremely important for industrial applications. However, most existing methods of text removal focus on deleting simple scene text which appears in images captured by a camera in an outdoor environment. There is little research dedicated to complex and practical images with dense text. Therefore, we created benchmark data for text removal from images including a large amount of text. From the data, we found that text‑removal performance becomes vulnerable against mask profile perturbation. Thus, for practical text‑removal tasks, precise tuning of the mask shape is essential. This study developed a method to model highly flexible mask profiles and learn their parameters using Bayesian optimization. The resulting profiles were found to be character‑wise masks. It was also found that the minimum cover of a text region is not optimal. Our research is expected to pave the way for a user‑friendly guideline for manual masking.

Abstract:
Video text‑based visual question answering (Video TextVQA) task aims to answer questions about videos by leveraging the visual text appearing within the videos. This task poses significant challenges, requiring models to accurately perceive and comprehend scene text that varies in scale, orientation, and clarity across frames, while effectively integrating temporal and semantic context to generate precise answers. Moreover, the model must identify question‑relevant textual cues and filter out redundant or irrelevant information to ensure answering is guided by the most relevant and informative cues. To address these challenges, we propose SFA, a training‑free framework and the first Video‑LLM‑based method tailored for Video TextVQA, motivated by the human process of answering questions. By adaptively scanning video frames, selectively focusing on key regions, and directly amplifying them, SFA effectively guides the Video‑LLM's attention toward essential cues, enabling it to generate more accurate answers. SFA achieves new state‑of‑the‑art results across several public Video TextVQA datasets and surpasses previous methods by a substantial margin, demonstrating its effectiveness and generalizability.

Abstract:
Vision‑Language Models (VLMs) often struggle with tasks that require fine‑grained image understanding, such as scene‑text recognition or document analysis, due to perception limitations and visual fragmentation. To address these challenges, we introduce CropVLM as an external low‑cost method for boosting performance, enabling VLMs to dynamically ''zoom in'' on relevant image regions, enhancing their ability to capture fine details. CropVLM is trained using reinforcement learning, without using human‑labeled bounding boxes as a supervision signal, and without expensive synthetic evaluations. The model is trained once and can be paired with both open‑source and proprietary VLMs to improve their performance. Our approach delivers significant improvements on tasks that require high‑resolution image understanding, notably for benchmarks that are out‑of‑domain for the target VLM, without modifying or fine‑tuning the VLM, thus avoiding catastrophic forgetting.

Abstract:
As VLMs are deployed in safety‑critical applications, their ability to abstain from answering when uncertain becomes crucial for reliability, especially in Scene Text Visual Question Answering (STVQA) tasks. For example, OCR errors like misreading "50 mph" as "60 mph" could cause severe traffic accidents. This leads us to ask: Can VLMs know when they can't see? Existing abstention methods suggest pessimistic answers: they either rely on miscalibrated output probabilities or require semantic agreement unsuitable for OCR tasks. However, this failure may indicate we are looking in the wrong place: uncertainty signals could be hidden in VLMs' internal representations. Building on this insight, we propose Latent Representation Probing (LRP): training lightweight probes on hidden states or attention patterns. We explore three probe designs: concatenating representations across all layers, aggregating attention over visual tokens, and ensembling single layer probes by majority vote. Experiments on four benchmarks across image and video modalities show LRP improves abstention accuracy by 7.6% over best baselines. Our analysis reveals: probes generalize across various uncertainty sources and datasets, and optimal signals emerge from intermediate rather than final layers. This establishes a principled framework for building deployment‑ready AI systems by detecting confidence signals from internal states rather than unreliable outputs.

Abstract:
Scene Text Editing (STE) is the task of modifying text content in an image while preserving its visual style, such as font, color, and background. While recent diffusion‑based approaches have shown improvements in visual quality, key limitations remain: lack of support for low‑resource languages, domain gap between synthetic and real data, and the absence of appropriate metrics for evaluating text style preservation. To address these challenges, we propose STELLAR (Scene Text Editor for Low‑resource LAnguages and Real‑world data). STELLAR enables reliable multilingual editing through a language‑adaptive glyph encoder and a multi‑stage training strategy that first pre‑trains on synthetic data and then fine‑tunes on real images. We also construct a new dataset, STIPLAR(Scene Text Image Pairs of Low‑resource lAnguages and Real‑world data), for training and evaluation. Furthermore, we propose Text Appearance Similarity (TAS), a novel metric that assesses style preservation by independently measuring font, color, and background similarity, enabling robust evaluation even without ground truth. Experimental results demonstrate that STELLAR outperforms state‑of‑the‑art models in visual consistency and recognition accuracy, achieving an average TAS improvement of 2.2% across languages over the baselines.

Abstract:
Scene Text Recognition (STR) remains challenging due to real‑world complexities, where decoupled visual‑linguistic optimization in existing frameworks amplifies error propagation through cross‑modal misalignment. Visual encoders exhibit attention bias toward background distractors, while decoders suffer from spatial misalignment when parsing geometrically deformed text‑collectively degrading recognition accuracy for irregular patterns. Inspired by the hierarchical cognitive processes in human visual perception, we propose OTSNet, a novel three‑stage network embodying a neurocognitive‑inspired Observation‑Thinking‑Spelling pipeline for unified STR modeling. The architecture comprises three core components: (1) a Dual Attention Macaron Encoder (DAME) that refines visual features through differential attention maps to suppress irrelevant regions and enhance discriminative focus; (2) a Position‑Aware Module (PAM) and Semantic Quantizer (SQ) that jointly integrate spatial context with glyph‑level semantic abstraction via adaptive sampling; and (3) a Multi‑Modal Collaborative Verifier (MMCV) that enforces self‑correction through cross‑modal fusion of visual, semantic, and character‑level features. Extensive experiments demonstrate that OTSNet achieves state‑of‑the‑art performance, attaining 83.5% average accuracy on the challenging Union14M‑L benchmark and 79.1% on the heavily occluded OST dataset‑establishing new records across 9 out of 14 evaluation scenarios.

Abstract:
Motion blur in scene text images severely impairs readability and hinders the reliability of computer vision tasks, including autonomous driving, document digitization, and visual information retrieval. Conventional deblurring approaches are often inadequate in handling spatially varying blur and typically fall short in modeling the long‑range dependencies necessary for restoring textual clarity. To overcome these limitations, we introduce a hybrid deep learning framework that combines convolutional neural networks (CNNs) with vision transformers (ViTs), thereby leveraging both local feature extraction and global contextual reasoning. The architecture employs a CNN‑based encoder‑decoder to preserve structural details, while a transformer module enhances global awareness through self‑attention. Training is conducted on a curated dataset derived from TextOCR, where sharp scene‑text samples are paired with synthetically blurred versions generated using realistic motion‑blur kernels of multiple sizes and orientations. Model optimization is guided by a composite loss that incorporates mean absolute error (MAE), squared error (MSE), perceptual similarity, and structural similarity (SSIM). Quantitative evaluations show that the proposed method attains 32.20 dB in PSNR and 0.934 in SSIM, while remaining lightweight with 2.83 million parameters and an average inference time of 61 ms. These results highlight the effectiveness and computational efficiency of the CNN‑ViT hybrid design, establishing its practicality for real‑world motion‑blurred scene‑text restoration.

Abstract:
Creation of large‑scale databases for Visual Question Answering tasks pertaining to the text data in a scene (text‑VQA) involves skilful human annotation, which is tedious and challenging. With the advent of foundation models that handle vision and language modalities, and with the maturity of OCR systems, it is the need of the hour to establish an end‑to‑end pipeline that can synthesize Question‑Answer (QA) pairs based on scene‑text from a given image. We propose a pipeline for automated synthesis for text‑VQA dataset that can produce faithful QA pairs, and which scales up with the availability of scene text data. Our proposed method harnesses the capabilities of multiple models and algorithms involving OCR detection and recognition (text spotting), region of interest (ROI) detection, caption generation, and question generation. These components are streamlined into a cohesive pipeline to automate the synthesis and validation of QA pairs. To the best of our knowledge, this is the first pipeline proposed to automatically synthesize and validate a large‑scale text‑VQA dataset comprising around 72K QA pairs based on around 44K images.

Abstract:
Image super‑resolution(SR) is fundamental to many vision system‑from surveillance and autonomy to document analysis and retail analytics‑because recovering high‑frequency details, especially scene‑text, enables reliable downstream perception. Scene‑text, i.e., text embedded in natural images such as signs, product labels, and storefronts, often carries the most actionable information; when characters are blurred or hallucinated, optical character recognition(OCR) and subsequent decisions fail even if the rest of the image appears sharp. Yet previous SR research has often been tuned to distortion (PSNR/SSIM) or learned perceptual metrics (LIPIS, MANIQA, CLIP‑IQA, MUSIQ) that are largely insensitive to character‑level errors. Furthermore, studies that do address text SR often focus on simplified benchmarks with isolated characters, overlooking the challenges of text within complex natural scenes. As a result, scene‑text is effectively treated as generic texture. For SR to be effective in practical deployments, it is therefore essential to explicitly optimize for both text legibility and perceptual quality. We present GLYPH‑SR, a vision‑language‑guided diffusion framework that aims to achieve both objectives jointly. GLYPH‑SR utilizes a Text‑SR Fusion ControlNet(TS‑ControlNet) guided by OCR data, and a ping‑pong scheduler that alternates between text‑ and scene‑centric guidance. To enable targeted text restoration, we train these components on a synthetic corpus while keeping the main SR branch frozen. Across SVT, SCUT‑CTW1500, and CUTE80 at x4, and x8, GLYPH‑SR improves OCR F1 by up to +15.18 percentage points over diffusion/GAN baseline (SVT x8, OpenOCR) while maintaining competitive MANIQA, CLIP‑IQA, and MUSIQ. GLYPH‑SR is designed to satisfy both objectives simultaneously‑high readability and high visual realism‑delivering SR that looks right and reds right.

Abstract:
Current image super‑resolution methods show strong performance on natural images but distort text, creating a fundamental trade‑off between image quality and textual readability. To address this, we introduce TIGER (Text‑Image Guided supEr‑Resolution), a novel two‑stage framework that breaks this trade‑off through a "text‑first, image‑later" paradigm. TIGER explicitly decouples glyph restoration from image enhancement: it first reconstructs precise text structures and uses them to guide full‑image super‑resolution. This ensures high fidelity and readability. To support comprehensive training and evaluation, we present the UZ‑ST (UltraZoom‑Scene Text) dataset, the first Chinese scene text dataset with extreme zoom. Extensive experiments show TIGER achieves state‑of‑the‑art performance, enhancing readability and image quality.

Abstract:
With the rapid development of diffusion models, style transfer has made remarkable progress. However, flexible and localized style editing for scene text remains an unsolved challenge. Although existing scene text editing methods have achieved text region editing, they are typically limited to content replacement and simple styles, which lack the ability of free‑style transfer. In this paper, we introduce SceneTextStylizer, a novel training‑free diffusion‑based framework for flexible and high‑fidelity style transfer of text in scene images. Unlike prior approaches that either perform global style transfer or focus solely on textual content modification, our method enables prompt‑guided style transformation specifically for text regions, while preserving both text readability and stylistic consistency. To achieve this, we design a feature injection module that leverages diffusion model inversion and self‑attention to transfer style features effectively. Additionally, a region control mechanism is introduced by applying a distance‑based changing mask at each denoising step, enabling precise spatial control. To further enhance visual quality, we incorporate a style enhancement module based on the Fourier transform to reinforce stylistic richness. Extensive experiments demonstrate that our method achieves superior performance in scene text style transformation, outperforming existing state‑of‑the‑art methods in both visual fidelity and text preservation.

Abstract:
Current text detection datasets primarily target natural or document scenes, where text typically appear in regular font and shapes, monotonous colors, and orderly layouts. The text usually arranged along straight or curved lines. However, these characteristics differ significantly from anime scenes, where text is often diverse in style, irregularly arranged, and easily confused with complex visual elements such as symbols and decorative patterns. Text in anime scene also includes a large number of handwritten and stylized fonts. Motivated by this gap, we introduce AnimeText, a large‑scale dataset containing 735K images and 4.2M annotated text blocks. It features hierarchical annotations and hard negative samples tailored for anime scenarios. %Cross‑dataset evaluations using state‑of‑the‑art methods demonstrate that models trained on AnimeText achieve superior performance in anime text detection tasks compared to existing datasets. To evaluate the robustness of AnimeText in complex anime scenes, we conducted cross‑dataset benchmarking using state‑of‑the‑art text detection methods. Experimental results demonstrate that models trained on AnimeText outperform those trained on existing datasets in anime scene text detection tasks. AnimeText on HuggingFace: https://huggingface.co/datasets/deepghs/AnimeText

Abstract:
Intention drives human movement in complex environments, but such movement can only happen if the surrounding context supports it. Despite the intuitive nature of this mechanism, existing research has not yet provided tools to evaluate the alignment between skeletal movement (motion), intention (text), and the surrounding context (scene). In this work, we introduce MonSTeR, the first MOtioN‑Scene‑TExt Retrieval model. Inspired by the modeling of higher‑order relations, MonSTeR constructs a unified latent space by leveraging unimodal and cross‑modal representations. This allows MonSTeR to capture the intricate dependencies between modalities, enabling flexible but robust retrieval across various tasks. Our results show that MonSTeR outperforms trimodal models that rely solely on unimodal representations. Furthermore, we validate the alignment of our retrieval scores with human preferences through a dedicated user study. We demonstrate the versatility of MonSTeR's latent space on zero‑shot in‑Scene Object Placement and Motion Captioning. Code and pre‑trained models are available at github.com/colloroneluca/MonSTeR.

Abstract:
Text removal is a crucial task in computer vision with applications such as privacy preservation, image editing, and media reuse. While existing research has primarily focused on scene text removal in natural images, limitations in current datasets hinder out‑of‑domain generalization or accurate evaluation. In particular, widely used benchmarks such as SCUT‑EnsText suffer from ground truth artifacts due to manual editing, overly simplistic text backgrounds, and evaluation metrics that do not capture the quality of generated results. To address these issues, we introduce an approach to synthesizing a text removal benchmark applicable to domains other than scene texts. Our dataset features text rendered on complex backgrounds using object‑aware placement and vision‑language model‑generated content, ensuring clean ground truth and challenging text removal scenarios. The dataset is available at https://huggingface.co/datasets/cyberagent/OTR .

Abstract:
Bronze inscriptions (BI), engraved on ritual vessels, constitute a crucial stage of early Chinese writing and provide indispensable evidence for archaeological and historical studies. However, automatic BI recognition remains difficult due to severe visual degradation, multi‑domain variability across photographs, rubbings, and tracings, and an extremely long‑tailed character distribution. To address these challenges, we curate a large‑scale BI dataset comprising 22454 full‑page images and 198598 annotated characters spanning 6658 unique categories, enabling robust cross‑domain evaluation. Building on this resource, we develop a two‑stage detection‑recognition pipeline that first localizes inscriptions and then transcribes individual characters. To handle heterogeneous domains and rare classes, we equip the pipeline with LadderMoE, which augments a pretrained CLIP encoder with ladder‑style MoE adapters, enabling dynamic expert specialization and stronger robustness. Comprehensive experiments on single‑character and full‑page recognition tasks demonstrate that our method substantially outperforms state‑of‑the‑art scene text recognition baselines, achieving superior accuracy across head, mid, and tail categories as well as all acquisition modalities. These results establish a strong foundation for bronze inscription recognition and downstream archaeological analysis.

Abstract:
Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integrity across diverse languages and formats. We introduce a structured framework that combines automated components with human oversight to address the complexities of advertisement localization. To the best of our knowledge, this is the first work to integrate scene text detection, inpainting, machine translation (MT), and text reimposition specifically for accelerating ad localization evaluation workflows. Qualitative results across six locales demonstrate that our approach produces semantically accurate and visually coherent localized advertisements, suitable for deployment in real‑world workflows.

Abstract:
Scene Text Recognition (STR) remains a challenging task due to complex visual appearances and limited semantic priors. We propose TEACH, a novel training paradigm that injects ground‑truth text into the model as auxiliary input and progressively reduces its influence during training. By encoding target labels into the embedding space and applying loss‑aware masking, TEACH simulates a curriculum learning process that guides the model from label‑dependent learning to fully visual recognition. Unlike language model‑based approaches, TEACH requires no external pretraining and introduces no inference overhead. It is model‑agnostic and can be seamlessly integrated into existing encoder‑decoder frameworks. Extensive experiments across multiple public benchmarks show that models trained with TEACH achieve consistently improved accuracy, especially under challenging conditions, validating its robustness and general applicability.

Abstract:
We introduce ModaRoute, an LLM‑based intelligent routing system that dynamically selects optimal modalities for multimodal video retrieval. While dense text captions can achieve 75.9% Recall@5, they require expensive offline processing and miss critical visual information present in 34% of clips with scene text not captured by ASR. By analyzing query intent and predicting information needs, ModaRoute reduces computational overhead by 41% while achieving 60.9% Recall@5. Our approach uses GPT‑4.1 to route queries across ASR (speech), OCR (text), and visual indices, averaging 1.78 modalities per query versus exhaustive 3.0 modality search. Evaluation on 1.8M video clips demonstrates that intelligent routing provides a practical solution for scaling multimodal retrieval systems, reducing infrastructure costs while maintaining competitive effectiveness for real‑world deployment.

Abstract:
Achieving precise word‑level typography control within generated images remains a persistent challenge. To address it, we newly construct a word‑level controlled scene text dataset and introduce the Text‑Image Alignment (TIA) framework. This framework leverages cross‑modal correspondence between text and local image regions provided by grounding models to enhance the Text‑to‑Image (T2I) model training. Furthermore, we propose WordCon, a hybrid parameter‑efficient fine‑tuning (PEFT) method. WordCon reparameterizes selective key parameters, improving both efficiency and portability. This allows seamless integration into diverse pipelines, including artistic text rendering, text editing, and image‑conditioned text rendering. To further enhance controllability, the masked loss at the latent level is applied to guide the model to concentrate on learning the text region in the image, and the joint‑attention loss provides feature‑level supervision to promote disentanglement between different words. Both qualitative and quantitative results demonstrate the superiority of our method to the state of the art. The datasets and source code will be available for academic use.

Abstract:
Scene text removal (STR) aims to erase textual elements from images. It was originally intended for removing privacy‑sensitiveor undesired texts from natural scene images, but is now also appliedto typographic images. STR typically detects text regions and theninpaints them. Although STR has advanced through neural networksand synthetic data, misuse risks have increased. This paper investi‑gates Inverse STR (ISTR), which analyzes STR‑processed images andfocuses on binary classification (detecting whether an image has un‑dergone STR) and localizing removed text regions. We demonstrate inexperiments that these tasks are achievable with high accuracies, en‑abling detection of potential misuse and improving STR. We also at‑tempt to recover the removed text content by training a text recognizerto understand its difficulty.

Abstract:
Large Multimodal Models (LMMs) have achieved impressive progress in visual perception and reasoning. However, when confronted with visually ambiguous or non‑semantic scene text, they often struggle to accurately spot and understand the content, frequently generating semantically plausible yet visually incorrect answers, which we refer to as semantic hallucination. In this work, we investigate the underlying causes of semantic hallucination and identify a key finding: Transformer layers in LLM with stronger attention focus on scene text regions are less prone to producing semantic hallucinations. Thus, we propose a training‑free semantic hallucination mitigation framework comprising two key components: (1) ZoomText, a coarse‑to‑fine strategy that identifies potential text regions without external detectors; and (2) Grounded Layer Correction, which adaptively leverages the internal representations from layers less prone to hallucination to guide decoding, correcting hallucinated outputs for non‑semantic samples while preserving the semantics of meaningful ones. To enable rigorous evaluation, we introduce TextHalu‑Bench, a benchmark of 1,740 samples spanning both semantic and non‑semantic cases, with manually curated question answer pairs designed to probe model hallucinations. Extensive experiments demonstrate that our method not only effectively mitigates semantic hallucination but also achieves strong performance on public benchmarks for scene text spotting and understanding.

Abstract:
Chinese scene text retrieval is a practical task that aims to search for images containing visual instances of a Chinese query text. This task is extremely challenging because Chinese text often features complex and diverse layouts in real‑world scenes. Current efforts tend to inherit the solution for English scene text retrieval, failing to achieve satisfactory performance. In this paper, we establish a Diversified Layout benchmark for Chinese Street View Text Retrieval (DL‑CSVTR), which is specifically designed to evaluate retrieval performance across various text layouts, including vertical, cross‑line, and partial alignments. To address the limitations in existing methods, we propose Chinese Scene Text Retrieval CLIP (CSTR‑CLIP), a novel model that integrates global visual information with multi‑granularity alignment training. CSTR‑CLIP applies a two‑stage training process to overcome previous limitations, such as the exclusion of visual features outside the text region and reliance on single‑granularity alignment, thereby enabling the model to effectively handle diverse text layouts. Experiments on existing benchmark show that CSTR‑CLIP outperforms the previous state‑of‑the‑art model by 18.82% accuracy and also provides faster inference speed. Further analysis on DL‑CSVTR confirms the superior performance of CSTR‑CLIP in handling various text layouts. The dataset and code will be publicly available to facilitate research in Chinese scene text retrieval.

Abstract:
Despite recent progress on the short‑video Text‑Visual Question Answering (ViteVQA) task ‑ largely driven by benchmarks such as M4‑ViteVQA ‑ existing datasets still suffer from limited video duration and narrow evaluation scopes, making it difficult to adequately assess the growing capabilities of powerful multimodal large language models (MLLMs). To address these limitations, we introduce TextVidBench, the first benchmark specifically designed for long‑video text question answering (>3 minutes). TextVidBench makes three key contributions: 1) Cross‑domain long‑video coverage: Spanning 9 categories (e.g., news, sports, gaming), with an average video length of 2306 seconds, enabling more realistic evaluation of long‑video understanding. 2) A three‑stage evaluation framework: "Text Needle‑in‑Haystack ‑> Temporal Grounding ‑> Text Dynamics Captioning". 3) High‑quality fine‑grained annotations: Containing over 5,000 question‑answer pairs with detailed semantic labeling. Furthermore, we propose an efficient paradigm for improving large models through: (i) introducing the IT‑Rope mechanism and temporal prompt engineering to enhance temporal perception, (ii) adopting non‑uniform positional encoding to better handle long video sequences, and (iii) applying lightweight fine‑tuning on video‑text data. Extensive experiments on multiple public datasets as well as TextVidBench demonstrate that our new benchmark presents significant challenges to existing models, while our proposed method offers valuable insights into improving long‑video scene text understanding capabilities.

Abstract:
While recent advancements in Image Super‑Resolution (SR) using diffusion models have shown promise in improving overall image quality, their application to scene text images has revealed limitations. These models often struggle with accurate text region localization and fail to effectively model image and multilingual character‑to‑shape priors. This leads to inconsistencies, the generation of hallucinated textures, and a decrease in the perceived quality of the super‑resolved text. To address these issues, we introduce TextSR, a multimodal diffusion model specifically designed for Multilingual Scene Text Image Super‑Resolution. TextSR leverages a text detector to pinpoint text regions within an image and then employs Optical Character Recognition (OCR) to extract multilingual text from these areas. The extracted text characters are then transformed into visual shapes using a UTF‑8 based text encoder and cross‑attention. Recognizing that OCR may sometimes produce inaccurate results in real‑world scenarios, we have developed two innovative methods to enhance the robustness of our model. By integrating text character priors with the low‑resolution text images, our model effectively guides the super‑resolution process, enhancing fine details within the text and improving overall legibility. The superior performance of our model on both the TextZoom and TextVQA datasets sets a new benchmark for STISR, underscoring the efficacy of our approach.

Abstract:
This study aims to investigate the challenge of insufficient three‑dimensional context in synthetic datasets for scene text rendering. Although recent advances in diffusion models and related techniques have improved certain aspects of scene text generation, most existing approaches continue to rely on 2D data, sourcing authentic training examples from movie posters and book covers, which limits their ability to capture the complex interactions among spatial layout and visual effects in real‑world scenes. In particular, traditional 2D datasets do not provide the necessary geometric cues for accurately embedding text into diverse backgrounds. To address this limitation, we propose a novel standard for constructing synthetic datasets that incorporates surface normals to enrich three‑dimensional scene characteristic. By adding surface normals to conventional 2D data, our approach aims to enhance the representation of spatial relationships and provide a more robust foundation for future scene text rendering methods. Extensive experiments demonstrate that datasets built under this new standard offer improved geometric context, facilitating further advancements in text rendering under complex 3D‑spatial conditions.

Abstract:
Diffusion‑based scene text synthesis has progressed rapidly, yet existing methods commonly rely on additional visual conditioning modules and require large‑scale annotated data to support multilingual generation. In this work, we revisit the necessity of complex auxiliary modules and further explore an approach that simultaneously ensures glyph accuracy and achieves high‑fidelity scene integration, by leveraging diffusion models' inherent capabilities for contextual reasoning. To this end, we introduce TextFlux, a DiT‑based framework that enables multilingual scene text synthesis. The advantages of TextFlux can be summarized as follows: (1) OCR‑free model architecture. TextFlux eliminates the need for OCR encoders (additional visual conditioning modules) that are specifically used to extract visual text‑related features. (2) Strong multilingual scalability. TextFlux is effective in low‑resource multilingual settings, and achieves strong performance in newly added languages with fewer than 1,000 samples. (3) Streamlined training setup. TextFlux is trained with only 1% of the training data required by competing methods. (4) Controllable multi‑line text generation. TextFlux offers flexible multi‑line synthesis with precise line‑level control, outperforming methods restricted to single‑line or rigid layouts. Extensive experiments and visualizations demonstrate that TextFlux outperforms previous methods in both qualitative and quantitative evaluations.

Abstract:
Scene text editing, a subfield of image editing, requires modifying texts in images while preserving style consistency and visual coherence with the surrounding environment. While diffusion‑based methods have shown promise in text generation, they still struggle to produce high‑quality results. These methods often generate distorted or unrecognizable characters, particularly when dealing with complex characters like Chinese. In such systems, characters are composed of intricate stroke patterns and spatial relationships that must be precisely maintained. We present GlyphMastero, a specialized glyph encoder designed to guide the latent diffusion model for generating texts with stroke‑level precision. Our key insight is that existing methods, despite using pretrained OCR models for feature extraction, fail to capture the hierarchical nature of text structures ‑ from individual strokes to stroke‑level interactions to overall character‑level structure. To address this, our glyph encoder explicitly models and captures the cross‑level interactions between local‑level individual characters and global‑level text lines through our novel glyph attention module. Meanwhile, our model implements a feature pyramid network to fuse the multi‑scale OCR backbone features at the global‑level. Through these cross‑level and multi‑scale fusions, we obtain more detailed glyph‑aware guidance, enabling precise control over the scene text generation process. Our method achieves an 18.02% improvement in sentence accuracy over the state‑of‑the‑art multi‑lingual scene text editing baseline, while simultaneously reducing the text‑region Fréchet inception distance by 53.28%.

Abstract:
Vision‑language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content. However, their robustness to common image corruptions remains under‑explored. In this work, we present the first comprehensive analysis of VLM robustness across 19 corruption types from the ImageNet‑C benchmark, spanning four categories: noise, blur, weather, and digital distortions. We introduce two new benchmarks, TextVQA‑C and GQA‑C, to systematically evaluate how corruptions affect scene text understanding and object‑based reasoning, respectively. Our analysis reveals that transformer‑based VLMs exhibit distinct vulnerability patterns across tasks: text recognition deteriorates most severely under blur and snow corruptions, while object reasoning shows higher sensitivity to corruptions such as frost and impulse noise. We connect these observations to the frequency‑domain characteristics of different corruptions, revealing how transformers' inherent bias toward low‑frequency processing explains their differential robustness patterns. Our findings provide valuable insights for developing more corruption‑robust vision‑language models for real‑world applications.

Abstract:
Recently supervised learning rapidly develops in scene text segmentation. However, the lack of high‑quality datasets and the high cost of pixel annotation greatly limit the development of them. Considering the well‑performed few‑shot learning methods for downstream tasks, we investigate the application of the few‑shot learning method to scene text segmentation. We propose TSAL, which leverages CLIP's prior knowledge to learn text attributes for segmentation. To fully utilize the semantic and texture information in the image, a visual‑guided branch is proposed to separately extract text and background features. To reduce data dependency and improve text detection accuracy, the adaptive prompt‑guided branch employs effective adaptive prompt templates to capture various text attributes. To enable adaptive prompts capture distinctive text features and complex background distribution, we propose Adaptive Feature Alignment module(AFA). By aligning learnable tokens of different attributes with visual features and prompt prototypes, AFA enables adaptive prompts to capture both general and distinctive attribute information. TSAL can capture the unique attributes of text and achieve precise segmentation using only few images. Experiments demonstrate that our method achieves SOTA performance on multiple text segmentation datasets under few‑shot settings and show great potential in text‑related domains.

Abstract:
Pursuing efficient text shape representations helps scene text detection models focus on compact foreground regions and optimize the contour reconstruction steps to simplify the whole detection pipeline. Current approaches either represent irregular shapes via box‑to‑polygon strategy or decomposing a contour into pieces for fitting gradually, the deficiency of coarse contours or complex pipelines always exists in these models. Considering the above issues, we introduce EdgeText to fit text contours compactly while alleviating excessive contour rebuilding processes. Concretely, it is observed that the two long edges of texts can be regarded as smooth curves. It allows us to build contours via continuous and smooth edges that cover text regions tightly instead of fitting piecewise, which helps avoid the two limitations in current models. Inspired by this observation, EdgeText formulates the text representation as the edge approximation problem via parameterized curve fitting functions. In the inference stage, our model starts with locating text centers, and then creating curve functions for approximating text edges relying on the points. Meanwhile, truncation points are determined based on the location features. In the end, extracting curve segments from curve functions by using the pixel coordinate information brought by truncation points to reconstruct text contours. Furthermore, considering the deep dependency of EdgeText on text edges, a bilateral enhanced perception (BEP) module is designed. It encourages our model to pay attention to the recognition of edge features. Additionally, to accelerate the learning of the curve function parameters, we introduce a proportional integral loss (PI‑loss) to force the proposed model to focus on the curve distribution and avoid being disturbed by text scales.

Abstract:
Scene text image super‑resolution (STISR) enhances the resolution and quality of low‑resolution images. Unlike previous studies that treated scene text images as natural images, recent methods using a text prior (TP), extracted from a pre‑trained text recognizer, have shown strong performance. However, two major issues emerge: (1) Explicit categorical priors, like TP, can negatively impact STISR if incorrect. We reveal that these explicit priors are unstable and propose replacing them with Non‑CAtegorical Prior (NCAP) using penultimate layer representations. (2) Pre‑trained recognizers used to generate TP struggle with low‑resolution images. To address this, most studies jointly train the recognizer with the STISR network to bridge the domain gap between low‑ and high‑resolution images, but this can cause an overconfidence phenomenon in the prior modality. We highlight this issue and propose a method to mitigate it by mixing hard and soft labels. Experiments on the TextZoom dataset demonstrate an improvement by 3.5%, while our method significantly enhances generalization performance by 14.8% across four text recognition datasets. Our method generalizes to all TP‑guided STISR networks.

Abstract:
Recent advancements in autoregressive and diffusion models have led to strong performance in image generation with short scene text words. However, generating coherent, long‑form text in images, such as paragraphs in slides or documents, remains a major challenge for current generative models. We present the first work specifically focused on long text image generation, addressing a critical gap in existing text‑to‑image systems that typically handle only brief phrases or single sentences. Through comprehensive analysis of state‑of‑the‑art autoregressive generation models, we identify the image tokenizer as a critical bottleneck in text generating quality. To address this, we introduce a novel text‑focused, binary tokenizer optimized for capturing detailed scene text features. Leveraging our tokenizer, we develop \ModelName, a multimodal autoregressive model that excels in generating high‑quality long‑text images with unprecedented fidelity. Our model offers robust controllability, enabling customization of text properties such as font style, size, color, and alignment. Extensive experiments demonstrate that \ModelName~significantly outperforms SD3.5 Large~\citesd3 and GPT4o~\citegpt4o with DALL‑E 3~\citedalle3 in generating long text accurately, consistently, and flexibly. Beyond its technical achievements, \ModelName~opens up exciting opportunities for innovative applications like interleaved document and PowerPoint generation, establishing a new frontier in long‑text image generating.

Abstract:
In recent years, vision transformers with text decoder have demonstrated remarkable performance on Scene Text Recognition (STR) due to their ability to capture long‑range dependencies and contextual relationships with high learning capacity. However, the computational and memory demands of these models are significant, limiting their deployment in resource‑constrained applications. To address this challenge, we propose an efficient and accurate STR system. Specifically, we focus on improving the efficiency of encoder models by introducing a cascaded‑transformers structure. This structure progressively reduces the vision token size during the encoding step, effectively eliminating redundant tokens and reducing computational cost. Our experimental results confirm that our STR system achieves comparable performance to state‑of‑the‑art baselines while substantially decreasing computational requirements. In particular, for large‑models, the accuracy remains same, 92.77 to 92.68, while computational complexity is almost halved with our structure.

Abstract:
Text images are unique in their dual nature, encompassing both visual and linguistic information. The visual component encompasses structural and appearance‑based features, while the linguistic dimension incorporates contextual and semantic elements. In scenarios with degraded visual quality, linguistic patterns serve as crucial supplements for comprehension, highlighting the necessity of integrating both aspects for robust scene text recognition (STR). Contemporary STR approaches often use language models or semantic reasoning modules to capture linguistic features, typically requiring large‑scale annotated datasets. Self‑supervised learning, which lacks annotations, presents challenges in disentangling linguistic features related to the global context. Typically, sequence contrastive learning emphasizes the alignment of local features, while masked image modeling (MIM) tends to exploit local structures to reconstruct visual patterns, resulting in limited linguistic knowledge. In this paper, we propose a Linguistics‑aware Masked Image Modeling (LMIM) approach, which channels the linguistic information into the decoding process of MIM through a separate branch. Specifically, we design a linguistics alignment module to extract vision‑independent features as linguistic guidance using inputs with different visual appearances. As features extend beyond mere visual structures, LMIM must consider the global context to achieve reconstruction. Extensive experiments on various benchmarks quantitatively demonstrate our state‑of‑the‑art performance, and attention visualizations qualitatively show the simultaneous capture of both visual and linguistic information.

Abstract:
Scaling architectures have been proven effective for improving Scene Text Recognition (STR), but the individual contribution of vision encoder and text decoder scaling remain under‑explored. In this work, we present an in‑depth empirical analysis and demonstrate that, contrary to previous observations, scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone. We also identify label noise as a key challenge in STR, particularly in real‑world data, which can limit the effectiveness of STR models. To address this, we propose Cloze Self‑Distillation (CSD), a method that mitigates label noise by distilling a student model from context‑aware soft predictions and pseudolabels generated by a teacher model. Additionally, we enhance the decoder architecture by introducing differential cross‑attention for STR. Our methodology achieves state‑of‑the‑art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.

Abstract:
Modern scene text recognition systems often depend on large end‑to‑end architectures that require extensive training and are prohibitively expensive for real‑time scenarios. In such cases, the deployment of heavy models becomes impractical due to constraints on memory, computational resources, and latency. To address these challenges, we propose a novel, training‑free plug‑and‑play framework that leverages the strengths of pre‑trained text recognizers while minimizing redundant computations. Our approach uses context‑based understanding and introduces an attention‑based segmentation stage, which refines candidate text regions at the pixel level, improving downstream recognition. Instead of performing traditional text detection that follows a block‑level comparison between feature map and source image and harnesses contextual information using pretrained captioners, allowing the framework to generate word predictions directly from scene context.Candidate texts are semantically and lexically evaluated to get a final score. Predictions that meet or exceed a pre‑defined confidence threshold bypass the heavier process of end‑to‑end text STR profiling, ensuring faster inference and cutting down on unnecessary computations. Experiments on public benchmarks demonstrate that our paradigm achieves performance on par with state‑of‑the‑art systems, yet requires substantially fewer resources.

Abstract:
During the steel billet production process, it is essential to recognize machine‑printed or manually written billet numbers on moving billets in real‑time. To address the issue of low recognition accuracy for existing scene text recognition methods, caused by factors such as image distortions and distribution differences between training and test data, we propose a billet number recognition method that integrates test‑time adaptation with prior knowledge. First, we introduce a test‑time adaptation method into a model that uses the DB network for text detection and the SVTR network for text recognition. By minimizing the model's entropy during the testing phase, the model can adapt to the distribution of test data without the need for supervised fine‑tuning. Second, we leverage the billet number encoding rules as prior knowledge to assess the validity of each recognition result. Invalid results, which do not comply with the encoding rules, are replaced. Finally, we introduce a validation mechanism into the CTC algorithm using prior knowledge to address its limitations in recognizing damaged characters. Experimental results on real datasets, including both machine‑printed billet numbers and handwritten billet numbers, show significant improvements in evaluation metrics, validating the effectiveness of the proposed method.

Abstract:
Designing datasets for Visual Question Answering (VQA) is a difficult and complex task that requires NLP for parsing and computer vision for analysing the relevant aspects of the image for answering the question asked. Several benchmark datasets have been developed by researchers but there are many issues with using them for methodical performance tests. This paper proposes a new benchmark dataset ‑‑ a pilot version called VQA‑Levels is ready now ‑‑ for testing VQA systems systematically and assisting researchers in advancing the field. The questions are classified into seven levels ranging from direct answers based on low‑level image features (without needing even a classifier) to those requiring high‑level abstraction of the entire image content. The questions in the dataset exhibit one or many of ten properties. Each is categorised into a specific level from 1 to 7. Levels 1 ‑ 3 are directly on the visual content while the remaining levels require extra knowledge about the objects in the image. Each question generally has a unique one or two‑word answer. The questions are 'natural' in the sense that a human is likely to ask such a question when seeing the images. An example question at Level 1 is, ``What is the shape of the red colored region in the image?" while at Level 7, it is, ``Why is the man cutting the paper?". Initial testing of the proposed dataset on some of the existing VQA systems reveals that their success is high on Level 1 (low level features) and Level 2 (object classification) questions, least on Level 3 (scene text) followed by Level 6 (extrapolation) and Level 7 (whole scene analysis) questions. The work in this paper will go a long way to systematically analyze VQA systems.

Abstract:
Multimodal large language models (MLLMs) have shown impressive capabilities across various domains, excelling in processing and understanding information from multiple modalities. Despite the rapid progress made previously, insufficient OCR ability hinders MLLMs from excelling in text‑related tasks. In this paper, we present Ocean‑OCR, a 3B MLLM with state‑of‑the‑art performance on various OCR scenarios and comparable understanding ability on general tasks. We employ Native Resolution ViT to enable variable resolution input and utilize a substantial collection of high‑quality OCR datasets to enhance the model performance. We demonstrate the superiority of Ocean‑OCR through comprehensive experiments on open‑source OCR benchmarks and across various OCR scenarios. These scenarios encompass document understanding, scene text recognition, and handwritten recognition, highlighting the robust OCR capabilities of Ocean‑OCR. Note that Ocean‑OCR is the first MLLM to outperform professional OCR models such as TextIn and PaddleOCR.

Abstract:
Vision‑Language Models (VLMs) have recently experienced significant advancements. However, challenges persist in the accurate recognition of fine details within high resolution images, which limits performance in multiple tasks. This work introduces Pheye, a novel architecture that efficiently processes high‑resolution images while training fewer parameters than similarly sized VLMs. Notably, Pheye achieves a high efficiency while maintaining strong performance, particularly in tasks that demand fine‑grained image understanding and/or the handling of scene‑text.